Part edge feature extraction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a part edge feature extraction method and device, electronic equipment and storage medium. The method comprises obtaining a target part image, inputting the target part image into a pre-trained saliency target detection model for feature extraction, and obtaining part edge features. The present application introduces an attention fusion structure between the encoder and the decoder, which better captures the differences in different regions of the image. Through the attention mechanism, the saliency target detection model can give different weights according to the differences in different regions of the image, thereby improving the detection performance of the saliency target detection model. A lightweight local large kernel convolution residual structure is proposed on the decoder output branch to help the detection model better capture the global context information in the image. Compared with small kernel convolution, large kernel convolution increases the network receptive field and improves the model accuracy and robustness. Based on this, the embodiments of the present application can accurately extract the edge features of the parts.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a part edge feature extraction method and device, an electronic device and a storage medium. BACKGROUND

[0002] The structure of electrical parts is complex and diverse, and there are many types. In the past, the method of manual measurement has been unable to meet the requirements of speed and accuracy of current industrial detection. In recent years, the machine vision measurement method based on has the advantages of non-contact, low cost and high efficiency, and is widely used in industrial part size measurement. Among them, edge contour feature extraction is the most critical link in size measurement. Once the edge feature extraction is improper (edge blur, shadow, etc.), it will directly affect the measurement accuracy.

[0003] At present, the existing machine vision method based on image processing is sensitive to light, noise and object shape changes, so the existing image processing method is difficult to accurately extract the edge features of the parts when facing complex structures, shape changes and complex electrical parts with different transparencies, so it is difficult to obtain accurate size measurement results. SUMMARY

[0004] The main purpose of the embodiment of the present application is to provide a part edge feature extraction method and device, an electronic device and a storage medium, which can accurately extract the edge features of the parts.

[0005] To achieve the above purpose, the first aspect of the embodiment of the present application provides a part edge feature extraction method, which comprises:

[0006] Obtaining a target part image;

[0007] Inputting the target part image into a pre-trained saliency target detection model for feature extraction to obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on the local details of the part edge features, the decoder comprises a local large kernel convolution residual structure PLR, and the part edge features are decoded by combining the local large kernel convolution residual structure PLR and bilinear interpolation.

[0008] In some embodiments, the training method of the saliency target detection model comprises:

[0009] Collecting part images, and constructing an initial sample data set according to the part images;

[0010] Building a basic detection model;

[0011] analyzing part features of the initial sample dataset, and improving the basic detection model according to the part features;

[0012] optimizing the basic detection model by introducing a cross-entropy loss, and updating network weights of the basic detection model;

[0013] obtaining a tuning sample and a new part dataset, training the basic detection model according to the tuning sample, the new part dataset and the network weights, and obtaining a trained saliency target detection model.

[0014] In some embodiments, the part image is collected, and an initial sample dataset is constructed according to the part image, comprising:

[0015] Defining categories of multiple parts, collecting high-resolution part images of different categories in an actual working scene, and storing the part images separately according to the categories;

[0016] According to the part to be measured position, the part images of different categories are labeled to obtain labeled images;

[0017] Randomly scaling, cropping, padding boundaries and randomly rotating the part images and the labeled images to obtain part datasets of different categories;

[0018] The obtained part datasets of different categories are respectively divided into training set, validation set and test set according to 6:2:2, and then combined into training set, validation set and test set containing all parts, to obtain the initial sample dataset.

[0019] In some embodiments, the calculation formula of the cross-entropy loss is as follows:

[0020] loss(p,gt)=weight[gt]*(-p[gt]+log(∑ j exp(p[j])))

[0021] Where p represents the predicted output of a certain pixel point, gt represents the true class of this pixel point, i.e. foreground or background, weight[gt] represents the weighting coefficient for weighting each class, p[gt] represents the predicted output of the pixel point with the true label gt, and p[j] represents the predicted output of the pixel point with the true label j.

[0022] In some embodiments, the outputs of different stages of the encoder have different distribution characteristics. The high-level features output by the deep stage have rich semantic information, and the low-level features output by the shallow stage have rich detail information.

[0023] In some embodiments, an attention fusion structure FAS is introduced between the encoder and the decoder, which is used to focus on the local details of the part edge features, including:

[0024] The high-level features are 2 times up-sampled to have the same resolution as the low-level features, and feature smoothing is performed through a 3x3 convolution. The attention fusion structure FAS first extracts the common features between the high-level features and the low-level features through element-wise multiplication; the high-level features are added to the common features to suppress the noise information in the common features fc; a spatial and channel attention mechanism CBAM module is introduced on the common features, which sequentially infers attention maps along two independent dimensions, multiplies the attention maps with the input feature maps for adaptive feature optimization, and fuses the high-level features with the common features fc again to obtain the output of the attention fusion structure FAS, the calculation formula is as follows:

[0025] f_c=f_l*f_H

[0026] f_l=f_l*f_H+A(f_c+f_H)

[0027] Where f_H represents the high-level features, f_l represents the low-level features, fc represents the common features, and A represents the channel and spatial attention mechanism CBAM.

[0028] In some embodiments, the feature decoding of the part edge features is implemented by combining the local large kernel convolution residual structure PLR and bilinear interpolation, including:

[0029] The feature map output by the deep stage is 2 times up-sampled using the bilinear interpolation method. Given the pixels of four pixel points (i, j), (i, j+1), (i+1, j), and (i+1, j+1), the pixel of the (i+u, j+v) point is obtained through the bilinear interpolation method as:

[0030] f(i+u,j+v)=(1-u)*(1-v)*f(i,j)+(1-u)*v*f(i,j+1)+u*(1-v)*f(i+1,j)+u*v*f(i+1,j+1);

[0031] The pixel of the (i+u, j+v) point is input to the local large kernel convolution residual structure PLR, which adopts a lightweight large kernel convolution to capture global features in the image, and adopts a residual connection to capture detailed information in the image. The feature map output by the attention fusion structure FAS is spliced with the low-level feature map output by the current adjacent stage, and the output of the local large kernel convolution residual structure PLR is added to realize multi-scale fusion of the features.

[0032] To achieve the above object, a second aspect of the embodiment of the present application provides a part edge feature extraction device, which comprises:

[0033] An acquisition module is configured to acquire a target part image.

[0034] An extraction module is configured to input the target part image into a pre-trained saliency target detection model to perform feature extraction and obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is configured to focus on local details of the part edge features, the decoder comprises a local large kernel convolution residual structure PLR, and the part edge features are decoded by combining the local large kernel convolution residual structure PLR and bilinear interpolation.

[0035] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0036] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0037] The part edge feature extraction method and device, electronic equipment and storage medium provided by the present application, by acquiring a target part image; input the target part image into a pre-trained saliency target detection model for feature extraction to obtain part edge features, wherein the saliency target detection model includes an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on the local details of the part edge features, and the decoder includes a local large kernel convolution residual structure PLR, and the part edge features are decoded by combining the local large kernel convolution residual structure PLR and the bilinear interpolation. Based on this, the present application introduces the attention fusion structure FAS between the encoder and the decoder, which better captures the differences of different regions in the image. Through the attention mechanism, the model can give different weights according to the differences of different regions in the image, thereby improving the detection performance of the model. The light local large kernel convolution residual structure PLR is proposed in the decoder part to help the model better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, which effectively alleviates the feature difference between the adjacent branches of the encoder, effectively suppresses the redundancy and background noise, and focuses on the local details of the features using the attention mechanism, thereby achieving the purpose of refining the edge. In the decoding stage, the feature decoding is realized by combining the local large kernel convolution residual structure PLR and the bilinear interpolation, so as to restore the information of the input image and obtain the final saliency target prediction map, and then the high-precision part edge features are extracted. Therefore, the part edge extraction method based on saliency target detection is proposed, which identifies and locates the measured part of the part pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the precision and detection efficiency of edge extraction in the part size measurement process. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of the part edge feature extraction method provided by the embodiment of the present application;

[0039] Figure 2 is a flowchart of the training method of the saliency target detection model provided by the embodiment of the present application;

[0040] Figure 3 is the overall flowchart of the training method of the saliency target detection model provided by another embodiment of the present application;

[0041] Figure 4 is a structure diagram of the improved reference detection model provided by the embodiment of the present application;

[0042] Figure 5is a structural schematic diagram of an attention fusion structure provided by an embodiment of the present application;

[0043] Figure 6 is a structural schematic diagram of a local large kernel convolution residual structure provided by an embodiment of the present application;

[0044] Figure 7 is a structural schematic diagram of a part edge feature extraction device provided by an embodiment of the present application;

[0045] Figure 8 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0047] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0048] In view of the technical problem in the prior art that it is difficult to accurately extract the edge features of a part, embodiments of the present application provide a part edge feature extraction method and device, an electronic device and a storage medium, which acquire a target part image; input the target part image into a pre-trained saliency target detection model for feature extraction to obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on local details of the part edge features, the decoder comprises a local large kernel convolution residual structure PLR, and feature decoding is realized by combining the local large kernel convolution residual structure PLR and bilinear interpolation on the part edge features. Based on this, the present application introduces the attention fusion structure FAS between the encoder and the decoder, better capturing the differences of different regions in the image. Through the attention mechanism, the model can give different weights according to the differences of different regions in the image, thereby improving the detection performance of the model. The light-weight local large kernel convolution residual structure PLR is proposed in the decoder part, which helps the model to better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the accuracy and robustness of the model. The attention fusion structure FAS is based on the attention fusion mechanism, effectively alleviating the feature difference between adjacent branches of the encoder, effectively suppressing redundancy and background noise, and focusing on local details of the features using the attention mechanism, thereby achieving the purpose of refining the edges. In the decoding stage, the present application realizes feature decoding by combining the local large kernel convolution residual structure PLR and bilinear interpolation, thereby restoring the information of the input image and obtaining the final saliency target prediction map, and then extracting high-precision part edge features. Therefore, the present application proposes a part edge extraction method based on saliency target detection, which identifies and locates the measurement part of the part pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the accuracy and detection efficiency of edge extraction in the part size measurement process.

[0049] The part edge feature extraction method and device, the electronic device and the storage medium provided by the embodiments of the present application are specifically described through the following embodiments. First, the part edge feature extraction method in the embodiments of the present application is described.

[0050] Figure 1 is an optional flowchart of the part edge feature extraction method provided by the embodiments of the present application, Figure 1 The method in the above embodiment can include but is not limited to steps S101 to S102.

[0051] Step S101, acquiring a target part image;

[0052] In step S102, the target part image is input into a pre-trained salient object detection model for feature extraction to obtain part edge features. The salient object detection model includes an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder. The attention fusion structure FAS is used to focus on the local details of the part edge features. The decoder includes a local large kernel convolution residual structure PLR. The feature decoding is realized by combining the local large kernel convolution residual structure PLR and the bilinear interpolation.

[0053] In some embodiments, by acquiring a target part image, the target part image is input into a pre-trained salient object detection model for feature extraction to obtain part edge features. It should be noted that the feature extraction process includes feature encoding and feature decoding. The present application introduces an attention fusion structure FAS between the encoder and the decoder, which better captures the differences in different regions of the image. Through the attention mechanism, the salient object detection model can give different weights according to the differences in different regions of the image, thereby improving the detection performance of the salient object detection model. A lightweight local large kernel convolution residual structure PLR is proposed in the decoder part to help the salient object detection model better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, which effectively alleviates the feature difference between adjacent branches of the encoder, effectively suppresses redundant and background noise, and focuses on local details of features using the attention mechanism, thereby achieving the purpose of refining edges. In the decoding stage, the feature decoding is realized by combining the local large kernel convolution residual structure PLR and the bilinear interpolation, so as to restore the information of the input image and obtain the final salient object prediction map, and then extract the high-precision part edge features.

[0054] Please refer to Figure 2 In some embodiments, the training method of the salient object detection model can include but is not limited to steps S201 to S205:

[0055] In step S201, part images are collected, and an initial sample data set is constructed according to the part images.

[0056] In step S202, a basic detection model is built.

[0057] In step S203, the part features of the initial sample data set are analyzed, and the basic detection model is improved according to the part features.

[0058] In step S204, the basic detection model is optimized by introducing a cross-entropy loss, and the network weights of the basic detection model are updated.

[0059] Step S205, obtain the tuning sample and the new part data set, train the basic detection model according to the tuning sample, the new part data set and the network weight, and obtain the trained saliency target detection model.

[0060] In some embodiments, as shown in Figure 3 The training method of the saliency target detection model specifically includes the following steps:

[0061] Step S1: collect part images and construct an initial sample data set.

[0062] The specific process of step S1 is as follows:

[0063] S1.1: first, define the categories of multiple parts, collect high-resolution part images of different categories in actual working scenes, and store them separately according to categories.

[0064] S1.2: according to the part measurement position to be measured, mark the relevant area of the parts of different categories. Labelme is used to mark each pixel point, and 2 pixel values (0 or 255) are used to distinguish the foreground (part measurement position to be measured) and the background, so as to obtain a binary label image as the true value of the output image.

[0065] S1.3: on the training set, the input and the marked image are randomly scaled, cropped, filled with boundaries and randomly rotated, so as to expand the training set, reduce the overfitting problem of the model and improve the robustness of the model.

[0066] S1.4: the obtained part data sets of different categories (including part original images and labels) are respectively divided into training set, validation set and test set according to 6:2:2, and then the training set, the validation set and the test set containing all parts are summarized, so as to obtain the initial part sample data set.

[0067] Step S2: build a basic detection model.

[0068] According to the detection requirements, high-precision extraction of part edge features is required, so multiple models focusing on edge detection quality are selected. Then the constructed part data set is input into the model and trained. One of the selected models with better performance is selected as the basic detection model.

[0069] Step S3: analyze the characteristics of the part data set and improve the existing basic detection model.

[0070] The characteristics of different types of parts are analyzed, and the base model is improved in combination with various features. In view of the problems of complex part structure, irregular shape and large scale change, an attention fusion mechanism is proposed between the encoder and the decoder, so that the differences of different regions in the image can be better captured. Through the attention mechanism, the model can give different weights according to the differences of different regions in the image, so as to improve the detection performance of the model. A light-weight large-core convolution structure is proposed in the decoder part, which helps the model to better capture the global context information in the image. Compared with the traditional small-core convolution, the large-core convolution increases the network receptive field and improves the model accuracy and robustness.

[0071] The specific process of step S3 is as follows:

[0072] S3.1 Encoder

[0073] ResNet50 is used as the encoder of the model, and a part image with a size of 512x512 is input into the encoder, and then sequentially passes through 5 stages (Conv1, Conv2, Conv3, Conv4, Conv5), and finally the part image is down-sampled to 16x16 size, and the number of feature map channels is expanded from 64 to 1024, as shown in Figure 4 .

[0074] S3.2 Attention fusion structure

[0075] An attention fusion structure (FAS) is introduced between the encoder and the decoder, as shown in Figure 5 . The outputs of different stages of the encoder have different distribution characteristics. The high-level features output by the deep stage have rich semantic information, but lack accurate position information. The low-level features output by the shallow stage have rich detail information, but are full of background noise. In order to generate a better saliency map, the output features of multiple stages need to be combined. However, if there is no fine control of information in the model, some redundant features, including noise from low layers and coarse boundary layers from high layers, will pass through and may cause performance degradation. Therefore, we introduce an attention fusion mechanism between the encoder and the decoder, which effectively alleviates the feature differences between adjacent branches of the encoder, effectively suppresses redundant and background noise, and focuses on local details of the features using the attention mechanism, so as to achieve the purpose of refining edges.

[0076] The specific implementation is as follows:

[0077] The high-level features and low-level features output by the adjacent stages of the encoder are input into the feature attention fusion structure (FAS) as two information flows. First, the high-level features f_h are upsampled by 2 times to have the same resolution as the low-level features f_l, i.e., f_H, and are smoothed by a 3x3 convolution. The feature attention fusion structure (FAS) first extracts the common part fc between the high-level features and the low-level features through element-wise multiplication. Since the high-level features have strong semantic information and less noise, the high-level features f_H are added to the common features fc (element-wise addition) to further strengthen the semantic feature representation and suppress the noise information in the common feature information. Then, a spatial and channel attention mechanism CBAM is introduced on the feature information. The CBAM module sequentially infers attention maps along two independent dimensions (channel and space), and then multiplies the attention maps with the input feature map to adaptively optimize the features. Finally, the output of the feature attention fusion structure (FAS) is obtained by fusing the common feature information fc again (element-wise addition). The formula is as follows:

[0078] f_c = f_l * f_H

[0079] f_l = f_l * f_H + A (f_c + f_H)

[0080] where f_H represents high-level features, f_l represents low-level features, fc represents common features, and A represents the channel and spatial attention mechanism CBAM.

[0081] S3.3 Decoder

[0082] In the decoding stage, the application realizes feature decoding by combining the local large kernel convolution residual structure (PLR) and bilinear interpolation to restore the information of the input image and obtain the output saliency map. The specific implementation is as follows:

[0083] S3.3.1: First, the feature map output by the deep stage is upsampled by 2 times using the bilinear interpolation method. Given the pixels of four pixel points (i, j), (i, j+1), (i+1, j), and (i+1, j+1), the pixel of the (i+u, j+v) point is obtained by bilinear interpolation as follows:

[0084] f(i+u,j+v) = (1-u)*(1-v)*f(i,j) + (1-u)*v*f(i,j+1) + u*(1-v)*f(i+1,j) + u*v*f(i+1,j+1);

[0085] The pixel of the (i+u, j+v) point is input into the local large kernel convolution residual structure (PLR), as shown in Figure 6As shown. Lightweight large kernel convolution is used to increase the feature receptive field and better capture global features in the image. The residual connection is used to promote gradient propagation and learn more detailed information. Then, the attention fusion structure output feature map is spliced with the low-level feature map output by the adjacent stage, and the output of the local large kernel convolution residual structure (PLR) is added to realize multi-scale fusion of features. Finally, the feature map output by the five local large kernel convolution residual structures (PLR) is compressed to 1 by 1x1 convolution, and O1, O2, O3, O4, and O5 are output. The five outputs are spliced, and the channel is compressed to 1 by 1x1 convolution to obtain the final output O6.

[0086] Step S4: optimize the improved model and constantly update the network weight

[0087] The improved network is optimized by introducing the cross-entropy loss, which is as follows:

[0088] loss(p,gt)=weight[gt]*(-p[gt]+log(∑ j exp(p[j])))

[0089] Where p represents the predicted output of a pixel point, gt represents the true class of the pixel point, i.e. foreground or background, weight[gt] represents the weighting coefficient for each class, p[gt] represents the predicted output of the pixel point with the true label gt, and p[j] represents the predicted output of the pixel point with the true label j.

[0090] Step S5: make tuning samples and add new part data set

[0091] S5.1: Continue to expand the part data set, collect the part categories already included in the data set, continue to perform labeling and data enhancement operations, and add to the original training set to increase the robustness of the model.

[0092] S5.2: Add new part types, from part definition, image acquisition, sample labeling to data enhancement, and add the new part data set to the original data set according to 6:2:2 respectively. Add to the training set, validation set, and test set. Then, constantly improve the model and optimize the network to increase the generalization ability of the model.

[0093] Step S6: improve and optimize the model to obtain the final saliency target detection model.

[0094] Input the test set, use P-R curve, ROC curve and MAE quantitative index to calculate the detection accuracy, continue to improve and optimize the model, obtain the final saliency target prediction map, and then extract high-precision part edge features.

[0095] In summary, the present application obtains a target part image, inputs the target part image into a pre-trained salient object detection model for feature extraction to obtain part edge features. The present application introduces an attention fusion structure FAS between the encoder and the decoder to better capture the differences in different regions of the image. Through the attention mechanism, the salient object detection model can give different weights according to the differences in different regions of the image, thereby improving the detection performance of the salient object detection model. A lightweight local large kernel convolution residual structure PLR is proposed in the decoder part to help the salient object detection model better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, which effectively alleviates the feature differences between adjacent branches of the encoder, effectively suppresses redundant and background noise, and focuses on local details of the features using the attention mechanism, thereby achieving the purpose of refining the edges. In the decoding stage, the present application realizes feature decoding by combining the local large kernel convolution residual structure PLR and the bilinear interpolation, thereby restoring the information of the input image and obtaining the final salient object prediction map, and then extracting high-precision part edge features. Therefore, the present application proposes a part edge extraction method based on salient object detection, which identifies and locates the measured part of the part pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the edge extraction precision and detection efficiency in the part size measurement process.

[0096] Referring to Figure 7 The embodiment of the present application also provides a part edge feature extraction device, which can realize the part edge feature extraction method described above, and the device comprises:

[0097] The acquisition module 710 is configured to acquire a target part image.

[0098] The extraction module 720 is configured to input the target part image into a pre-trained salient object detection model for feature extraction to obtain part edge features, wherein the salient object detection model comprises an encoder and a decoder, an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on local details of the part edge features, the decoder comprises a local large kernel convolution residual structure PLR, and feature decoding is realized on the part edge features by combining the local large kernel convolution residual structure PLR and the bilinear interpolation.

[0099] Based on this, the part edge feature extraction device of the embodiment of the present application acquires the target part image through the acquisition module 710; the extraction module 720 inputs the target part image into the pre-trained saliency target detection model to perform feature extraction, and obtains the part edge feature, wherein the saliency target detection model includes an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on the local details of the part edge feature, the decoder includes a local large kernel convolution residual structure PLR, and the feature decoding is realized on the part edge feature by combining the local large kernel convolution residual structure PLR and the bilinear interpolation. Based on this, the present application introduces the attention fusion structure FAS between the encoder and the decoder, and better captures the differences of different regions in the image. Through the attention mechanism, the model can give different weights according to the differences of different regions in the image, thereby improving the detection performance of the model. The light-weight local large kernel convolution residual structure PLR is proposed in the decoder part, which helps the model to better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, which effectively alleviates the feature difference between the adjacent branches of the encoder, effectively suppresses the redundancy and background noise, and focuses on the local details of the feature through the attention mechanism, thereby achieving the purpose of refining the edge. In the decoding stage, the feature decoding is realized by combining the local large kernel convolution residual structure PLR and the bilinear interpolation, so as to restore the information of the input image and obtain the final saliency target prediction map, and then the high-precision part edge feature is extracted. Therefore, the part edge extraction method based on saliency target detection is proposed, which identifies and locates the part to be measured pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the precision and detection efficiency of edge extraction in the part size measurement process.

[0100] The specific embodiment of the part edge feature extraction device is basically the same as the specific embodiment of the part edge feature extraction method described above, and will not be repeated here.

[0101] The embodiment of the present application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the part edge feature extraction method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer, a vehicle-mounted computer or the like.

[0102] Please refer to Figure 8 , Figure 8 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0103] The processor 801 can be implemented in a manner of a general CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute a related program to implement the technical solutions provided by the embodiment of the present application.

[0104] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM), etc. The memory 802 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 802 and are called and executed by the processor 801 to perform the part edge feature extraction method of the embodiments of the present application, that is, by acquiring a target part image; inputting the target part image into a pre-trained salient object detection model to extract features to obtain part edge features, wherein the salient object detection model includes an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on the local details of the part edge features, the decoder includes a local large kernel convolution residual structure PLR, and the part edge features are decoded by combining the local large kernel convolution residual structure PLR and the bilinear interpolation. Based on this, the present application introduces the attention fusion structure FAS between the encoder and the decoder, which better captures the differences in different regions of the image. Through the attention mechanism, the model can give different weights according to the differences in different regions of the image, thereby improving the detection performance of the model. In the decoder part, a light-weight local large kernel convolution residual structure PLR is proposed to help the model better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, which effectively alleviates the feature difference between the adjacent branches of the encoder, effectively suppresses the redundancy and background noise, and uses the attention mechanism to focus on the local details of the features, thereby achieving the purpose of refining the edges. In the decoding stage, the present application realizes feature decoding by combining the local large kernel convolution residual structure PLR and the bilinear interpolation, so as to restore the information of the input image and obtain the final salient object prediction map, and then extract high-precision part edge features. Therefore, the present application proposes a part edge extraction method based on salient object detection, which identifies and locates the measured part of the part pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the precision and detection efficiency of edge extraction in the part size measurement process.

[0105] The input / output interface 803 is used to realize information input and output.

[0106] The communication interface 804 is used to realize the communication interaction between the device and other devices, which can realize communication through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0107] A bus transmits information between various components (for example, the processor 801, the memory 802, the input / output interface 803, and the communication interface 804) of the device.

[0108] The processor 801, the memory 802, the input / output interface 803, and the communication interface 804 are communicatively connected to each other inside the device through the bus.

[0109] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the part edge feature extraction method.

[0110] The memory is a non-transitory computer readable storage medium, and can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0111] The part edge feature extraction method, part edge feature extraction device, electronic equipment and storage medium provided by the embodiment of the application, by acquiring a target part image; inputting the target part image into a pre-trained saliency target detection model for feature extraction to obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, and an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on local details of the part edge features, the decoder comprises a local large kernel convolution residual structure PLR, and feature decoding is realized on the part edge features by combining the local large kernel convolution residual structure PLR and bilinear interpolation. Based on this, the application introduces the attention fusion structure FAS between the encoder and the decoder, better capturing the differences of different regions in the image. Through the attention mechanism, the model can give different weights according to the differences of different regions in the image, thereby improving the detection performance of the model. The light-weight local large kernel convolution residual structure PLR is proposed in the decoder part, which helps the model to better capture the global context information in the image. Compared with the traditional small kernel convolution, the large kernel convolution increases the network receptive field and improves the model accuracy and robustness. The attention fusion structure FAS is based on the attention fusion mechanism, effectively alleviating the feature difference between adjacent branches of the encoder, effectively suppressing redundancy and background noise, and focusing on local details of the features by using the attention mechanism, thereby achieving the purpose of refining the edge. In the decoding stage, the application realizes feature decoding by combining the local large kernel convolution residual structure PLR and bilinear interpolation, thereby restoring the information of the input image and obtaining the output final saliency target prediction map, and then extracting high-precision part edge features. Therefore, the part edge extraction method based on saliency target detection is proposed, which identifies and locates the measured part of the part pixel by pixel, solves the edge extraction problem of various complex parts, extracts high-precision part edge features, and effectively improves the edge extraction precision and detection efficiency in the part size measurement process.

[0112] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented by software, firmware, hardware, and appropriate combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable programs, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable programs, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0113] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems as technology evolves and new application scenarios appear.

[0114] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0115] The device embodiments described above are only schematic, and units described as separate components can or can not be physically separate, i.e. can be located in one place, or can be distributed over multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0116] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0117] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on completion of some desired or other convenient events, or based on other modification that can be wished to those with average skill in the art. Additionally, the terms "comprising", "having", "including", and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to", "comprising, but not limited to", "having, but not limited to", or "including, but not limited to") unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein and each separate value is incorporated into the specification as if it were individually recited herein. The use of any of the following terms "coupled", "connected", or "communicatively coupled", means the elements so connected are electrically or otherwise directly connected without any additional intervening elements. The use of any of the following terms "in communication with", "in connection with", or "in communication therewith", means the elements so connected are electrically or otherwise directly connected without any additional intervening elements.

[0118] It should be understood that, in this application, "at least one", means one or more, "multiple" means two or more. "And / or", used to describe the relationship between associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including single or multiple items. For example, at least one of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0119] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0120] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0121] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0122] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0123] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method of part edge feature extraction, the method comprising: The method comprises: acquiring a target part image; inputting the target part image into a pre-trained saliency target detection model to extract features, to obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on local details of the part edge features, the outputs of different stages of the encoder have different distribution features, high-level features output by a deep stage have rich semantic information, low-level features output by a shallow stage have rich detail information, the decoder comprises a local large kernel convolution residual structure PLR, and feature decoding is realized on the part edge features by combining the local large kernel convolution residual structure PLR and bilinear interpolation; wherein an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is used to focus on local details of the part edge features, and comprises: performing 2 times up-sampling on the high-level features to make the high-level features have the same resolution as the low-level features, and performing feature smoothing through a 3x3 convolution, the attention fusion structure FAS first extracts common features between the high-level features and the low-level features through element-wise multiplication; the high-level features are continuously added to the common features to suppress noise information in the common features fc; a spatial and channel attention mechanism CBAM module is introduced on the common features, the CBAM module sequentially infers attention maps along two independent dimensions, the attention maps are multiplied with input feature maps to perform adaptive feature optimization, the high-level features are fused with the common features fc again, to obtain an output of the attention fusion structure FAS, and a calculation formula is as follows: f_c=f_l*f_H f_l=f_l*f_H+A(f_c+f_H) wherein f_H represents high-level features, f_l represents low-level features, fc represents common features, and A represents a channel and spatial attention mechanism CBAM; wherein the feature decoding on the part edge features is realized by combining the local large kernel convolution residual structure PLR and bilinear interpolation, and comprises: performing 2 times up-sampling on feature maps output by a deep stage using a bilinear interpolation method, given pixels of four pixel points (i,j), (i,j+1), (i+1,j), and (i+1,j+1), a pixel of a (i+u,j+v) point is obtained through bilinear interpolation, and the pixel is as follows: f(i+u,j+v)=(1-u)*(1-v)*f(i,j)+(1-u)*v*f(i,j+1)+u*(1-v)*f(i+1,j)+u*v*f(i+1,j+1) The pixel of the (i+u, j+v) point is input to a local large kernel convolution residual structure PLR, and a lightweight large kernel convolution is adopted to capture global features in the image; and a residual connection mode is adopted to capture detailed information in the image; a feature map output by an attention fusion structure FAS is spliced with a low-level feature map output by a current adjacent stage, and an output of the local large kernel convolution residual structure PLR is added, so that multi-scale fusion of features is realized.

2. The method of claim 1, wherein, The training method of the saliency object detection model comprises: Collecting a part image, and constructing an initial sample data set according to the part image; Building a basic detection model; Analyzing part features of the initial sample data set, and improving the basic detection model according to the part features; Optimizing the basic detection model by introducing a cross-entropy loss, and updating network weights of the basic detection model; Obtaining a tuning sample and a new part data set, training the basic detection model according to the tuning sample, the new part data set and the network weights, and obtaining the trained saliency object detection model.

3. The method of claim 2, wherein, The collecting of the part image and the constructing of the initial sample data set according to the part image comprise: Defining categories of multiple parts, collecting high-resolution part images of different categories in an actual working scene, and storing the part images separately according to the categories; According to the part measurement position, the part images of different categories are labeled to obtain labeled images; The part images and the labeled images are randomly scaled, cropped, boundary filled and randomly rotated to obtain part data sets of different categories; The obtained part data sets of different categories are respectively divided into a training set, a verification set and a test set according to 6:2:2, and then the training set, the verification set and the test set containing all parts are summarized to obtain the initial sample data set.

4. The method of claim 2, wherein, The calculation formula of the cross-entropy loss is as follows: loss(p, gt) = weight[gt] * (-p[gt] + log(∑ j exp(p[j]))) Wherein, p represents the predicted output of a certain pixel point, gt represents the real class of the pixel point, that is, foreground or background, weight[gt] represents a weighting coefficient for weighting each class, p[gt] represents the predicted output of the pixel point with the real label being gt, and p[j] represents the predicted output of the pixel point with the real label being j.

5. A part edge feature extraction apparatus characterized by, The device comprises: An acquisition module configured to acquire a target part image; The extraction module is configured to input the target part image into a pre-trained saliency target detection model to perform feature extraction and obtain part edge features, wherein the saliency target detection model comprises an encoder and a decoder, an attention fusion structure FAS is introduced between the encoder and the decoder, the attention fusion structure FAS is configured to focus on local details of the part edge features, the outputs of different stages of the encoder have different distribution features, high-level features output by a deep stage have rich semantic information, and low-level features output by a shallow stage have rich detail information, and the decoder comprises a local large kernel convolution residual structure PLR, and the part edge features are decoded by combining the local large kernel convolution residual structure PLR and bilinear interpolation; The attention fusion structure FAS is introduced between the encoder and the decoder, and the attention fusion structure FAS is configured to focus on local details of the part edge features, and comprises: The high-level features are 2 times up-sampled to have the same resolution as the low-level features, and a 3x3 convolution is performed for feature smoothing, the attention fusion structure FAS first extracts common features between the high-level features and the low-level features through element-wise multiplication; the high-level features are continuously added to the common features to suppress noise information in the common features fc; a spatial and channel attention mechanism CBAM module is introduced on the common features, the CBAM module sequentially infers attention maps along two independent dimensions, the attention maps are multiplied with input feature maps for adaptive feature optimization, the high-level features are fused with the common features fc again to obtain an output of the attention fusion structure FAS, and a calculation formula is as follows: f_c=f_l*f_H f_l=f_l*f_H+A(f_c+f_H) Wherein, f_H represents high-level features, f_l represents low-level features, fc represents common features, and A represents a channel and spatial attention mechanism CBAM; The feature decoding of the part edge features by combining the local large kernel convolution residual structure PLR and bilinear interpolation comprises: The feature maps output by the deep stage are 2 times up-sampled using the bilinear interpolation method, four pixel points (i, j), (i, j+1), (i+1, j) and (i+1, j+1) are known, and the pixel of (i+u, j+v) is obtained by the bilinear interpolation method as follows: f(i+u,j+v)=(1-u)*(1-v)*f(i,j)+(1-u)*v*f(i,j+1)+u*(1-v)*f(i+1,j)+u*v*f(i+1,j+1) The pixel of the (i+u, j+v) point is input to a local large kernel convolution residual structure PLR, a lightweight large kernel convolution is adopted to capture global features in the image, and a residual connection mode is adopted to capture detailed information in the image; a feature map output by an attention fusion structure FAS is spliced with a low-level feature map output by a current adjacent stage, and an output of the local large kernel convolution residual structure PLR is added, so that multi-scale fusion of features is realized.

6. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the part edge feature extraction method in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor to implement the part edge feature extraction method in any one of claims 1 to 4.

Images