A workpiece inner hole defect detection method and system based on multi-modal feature fusion

Abstract

The application provides a workpiece inner hole defect detection method and system based on multi-form feature fusion, and relates to the technical field of image processing.The method comprises the following steps: acquiring a surround-view image of a workpiece inner hole, and obtaining a rectangular development image according to the surround-view image; performing double-branch feature extraction on the surround-view image and the rectangular development image to obtain a double-form feature pair, wherein the double-form feature pair comprises a circular ring feature and a rectangular feature; obtaining a circular ring development feature according to the circular ring feature; performing multi-form feature fusion on the circular ring development feature and the rectangular feature to obtain a fusion feature; performing defect detection according to the fusion feature to obtain a corresponding defect detection result, thereby ensuring the accuracy of inner hole defect detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and system for detecting defects in the inner hole of a workpiece based on multi-morphological feature fusion. Background Technology

[0002] In modern engineering manufacturing, the inner bore of a workpiece is a crucial component, undertaking core functions such as connection, transmission, and sealing. However, in actual industrial production, inner bores may face threats from various potential defects such as corrosion, scaling, wear, and cracks. These defects can not only affect the normal function and safety performance of the workpiece but also seriously impact the overall product quality and user experience. Therefore, accurate defect detection of the inner bore of workpieces is essential. It is not only a key link in ensuring the safety and quality of the overall manufacturing process but also an important measure to guarantee product reliability and user satisfaction.

[0003] In the field of industrial visual inspection, convolutional neural networks have demonstrated superior performance compared to traditional methods due to their efficient image processing capabilities and accuracy. However, for defect detection using panoramic imaging of workpiece inner holes, existing visual inspection methods still require optimization and improvement. Some solutions use panoramic imaging as the input source for feature extraction, but distortion in panoramic imaging results in smaller defects near the extreme points, making it difficult to detect small internal defects. Other solutions use rectangular features for defect detection, leading to feature loss during feature extraction and pixel distortion in the original image during unfolding, resulting in false detections. Summary of the Invention

[0004] The problem addressed by this invention is how to increase the accuracy of internal hole defect detection.

[0005] To address the above problems, this invention provides a method for detecting defects in the inner hole of a workpiece based on multi-morphological feature fusion, comprising:

[0006] Obtain a panoramic image of the inner hole of the workpiece, and obtain a rectangular unfolded image based on the panoramic image;

[0007] The panoramic imaging and the rectangular unfolded image are subjected to bi-branch feature extraction to obtain a dual-morphological feature pair, wherein the dual-morphological feature pair includes a ring feature and a rectangle feature;

[0008] The circular unfolding feature is obtained based on the circular ring feature;

[0009] The circular unfolded feature and the rectangular feature are fused into a multi-morphological feature to obtain a fused feature. Defect detection is performed based on the fused feature to obtain the corresponding defect detection result.

[0010] Compared to existing technologies, this invention performs rectangular unfolding processing on the initial panoramic imaging to obtain a rectangular unfolded image. This step forms two independent branches: the panoramic imaging branch and the rectangular unfolded image branch. By extracting features from these two branches, the integrity of the annular features can be preserved, while undistorted features can be retained through the rectangular features. Unfolding the extracted annular features yields annular unfolded features at a different level from the original annular features. This step ensures more accurate defect detection results. Fusing the rectangular features with the annular unfolded features effectively compensates for the imaging distortion and feature loss that may occur with single features in defect detection. By fusing the two types of features, the detection model can more comprehensively and accurately capture the characterization of the workpiece's inner hole, thereby significantly improving the accuracy of detecting defects in inner hole workpieces.

[0011] Optionally, obtaining a panoramic image of the workpiece's inner hole and obtaining a rectangular unfolded image based on the panoramic image includes:

[0012] The target size of the rectangular unfolded image is determined based on the input parameters of the surround-view imaging;

[0013] Based on the target size, the surround view image is mapped to the original surround view image coordinates to obtain a rectangular unfolded image.

[0014] Optionally, the step of mapping the surround view image to the original surround view image coordinates according to the target size to obtain a rectangular unfolded image includes:

[0015] Based on the target size, the surround view image is mapped to a Cartesian coordinate system to obtain the first coordinates;

[0016] Convert each point in the first coordinate system to polar coordinates to obtain the second coordinate system;

[0017] Each point in the second coordinate system is converted to Cartesian coordinates under the annular image to obtain the mapping relationship between each point of the panoramic imaging and the rectangular unfolded image.

[0018] By using bilinear interpolation, the pixels of the panoramic imaging are mapped to the Cartesian coordinate system according to the mapping relationship, thereby obtaining the rectangular unfolded image.

[0019] Optionally, the step of performing bi-branch feature extraction on the surround-view imaging and the rectangular unfolded image to obtain dual-morphological feature pairs includes:

[0020] At least two parallel backbone networks are constructed based on convolutional layers, residual blocks, and feature pyramids, wherein the backbone networks are used for feature extraction independently.

[0021] The features of the panoramic imaging and the rectangular unfolded image are extracted according to the dual-branch feature extraction network to obtain the dual-morphological feature pair, wherein the dual-branch feature extraction network is composed of the backbone network.

[0022] Optionally, the backbone network includes a first module, a second module, and an SPPF module from top to bottom. The first module includes at least two first convolutional layers and a computer vision module, and the second module includes at least one second convolutional layer and a computer vision module corresponding to the second convolutional layer.

[0023] Optionally, the step of performing bi-branch feature extraction on the surround-view imaging and the rectangular unfolded image to obtain dual-morphological feature pairs includes:

[0024] The surround view image and the rectangular unfolded image are subjected to bi-branch feature extraction to obtain at least three bimorphic feature pairs, wherein each bimorphic feature pair is obtained from the same level output of the bi-branch feature extraction network.

[0025] Optionally, obtaining the annular unfolding feature based on the annular feature includes:

[0026] The annular feature is processed by the first geometric reshaping network, the second geometric reshaping network, and the third geometric reshaping network, respectively, to obtain the first unfolded feature, the second unfolded feature, and the third unfolded feature, wherein the ratio between the size of the first unfolded feature and the size of the surround view image, the ratio between the size of the second unfolded feature and the size of the surround view image, and the ratio between the size of the third unfolded feature and the size of the surround view image are respectively...

[0027] Optionally, the step of fusing the expanded annular feature and the rectangular feature with multiple morphological features to obtain the fused feature includes:

[0028] The circular unfolded feature and the rectangular feature are convolved respectively. After convolution, the corresponding circular unfolded feature and the rectangular feature are concatenated to obtain the concatenated feature.

[0029] The spliced ​​features are input into different residual networks to obtain feature extraction results respectively;

[0030] The fused features are obtained based on the feature extraction results.

[0031] Optionally, the input parameters include the outer circumference radius, the inner circumference radius, and the center of the circle.

[0032] Secondly, the present invention also provides a workpiece internal hole defect detection system based on multi-morphological feature fusion, comprising:

[0033] A rectangular unfolding unit is used to acquire a panoramic image of the inner hole of a workpiece, and to obtain a rectangular unfolded image based on the panoramic image.

[0034] A dual-branch feature extraction unit is used to perform dual-branch feature extraction on the panoramic imaging and the rectangular unfolded image to obtain dual-morphological feature pairs, wherein the dual-morphological feature pairs include annular features and rectangular features;

[0035] A ring unfolding unit, used to obtain ring unfolding features based on the ring features;

[0036] The feature fusion unit is used to fuse the circular unfolded feature and the rectangular feature into multi-morphological features to obtain fused features, and to perform defect detection based on the fused features to obtain corresponding defect detection results.

[0037] The beneficial effects of the workpiece internal hole defect detection system based on multi-morphological feature fusion compared to the prior art are the same as those of the workpiece internal hole defect detection method based on multi-morphological feature fusion, and will not be repeated here. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the workpiece internal hole defect detection method based on multi-morphological feature fusion according to an embodiment of the present invention.

[0039] Figure 2 This is a detailed flowchart of step S100 of the workpiece internal hole defect detection method based on multi-morphological feature fusion according to an embodiment of the present invention.

[0040] Figure 3 This is a detailed flowchart of step S200 of the workpiece internal hole defect detection method based on multi-morphological feature fusion according to an embodiment of the present invention.

[0041] Figure 4 This is a detailed flowchart of step S400 of the workpiece internal hole defect detection method based on multi-morphological feature fusion according to an embodiment of the present invention.

[0042] Figure 5 This is a schematic diagram of the steps of the workpiece internal hole defect detection method based on multi-morphological feature fusion according to an embodiment of the present invention;

[0043] Figure 6 This is an example diagram of a dual-branch feature extraction network based on multi-morphological feature fusion according to an embodiment of the present invention. Detailed Implementation

[0044] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0045] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0046] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0047] It should be noted that the terms "one" and "more" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0048] like Figure 1 and Figure 5 As shown, the present invention provides a method for detecting defects in the inner hole of a workpiece based on multi-morphological feature fusion, comprising:

[0049] Step S100: Obtain a panoramic image of the inner hole of the workpiece, and obtain a rectangular unfolded image based on the panoramic image.

[0050] In one embodiment, an endoscope is inserted radially along the inner hole of the workpiece to capture an image of the inner wall portion of the workpiece, which is then used as a surround view image. In this embodiment, the size of the surround view image is 640×640. The geometric parameters of the surround view images captured by the endoscope for the same diameter are identical. Because the original surround view image is distorted when used as an input source, the features extracted from pixels near and far from the poles differ. Features near the poles are smaller, making it difficult to detect small internal defects. Expanding the surround view image to obtain a rectangular unfolded image can reduce the impact of distortion and increase the accuracy of feature extraction.

[0051] Step S200: Perform bi-branch feature extraction on the surround view imaging and the rectangular unfolded image to obtain a dual-morphological feature pair, wherein the dual-morphological feature pair includes annular features and rectangular features.

[0052] Step S300: Obtain the circular unfolding feature based on the circular ring feature.

[0053] Specifically, the dual-branch feature extraction includes feature extraction from the surround view imaging branch and feature extraction from the rectangular unfolded image. Feature extraction is performed on the two types of imaging separately to obtain dual-morphological feature pairs. By extracting the annular features, feature loss during the feature extraction process of the rectangular unfolded image is avoided, and the drawback of noise amplification caused by the unfolding process of the rectangular unfolded image is also avoided. By extracting the rectangular features of the rectangular unfolded image, distortion in the surround view image is avoided, ensuring the accuracy of detecting small internal defects.

[0054] Step S400: The circular unfolded feature and the rectangular feature are fused using multi-morphological features to obtain fused features. Defect detection is performed based on the fused features to obtain corresponding defect detection results.

[0055] Specifically, a circular feature map is obtained by expanding the circular feature map, and a rectangular feature map is obtained by expanding the rectangular feature map. Since the feature maps obtained by different features represent different defect features with different advantages, the two are fused to obtain a fused feature map that is suitable for detecting small internal defects and does not lose some defect features. The fused feature map is input into the detection model used for defect detection to obtain the corresponding defect detection results, ensuring that the feature information in the surround view imaging is fully utilized and improving the reliability of defect detection.

[0056] In an embodiment of the invention, a rectangular unfolded image is obtained by performing rectangular unfolding processing on the initial surround view imaging. This step forms two independent branches: the surround view imaging branch and the rectangular unfolded image branch. By extracting features from these two branches, the advantage of preserving the integrity of the annular features can be maintained, while undistorted features can be preserved through the rectangular features. Unfolding the extracted annular features yields annular unfolded features at a different level from the original annular features. This step ensures that defect detection can obtain more accurate results. Fusing the rectangular features and the annular unfolded features can effectively compensate for the imaging distortion and feature loss that may exist in defect detection with a single feature. By fusing the two features, the detection model can more comprehensively and accurately capture the characterization of the inner hole of the workpiece, thereby significantly improving the detection accuracy of defects in inner hole workpieces.

[0057] Optionally, such as Figure 2 As shown, the step of obtaining a panoramic image of the workpiece's inner hole, and obtaining a rectangular unfolded image based on the panoramic image, includes:

[0058] Step S110: Determine the target size of the rectangular unfolded image based on the input parameters of the surround view imaging.

[0059] Step S120: Based on the target size, map the surround view image to the original surround view image coordinates to obtain a rectangular unfolded image.

[0060] Optionally, the input parameters include the outer circumference radius, the inner circumference radius, and the coordinates of the center.

[0061] In one embodiment, a geometric reshaping network processes the surround view image, unfolding the surround view image and its feature map to obtain a rectangular unfolded image. Since the surround view image is formed by radially penetrating the inner wall of the workpiece through an endoscope, it has specific dimensions. Surround view images taken by the same diameter endoscope and with the same diameter share the same input parameters, which can be obtained by capturing sample images. The workflow of the geometric reshaping network includes first determining the size of the rectangular image to be unfolded based on the input parameters of the surround view image, i.e., determining a rectangular image with a length of 2πR and a width of Rr, where R represents the outer circumference radius and r represents the inner circumference radius. Based on the target size, the pixels of the surround view image are represented using a coordinate system, and the coordinates are mapped to a rectangle of the target size to obtain the rectangular unfolded image.

[0062] Optionally, the step of mapping the surround view image to the original surround view image coordinates according to the target size to obtain a rectangular unfolded image includes:

[0063] Based on the target size, the surround view imaging is mapped to a Cartesian coordinate system to obtain the first coordinates.

[0064] Convert each point in the first coordinate system to polar coordinates to obtain the second coordinate system.

[0065] Each point in the second coordinate system is converted to Cartesian coordinates under the annular image to obtain the mapping relationship between each point in the panoramic imaging and the rectangular unfolded image.

[0066] By using bilinear interpolation, the pixels of the panoramic imaging are mapped to the Cartesian coordinate system according to the mapping relationship, thereby obtaining the rectangular unfolded image.

[0067] In one embodiment, based on the target size, the surround-view image is mapped to a Cartesian coordinate system to obtain a first coordinate represented in the Cartesian coordinate system. Specifically, the upper left corner of the image is taken as the origin, and points with uniform intervals are generated along the horizontal and vertical directions, and represented by coordinates. Each point represented by the first coordinate is transformed into polar coordinates, which serve as the second coordinate. Further, each point represented by the second coordinate is converted to Cartesian coordinates under the annular image to obtain the mapping relationship between each point of the surround-view image and the rectangular unfolded image. The rectangular unfolded image is mapped to the annular image using bilinear interpolation, and the corresponding pixel values ​​are obtained, ultimately obtaining the rectangular unfolded image. The bilinear interpolation mapping can be expressed as follows:

[0068] x cir =(r+y rec cos(x) rec / R)+x0,

[0069] y cir =(r+y rec sin(x) rec / R)+y0,

[0070] Where, x rec Represents the unfolded rectangular image I rec any point P rec x-coordinate, y rec Represents the unfolded rectangular image I rec any point P on rec The ordinate, x cir Indicates surround view imaging I cir any point P on cir x-coordinate, y cir Indicates surround view imaging I cir any point P on cir The ordinate of the circle is r, R, x0 and y0, which represent the inner circumference radius, outer circumference radius and center coordinates of the circle in the panoramic image, respectively.

[0071] Optionally, such as Figure 3As shown, the step of performing bi-branch feature extraction on the surround-view imaging and the rectangular unfolded image to obtain dual-morphological feature pairs includes:

[0072] Step S210: Construct at least two parallel backbone networks based on convolutional layers, residual blocks, and feature pyramids, wherein the backbone networks are used for feature extraction individually.

[0073] Step S211: Extract features from the surround view image and the rectangular unfolded image using a dual-branch feature extraction network to obtain the dual-morphological feature pair, wherein the dual-branch feature extraction network is composed of the backbone network.

[0074] In one embodiment, convolutional layers, residual blocks, and feature pyramids are connected from top to bottom to form a backbone network. Two parallel backbone networks ensure that features can be extracted from both the surround-view image and the unfolded rectangular image separately, better preserving the advantages of features from both and ensuring the accuracy of defect detection. Features from the surround-view image and the unfolded rectangular image are extracted using a dual-branch feature extraction network. The extracted features are used as a dual-morphological feature pair, which includes circular and rectangular features.

[0075] Optionally, such as Figure 6 As shown, the backbone network includes a first module, a second module, and an SPPF module from top to bottom. The first module includes at least two first convolutional layers and a computer vision module. The second module includes at least one second convolutional layer and a computer vision module corresponding to the second convolutional layer.

[0076] In one embodiment, each backbone of the dual-branch feature extraction network is composed of a first module, a second module, and an SPPF module connected sequentially. Preferably, the first module includes two identical first convolutional layers (conv) and a C2f module connected sequentially, and the second module includes a second convolutional layer and a C2f module connected sequentially. Thus, the backbone network structure consists of a first module, multiple second modules, and an SPPF module from top to bottom. Preferably, there is one first module and three second modules, with the first convolutional layer being a 3×3 convolutional layer and the second convolutional layer being a 3×3 convolutional layer.

[0077] The C2f module, or Convolution to Feature module, is a module that combines Convolutional Neural Networks (CNNs) and Fully Connected Neural Networks (FNNs). This module aims to simultaneously preserve local image features and learn global features, thereby improving model performance. The SPPF module is an improvement on the original Spatial Pyramid Pooling structure, designed to address image distortion caused by image region cropping and scaling, thus improving the accuracy and stability of image processing.

[0078] Optionally, the step of performing bi-branch feature extraction on the surround-view imaging and the rectangular unfolded image to obtain dual-morphological feature pairs includes:

[0079] The surround view image and the rectangular unfolded image are subjected to bi-branch feature extraction to obtain at least three bimorphic feature pairs, wherein each bimorphic feature pair is obtained from the same level output of the bi-branch feature extraction network.

[0080] In one embodiment, surround imaging I cir and unfolded rectangular image I rec Input the bi-branch feature extraction network, and obtain the ring features through the fifth, seventh, and tenth layers of the feature extraction network, respectively. and rectangular features X = 5, 7, 10, where X represents the number of feature output layers. Considering that lower-level features focus more on capturing local information, while higher-level features focus more on global information analysis, a dual-branch feature extraction strategy is adopted, and processing is performed separately for different layers. In this way, dual-morphological feature pairs at different layers are output, which not only makes full use of the unique advantages of panoramic imaging and rectangular unfolded images, but also significantly improves the accuracy of defect detection through feature information from different layers.

[0081] Optionally, obtaining the annular unfolding feature based on the annular feature includes:

[0082] The annular feature is processed by the first geometric reshaping network, the second geometric reshaping network, and the third geometric reshaping network, respectively, to obtain the first unfolded feature, the second unfolded feature, and the third unfolded feature, wherein the ratio between the size of the first unfolded feature and the size of the surround view image, the ratio between the size of the second unfolded feature and the size of the surround view image, and the ratio between the size of the third unfolded feature and the size of the surround view image are respectively...

[0083] In one embodiment, since the original surround view image size is 640×640, the ratio of the output first unfolded feature, second unfolded feature, and third unfolded feature to the surround view image is: Therefore, the size of the first expanded feature is 80×80, the size of the second expanded feature is 40×40, and the size of the third expanded feature is 20×20. Through a variable-resolution geometric reshaping network, internal aperture panoramic imaging at different resolutions can be reshaped into rectangular imaging, acquiring data and features under different morphologies. This allows the extracted features to enable the detection model to more comprehensively and accurately capture the target's representation.

[0084] like Figure 6 As shown, the circular feature The first expanded feature is output when the data is input into the first geometric reshaping network PSTNet-l, the second geometric reshaping network PSTNet-m, and the third geometric reshaping network PSTNet-s, respectively. The size is 80×80, and the second unfolding feature The size is 40×40, and the third unfolding feature is... The size is 20×20.

[0085] Optionally, such as Figure 4 As shown, the process of fusing the expanded circular feature and the rectangular feature with multiple morphological features to obtain the fused feature includes:

[0086] Step S410: Perform convolution processing on the circular unfolded feature and the rectangular feature respectively. After convolution, concatenate the corresponding circular unfolded feature and the rectangular feature to obtain the concatenated feature.

[0087] Step S411: Input the spliced ​​features into different residual networks to obtain feature extraction results respectively.

[0088] Step S412: Obtain the fused features based on the feature extraction results.

[0089] In one embodiment, circular ring unfolded features and rectangular features are obtained by processing different layers of the geometric reshaping network in pairs. After performing convolution processing on the circular ring unfolded features and rectangular features respectively, the circular ring unfolded features are concatenated with the corresponding rectangular features to obtain concatenated features. The number of concatenated features is consistent with the number of circular ring unfolded features / rectangular features.

[0090] Optionally, paired circular ring unfolding features and rectangular features are obtained through the first geometric reshaping network PSTNet-l, the second geometric reshaping network PSTNet-m, and the third geometric reshaping network PSTNet-s, respectively.

[0091] Preferably, the number of circular unfolded features and rectangular features is 3, and the number of spliced ​​features after splicing is 3.

[0092] In one embodiment, the circular unfolding feature is... (X = 5, 7, 10) and rectangular features The fused features are obtained by inputting each feature into the Multimodal Feature Fusion Network (MFFNet). (X = 5, 7, 10). Specifically, the features of the torus are unfolded respectively. and rectangular features The input to the multi-morphological feature fusion network first undergoes convolution processing through a 1×1 third convolutional layer, halving the number of channels in the feature maps of the circular unfolded feature and the rectangular feature. Then, the halved circular unfolded feature and rectangular feature are concatenated, which can be represented as:

[0093]

[0094] in, c(x) represents the concatenated features, and c(x) represents the convolution operation. Indicates feature splicing, Represents rectangular features. X represents the toroidal unfolded feature, and X represents the number of layers in the feature output.

[0095] The concatenated features are input into multiple independent residual networks to obtain feature extraction results. Based on the feature extraction results, the fused features are obtained, represented as follows:

[0096]

[0097] in, Indicates fusion characteristics, This indicates the residual network's characteristics for the input. The role of n is the role of n independent residual networks, and X is the number of layers in the feature output.

[0098] By leveraging residual modules, the network can learn and understand pattern correlation information between different features. Simultaneously, by improving the network structure, the new network achieves performance optimization and enhancement while retaining the original network's powerful feature extraction capabilities.

[0099] The fused features are considered the final output of feature extraction and are then input into other modules of the detection model. These modules process the fused features to obtain the corresponding defect detection results. Through a multi-morphological feature fusion network, dual-morphological feature pairs of different resolutions can be fully fused, compensating for the shortcomings of single-morphological feature models and ensuring detection accuracy.

[0100] Another embodiment of the present invention provides a workpiece internal hole defect detection system based on multi-morphological feature fusion, comprising:

[0101] A rectangular unfolding unit is used to acquire a panoramic image of the inner hole of a workpiece, and to obtain a rectangular unfolded image based on the panoramic image.

[0102] A dual-branch feature extraction unit is used to perform dual-branch feature extraction on the panoramic imaging and the rectangular unfolded image to obtain dual-morphological feature pairs, wherein the dual-morphological feature pairs include annular features and rectangular features;

[0103] A ring unfolding unit, used to obtain ring unfolding features based on the ring features;

[0104] The feature fusion unit is used to fuse the circular unfolded feature and the rectangular feature into multi-morphological features to obtain fused features, and to perform defect detection based on the fused features to obtain corresponding defect detection results.

[0105] Another embodiment of the present invention provides an electronic device, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the workpiece internal hole defect detection method based on multi-morphological feature fusion as described above when the computer program is executed.

[0106] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the workpiece internal hole defect detection method based on multi-morphological feature fusion as described above.

[0107] The present invention will now describe electronic devices that can serve as servers or clients of the present invention, which are examples of hardware devices that can be applied to various aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0108] Electronic devices include a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0109] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0110] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0111] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for detecting internal hole defects in workpieces based on multi-morphological feature fusion, characterized in that, include: Obtain a panoramic image of the inner hole of the workpiece, and obtain a rectangular unfolded image based on the panoramic image; The method involves performing bi-branch feature extraction on the surround-view image and the rectangular unfolded image to obtain bimorphic feature pairs. This includes: constructing at least two parallel backbone networks based on convolutional layers, residual blocks, and feature pyramids, wherein the backbone networks are used for feature extraction independently; extracting features from the surround-view image and the rectangular unfolded image using a bi-branch feature extraction network to obtain the bimorphic feature pairs; and performing bi-branch feature extraction on the surround-view image and the rectangular unfolded image to obtain at least three bimorphic feature pairs, wherein the bimorphic feature pairs include annular features and rectangular features. The bi-branch feature extraction network is composed of the backbone network, and each bimorphic feature pair is obtained from the same level output of the bi-branch feature extraction network. Obtaining an annular unfolded feature based on the annular feature includes: processing the annular feature using a first geometric reshaping network, a second geometric reshaping network, and a third geometric reshaping network respectively to obtain a first unfolded feature, a second unfolded feature, and a third unfolded feature, wherein the ratio between the size of the first unfolded feature and the size of the surround view image, the ratio between the size of the second unfolded feature and the size of the surround view image, and the ratio between the size of the third unfolded feature and the size of the surround view image are respectively... , , ; The circular unfolded feature and the rectangular feature are fused using multi-morphological feature fusion to obtain a fused feature. Defect detection is performed based on the fused feature to obtain the corresponding defect detection result. The backbone network comprises, from top to bottom, a first module, a second module, and an SPPF module. The first module includes at least two first convolutional layers and a C2f module. The second module includes at least one second convolutional layer and a C2f module corresponding to the second convolutional layer.

2. The method for detecting internal defects in workpieces based on multi-morphological feature fusion according to claim 1, characterized in that, The step of obtaining the rectangular unfolded image based on the surround-view imaging includes: The target size of the rectangular unfolded image is determined based on the input parameters of the surround-view imaging; Based on the target size, the surround view image is mapped to the original surround view image coordinates to obtain a rectangular unfolded image.

3. The method for detecting internal defects in workpieces based on multi-morphological feature fusion according to claim 2, characterized in that, The step of mapping the surround view image to the original surround view image coordinates based on the target size to obtain a rectangular unfolded image includes: Based on the target size, the surround view image is mapped to a Cartesian coordinate system to obtain the first coordinates; Convert each point in the first coordinate system to polar coordinates to obtain the second coordinate system; Each point in the second coordinate system is converted to Cartesian coordinates under the annular image to obtain the mapping relationship between each point of the panoramic imaging and the rectangular unfolded image. By using bilinear interpolation, the pixels of the panoramic imaging are mapped to the Cartesian coordinate system according to the mapping relationship, thereby obtaining the rectangular unfolded image.

4. The method for detecting internal defects in workpieces based on multi-morphological feature fusion according to claim 1, characterized in that, The step of fusing the expanded circular feature and the rectangular feature with multiple morphological features to obtain the fused feature includes: The circular unfolded feature and the rectangular feature are convolved respectively. After convolution, the corresponding circular unfolded feature and the rectangular feature are concatenated to obtain the concatenated feature. The spliced ​​features are input into different residual networks to obtain feature extraction results respectively; The fused features are obtained based on the feature extraction results.

5. The method for detecting internal defects in workpieces based on multi-morphological feature fusion according to claim 2, characterized in that, The input parameters include the outer circumference radius, the inner circumference radius, and the center of the circle.

6. A workpiece internal hole defect detection system based on multi-morphological feature fusion, characterized in that, include: A rectangular unfolding unit is used to acquire a panoramic image of the inner hole of a workpiece, and to obtain a rectangular unfolded image based on the panoramic image. A dual-branch feature extraction unit is used to perform dual-branch feature extraction on the surround view image and the rectangular unfolded image to obtain dual-morphological feature pairs. This includes: constructing at least two parallel backbone networks based on convolutional layers, residual blocks, and feature pyramids, wherein the backbone networks are used for feature extraction independently; extracting features from the surround view image and the rectangular unfolded image using the dual-branch feature extraction network to obtain the dual-morphological feature pairs; and performing dual-branch feature extraction on the surround view image and the rectangular unfolded image to obtain at least three dual-morphological feature pairs, wherein the dual-morphological feature pairs include annular features and rectangular features. The dual-branch feature extraction network is composed of the backbone network, and each dual-morphological feature pair is obtained from the same level output of the dual-branch feature extraction network. A ring unfolding unit, configured to obtain ring unfolding features based on the ring features, includes: processing the ring features according to a first geometric reshaping network, a second geometric reshaping network, and a third geometric reshaping network respectively, to obtain a first unfolding feature, a second unfolding feature, and a third unfolding feature respectively, wherein the ratio between the size of the first unfolding feature and the size of the surround view image, the ratio between the size of the second unfolding feature and the size of the surround view image, and the ratio between the size of the third unfolding feature and the size of the surround view image are respectively... , , ; The feature fusion unit is used to fuse the circular unfolded feature and the rectangular feature into multi-morphological features to obtain fused features, and to perform defect detection based on the fused features to obtain corresponding defect detection results. The backbone network comprises, from top to bottom, a first module, a second module, and an SPPF module. The first module includes at least two first convolutional layers and a C2f module. The second module includes at least one second convolutional layer and a C2f module corresponding to the second convolutional layer.

Images