Intelligent detection system for forging defects of a forged product

Abstract

The application provides a forging defect intelligent detection system for a forged product, and relates to the technical field of industrial intelligent detection, comprising: splicing after collecting multi-angle images of the workpiece to be detected to generate a complete surface development map of the product; obtaining the defect type of the defect candidate area through the area coordinates and area size of the defect candidate; obtaining the confidence value of the corresponding defect type judgment result; calling a lightweight deep network recognition model for secondary judgment, supporting artificial labeling to generate new sample data, connecting a programmable controller at the equipment end for intelligent detection of defects of the workpiece to be detected. Through the application, the problem that in the prior art, due to the variety and complex form of the surface defects of forgings, the limited image collection angle leads to incomplete defect coverage, and high-precision automatic identification of multiple types of defects cannot be achieved, affecting the detection accuracy and production efficiency, can be solved, multi-angle image comprehensive collection and splicing are achieved, and the technical effect of improving the defect detection accuracy of forged products is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial intelligent detection, and particularly relates to a forging defect intelligent detection system for a forged product. BACKGROUND

[0002] As important structural basic parts, forgings are widely used in the fields of automobiles, aerospace, energy, rail transportation and heavy equipment manufacturing, and the quality stability of the forgings is directly related to the safety and service life of the whole machine. In the forging process, the surface of the forgings is prone to defects such as bubbles, scratches, depressions, cold partitions and flow marks due to the influence of various factors such as purity of raw materials, mold state, process temperature control, lubrication conditions and operation mode. These defects not only affect the appearance quality, but also may cause local stress concentration and early emergence of fatigue cracks, thereby affecting the product performance and even causing failure accidents.

[0003] At present, the detection of surface defects of forgings in industrial sites mainly relies on manual visual inspection, which has low detection efficiency, limited accuracy, poor stability, is easily affected by the experience level of operators, and is difficult to meet the requirements of modern manufacturing for high consistency and mass quality control.

[0004] In summary, in the prior art, due to the complex surface structure of the forgings, the various types of defects and the irregular shapes, a stable and reliable multi-angle visual fusion and intelligent recognition mechanism cannot be constructed, which leads to incomplete defect detection coverage, low classification accuracy and inability to dynamically adapt to actual working condition changes, further affecting the intelligent level of forging product quality control and the automation efficiency of the manufacturing process. SUMMARY

[0005] The purpose of the present application is to provide a forging defect intelligent detection system for a forged product, which solves the technical problem in the prior art that due to the complex surface structure of the forgings, the various types of defects and the various image interference factors, efficient fusion of multi-angle visual information and high-precision defect recognition and determination cannot be achieved, which leads to incomplete defect detection coverage, low recognition accuracy and lack of self-adaptation, further affecting the quality stability of the forged product and the automation level of the manufacturing process.

[0006] In view of the above problems, the present application provides a forging defect intelligent detection system for a forged product, comprising: an image acquisition module, the image acquisition module is used for acquiring multi-angle images of a workpiece to be detected through a high-resolution industrial camera and a rotary drive assembly, and indexing the obtained images in order, the industrial camera is installed on a controllable rotating platform, and the workpiece to be detected is fixed on a center shaft position through a positioning clamp; an image stitching module, the image stitching module is used for stitching the acquired multi-angle images, performing coordinate unification and surface unfolding on the complete image after stitching, and generating a product complete surface unfolding image; a defect candidate region extraction module, the defect candidate region extraction module is used for extracting a defect candidate region with defects through an image segmentation algorithm, obtaining region coordinates and region size of the defect candidate region, and the region size includes defect area and maximum boundary size of the defect region; a defect type determination module, the defect type determination module is used for matching and comparing image features of the extracted defect candidate region with a pre-constructed defect image sample library, obtaining a defect type of the defect candidate region, the defect image sample library includes five types of standardized defect image feature templates, and the five types of defects include bubbles, scratches, depressions, cold shut and flow marks; a defect confidence determination module, the defect confidence determination module is used for calculating feature similarity, category separation degree, region structure quality and sample coverage for each candidate defect region, inputting a defect confidence calculation formula, and obtaining a confidence value of a corresponding defect type determination result; an abnormal region secondary determination module, the abnormal region secondary determination module is used for calling a lightweight deep network recognition model for secondary determination for low confidence samples, while supporting artificial labeling to generate new sample data and adding the new sample data to the defect sample library; an automatic control module, the automatic control module is used for connecting a programmable controller at a device end, cooperating with an operation panel, and performing intelligent detection and feedback adjustment management of defects of the workpiece to be detected.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] By multi-angle visual acquisition and image registration splicing, combined with surface unfolding algorithm, complete and continuous surface visual information of the forging product can be obtained, and defect detection with no blind area coverage is realized; based on image segmentation algorithm, defect candidate regions are extracted, combined with image matching and similarity comparison of multiple feature dimensions (such as texture, edge, spatial vector), the recognition accuracy of multiple defects (bubbles, scratches, depressions, cold shut, flow marks) is effectively improved; through defect confidence score mechanism, factors such as feature similarity, region structure quality and sample coverage are comprehensively judged, and the classification credibility of boundary samples and complex morphological defects is effectively improved; when the confidence is lower than the preset threshold, the system automatically calls the lightweight deep neural network for secondary determination, and supports the artificial annotation results into the sample library, realizes the continuous optimization and self-evolution of the recognition model; that is, by fusing multi-angle image acquisition, image registration splicing and surface unfolding processing, defect candidate regions are extracted and combined with multi-feature image matching algorithm for defect type recognition, and confidence evaluation and low confidence secondary determination mechanism are introduced, to achieve the technical effects of fully covering the surface of the forging, accurately identifying multiple typical forging defects, and improving the recognition stability and system adaptive ability.

[0009] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more apparent and easy to understand, the following specific embodiments of the present application are described. It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only exemplary, and other drawings can be obtained by those skilled in the art without creating any inventive labor on the basis of the provided drawings.

[0011] Figure 1 FIG. 1 is a structural schematic diagram of a forging product forging defect intelligent detection system according to the present application.

[0012] Explanation of reference signs:

[0013] 11 image acquisition module, 12 image splicing module, 13 defect candidate region extraction module, 14 defect type determination module, 15 defect confidence determination module, 16 abnormal region secondary determination module, 17 automatic control module. DETAILED DESCRIPTION

[0014] The application solves the problem in the prior art that due to the complex surface structure of the forging, the various types of defects and the many image interference factors, efficient fusion of multi-angle image information and high-confidence defect recognition and determination cannot be achieved, resulting in limited defect detection range, unstable recognition accuracy, and further affecting the consistency and intelligent level of product quality control. The technical goal of comprehensive extraction, accurate classification and dynamic recognition of typical defects on the surface of the forging is achieved, and the technical effect of improving the defect detection coverage, recognition accuracy and system adaptive ability is achieved.

[0015] The technical solutions in the application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. It should be understood that the application is not limited by the example embodiments described herein. Based on the embodiments of the application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the application. In addition, it should be noted that, for the convenience of description, only parts related to the application are shown in the drawings, rather than all parts.

[0016] Please refer to the drawings Figure 1 The application provides a forging product forging defect intelligent detection system, specifically comprising:

[0017] The image acquisition module 11 is configured to acquire multi-angle images of the workpiece to be detected by configuring a high-resolution industrial camera and a rotating drive assembly, and to sort the obtained images in order according to the index. The industrial camera is installed on a controllable rotating platform, and the workpiece to be detected is fixed on the central axis position by a positioning clamp.

[0018] Specifically, a high-resolution industrial camera and a rotating drive assembly are configured to acquire multi-angle images of the workpiece to be detected. The industrial camera is installed on a rotating platform with controllable rotation capability, and the rotating platform can perform equiangular or continuous rotation scanning around the workpiece to be detected, forming a full-range visual acquisition path. The workpiece to be detected is fixed on the central axis position of the rotating platform by a positioning clamp, ensuring that the posture of the workpiece is stable and the axis is consistent during rotation, avoiding image distortion or loss caused by deviation. During image acquisition, the rotating platform is driven to rotate according to the set angle step value, and an industrial camera acquires an image at each predetermined angle or during continuous movement, obtaining a sequence of images covering different angles of the workpiece surface. Each acquired image corresponds to its acquisition order number and rotation angle parameter, and the image file is automatically indexed according to the acquisition order for subsequent image registration, splicing and surface unfolding processing.

[0019] By the above structure and method, multi-angle image acquisition of the workpiece to be inspected under stable positioning conditions can be realized, ensuring that the imaging view angles are uniformly covered without significant occlusion and missing areas, while ensuring that the images have clear sequential indexing relationship and angle reference information, providing continuous, standardized and traceable image input basis for subsequent image registration, stitching and surface unfolding processing, thereby improving the integrity and recognition accuracy of the overall defect detection.

[0020] An image stitching module 12 is configured to stitch the acquired multi-angle images, unify the coordinates of the stitched complete image and unfold the curved surface, to generate a product complete surface unfolding image.

[0021] Specifically, the acquired multi-angle images are first aligned in space by an image registration algorithm, feature points are extracted from the overlapping areas between adjacent images, accurate matching between images and transformation matrix calculation are realized, and the image stitching fusion is completed. After stitching, an overall image covering the surface of the forged product is formed, but the overall image still remains in the two-dimensional coordinate system of each view angle, and there is spatial inconsistency. In order to realize a unified spatial reference framework, the pixel coordinates of the stitched image are processed by coordinate conversion, and each part of the image is mapped to a unified two-dimensional plane coordinate system. The coordinate unification process is based on the geometric characteristics of the curved surface of the forged product, and the mapping relationship between the curved surface and the plane is established to complete the unfolding operation from the curved surface to the plane. The unfolding operation adopts geometric mapping and pixel resampling technology to convert the forged product surface curved surface coordinates to unfolded plane coordinates, eliminates the curved surface distortion and view angle difference, and generates a seamless, continuous and complete surface unfolding image. The surface unfolding image fully reflects the distribution of the external defects of the forged product, and facilitates the subsequent defect detection algorithm to extract and identify the region. Through the above stitching, coordinate unification and curved surface unfolding process, the multi-angle image fusion and reconstruction of the forged product surface are realized, providing a high-quality overall surface unfolding visual basis.

[0022] A defect candidate region extraction module 13 is configured to extract a defect candidate region where defects exist by an image segmentation algorithm, to obtain the region coordinates and the region size of the defect candidate, and the region size includes the defect area and the maximum boundary size of the defect region.

[0023] Specifically, in the defect detection process, first, the image segmentation algorithm is applied to the collected complete surface development map to identify the possible defect area in the image. The image segmentation algorithm can adopt threshold-based segmentation, edge detection, region growing or semantic segmentation model based on deep learning, adapt to the surface features of forgings, and accurately distinguish the defect area from the normal surface. Through segmentation processing, the system extracts a group of defect candidate regions from the image. For each defect candidate region, calculate its specific position coordinates in the image coordinate system as the spatial identifier of the region. Further, analyze the geometric properties of the defect candidate region, calculate its area size, that is, the total number of pixels in the region multiplied by the actual size of a single pixel, reflecting the actual area occupied by the defect. At the same time, measure the maximum boundary size of the defect candidate region, that is, the longest boundary distance of the region in the horizontal or vertical direction, which is used to describe the linear size characteristics of the defect. The maximum boundary size helps to distinguish the morphology and severity of the defect, supporting subsequent classification and confidence assessment. By obtaining the coordinates and size information of the defect candidate region, accurate basic data support is provided for subsequent defect feature extraction, type determination and positioning feedback.

[0024] The defect type determination module 14 is used to match and compare the extracted image features of the defect candidate region with a pre-constructed defect image sample library to obtain the defect type of the defect candidate region. The defect image sample library includes standardized five types of defect image feature templates, and the five types of defects include bubbles, scratches, depressions, cold partitions, and flow marks.

[0025] Specifically, for the defect candidate region extracted by image segmentation, first, feature extraction is performed, which includes but is not limited to texture features, edge shape, gray distribution, color information, and spatial geometric features of the region, etc., forming a multi-dimensional image feature vector. The extracted features aim to comprehensively reflect the visual and structural features of the defect, providing sufficient information for subsequent matching. The obtained defect candidate region features are input into a pre-set defect image matching module, which includes a pre-constructed defect image sample library. The defect image sample library stores standardized image feature templates for typical forging defects, covering five types of common defect types, including bubbles, scratches, depressions, cold partitions, and flow marks. The matching module calculates the similarity between the defect candidate region features and the defect templates in the sample library, and uses multiple image similarity algorithms (such as cosine similarity, edge matching, spatial distance measurement, etc.) to comprehensively evaluate the matching degree. In the matching result, the defect type is determined by the template category with the highest similarity, thereby realizing the automatic classification of the defect type of the defect candidate region. This matching process effectively combines the rich defect feature information of the sample library and the multi-dimensional feature representation of the defect candidate region, improving the accuracy and robustness of defect recognition, and adapting to the intelligent detection needs of diversified forging defects.

[0026] The defect confidence determination module 15 is configured to calculate feature similarity, category separation degree, region structure quality, and sample coverage for each candidate defect region, and input a defect confidence calculation formula to obtain a confidence value of the corresponding defect type determination result.

[0027] Specifically, for each defect candidate region, the system first calculates the feature similarity between the region and each category template in the defect sample library. The feature similarity comprehensively considers multi-dimensional features such as texture, shape, and color, and quantifies the similarity through methods such as cosine similarity, edge matching distance, and high-dimensional space distance, to reflect the matching degree of the defect candidate region and the standard defect. At the same time, the category separation degree is evaluated, that is, the difference between the similarity of the defect candidate region features and the most similar category template and the similarity between other category templates. The greater the separation degree, the more explicit the defect type determination, and the higher the category separation degree. In addition, the region structure quality is calculated to measure the shape integrity and smoothness of the defect candidate region, to exclude abnormal regions caused by noise or image missegmentation, and to ensure the rationality and stability of the defect region. Further, the sample coverage is evaluated to analyze the representativeness of the current defect candidate region features in the defect sample library, to determine whether there is a typical sample similar to the defect and sufficiently covering in the sample library, to assist in judging the reliability of defect recognition. The above multiple indexes are used as input variables, and a preset defect confidence calculation formula is combined for comprehensive calculation to output the confidence value of the defect type determination result corresponding to the defect candidate region. The confidence value reflects the credibility of the recognition result, and provides a basis for subsequent determination optimization, manual review, or secondary recognition. Through the confidence calculation mechanism, the recognition stability and accuracy of the forging defect detection system for complex and variable defects are effectively improved, and the adaptive ability and robustness of the system are enhanced.

[0028] The abnormal region secondary determination module 16 is configured to call a lightweight deep network recognition model for secondary determination for low-confidence samples, and support manual annotation to generate new sample data and add it to the defect sample library.

[0029] Specifically, for candidate defect regions with defect confidence calculation results below the preset threshold, the system automatically triggers a secondary judgment process. This process calls a lightweight deep neural network recognition model to analyze the low-confidence samples again. The lightweight deep network model has a compact structure and low computational resource consumption, and can adapt to real-time detection requirements in industrial fields while ensuring accuracy. In the secondary judgment process, the deep network comprehensively extracts complex texture, shape and local detail information through deep semantic understanding of the input defect candidate region image features, making up for the limitations of traditional feature matching in preliminary recognition and improving the accuracy and robustness of defect classification. If the secondary judgment result still cannot meet the clear recognition standard, the system supports starting the manual annotation module, and professional personnel manually confirm and annotate the difficult defect regions. After the manual annotation result is reviewed, it is added to the defect image sample library as high-quality sample data, dynamically enriching the content of the sample library. The continuous updating of the sample library not only improves the representativeness and coverage of the defect template, but also promotes the iterative training and performance improvement of the recognition model, realizes the self-learning and evolution of the system, and ensures the long-term stable and efficient operation of the defect detection system.

[0030] The automatic control module 17 is used for connecting a programmable controller at the equipment end, cooperating with an operation panel, and performing intelligent detection and feedback regulation management of defects of a workpiece to be detected.

[0031] Specifically, the equipment end is connected with a programmable logic controller (PLC) through a high-speed communication interface, realizing real-time transmission of data and interaction of instructions. The programmable controller serves as a core control unit of the system and is responsible for coordinating the running states and process management of each detection module. The operation panel is provided with a touch display interface for realizing man-machine interaction. An operator can monitor the defect detection progress in real time, view the detection results, adjust the detection parameters, and control the system operation through the panel. The operation panel and the programmable controller realize bidirectional communication through an industrial bus or Ethernet, ensuring instant transmission of instructions and feedback information. During the defect detection process, the system feeds back the recognition results and defect positioning information to the programmable controller in real time. The controller automatically adjusts the parameters of the detection equipment or triggers corresponding process adjustment measures, such as adjusting the shooting angle, light intensity, or controlling the workpiece conveying speed, according to the preset feedback regulation strategy, so as to optimize the detection effect and production quality. This cooperative management mechanism realizes closed-loop linkage of intelligent detection and production control, improves the accuracy and real-time performance of defect recognition, and effectively supports the automatic operation and dynamic quality control of the production line.

[0032] The intelligent forging defect detection system for the forging product can achieve the technical goal of accurate multi-class recognition and positioning of forging surface defects, and achieve the technical effect of improving the accuracy, coverage and automation level of defect detection.

[0033] Further, the present application also includes:

[0034] Based on the overlapping area between adjacent images in the multi-angle images, an image registration operation is performed, the multi-angle images are spliced using an image splicing algorithm, and a spliced image covering the complete product surface is generated; a coordinate unification operation is performed on the spliced image, pixel coordinates in different image sources are converted to a unified image plane coordinate system, and positional deviations caused by different shooting angles are eliminated; based on the unfolding rule of the product, pixel information in the spliced image is sequentially unfolded along the direction of the product outer surface, and a product surface unfolding diagram with a unified plane structure is generated, which is used for subsequent defect identification processing.

[0035] Specifically, first, based on the overlapping area between adjacent images in the multi-angle images, feature points are extracted and matched, and image registration is achieved by calculating a transformation matrix. The image registration process ensures that the corresponding relationship of the spatial positions between images is accurately established, laying a foundation for seamless splicing of images. Subsequently, the multi-angle images are fused using an image splicing algorithm, a complete spliced image covering the surface of the product under inspection is generated, and the continuity and integrity of the visual information are achieved. After splicing, a coordinate unification operation is performed on the spliced image. This operation effectively eliminates spatial positional deviations caused by differences in shooting angles and viewpoints by converting pixel coordinates from different image sources to a unified two-dimensional image plane coordinate system, ensuring the spatial consistency and measurement accuracy of the spliced image.

[0036] Next, based on the unfolding rule of the forging product, pixel information in the spliced image is sequentially arranged and unfolded along the direction of the product outer surface, the mapping conversion from a curved surface to a plane is completed, and a product surface unfolding diagram with a unified plane structure is generated. The unfolding diagram truly reflects the spatial distribution relationship of the product surface defects, facilitating accurate positioning and classification processing of the defect regions by subsequent defect identification algorithms.

[0037] Finally, through the above image registration, splicing, coordinate unification, and unfolding processing procedures, the system realizes efficient fusion and structural optimization of multi-angle acquisition images, providing a solid image foundation for intelligent identification of forging defects.

[0038] Through the feature matching and image registration algorithm based on the overlapping area of multi-angle images, accurate splicing and spatial position correction of multiple images can be achieved, and finally a unified unfolding diagram covering the complete surface of the forging product is generated, which is used for subsequent efficient and accurate defect identification and positioning.

[0039] Further, the present application also includes:

[0040] According to a preset shooting order, adjacent images are determined, and in combination with indexes of the images, an image feature extraction operation is performed on parts in the adjacent images located in an overlapping area, the features including edge features, corner point features and texture features; feature points extracted in the overlapping area of an image are matched with feature points in a corresponding area of an adjacent image of the image, a descriptor matching algorithm SIFT is used to calculate distances between the feature points, and an initial matching point set is screened based on a distance threshold; a RANSAC algorithm is used to robustly screen the initial matching point set, and abnormal matches are eliminated to obtain a consistent matching pair set; a homography matrix between the images is estimated based on the consistent matching pair set, spatial alignment of image coordinate systems is realized; an iterative optimization is performed on a transformation result, and registered images are superimposed according to a unified coordinate system; a multi-resolution fusion strategy is used to perform smooth transition processing on pixels in the overlapping area, obvious splicing gaps are avoided, and a spliced image covering a complete product surface, continuous texture and complete contour is generated.

[0041] Specifically, according to a preset shooting order, a plurality of collected images are sorted, and combination relationships and index information of adjacent images are determined. For overlapping areas between adjacent images, an image feature extraction operation is first performed. The extracted features include edge features for capturing obvious contour lines in the image, corner point features for identifying corners or significant change points in the image, and texture features for reflecting detailed structures and surface changes in the area. Preferably, a scale-invariant feature transform (SIFT) algorithm is used to construct feature descriptors for the above-mentioned feature points, and the similarity between the feature points is calculated based on the Euclidean distance between the descriptors. According to a set distance threshold, all feature point pairs are screened to obtain an initial matching point set. To further improve the robustness of matching, a random sample consensus (RANSAC) algorithm is used to iteratively screen the initial matching point set, and abnormal matching point pairs are eliminated, and finally a consistent matching pair set is obtained. Based on the consistent matching pair set, a homography matrix estimation method is used to model the geometric transformation relationship between adjacent images, and a spatial alignment operation of the image coordinate system is completed. Further, local iterative optimization processing is performed on the estimated transformation parameters to eliminate local distortion and improve image registration accuracy. After image registration, all images are superimposed according to a unified coordinate system to form a preliminary spliced image. To eliminate visual artifacts such as brightness inconsistency and texture discontinuity that may exist at the image boundary overlap, a multi-resolution fusion strategy is used to perform smooth transition processing on pixel information in the overlapping area to realize seamless splicing. Finally, a high-quality spliced image covering a complete product surface, continuous texture and complete contour is generated.

[0042] Further, the present application also includes:

[0043] The image region in the unwinding diagram is processed by an image segmentation algorithm to extract the image region where defects may exist as a defect candidate region, the image segmentation algorithm includes a segmentation network based on pixel-level classification, a region growing method based on edge detection, and is used to separate the region with abnormal texture, color or boundary features in the image from the background; in each extracted defect candidate region, the spatial coordinate information of the region is obtained, including the upper left corner coordinate point and the lower right corner coordinate point, to determine the position of the defect region in the image; the size of each defect candidate region is calculated, including the defect area and the maximum boundary size of the defect region, the defect area is calculated according to the number of effective pixels in the region, and the maximum boundary size of the defect region is the longest side length of the defect in the horizontal direction or the vertical direction.

[0044] Specifically, the image region in the unwinding diagram is first processed by an image segmentation algorithm to extract the image region where defects may exist as a defect candidate region. The image segmentation algorithm includes a deep segmentation network based on pixel-level classification, which can classify and judge each pixel to realize accurate segmentation of the region with abnormal texture, color or boundary features; and a region growing method based on edge detection is combined to further enhance the boundary definition of the defect region by detecting the obvious edge information in the image and expanding along the boundary.

[0045] Then, the combined segmentation method effectively separates the abnormal feature region from the background, improving the extraction accuracy and integrity of the defect region. For each extracted defect candidate region, the system obtains its spatial coordinate information, including the upper left corner coordinate point and the lower right corner coordinate point, to accurately locate the position of the defect region in the entire unwinding diagram.

[0046] Subsequently, the size index of each defect candidate region is calculated. The size includes two parts of the defect area and the maximum boundary size, wherein the defect area is obtained by counting the number of effective pixels in the region and converting the actual size of the pixels, reflecting the actual coverage area of the defect; the maximum boundary size is measured by measuring the longest boundary length of the defect region in the horizontal direction or the vertical direction, reflecting the linear size feature of the defect. This size information is helpful for subsequent defect classification and severity determination.

[0047] Further, the present application also includes:

[0048] The image features of the defect candidate region are input into an image similarity comparison algorithm to perform feature matching and similarity comparison with a pre-constructed defect image sample library to determine the defect type of the defect candidate region.

[0049] The image similarity comparison algorithm calls a pre-constructed defect image sample library, the defect image sample library is a standard image template set established according to different defect types, each template corresponds to a defect type, and the corresponding feature encoding result is stored; the feature matching operation between the defect candidate region and each defect template in the sample library is performed, the image similarity comparison algorithm is used to calculate the similarity score, and the similarity score is used to measure the matching degree of the defect candidate region and each defect template; the defect type of the defect candidate region is determined according to the defect template category corresponding to the maximum value of the similarity score.

[0050] Specifically, the image features of the defect candidate region are input into the image similarity comparison algorithm, and the image features in the pre-constructed defect image sample library are matched and similarity scores are calculated to determine the defect type corresponding to the defect candidate region. The defect image sample library establishes a standardized image template set according to different defect types, each template corresponds to a defect type, and the feature encoding result related to the template is stored to represent the multi-dimensional feature information of the template such as texture, shape and color.

[0051] The similarity comparison algorithm quantifies the similarity between the defect candidate region and each defect template by calculating the feature matching degree between the defect candidate region and each defect template in the sample library, using multiple image similarity evaluation indicators such as cosine similarity, edge matching distance and high-dimensional feature space distance. According to the similarity score calculation result, the matching strength between the defect candidate region and each defect template is determined.

[0052] By comparing the similarity scores of each defect template, the template category with the highest score is selected as the final defect type determination result of the defect candidate region. This method combines rich defect sample features and multi-dimensional similarity calculation, effectively improving the accuracy and robustness of defect type recognition.

[0053] Further, the present application also includes:

[0054] The multi-dimensional image features of the defect candidate region and the template images in the sample library are extracted respectively, the features include texture features, shape features, semantic features and spatial distribution features; the extracted multi-dimensional image features are vectorized and normalized to unify the dimension and scale, forming a standard feature code; feature matching is performed based on the feature vectors of the image features of the defect candidate region and each defect template in the defect image sample library, and the feature vectors of the image features of the defect candidate region and each defect template in the defect image sample library are matched to obtain a feature similarity , including cosine similarity , edge shape matching distance and high-dimensional space distance ; the feature similarity The preset similarity fusion score function is inputted to obtain a fusion score between the defect candidate region and each type of defect template to obtain a similarity fusion score; the similarity fusion score calculation formula is: = a x + b x (1- ) + g x (1- ); wherein, is a cosine similarity, is an edge shape matching distance, is a high-dimensional space distance, a, b, g are preset weighting coefficients, and a + b + g = 1; the fusion scores of the defect candidate region and each type of defect template are sorted, and the defect type corresponding to the highest score is taken as the recognition result.

[0055] Specifically, image preprocessing and detection algorithms (such as edge detection, heat map, abnormal region detection, etc.) are used to identify candidate regions that may contain defects. Each candidate region is cropped and the image size is unified (such as being adjusted to a fixed size) to facilitate subsequent feature extraction. The intermediate layer output of a pre-trained convolutional neural network (such as ResNet, VGG, MobileNet, etc.) is used as the feature extraction feature for the cropped image region. For the defect candidate region and the template image in the sample library, multi-dimensional image features are extracted by an image feature extraction algorithm. The extracted multi-dimensional features are converted into one-dimensional vectors and normalized (such as L2 normalization or maximum and minimum value normalization) to form standardized candidate region feature vectors. Collecting various typical defect image samples and storing them by defect type, a defect image sample library is constructed. Repeat the image preprocessing, size normalization, feature extraction and normalization process for the template image to ensure that the candidate region and the template feature are comparable. Store the feature vectors of each type of defect template in the feature database. Single representative template (Prototype-based), multi-sample average feature vector (Centroid), and vector set of all samples (for clustering or distance weighting) can be used. The extracted features mainly include texture features for describing surface details and texture changes, shape features reflecting the geometric contour and edge information of the defect region, semantic features representing the high-level semantic attributes of the defects in the image, such as abstract information related to the defect category, and spatial distribution features revealing the position relationship and layout pattern of the defect features in the image. The extracted image features of various types are vectorized and converted into numerical vectors of a unified format, and are normalized to ensure the consistency of different feature dimensions and scales. The feature vectors of the defect candidate region and the feature vectors of the defect templates are obtained. Through this process, standardized image feature codes are formed, providing a unified input basis for subsequent feature matching.

[0056] Then, based on the standardized image feature code, the image features of the defect candidate region and the feature vectors of each type of defect template in the defect image sample library are matched. In the feature matching process, multiple similarity calculation methods are used to quantify the feature similarity between the defect candidate region and each defect template. The feature similarity includes cosine similarity , edge shape matching distance and high-dimensional space distance ; the feature similarity is input into a preset similarity fusion scoring function to obtain the fusion score between the defect candidate region and each type of defect template to obtain the similarity fusion score. =α× +β×(1- )+γ×(1- );wherein, is the cosine similarity, is the edge shape matching distance, is the high-dimensional space distance, and α, β, γ are preset weighting coefficients, α+β+γ=1; the fusion scores of the defect candidate region and each type of defect template are sorted, and the defect type corresponding to the highest score is taken as the recognition result.

[0057] The calculated multi-dimensional feature similarity is input into a preset similarity fusion scoring function, and through a weighted or nonlinear fusion strategy, a comprehensive fusion score between the defect candidate region and each type of defect template is obtained. The fusion score reflects the overall matching degree and comprehensively considers the feature similarity of multiple dimensions such as texture, shape, semantics and spatial distribution. The fusion score results are sorted according to the numerical value, and the defect template category with the highest score is selected as the final recognition result of the defect candidate region. This method effectively integrates multi-dimensional feature information, improves the accuracy and robustness of defect recognition, and is suitable for intelligent detection of multiple types of forging defects.

[0058] Further, the present application also includes:

[0059] calculate the cosine similarity , edge shape matching distance and high-dimensional space distance between the feature vectors; based on the cosine similarity between the feature vectors extracted based on the image texture, shape or deep semantic features, the consistency of the image feature direction is measured; based on the edge contour features of the defect candidate region and the template image, the Hausdorff distance is used to calculate the edge shape matching distance, which is used to measure the geometric structure similarity; based on the deep feature encoding result extracted by the convolutional neural network, the high-dimensional space distance is calculated, which is used to describe the image difference in the semantic space, and the high-dimensional space distance is the Euclidean distance.

[0060] Specifically, first, the texture, shape and deep semantic features extracted from the image are used to generate corresponding feature vectors. The cosine similarity between the feature vectors is calculated to measure the consistency of the two sets of feature vectors in high-dimensional space. The cosine similarity reflects the similarity of image features, and the closer the value is to 1, the more consistent the direction is.

[0061] Then, for the edge contour features of the defect candidate region and the template image, the Hausdorff distance is used to calculate the edge shape matching distance. This distance measures the maximum and minimum distance between the two sets of edge points, accurately reflecting the similarity of the geometric structure. In the calculation process, the mutual shortest distance of the edge point set is considered, and the smaller the value is, the more matched the shape is.

[0062] Next, based on the deep feature encoding results extracted by the convolutional neural network (CNN), the distance between the defect candidate region and the template image in the high-dimensional semantic space is calculated, and the Euclidean distance is used as the measurement standard. This high-dimensional space distance reflects the difference between images at the semantic level.

[0063] By comprehensively using the cosine similarity, Hausdorff distance and high-dimensional space Euclidean distance, the system realizes multi-dimensional feature similarity evaluation of forging defect images, providing multi-angle judgment basis for defect recognition.

[0064] Further, the present application also includes:

[0065] The image features of the defect candidate region are matched with the feature vectors of each type of defect template in the defect image sample library to obtain feature similarity , class separation degree , region structure quality , sample coverage ;

[0066] The class separation degree is based on the similarity difference of the defect candidate region in each type of defect template, which measures the clarity of classification discrimination and reflects the distribution concentration of classification confidence;

[0067] The region structure quality is an evaluation of the edge sharpness, contour closure, texture contrast and other image quality features of the defect candidate region image, which is used to avoid misjudgment caused by image blur or noise;

[0068] The sample coverage is based on the distribution density and representativeness of the matching template in the sample library, which judges whether the defect candidate region is within the effective coverage range of the sample library, and is used to improve the robustness in small sample scenarios;

[0069] The defect confidence calculation formula is:

[0070]

[0071] wherein, is a defect confidence score, is a feature similarity, is a category separation degree, is a region structure quality, is a sample coverage, and a, b, c, and d are preset weight coefficients, and a+b+c+d=1.

[0072] Specifically, the image features of the defect candidate region are matched with the feature vectors of each defect template in the defect image sample library in multiple dimensions to obtain comprehensive evaluation indexes including feature similarity, category separation degree, region structure quality, and sample coverage. The feature similarity quantifies the closeness of the defect candidate region to different defect types by comparing the similarity between the feature vectors of the defect candidate region and each defect template, and is a basic index for determining the defect type. The category separation degree is based on the similarity difference between the defect candidate region and each defect template, and measures the definiteness of classification discrimination. Specifically, by calculating the similarity gap between the defect candidate region and the most similar category and the second most similar category, the distribution concentration of the classification confidence is reflected. The greater the category separation degree, the higher the distinguishing degree of the defect candidate region features between different categories, and the more reliable the classification result. The region structure quality evaluates the image quality of the defect candidate region, covering edge sharpness, contour closure, and texture contrast. This index is used to exclude misjudgments caused by image blur, noise, or abnormal acquisition, and to ensure that the defect determination is based on high-quality image regions. The sample coverage evaluates the distribution density and representativeness of the matching template in the sample library, and determines whether the defect candidate region falls within the effective coverage range of the sample library. By analyzing the distribution characteristics of related defect templates in the sample library, the recognition robustness of the system for small samples or rare defects is improved. wherein, is a defect confidence score, is a feature similarity, is a category separation degree, is a region structure quality, is a sample coverage, and a, b, c, and d are preset weight coefficients, and a+b+c+d=1.

[0073] The above indexes are input into the defect confidence calculation model, and through weighted fusion or nonlinear mapping, the confidence value of the corresponding defect type determination result is obtained. This confidence value reflects the credibility of defect recognition, guiding the subsequent automatic determination or manual review process.

[0074] Further, the present application also includes: ​

[0075] When the confidence value is lower than a preset confidence value, a lightweight deep neural network recognition model is called to perform secondary defect type determination on the current defect candidate region; if the output confidence of the lightweight deep recognition model is lower than a preset secondary confidence threshold, it is determined that the recognition result of the defect candidate region is not clear, and manual defect type labeling is performed on the defect candidate region; the defect candidate region and defect category information after manual labeling are added to a defect image sample library.

[0076] Specifically, a lightweight deep neural network recognition model is called to perform secondary defect type determination on the current defect candidate region. The lightweight deep neural network model has efficient feature extraction and classification capabilities through a multi-layer convolution and pooling structure combined with a simplified full connection layer, and is suitable for quickly performing defect recognition tasks in a resource-limited environment. In the secondary determination process, the model outputs a confidence value corresponding to each defect category. If the output confidence is still lower than the set secondary confidence threshold, it is determined that the recognition result of the defect candidate region is not clear, and a manual review link is needed. The defect region image with unclear recognition is submitted to a manual labeling module, and a professional person accurately labels the defect type. After labeling is completed, the sample image and its defect category information confirmed by the human are recorded in the defect image sample library, which is used for subsequent model training and updating, to improve the richness of the sample library and the accuracy of the recognition model. Through this mechanism, the low-confidence defect samples are effectively supplemented and the model is self-adaptively optimized, and the recognition reliability and application range of the forging product defect intelligent detection system are enhanced.

[0077] In summary, the forging product defect intelligent detection system provided by the present application has the following technical effects:

[0078] By configuring a high-resolution industrial camera with a rotary drive assembly, multi-angle image acquisition of the forging product to be inspected is realized, ensuring comprehensive coverage and high-precision capture of surface defects, using multi-angle image stitching, coordinate unification and curved surface unfolding technology to generate a complete and expanded product surface image, providing an accurate image basis for defect detection; using an advanced image segmentation algorithm to extract defect candidate regions, combined with region coordinates and size information, precise positioning and quantitative analysis of defects are realized, a standardized multi-type defect image sample library is constructed, efficient feature matching and similarity fusion scoring are performed based on multi-dimensional image features, improving the accuracy and robustness of defect recognition; an innovative defect confidence calculation method is designed, considering feature similarity, class separation, region structure quality and sample coverage, significantly improving the reliability of defect determination; through a lightweight deep neural network, secondary determination of low-confidence samples is performed, and combined with an artificial labeling mechanism, the sample library and recognition model are continuously optimized, enhancing the system's ability to adapt to complex and variable detection scenarios; that is, by achieving the technical goals of multi-angle high-precision image acquisition and stitching processing, multi-dimensional feature fusion defect recognition algorithm, and confidence-based multi-level determination and dynamic optimization of the sample library, the accuracy, reliability and intelligent level of forging product forging defect detection are improved.

[0079] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0080] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the present application. Accordingly, the present application is intended to embrace all such modifications and changes as fall within the scope of the present application and its equivalents.

Claims

1. An intelligent detection system for forging defects of forging products, characterized in that: include: An image acquisition module, configured to acquire multi-angle images of the workpiece to be inspected by configuring a high-resolution industrial camera and a rotary drive assembly, and indexing the acquired images in chronological order. The industrial camera is mounted on a controllable rotating platform, and the workpiece to be inspected is fixed to the central axis position by a positioning fixture; An image stitching module is used to stitch the collected multi-angle images, unify the coordinates and unfold the surface of the stitched complete image, and generate a complete surface unfolding diagram of the product; A defect candidate region extraction module is used to extract defect candidate regions where defects exist through an image segmentation algorithm, and obtain the region coordinates and region size of the defect candidate, wherein the region size includes the defect area and the maximum boundary size of the defect region; a defect type determination module, which is used to match and compare the extracted image features of the defect candidate area with a pre-built defect image sample library to obtain the defect type of the defect candidate area. The defect image sample library includes five standardized defect image feature templates, and the five defect categories include bubbles, scratches, dents, cold shuts, and flow marks; A defect confidence determination module is used to calculate feature similarity, category separation, regional structure quality, and sample coverage for each candidate defect region, and input a defect confidence calculation formula to obtain a confidence value for the corresponding defect type determination result; Abnormal area secondary judgment module, which is used to call a lightweight deep network recognition model for secondary judgment of low-confidence samples, and supports manual annotation to generate new sample data and add it to the defect sample library; Automation control module, which is used to connect the device end to the programmable controller and cooperate with the operation panel to perform intelligent detection of defects in the workpiece to be inspected and feedback adjustment management; Among them, the category separation It is a measure of the similarity difference between defect candidate regions in various defect templates, which measures the clarity of classification discrimination and reflects the distribution concentration of classification confidence. The structural quality of the area To evaluate the edge clarity, contour closure, and texture contrast image quality characteristics of defect candidate area images to avoid misjudgment due to image blur or noise; The sample coverage In order to judge whether the defect candidate area is within the effective coverage range of the sample library based on the distribution density and representativeness of the matching template in the sample library, it is used to improve the robustness in small sample scenarios.

2. The intelligent forging defect detection system for forging products according to claim 1, characterized in that: The method includes stitching the collected multi-angle images, unifying the coordinates and unfolding the surface of the stitched complete image to generate a complete surface unfolding diagram of the product, including: Based on the overlapping areas between adjacent images in the multi-angle image, the image registration operation is performed, and the multi-angle images are stitched using an image stitching algorithm to generate a stitched image covering the entire product surface; Performing a coordinate unification operation on the stitched image to convert pixel coordinates in different image sources into a unified image plane coordinate system to eliminate position deviations caused by different shooting angles; Based on the product expansion rule, the pixel information in the spliced ​​image is sequentially expanded along the product outer surface direction to generate a product surface expansion map with a unified plane structure for subsequent defect recognition processing.

3. The intelligent forging defect detection system for forging products according to claim 2, characterized in that: The method includes performing an image registration operation based on the overlapping areas between adjacent images in the multi-angle image, and using an image stitching algorithm to stitch the multi-angle images to generate a stitched image covering the entire product surface, including: Determine adjacent images according to a preset shooting order, and perform image feature extraction on portions of the adjacent images located in overlapping areas in combination with image indexes, wherein the features include edge features, corner features, and texture features; The feature points extracted from the overlapping area of ​​one image are matched with the feature points of the corresponding area in its adjacent image. The distance between the feature point pairs is calculated using the descriptor matching algorithm SIFT, and the initial matching point set is screened based on the distance threshold. Using the RANSAC algorithm to perform robustness screening on the initial matching point set, eliminating abnormal matches, and obtaining a consistent matching pair set; Based on the consistent matching pair set, the homography matrix between the images is estimated to achieve spatial alignment of the image coordinate system; Iteratively optimize the transformation results and superimpose the registered images according to a unified coordinate system; A multi-resolution fusion strategy is used to smoothly transition the pixels in the overlapping areas to avoid obvious stitching gaps and generate a stitching image that covers the entire product surface, has continuous texture, and complete contours.

4. The intelligent forging defect detection system for forging products according to claim 1, characterized in that: Extract defect candidate areas with defects through image segmentation algorithms, and obtain the area coordinates and area size of the defect candidate. The area size includes the defect area and the maximum boundary size of the defect area, including: Processing the image regions in the expanded image using an image segmentation algorithm to extract image regions that may have defects as candidate defect regions. The image segmentation algorithm includes a segmentation network based on pixel-level classification and a region growing method based on edge detection, which is used to separate regions with abnormal texture, color, or boundary features in the image from the background; In each extracted defect candidate area, the spatial coordinate information of the area is obtained, including the coordinate points of the upper left corner and the lower right corner, to determine the position of the defect area in the image; Calculate the region size of each defect candidate region, where the region size includes the defect area and the maximum boundary size of the defect region. The defect area is calculated based on the number of valid pixels in the region, and the maximum boundary size of the defect region is the longest side length of the defect in the horizontal or vertical direction.

5. The intelligent forging defect detection system for forging products according to claim 1, characterized in that: The extracted image features of the defect candidate area are matched and compared with the pre-built defect image sample library to obtain the defect type of the defect candidate area. The defect image sample library includes five standardized defect image feature templates. The five types of defects include bubbles, scratches, dents, cold shuts, and flow marks, including: Input the image features of the defect candidate area into the image similarity comparison algorithm, perform feature matching and similarity comparison with the pre-built defect image sample library, and determine the defect type of the candidate area; The image similarity comparison algorithm uses a pre-built defect image sample library, which is a set of standard image templates established according to different defect types. Each template corresponds to a defect type and stores the corresponding feature encoding results. Perform feature matching operations between the defect candidate area and various defect templates in the defect image sample library, and use an image similarity comparison algorithm to calculate a similarity score, which is used to measure the degree of matching between the defect candidate area and various defect templates; The defect type of the defect candidate area is determined according to the defect template category corresponding to the maximum similarity score.

6. The intelligent forging defect detection system for forging products according to claim 5, characterized in that: The image similarity comparison algorithm includes: Extract multi-dimensional image features from defect candidate areas and template images in the defect image sample library, including texture features, shape features, semantic features, and spatial distribution features; The extracted multi-dimensional image features are vectorized and normalized to unify the dimensions and scales to form a standard feature code; Based on the extracted image features of the defect candidate area and the feature vectors of various defect templates in the defect image sample library, feature similarity is obtained. , including cosine similarity , edge shape matching distance Distance to high-dimensional space ; The feature similarity Input a preset similarity fusion scoring function to obtain a fusion score between the defect candidate area and the various defect templates to obtain a similarity fusion score; The similarity fusion score calculation formula is: ； in, is the cosine similarity, is the edge shape matching distance, is the high-dimensional space distance, α, β, γ are preset weighting coefficients, α+β+γ=1; The defect candidate areas and the fusion scores of the various defect templates are sorted, and the defect type corresponding to the highest score is taken as the recognition result.

7. The intelligent forging defect detection system for forging products according to claim 6, characterized in that: Feature matching is performed based on the extracted image features of the defect candidate area and the feature vectors of various defect templates in the defect image sample library. The feature matching includes cosine similarity, edge shape matching distance and high-dimensional space distance, including: Calculate the cosine similarity between feature vectors , edge shape matching distance Distance to high-dimensional space ; Cosine similarity between feature vectors extracted based on image texture, shape, or deep semantic features Calculation, used to measure the consistency of image feature directions; Based on the edge contour features of the defect candidate area and the template image, the Hausdorff distance is used to calculate the edge shape matching distance to measure the geometric structure similarity; Based on the deep feature encoding results extracted by the convolutional neural network, a high-dimensional space distance is calculated to describe the image differences in the semantic space. The high-dimensional space distance is the Euclidean distance.

8. The intelligent forging defect detection system for forging products according to claim 1, characterized in that: Calculate the feature similarity, category separation, regional structure quality, and sample coverage for each candidate defect area, and input the defect confidence calculation formula to obtain the confidence value of the corresponding defect type judgment result, including: The image features of the defect candidate area are matched with the feature vectors of various defect templates in the defect image sample library to obtain feature similarity , category separation , regional structural quality , sample coverage ; The defect confidence calculation formula is: ； in, Score the defect confidence level, is the feature similarity, is the degree of class separation, is the regional structural quality, is the sample coverage, a, b, c, d are the preset weight coefficients, and a+b+c+d=1.

9. The intelligent forging defect detection system for forging products according to claim 1, characterized in that: For low-confidence samples, a lightweight deep network recognition model is called for secondary judgment. At the same time, manual annotation is supported to generate new sample data and add it to the defect sample library, including: When the confidence value is lower than a preset confidence value, a lightweight deep neural network recognition model is called to perform a secondary defect type determination on the current defect candidate area; If the output confidence of the lightweight deep recognition model is lower than the preset quadratic confidence threshold, it is determined that the recognition result of the defect candidate area is unclear, and the defect candidate area is manually labeled with the defect type; The manually annotated defect candidate areas and defect category information are added to the defect image sample library.

Images