Wire harness defect detection method and system based on visual recognition

Abstract

The present invention relates to the technical field of industrial defect detection, and in particular to a wire harness defect detection method and system based on visual recognition. The method comprises the following steps: acquiring wire harness image data, and performing image segmentation according to the wire harness image data to obtain wire harness image segmentation data; performing morphological feature extraction according to the wire harness image segmentation data to obtain wire harness morphological feature data, and performing three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data; performing microstructure defect detection according to the wire harness morphological feature data to obtain a first wire harness defect detection to obtain first wire harness defect detection data, and performing micromorphological defect detection on the wire harness three-dimensional reconstruction data to obtain a second wire harness defect detection data; marking the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data. The present invention can identify potential defects in subtle morphological changes of the wire harness.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial defect detection, and in particular to a wire harness defect detection method and system based on visual recognition. Background Art

[0002] In the field of industrial manufacturing, wire harnesses, as an important part of the electrical system, are widely used in high-tech fields such as automobiles, aviation, and electronic equipment. These wire harnesses have complex structures and usually contain a large number of cables, connectors, and sheaths for transmitting electrical signals or data. Since wire harnesses usually work in harsh environments (such as high temperature, vibration, or chemical corrosion), any tiny defects (such as surface cracks, internal fractures, wear, etc.) will lead to electrical failures, equipment downtime, and even safety accidents. Therefore, wire harness defect detection is crucial to ensure product quality and equipment safety.

[0003] Traditional wire harness inspection methods mainly rely on manual visual inspection and simple mechanical testing. Manual inspection has problems such as large errors, low efficiency, and strong subjectivity, making it difficult to achieve accurate inspection. Especially when detecting complex defects such as tiny cracks and internal fractures, manual inspection often cannot meet high-precision requirements. In addition, mechanical testing methods (such as tensile testing) can only detect obvious structural damage, but have limited detection capabilities for microscopic cracks and potential faults. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a wire harness defect detection method and system based on visual recognition to solve at least one of the above technical problems.

[0005] The present application provides a wire harness defect detection method based on visual recognition, comprising the following steps:

[0006] Step S1: acquiring wire beam image data, and performing image segmentation according to the wire beam image data to obtain wire beam image segmentation data;

[0007] Step S2: extracting morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and performing three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data;

[0008] Step S3: performing microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and performing micromorphological defect detection on the wire harness 3D reconstruction data to obtain second wire harness defect detection data;

[0009] Step S4: marking the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0010] In the present invention, the image data of the wire harness is first obtained, and the image segmentation technology is used to extract the wire harness area, so as to effectively isolate the wire harness from the background and reduce noise and interference. Two different detection methods, microstructure and micromorphology, are used to detect 2D plane features and 3D morphological features respectively, so that more subtle and complex defects can be identified, greatly improving the accuracy of detection. By reconstructing the wire harness in three dimensions after morphological feature extraction, a three-dimensional model of the wire harness is obtained, so that defect detection is not limited to the plane, but anomalies in the three-dimensional structure can be detected, and it can effectively identify, for example, wear on the surface of the wire harness and internal structural defects. A defect marking map is automatically generated based on the first and second wire harness defect detection data, and the identified defects are directly marked in the image data, so that the detection results can be directly visualized and provide clear defect location information in production or maintenance. Through dual detection of 2D and 3D data, the system can cross-verify the detection results and improve the reliability and consistency of the detection results.

[0011] Preferably, step S1 specifically includes:

[0012] Ambient light data is obtained by collecting ambient light through an ambient light sensor;

[0013] Get harness material data;

[0014] Evaluate the material surface characteristics according to the wire harness material data to obtain the wire harness surface characteristics data;

[0015] Setting visual parameters according to ambient light data and harness surface characteristic data to obtain visual parameter data;

[0016] According to the visual parameter data, the wire harness multi-view image acquisition is performed through a preset visual device to obtain the wire harness image data;

[0017] Perform geometric calibration according to the wire harness image data to obtain wire harness geometric calibration data;

[0018] Perform edge detection according to the wire harness geometry calibration data to obtain wire harness edge data;

[0019] Perform region segmentation and marking according to the wire harness edge data to obtain wire harness region segmentation data;

[0020] Smooth detail reconstruction is performed based on the wire harness region segmentation data to obtain wire harness image segmentation data.

[0021] In the present invention, the current ambient light data is collected by an ambient light sensor, and the visual parameters are flexibly adjusted according to the lighting conditions, which can significantly reduce the instability of image quality caused by lighting changes, thereby obtaining a clearer image. According to the data of the harness material, the reflectivity, light absorption and other characteristics of the surface are evaluated, and the visual parameters are optimized to cope with different material properties. The multi-view images are geometrically calibrated to correct the shooting angle and position differences to form a set of geometrically precise image data, which provides support for precise edge detection and segmentation. After geometric calibration, the outline of the harness is extracted by an accurate edge detection algorithm. Due to the high image quality and proper calibration, the edge detection process can efficiently capture the outline of the harness and reduce errors and blurred areas. Smoothing and detail enhancement are performed based on the regional segmentation data to remove noise and ensure that the details and texture of the image are clear and the edges are smooth.

[0022] Preferably, the smooth detail reconstruction is specifically:

[0023] Perform multi-resolution analysis based on the wire harness area segmentation data to obtain wire harness multi-resolution level data;

[0024] Perform edge enhancement according to the multi-resolution level data of the wire bundle to obtain multi-level edge enhancement data of the wire bundle;

[0025] Performing texture restoration on the wire harness multi-level edge enhancement data to obtain the wire harness multi-level texture restoration data;

[0026] Perform super-resolution restoration based on the multi-level texture restoration data of the wire harness to obtain super-resolution data of the wire harness;

[0027] Performing structure-preserving mapping on the wire bundle super-resolution data to obtain wire bundle structure mapping data;

[0028] Performing feature smoothing processing on the wire harness structure mapping data to obtain wire harness feature smoothing data;

[0029] Perform local detail alignment based on the wire harness feature smoothing data to obtain multi-level detail alignment data of the wire harness;

[0030] The wire harness multi-level detail alignment data is reconstructed and segmented and outputted to obtain wire harness image segmentation data.

[0031] In the present invention, the wire harness image is subdivided into hierarchical data of different resolutions through multi-resolution analysis. This method can capture the detailed features of the wire harness at different resolution levels. The high-resolution layer captures fine features, and the low-resolution layer captures the overall contour information, thereby achieving a balance between fine features and global contours. By performing multi-level enhancement on the edges of different resolution levels, the edges of the wire harness contours are clearly visible. Edge enhancement is particularly beneficial for complex structures and interlaced wire harnesses, so that each wire harness contour in the image has a clear boundary. After edge enhancement, texture restoration is performed to restore or enhance the natural texture of the wire harness surface, ensuring that the texture details of the surface are retained while the edges are clear. The image details are further enhanced by a high-resolution algorithm, so that the fine features in the low-resolution image can be clearly presented. After super-resolution restoration, the structural deformation introduced by the image enhancement process is avoided through structure-preserving mapping, ensuring that the shape and structure of the wire harness remain original after magnification. The texture and shape are delicately processed through feature smoothing, so that the transition between different areas of the image is more natural, and unnecessary noise and sharp boundaries are eliminated. After feature smoothing, detail alignment is performed to recalibrate feature data at different resolution levels to the same alignment standard to ensure that data at each level are spatially unified and accurate.

[0032] Preferably, the feature smoothing process is specifically:

[0033] Performing multi-scale similar block search on the wire harness structure mapping data to obtain multi-scale similar block data of the wire harness;

[0034] A lightweight convolutional network similarity weight calculation is performed based on the multi-scale similarity block data of the wire bundle to obtain the wire bundle similarity weight data;

[0035] The wire harness structure mapping data is weighted averaged according to the wire harness similarity weight data to obtain the wire harness similarity weighted data;

[0036] Perform pixel update according to the wire bundle similarity weighted data to obtain wire bundle pixel update data;

[0037] An edge preservation check is performed based on the wire harness pixel update data to obtain the wire harness feature smoothing data.

[0038] In the present invention, similar areas in the image can be identified by searching for similar blocks at different scales, and the complexity and diversity in the wire harness structure can be more flexibly adapted, especially for wire harnesses with complex textures or tiny features. The use of lightweight convolutional networks makes the similarity weight calculation more efficient and accurate. The convolutional network can automatically learn the complex feature relationship between similar blocks, thereby avoiding the deviation problem of traditional manual similarity calculation. The weighted mean calculation based on the similarity weight data can effectively smooth the pixel values ​​of similar areas without destroying the original detail information, so that the smoothing process can remove noise while maintaining the protection of key details. After the weighted mean result is updated to the pixel position in the image, the smoothness and consistency of the image are significantly improved, ensuring that the result of the smoothing process can be accurately reflected in the wire harness structure mapping data, reducing the loss of details caused by noise during the smoothing process, and compared with the simple mean or Gaussian smoothing process of the existing method, the weighted mean calculation of the present invention retains the edges and tiny features while denoising, and improves the overall clarity of the wire harness detection. After the pixel is updated, the edge retention check is used to ensure that the key edges of the wire harness are not blurred. Edge preservation can specifically protect the edges of structures, so that the final smoothed image has both high-quality smoothing effects and retains edge clarity.

[0039] Preferably, the morphological feature extraction is specifically:

[0040] Capturing the area contour according to the wire harness image segmentation data to obtain the wire harness area contour data;

[0041] Performing contour reconstruction according to the wire harness area contour data to obtain wire harness contour reconstruction data;

[0042] Performing coupled convolution feature extraction and interpreted convolution feature extraction on the wire beam contour reconstruction data to obtain first wire beam image feature data and second wire beam image feature data respectively;

[0043] Performing wire harness shape mapping on the first wire harness image feature data and the second wire harness image feature data to obtain first wire harness shape mapping data and second wire harness shape mapping data;

[0044] Perform detail structure difference mapping according to the first wire harness shape mapping data and the second wire harness shape mapping data to obtain wire harness shape detail structure difference data;

[0045] Generate a shape feature map based on the wire harness morphology detail structure difference data to obtain wire harness morphology feature data;

[0046] The coupled convolution feature extraction is specifically as follows:

[0047] Performing initial convolution layer processing on the wire harness contour reconstruction data to obtain wire harness initial convolution layer data;

[0048] Separate the coupled feature channels of the initial convolutional layer data of the wire harness to obtain the wire harness geometric structure feature data and the wire harness texture feature data respectively;

[0049] Perform residual connection convolution layer processing on the wire harness geometric structure feature data to obtain high-order feature data of the wire harness geometric structure, and perform dense connection convolution layer processing on the wire harness texture feature data to obtain high-order feature data of the wire harness texture;

[0050] Performing feature fusion on the high-order feature data of the wire harness geometry structure and the high-order feature data of the wire harness texture to obtain coupled fusion feature map data;

[0051] The coupled fusion feature map data will be subjected to local receptive field enhancement to obtain coupled local enhanced feature map data;

[0052] Performing feature compression on the coupled local enhanced feature map data to obtain first line beam image feature data;

[0053] The explanation of convolution feature extraction is as follows:

[0054] Calculate the confidence of the wire harness contour reconstruction data according to the coupled fusion feature map data to obtain the wire harness area confidence data;

[0055] Performing feature selective convolution layer processing on the wire harness contour reconstruction data according to the wire harness region confidence data to obtain wire harness selective convolution layer data;

[0056] Perform feature comparison on the wire bundle selective convolution layer data and the coupled fusion feature map data to obtain wire bundle comparison feature data;

[0057] Performing a significance analysis on the wire harness contrast feature data to obtain the wire harness significance feature data;

[0058] Perform high-level structural aggregation on the wire harness significant feature data to obtain wire harness structural aggregation data;

[0059] Perform explanatory self-attention map calculation on the bundle salient feature data and the bundle structure aggregation data to obtain the bundle weighted explanatory feature map data;

[0060] Global morphological mapping is performed according to the line bundle weighted explanatory feature map data to obtain second line bundle image feature data.

[0061] In the present invention, the basic outline of the wire harness is accurately identified by capturing and reconstructing the outline of the wire harness image segmentation data, thereby ensuring the accuracy of the basic structure. The geometric structure features are separated from the texture features, and residual connections and dense connection convolution layers are applied respectively, so that each type of feature can be deeply extracted in its own space. The geometric features provide shape information, and the texture features highlight the details, thereby ensuring the comprehensiveness of feature capture. Through feature fusion and local receptive field enhancement, the geometric and texture features are integrated into a unified coupled feature map, while enhancing the parsing ability of local features. Local receptive field enhancement ensures that tiny edges and morphological changes are not ignored. Convolutional layer processing is selectively performed according to confidence data, so that important areas receive higher calculation weights. Selective convolution increases the emphasis on important feature areas and reduces the interference of unimportant information. The features are analyzed for significance, the most distinguishing feature areas are identified, and these areas are highlighted in the feature map through weighting by the self-attention mechanism, thereby ensuring the high visibility of key areas and significant features in the data. Global morphological mapping is performed on the basis of multi-layer feature data, and key morphological features are aggregated into the global space, thereby realizing accurate recognition of the overall morphology of the wire harness. High-level structural aggregation helps the system generate higher-level structural information.

[0062] Preferably, the three-dimensional reconstruction is specifically:

[0063] Performing morphological classification according to the wire harness morphological feature data to obtain wire harness morphological classification data;

[0064] Perform feature map fusion according to the wire harness morphology classification data to obtain wire harness morphology feature map fusion data;

[0065] Performing depth estimation on the fusion data of the wire harness morphological feature map to obtain the wire harness feature depth data;

[0066] Performing perspective matching according to the wire harness feature depth data to obtain wire harness perspective matching data;

[0067] Generate three-dimensional point cloud according to the wire harness view matching data to obtain wire harness point cloud data;

[0068] Three-dimensional mesh reconstruction is performed based on the wire harness point cloud data to obtain three-dimensional reconstruction data of the wire harness.

[0069] In the present invention, before reconstruction begins, the wire harness morphology feature data is classified to generate wire harness morphology classification data. This step can effectively distinguish different morphological features, ensuring that the reconstruction process is more targeted, especially when dealing with complex structures or diversified wire harness morphologies. The classified morphological feature maps are fused so that different morphological information is integrated into a complete feature map to ensure that all morphological features remain consistent in space. The fusion process can enhance the integrity of the overall features and provide clear morphological data for three-dimensional reconstruction. By performing depth estimation on the fused morphological feature map, the depth information of different features is calculated. Depth estimation can accurately obtain the spatial position of different parts of the wire harness at this stage, providing reliable depth data for perspective matching and point cloud generation. By performing multi-perspective matching on the depth data, the morphological features under different perspectives are aligned to ensure the accuracy of three-dimensional reconstruction. Perspective matching can compensate for errors in multi-perspective shooting, making the reconstructed data more consistent. A three-dimensional point cloud is generated based on the perspective matching data to capture the three-dimensional structure of the wire harness. The point cloud data contains rich three-dimensional coordinate information, providing high-quality spatial data for grid reconstruction.

[0070] Preferably, the microstructure defect detection is specifically:

[0071] Perform feature block division according to the wire harness morphology feature data to obtain the wire harness morphology local feature data;

[0072] Identify defect candidate areas according to the local feature data of the wire harness morphology to obtain wire harness defect candidate area data;

[0073] Perform texture feature matching based on the wire harness defect candidate area data and the preset conventional wire harness area data to obtain the wire harness texture abnormality data;

[0074] According to the abnormal data of wire harness texture, local deformation similarity calculation and relative deformation measurement processing are performed to obtain the wire harness local deformation similarity data and the wire harness relative deformation measurement data;

[0075] Performing shape comparison based on the wire harness local deformation similarity data and the wire harness relative deformation measurement data to obtain wire harness shape comparison data;

[0076] Defect classification and confidence estimation are performed based on the wire harness morphology comparison data to obtain first wire harness defect detection data.

[0077] In the present invention, the wire harness morphological feature data is divided into local feature blocks so as to analyze local details, ensure that the system can focus on feature details in a local range, and is particularly suitable for capturing small-scale defects and subtle anomalies. By screening the local feature data to identify potential defect candidate areas, the amount of areas processed by the system can be effectively reduced, and the computing resources can be concentrated on the areas with defects. By matching the defect candidate areas with the preset conventional wire harness texture data, the texture abnormal areas that are different from the normal areas are detected, the potential defect areas are quickly screened out, and the misjudgment rate is significantly reduced. Similarity and deformation measurement analysis are performed on the texture abnormal areas to identify subtle deformation anomalies. Deformation similarity analysis can accurately detect the degree of deformation of the abnormal area, while relative deformation measurement provides a comparison of deformation position and amplitude. The deformation similarity data and relative deformation data are compared to further clarify the location and shape characteristics of the defects and ensure accurate identification of the defect areas. By classifying the morphological comparison results, dividing the defect areas according to types, and estimating the confidence of the detection results, detailed defect type information can be generated to ensure the reliability of the classification and provide a credibility evaluation of the recognition results.

[0078] Preferably, the micro-morphological defect detection is specifically:

[0079] Extracting surface features from the wire harness three-dimensional reconstruction data to obtain wire harness surface feature data;

[0080] Performing local area division and mesh refinement on the surface feature data of the harness to obtain local refined mesh data;

[0081] Perform morphological anomaly detection based on local refined grid data to obtain harness morphological anomaly data;

[0082] Perform structural integrity analysis based on the wire harness 3D reconstruction data and wire harness morphology abnormality data to obtain wire harness structural integrity data;

[0083] Perform fine texture detection according to the abnormal data of wire harness morphology to obtain abnormal data of wire harness fine texture;

[0084] Defect category classification and positioning processing are performed based on the wire harness fine texture abnormality data to obtain second wire harness defect detection data.

[0085] In the present invention, surface feature extraction is performed based on the three-dimensional reconstruction data of the wire harness, ensuring that the system can identify the appearance features of each structure, so that the detailed information of the wire harness surface can be fully captured, thereby laying a high-quality input foundation for detection. The surface feature data is locally divided, and the spatial resolution of the local area is increased by mesh refinement, which can effectively improve the resolution of the three-dimensional model so that even tiny defect features can be identified. Based on the refined mesh data, the system can efficiently identify areas with morphological anomalies, which can not only quickly screen out abnormal areas, but also reduce the scope of subsequent processing and improve overall detection efficiency. Combining three-dimensional reconstruction data and morphological anomaly data, the system can analyze the integrity of the structure, identify damaged areas such as fractures and cracks, and ensure the comprehensiveness of the detection.

[0086] Preferably, step S4 is specifically:

[0087] Initialize the marking position according to the first wire harness defect detection data and the second wire harness defect detection data to obtain defect position marking data;

[0088] Performing plane marking on the wire harness image data according to the first wire harness defect detection data to obtain first wire harness image defect marking data;

[0089] Performing three-dimensional superposition on the wire harness image data according to the defect position marking data and the wire harness three-dimensional reconstruction data to obtain second wire harness image defect marking data;

[0090] Defect visualization rendering is performed according to the first wire harness image defect marking data and the second wire harness image defect marking data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0091] In the present invention, the defect position mark is initialized according to the first and second wire harness defect detection data, and the defect position mark data is generated to ensure that the mark position corresponds accurately to the detected defect position, which can eliminate the position deviation and make the marking process more accurate. By marking the first wire harness defect detection data onto the original image, the first wire harness image defect mark data is generated, so that the detection results can be intuitively displayed on a two-dimensional plane. This step quickly generates clear plane defect marks, which is helpful for the display and rapid feedback of preliminary detection results. The defect position mark data is superimposed with the three-dimensional reconstruction data to generate three-dimensional defect mark image data (second wire harness image defect mark data). Three-dimensional superposition can accurately display the distribution of defects in three-dimensional space, making the marking more three-dimensional and accurate.

[0092] Preferably, the present application also provides a wire harness defect detection system based on visual recognition, which is used to execute the wire harness defect detection method based on visual recognition as described above. The wire harness defect detection system based on visual recognition includes:

[0093] A wire harness image segmentation module, used for acquiring wire harness image data, and performing image segmentation according to the wire harness image data to obtain wire harness image segmentation data;

[0094] A wire harness morphological feature extraction and three-dimensional reconstruction module is used to extract morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and to perform three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data;

[0095] A wire harness defect detection module is used to perform microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and perform micromorphological defect detection on the wire harness three-dimensional reconstruction data to obtain second wire harness defect detection data;

[0096] The wire harness image defect marking module is used to mark the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0097] The beneficial effect of the present invention is that after acquiring the image data, the regional information of the wire harness is extracted by image segmentation, and the background interference is effectively isolated, so as to ensure that the feature extraction can focus on the structure of the wire harness itself. The morphological feature extraction further enhances the detailed information of the wire harness, ensuring that the system can identify potential defects in subtle morphological changes, which not only improves the clarity of the initial data, but also shows extremely high adaptability in complex detection environments. Through the separated microstructure and micromorphological defect detection, not only the high-precision recognition of surface defects (microstructure defects) is achieved on the two-dimensional plane, but also the abnormalities (micromorphological defects) in the structure or in the three-dimensional morphology are identified through the deep analysis after three-dimensional reconstruction. The dual detection strategy effectively reduces the detection blind area and greatly improves the comprehensiveness and depth of defect recognition. The system separates and fuses the geometric and texture features through coupled convolution to capture more subtle features; the interpretation convolution feature extraction further focuses on the abnormal area through confidence and significance analysis. The layered feature extraction method not only improves the recognition accuracy of details, but also makes the system more sensitive when dealing with complex textures and diverse structures. The marking position is initialized according to the first and second wire harness defect detection data, and the defect information is accurately located in the wire harness image data through the combination of two-dimensional plane marking and three-dimensional superposition marking. The multi-dimensional marking process can not only display the plane position of the defect on the two-dimensional image, but also more intuitively display the position relationship of the defect in three-dimensional space through three-dimensional superposition, thereby achieving the unity of three-dimensional marking and spatial accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0099] Figure 1 A flowchart of a wiring harness defect detection method based on visual recognition according to an embodiment is shown;

[0100] Figure 2 A flowchart of a line beam image segmentation method according to an embodiment is shown. DETAILED DESCRIPTION

[0101] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0102] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0103] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0104] Based on the ambient light parameters detected by the sensor, the visual parameters are adjusted to an exposure value of 300 and a white balance of 4500K. A high-resolution camera is used for multi-view shooting in four directions to generate four images with a resolution of 4000×3000. Based on the edge features of the wire harness, the wire harness is segmented from the background through edge detection and segmentation algorithms, and the image segmentation data obtained accounts for 75% of the original image.

[0105] The segmented wire harness image is processed by the initial convolution layer. The initial convolution feature is 120 dimensions, and then the coupled features are separated into geometric features and texture features, generating 60-dimensional geometric structure feature data and 60-dimensional texture feature data respectively. The geometric and texture features are fused to generate a coupled fusion feature map for deeper defect identification. The system calculates the depth estimate value between 0.3mm and 2.5mm, and aligns the spatial positions of each image based on multi-view matching. The matched data is converted into 3D point cloud data to obtain a point cloud of 300,000 points, representing the 3D structure of the wire harness. The point cloud data is converted into a 3D model through 3D mesh reconstruction. The side length of each mesh unit is 0.1mm, and the total number of reconstructed mesh structures reaches 20,000, forming a high-precision 3D model of the wire harness.

[0106] The morphological features were segmented into local feature blocks, each with a size of 20×20 pixels, and 500 local feature data blocks were generated. 50 blocks with defects were identified among the 500 feature blocks. By comparing the normal texture features, 15 feature blocks were detected to have abnormal deformation, with a texture difference of more than 10%. Through relative deformation measurement and morphological comparison, 10 blocks were confirmed to have defects and were classified as "surface cracks" with a confidence level of 95%, generating the first harness defect detection data.

[0107] The mesh of key areas in the 3D reconstruction model was refined to increase the resolution to 0.05mm, and 40,000 meshes were obtained after refinement. The refined mesh was tested for morphological anomalies, and 5 abnormal areas were found. Further structural integrity analysis confirmed that 3 of the areas had internal fractures. Micro-texture detection was performed in the abnormal areas, and it was found that 2 of them had serious texture damage, which was located as internal fractures. The second harness defect detection data was generated with a confidence level of 98%.

[0108] According to the position information of the first and second wire harness defect detection data, the defect position is preliminarily marked and defect position marking data is generated. The first wire harness defect data is marked on the two-dimensional plane image, and the position of the crack is marked with a red marking box. The two-dimensional coordinate information of the 10 crack defect marking points is recorded on the image. Combined with the three-dimensional reconstruction data, the two-dimensional crack marking data and the three-dimensional fracture marking are superimposed on the three-dimensional model to generate a three-dimensional defect marking map, which accurately shows the position of cracks and fractures in space. The two-dimensional and three-dimensional defect data are comprehensively rendered to generate a wire harness defect marking map containing different types of defects, with red representing cracks and blue representing fractures, generating clear defect visualization data.

[0109] See also Figure 1 to Figure 2 , the present application provides a wire harness defect detection method based on visual recognition, comprising the following steps:

[0110] Step S1: acquiring wire beam image data, and performing image segmentation according to the wire beam image data to obtain wire beam image segmentation data;

[0111] In one embodiment, a camera device (such as an industrial camera) is used to collect multi-angle images of the wire harness to ensure that the data covers all parts of the wire harness to ensure comprehensive detection. The wire harness parts are segmented using features such as edge detection and color difference to obtain a clear image of the wire harness area.

[0112] In one embodiment, the diameter of the wire harness to be inspected is 5 mm, the length is 100 mm, and the defect is a small break or twist. A high-resolution camera is used to capture the image of the wire harness, and the image resolution is 1920x1080 pixels, and the unit pixel represents 0.05 mm. A high-resolution camera is used to capture multi-angle images of the wire harness to ensure that all areas of the wire harness are covered. Five pictures are collected, each of which is 1920x1080 pixels in size. The image is grayed to reduce the amount of processing calculations. Gaussian filtering is used to eliminate noise. A color- and edge-based segmentation algorithm is used to extract the area of ​​the wire harness from the image. Regional segmentation is performed by pixel intensity and wire harness color difference. For example, the threshold is set to 120 (within the grayscale range of 0-255), and the part with a grayscale value greater than 120 is regarded as the wire harness area. The size of the wire harness image segmentation data matrix obtained after segmentation is 1000x200 pixels.

[0113] Step S2: extracting morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and performing three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data;

[0114] In one embodiment, the wire harness region in the wire harness segmentation image is further analyzed to extract its morphological features, including extracting information such as contours, corner points and curve features, to describe the geometric structure of the wire harness. Combined with these morphological feature data, a complete set of wire harness morphological description information is formed to provide data support for the next step of three-dimensional reconstruction. By analyzing the morphological feature data of the wire harness, a three-dimensional model of the wire harness is established, and by performing feature registration on image data from different perspectives, the spatial coordinates and three-dimensional morphological information of the wire harness are generated, thereby obtaining the three-dimensional reconstruction data of the wire harness.

[0115] In one embodiment, the edge contour and geometric features (such as length, diameter, etc.) of the wire harness are extracted. For example, the edge contour of the wire harness is obtained by using an edge detection method. By measuring the length and width of the edge contour, it is confirmed whether the wire harness maintains a straight line and uniform diameter. Each feature information is combined into a morphological feature data matrix, for example, the edge contour length is 2000 pixels and the width is 100 pixels. According to the edge contours in the images of different viewing angles, matching and spatial positioning are performed to generate three-dimensional point cloud data. For example, based on the matching of corresponding points of two viewing angle images, a three-dimensional model of the wire harness is obtained by triangulation calculation, and three-dimensional reconstruction data of the wire harness is generated, and the size of the formed three-dimensional point cloud data is 1000x3 point coordinates.

[0116] Step S3: performing microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and performing micromorphological defect detection on the wire harness 3D reconstruction data to obtain second wire harness defect detection data;

[0117] In one embodiment, based on the previously extracted morphological feature data, the microstructure of the wire harness is analyzed in detail to identify the defective areas. First, the morphological feature data is analyzed to detect small defects by comparing the microstructure features of the normal wire harness. Using the 3D reconstructed data, the microstructure of each part in the 3D model is analyzed to identify abnormal changes in the 3D morphology. Based on the 3D reconstructed data, the 3D structure of the wire harness is carefully compared to further discover defects in the 3D model.

[0118] In one embodiment, the morphological feature data is used to detect defects by comparing the slight curvature changes of the wire harness. At the 300th pixel, the curvature changes abnormally (higher than the threshold of 0.02), a microcrack is detected, and the first wire harness defect detection data is generated to record the defect location and characteristics. The wire harness surface is further analyzed using the three-dimensional reconstruction data. Defects are further detected by detecting abnormal curvature or uneven areas of the three-dimensional point cloud. In the three-dimensional model, the curvature changes greatly between the 300th point and the 320th point, and a twist defect is detected. The second wire harness defect detection data is generated to record the three-dimensional defect location and characteristics.

[0119] Step S4: marking the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0120] In one embodiment, the first and second wire harness defect detection data are combined to obtain comprehensive defect data for marking and analysis. Based on the above defect data, the defect location is marked on the original image data. For example, different colors or symbols are used to mark the defect location on the image for easy analysis and inspection, and image data with defect markings is generated to provide visual inspection results for inspection personnel and for further manual or automated auxiliary analysis.

[0121] In one embodiment, the first and second wire harness defect detection data are aggregated. There is a microcrack at the 300-320th pixel, and there is a distortion defect at the 320-340th pixel. Defects are marked using different colors on the original image. For example, microcracks are marked in red, and distortions are marked in yellow. The marking data is represented as: position (300, 320), color (red / yellow), and an image with defect markings is generated for detection and analysis. The defect location is clearly marked on the output image, and detailed defect data is recorded.

[0122] Preferably, step S1 specifically includes:

[0123] Step S11: collecting ambient light through an ambient light sensor to obtain ambient light data;

[0124] In one embodiment, an ambient light sensor is used to detect ambient light conditions, collect ambient light intensity and light source direction data, and ensure that image quality does not fluctuate due to changes in illumination during image acquisition. The light source direction and brightness value (for example, the brightness is 500 lux and the light source angle is 45 degrees) are recorded.

[0125] Step S12: obtaining wire harness material data;

[0126] In one embodiment, the material type data of the wire harness (e.g., copper, aluminum, or plastic) and its surface characteristics (smooth, rough, reflective, etc.) are obtained. The reflectivity and surface texture data of the material are recorded. For example, the reflectivity is 0.3 and the surface roughness is 0.05.

[0127] Step S13: evaluating the material surface characteristics according to the wire harness material data to obtain the wire harness surface characteristic data;

[0128] In one embodiment, based on the reflectivity and roughness data of the material, the surface characteristics of the wire beam, such as light absorptivity and reflection degree, are evaluated to adjust the visual parameters. The surface reflection coefficient is calculated. If the wire beam has a rough surface, the light absorptivity is set to 0.7 and the reflectivity is set to 0.3 to generate surface characteristic data.

[0129] Step S14: setting visual parameters according to the ambient light data and the wire harness surface characteristic data to obtain visual parameter data;

[0130] In one embodiment, the ambient light data and the surface characteristic data of the wire harness are used to set the visual acquisition parameters (such as exposure time, gain, contrast, etc.) to adapt to the current lighting and material characteristics. The parameters of the camera device are adjusted. For example, according to the ambient light of 500 lux and the surface reflectivity of 0.3, the exposure time is set to 10 ms, the gain is set to 1.2, and the contrast is set to 1.1. The setting principle can be generated by mapping based on the database query or the preset parameter matching table.

[0131] Step S15: collecting wire harness multi-view images through a preset visual device according to the visual parameter data to obtain wire harness image data;

[0132] In one embodiment, the wire harness is photographed from multiple viewing angles using the preset visual parameters through a preset camera device to ensure comprehensive image data is obtained. Five images are taken from different angles and collected at a resolution of 1920x1080 pixels to ensure that the wire harness is in a clear focus range in each image.

[0133] Step S16: performing geometric calibration according to the wire harness image data to obtain wire harness geometric calibration data;

[0134] In one embodiment, based on the collected wire beam image, the image is geometrically calibrated to correct the distortion caused by the viewing angle of the camera device. By identifying the position offset of the reference wire beam point in the image, a coordinate transformation is performed to ensure that the actual position of each part of the wire beam is consistent with the image position. Geometric calibration data, such as a calibration matrix and coordinate offset values, are generated.

[0135] Step S17: performing edge detection according to the wire harness geometry calibration data to obtain wire harness edge data;

[0136] In one embodiment, the geometric calibration data is used to perform edge detection on the wire bundle in the image to extract the outline of the wire bundle. The edge position is detected by analyzing the pixel brightness difference. If a continuous brightness change is detected within a range of 200-400 pixels, this represents the edge of the wire bundle, and edge data is generated.

[0137] Step S18: performing region segmentation and marking according to the wire harness edge data to obtain wire harness region segmentation data;

[0138] In one embodiment, edge data is used to separate the portion of the image belonging to the wire bundle from the background area, and the wire bundle area is marked. The coordinate range of the wire bundle area is identified and the wire bundle mark is set. For example, the area in the range of (200, 300)-(400, 500) is marked as the wire bundle part in the image, and the area segmentation data is generated.

[0139] Step S19: reconstruct smooth details according to the wire harness region segmentation data to obtain wire harness image segmentation data.

[0140] In one embodiment, the region segmentation data is smoothed to eliminate small noise and detail loss in edge segmentation. The edges in the segmented image are smoothed to remove overly sharp or discontinuous edges, and smoothed line image segmentation data is generated to ensure that the line boundaries are clear and continuous.

[0141] Preferably, the smooth detail reconstruction is specifically:

[0142] Perform multi-resolution analysis based on the wire harness area segmentation data to obtain wire harness multi-resolution level data;

[0143] In one embodiment, a multi-resolution representation of a wire harness image is constructed, and multiple hierarchical images are generated by gradually reducing the resolution. For example, the original image is reduced to different scales such as 1 / 2, 1 / 4, and 1 / 8, and each hierarchical image captures the details of the wire harness at different resolutions.

[0144] Perform edge enhancement according to the multi-resolution level data of the wire bundle to obtain multi-level edge enhancement data of the wire bundle;

[0145] In one embodiment, at each resolution level, based on the image edge information, the edge of the line bundle is enhanced to highlight the structural features. For example, the edge pixel contrast at different scales is improved to highlight the outline of the line bundle, so that the edge of the line bundle is clearer at each level, and multi-level edge enhancement data is obtained, including each resolution level image after enhancement, which is recorded as multi-level edge enhancement = enhanced multi-resolution image.

[0146] Performing texture restoration on the wire harness multi-level edge enhancement data to obtain the wire harness multi-level texture restoration data;

[0147] In one embodiment, multi-resolution analysis is first performed on the multi-level edge enhancement data of the line beam. The enhanced data is divided into different resolution levels (such as low, medium and high resolution). At each resolution level, different texture features (such as roughness and smoothness) are extracted. The distribution and direction of the texture features are analyzed using methods based on gray level co-occurrence matrix (GLCM), local binary pattern (LBP) and the like. For edge areas, texture details near the edge are extracted to maintain a natural transition between the edge and the texture. For example, for low-frequency areas, smoothness is maintained, while tiny texture details are identified in high-frequency areas. In each level, the subtle texture lost due to the edge enhancement process is restored by an interpolation method. Bilinear interpolation or wavelet transform can be used to gradually restore the texture details of the high-frequency and medium-frequency levels, so that they are combined with the original edge enhancement effect to generate a texture with distinct detail levels. For the medium-frequency level, a relatively fine edge texture is added to the texture of the low-frequency level to maintain the continuity of the transition. After completing the texture restoration of each level, the restored data of different resolution levels are superimposed step by step. First, the overall texture of the low-resolution layer is merged as the base layer; then the local details of the mid-frequency layer are superimposed on it, and higher-resolution texture information is gradually added. Each layer of texture information is synthesized according to the weight of its resolution layer, ensuring that the high-level texture details do not affect the overall texture effect of the low-frequency.

[0148] Perform super-resolution restoration based on the multi-level texture restoration data of the wire harness to obtain super-resolution data of the wire harness;

[0149] In one embodiment, first, the multi-level texture restoration data of the wire beam is input into the super-resolution model, and the texture details, edge information and layering in the image are preliminarily analyzed. A convolutional neural network (CNN) can be used for feature extraction, and the edge and detail features in the wire beam texture restoration data are extracted through a series of convolution layers. The initial feature extraction layer strengthens the different detail areas of the image through a small convolution kernel (for example, 3x3 or 5x5) to highlight important features in the image, such as contours and texture changes. After feature extraction, the image is enhanced in resolution through an upsampling layer. Bilinear interpolation or deconvolution layers can be used to enlarge the resolution of the image by 1 to 4 times while maintaining the smoothness of the edges and the fineness of the texture. In the layer-by-layer reconstruction process, the low-frequency level is responsible for the overall contour of the image, ensuring clarity on a large scale, while the medium-frequency level further refines the texture features. The high-frequency level processes detail textures and high-contrast areas to ensure that the enlarged image retains tiny details. In the super-resolution enlarged image, detail compensation is performed on the edge area to ensure detail clarity in different areas. Methods based on wavelet transform or Fourier analysis can be applied to perform texture interpolation compensation on high-frequency areas to maintain the clarity of edge contours while restoring detailed textures. In flat areas of the image, smooth interpolation is used to reduce high-frequency noise so that textures and edges transition smoothly. The enhanced features of each level are gradually superimposed to form a complete super-resolution image. The low-frequency, medium-frequency and high-frequency level features are fused separately to ensure the natural and delicate texture restoration effect. During the fusion process, the high-frequency level features are moderately smoothed to prevent noise caused by excessive high-frequency information. The layering and contrast of the image are enhanced in the low-frequency layer to highlight the overall shape.

[0150] Performing structure-preserving mapping on the wire bundle super-resolution data to obtain wire bundle structure mapping data;

[0151] In one embodiment, based on the super-resolution data, the global structural features of the line bundle are first analyzed, including the distribution of the main contours, edges and texture areas. The main edge contours of the line bundle are extracted by edge detection (such as Canny edge detection) to identify the structurally prominent areas. These initially extracted edges will be used as the focus of subsequent structure preservation to ensure that the key contours are not lost during the magnification and mapping process. By dividing the super-resolution data into multiple scale levels, the edges and texture details of different levels are extracted in layers. For example, the overall morphology of the line bundle is extracted in the low-scale layer, the local structural details are extracted in the middle-scale layer, and the fine edge texture is retained in the high-scale layer. In this way, the hierarchical division can ensure that the overall structure and subtle features of the image can be reflected in the structural mapping. Edge preservation technology is applied in structure-preserving mapping to ensure that the sharpness of the edge is not lost when mapping to different resolutions or scales. An adaptive edge-preserving filter or bilateral filtering can be used to process the edge area and the non-edge area separately. The edge area maintains high sharpness, while the non-edge area is moderately smoothed to ensure that the edges of the entire image are clear and the texture is delicate. This can prevent the blurring and distortion of the edges during the mapping process. By analyzing the texture and edge features in the super-resolution data, a mapping relationship from the super-resolution image to the structure-preserving mapping image is established. Specifically, a set of structure-preserving weight matrices are generated by calculating the relative position of each pixel and its edge similarity with the neighborhood. These weight matrices are used to control the structural integrity of the image during the mapping process, so that when the resolution of the image changes, the key structure can be given a higher weight, while the less important parts can be weakened. The super-resolution data is mapped according to the structure-preserving weight matrix to generate the wire bundle structure mapping data. During the mapping process, in order to further enhance the details of the key areas, detail enhancement algorithms such as Laplace sharpening or adaptive sharpening algorithms can be applied to the edge areas of the important structures. This step ensures that in the data after structural mapping, not only the overall morphology is clear, but also the local details are enriched. After the mapping is completed, a global structural consistency check is performed to ensure that the morphology of the wire bundle is not distorted or deformed after mapping. By comparing the key point positions in the original super-resolution data and the mapped data, it is confirmed that no morphological shift occurs during the mapping process. If it is found that the mapping of a local area deviates too much from the original structure, the backtracking algorithm can be used to perform secondary processing on the area to ensure the accuracy and consistency of the mapped data.

[0152] Performing feature smoothing processing on the wire harness structure mapping data to obtain wire harness feature smoothing data;

[0153] In one embodiment, the structure mapping data is smoothed to eliminate local noise and discontinuity to ensure visual coherence of the image. Smoothing technology is used to reduce small noise in the image caused by edge enhancement and other operations to improve the overall visual effect.

[0154] Perform local detail alignment based on the wire harness feature smoothing data to obtain multi-level detail alignment data of the wire harness;

[0155] In one embodiment, local alignment is performed on the smoothed image data to ensure the coherence of details at different resolutions. Local position alignment is performed on multi-level images to make the details at different resolution levels consistent with each other.

[0156] The wire harness multi-level detail alignment data is reconstructed and segmented and outputted to obtain wire harness image segmentation data.

[0157] In one embodiment, a smooth, noise-free, and structurally coherent segmentation result is generated based on the aligned multi-level detail images, and the segmentation data of each level is reconstructed into a single line image segmentation data to ensure clear edges and textures.

[0158] Preferably, the feature smoothing process is specifically:

[0159] Performing multi-scale similar block search on the wire harness structure mapping data to obtain multi-scale similar block data of the wire harness;

[0160] In one embodiment, in the wire structure mapping data, a multi-scale block search is performed on each pixel to find a region block with similar texture or structure in the image. For example, neighborhood blocks of different sizes such as 3x3, 5x5, 7x7 are selected to search for image regions similar to the central pixel block at different scales. The structure mapping data is divided into multiple scales (such as 3x3, 5x5, 7x7, etc.), and all local regions in the image are scanned layer by layer. For each specific scale, a set of similar block data is generated by searching for block regions with similar features to the target block. For example, at a scale of 3x3, the pixel intensity, texture pattern and local gradient of each block are compared, and blocks similar to the target area are marked.

[0161] A lightweight convolutional network similarity weight calculation is performed based on the multi-scale similarity block data of the wire bundle to obtain the wire bundle similarity weight data;

[0162] In one embodiment, the similar block data of each scale is input into a lightweight convolutional network. The convolutional network consists of several shallow convolutional layers and a fully connected layer. The convolutional layer is responsible for extracting the key features of similar blocks, and the fully connected layer outputs similarity weights based on the results of feature extraction. Through this process, the network can learn the similarities of different regions and give higher weights to blocks with higher similarity. The similarity weight value is between 0 and 1, and the higher the value, the greater the similarity.

[0163] The wire harness structure mapping data is weighted averaged according to the wire harness similarity weight data to obtain the wire harness similarity weighted data;

[0164] In one embodiment, a weighted mean calculation is performed on each pixel block in the structure mapping data, and the information of similar regions is fused to achieve smoothing. By weighted mean, the grayscale or color values ​​of adjacent pixels with high similarity are superimposed to obtain a smoother pixel value. According to the similarity weight, the pixel values ​​of multiple similar blocks are weighted averaged to obtain a smoother pixel value. The pixel values ​​of all similar blocks corresponding to the target pixel position are taken out, and then these values ​​are weighted summed according to the similarity weight and the mean is calculated.

[0165] Perform pixel update according to the wire bundle similarity weighted data to obtain wire bundle pixel update data;

[0166] In one embodiment, after the weighted mean calculation, the pixel update is then performed to generate a smoother and more continuous image. The pixel update process replaces the value of the corresponding pixel in the original image with the weighted mean value based on the result of the weighted mean. In this way, noise and mutations in the original image are eliminated while maintaining the continuity and consistency of the region. For example, in areas with large texture changes, the updated pixel values ​​will present a more natural transition effect, thereby improving the visual quality of the image.

[0167] An edge preservation check is performed based on the wire harness pixel update data to obtain the wire harness feature smoothing data.

[0168] In one embodiment, an edge preservation check is performed on the image after pixel update to ensure that the edge features are not lost during the smoothing process. The edge area of ​​the image is checked, and if the edge is blurred, appropriate adjustments are made to maintain the clarity of the edge. The specific steps are edge detection and edge restoration. First, edge detection is performed on the pixel update data through an edge detection algorithm (such as Canny edge detection) to mark the main edge area in the image. Then, the detected edges are compared with the edges in the original image to restore the edges weakened by the smoothing operation. Specifically, the high-frequency information of the pixels in the edge area is restored to make the outline of the edge part clearer, and the generated line feature smoothing data contains both smoothed image features and retains clear edges.

[0169] Preferably, the morphological feature extraction is specifically:

[0170] Step S21: capturing the area contour according to the wire harness image segmentation data to obtain the wire harness area contour data;

[0171] In one embodiment, based on the wire harness image segmentation data, the outer contour of the wire harness is identified and extracted to form a continuous edge, for example, by scanning pixels at the edge of the segmented area and forming a complete wire harness contour path through coordinate recording.

[0172] Step S22: reconstructing the contour according to the wire harness area contour data to obtain wire harness contour reconstruction data;

[0173] In one embodiment, the complete wire harness contour is reconstructed based on the contour data to ensure that there are no breakpoints on the edge and the contour is smooth. The broken edge points are reconnected and smoothed to form a continuous and clear contour path.

[0174] Step S23: performing coupled convolution feature extraction and interpreted convolution feature extraction on the wire beam contour reconstruction data to obtain first wire beam image feature data and second wire beam image feature data respectively;

[0175] In one embodiment, local and global line beam morphological features are extracted from the contour reconstruction data to capture the overall shape and detail features of the contour, and the contour feature information is extracted layer by layer to form the first line beam image feature data, which represents the comprehensive information of the local and global contours. The contour data is further feature decomposed to capture the texture and edge features of the local details, and more detailed line beam contour information is extracted to form the second line beam image feature data, which mainly includes the microscopic features of the edge.

[0176] Step S24: performing wire harness shape mapping on the first wire harness image feature data and the second wire harness image feature data to obtain first wire harness shape mapping data and second wire harness shape mapping data;

[0177] In one embodiment, the first wire harness image feature data is converted into a two-dimensional shape map to reflect the overall shape of the wire harness, for example, the coupling feature data is mapped to a two-dimensional map to form a basic shape structure map of the wire harness. The second wire harness image feature data is converted into a more detailed shape map to highlight local texture and edge changes, and the interpretation feature data is mapped to a detail layer to form a microscopic feature map of the contour.

[0178] Step S25: performing detail structure difference mapping according to the first wire harness shape mapping data and the second wire harness shape mapping data to obtain wire harness shape detail structure difference data;

[0179] In one embodiment, the first and second shape mapping data are compared to find the difference in shape details between the local and the whole, and to identify inconsistent feature points in the overlapping area to reflect the difference between the overall contour and the local texture.

[0180] Step S26: generating a shape feature map according to the wire harness morphology detail structure difference data to obtain wire harness morphology feature data;

[0181] In one embodiment, based on the structural difference data, a wire harness feature map with rich morphological information is generated to clearly reflect the structure and morphological characteristics of the wire harness. The difference data and the mapping data are combined to generate a shape feature map with global and detailed information to obtain the wire harness morphological feature data.

[0182] The coupled convolution feature extraction is specifically as follows:

[0183] Performing initial convolution layer processing on the wire harness contour reconstruction data to obtain wire harness initial convolution layer data;

[0184] In one embodiment, convolution processing is performed on the wire bundle contour reconstruction data to extract basic feature information, including contour edges and local structures, and generate the wire bundle initial convolution layer data. The initial convolution layer data captures the basic geometric contour and texture features of the wire bundle to form a basic feature map.

[0185] Separate the coupled feature channels of the initial convolutional layer data of the wire harness to obtain the wire harness geometric structure feature data and the wire harness texture feature data respectively;

[0186] In one embodiment, the initial convolutional layer data is divided into two different feature channels, one for geometric structure features and the other for texture features, to obtain two types of feature data: line bundle geometric structure feature data and line bundle texture feature data. The geometric structure channel focuses on edge and shape information, and the texture channel captures detailed texture and surface features.

[0187] Perform residual connection convolution layer processing on the wire harness geometric structure feature data to obtain high-order feature data of the wire harness geometric structure, and perform dense connection convolution layer processing on the wire harness texture feature data to obtain high-order feature data of the wire harness texture;

[0188] In one embodiment, in the geometric structure feature channel, a residual connection layer is used to further extract high-order geometric information of the contour, retain the basic features while capturing complex edge changes, and generate high-order feature data of the geometric structure of the wire harness, which includes the complex shape information and structural changes of the wire harness contour. The core of the residual connection layer is to construct a residual block, each of which contains two or more convolutional layers and a jump connection. The specific residual block structure is as follows: Convolution layer 1: Use a 3x3 convolution layer with a step size of 1, and the number of convolution kernels is set according to demand (for example, 64). This convolution layer extracts preliminary information of geometric features. Batch normalization and activation: After the convolution operation, batch normalization is performed to stabilize network training, and nonlinearity is increased through the ReLU activation function so that the model can learn complex geometric features. Convolution layer 2: Followed by a 3x3 convolution layer with a step size of 1 and the same number of convolution kernels as convolution layer 1. The second layer of convolution further extracts high-order geometric information, such as more complex contour changes. Jump connection: Add the input of the residual block directly to the output of convolution layer 2 to form a "residual connection". The input feature is x, and the feature after convolution is F(x). Then the output of the residual connection is y=F(x)+x. This connection operation can avoid information loss and ensure that the initial feature is retained in the output. In the texture feature channel, a dense connection layer is used to obtain high-order texture features of surface details, generate high-order feature data of wire texture, and record the detailed surface texture and small-scale texture changes. The dense connection block contains multiple convolution layers. The output of each convolution layer is not only passed to the next convolution layer, but also connected to the input of all subsequent convolution layers. This structure allows the network to capture complex texture details more effectively. The structure of the dense connection block is as follows: Convolution layer 1: Use a 3x3 convolution layer to extract the basic texture features of the surface. The output of this layer is directly passed to the next convolution layer and is also connected to all subsequent convolution layers. Convolution layer 2: Based on convolution layer 1, a second 3x3 convolution layer is connected, and the output and input of convolution layer 1 are connected to this layer to form a "dense connection". Through this connection, convolutional layer 2 not only uses its own input, but also obtains the texture feature information of the first layer, allowing it to extract small-scale texture changes at a higher level. Convolutional layer N: A similar structure continues to layer N. The input of layer N will contain all the outputs of the previous N-1N-1N-1 layers, ensuring that the high-order texture features of each layer are superimposed on the final output, thereby generating a detailed, high-order texture map composed of multiple layers of features.

[0189] Performing feature fusion on the high-order feature data of the wire harness geometry structure and the high-order feature data of the wire harness texture to obtain coupled fusion feature map data;

[0190] In one embodiment, the geometric structure high-order features and the texture high-order features are fused to comprehensively present the shape and details of the wire bundle, generate coupled fusion feature map data, and form a comprehensive feature map including geometric and texture features.

[0191] The coupled fusion feature map data will be subjected to local receptive field enhancement to obtain coupled local enhanced feature map data;

[0192] In one embodiment, in a convolutional neural network, the receptive field defines which pixels in the input image a neuron in the network has an effect on or responds to. For example, the size of a neuron's receptive field determines how wide a range of image features it can capture: a small receptive field can only capture subtle local details, while a large receptive field can integrate a wider range of contextual information. Local receptive field enhancement expands and optimizes the network's receptive field so that it can capture richer detail information and local features when processing images, while maintaining the global features of the image, while enhancing the recognition and expression of detail areas. Through an attention mechanism or other weight calculation method, the response weights of the pixels in each feature map (coupled fusion feature map data) are calculated to determine the importance of the details of each local area, and a set of response weight matrices are obtained. Each feature map (coupled fusion feature map data) is weighted and merged according to the calculated weights. The eigenvalues ​​at each pixel position are adjusted according to the weight matrix to form an enhanced feature map, i.e., coupled local enhanced feature map data. In the coupled fusion feature map, the local perception ability of the specific area is enhanced to better capture the local details of the wiring harness, and the coupled local enhanced feature map data is obtained, which makes the fusion feature more responsive in the detail area and further enhances the recognition of specific details.

[0193] Performing feature compression on the coupled local enhanced feature map data to obtain first line beam image feature data;

[0194] In one embodiment, each feature channel is globally pooled, for example, using global average pooling or global maximum pooling. Global pooling aggregates all feature values ​​of each channel into a single value, representing the main features of the channel. Taking global average pooling as an example, the pixel values ​​of each channel are averaged to generate a low-dimensional feature vector. After global pooling, if the feature dimension is still high, the dimension is further reduced by a 1x1 convolution layer before tightening. The 1x1 convolution kernel can merge multiple feature channels and reduce the number of output features, thereby obtaining a more streamlined feature map. In actual operation, the number of output channels of the 1x1 convolution can be set as needed to control the dimension of the final feature data. For example, if the number of channels of the original feature map is 128, a 1x1 convolution layer with a channel number of 64 can be used to reduce it to 64 channels.

[0195] The explanation of convolution feature extraction is as follows:

[0196] Calculate the confidence of the wire harness contour reconstruction data according to the coupled fusion feature map data to obtain the wire harness area confidence data;

[0197] In one embodiment, in the initial stage of interpreting the convolution feature extraction, it is first necessary to perform confidence calculations on different regions of the wire profile reconstruction data based on the coupled fusion feature map data. The purpose of the confidence calculation is to evaluate the feature reliability of each region to determine which regions should be analyzed in detail. Specifically, a convolution calculation is performed on each region to capture specific edge or texture feature patterns, and the confidence is assigned according to the intensity of these feature patterns in different regions. For example, regions with higher edge intensity can be assigned higher confidence, while flat regions have lower confidence.

[0198] Performing feature selective convolution layer processing on the wire harness contour reconstruction data according to the wire harness region confidence data to obtain wire harness selective convolution layer data;

[0199] In one embodiment, a deeper convolutional network is applied to a high-confidence area to capture the detailed features of the area, while a shallower convolutional network is applied to a low-confidence area to reduce the amount of calculation. The model efficiency is effectively improved without affecting the overall feature accuracy. Through selective convolution processing, the line bundle selective convolution layer data is obtained, which contains richer high-confidence area features and provides detailed local texture and morphological information.

[0200] Perform feature comparison on the wire bundle selective convolution layer data and the coupled fusion feature map data to obtain wire bundle comparison feature data;

[0201] In one embodiment, after the selective convolution layer data is generated, feature comparison is then performed on the line beam selective convolution layer data and the coupled fusion feature map data. The purpose of feature comparison is to identify abnormal areas that deviate from standard features through the difference between the two. Specifically, for the feature data at the same pixel position, the difference value between the selective convolution layer data and the fusion feature map data is calculated, and the area with large difference is marked as abnormal. During the comparison process, calculation methods based on pixel difference, feature similarity, etc. can be used to quantify the deviation value of each pixel.

[0202] Performing a significance analysis on the wire harness contrast feature data to obtain the wire harness significance feature data;

[0203] In one embodiment, the difference value distribution in the feature data is calculated and compared, and the significance of the area with higher difference value is enhanced. Usually, through pixel-by-pixel detection, the area with difference value exceeding a certain threshold is marked as a significant feature, and a higher weight is given to the area. In addition, in order to ensure the continuity of the significant feature, the significant area can be smoothed and connected to form a more complete significant feature area.

[0204] Perform high-level structural aggregation on the wire harness significant feature data to obtain wire harness structural aggregation data;

[0205] In one embodiment, important regions in the salient feature data are spatially aggregated, and adjacent salient feature regions are connected into an overall structure. For example, a method based on region growing or graph aggregation is used to connect edges or important features to form a structural feature map with more continuity and integrity. Through this aggregation process, the generated wire structure aggregation data not only contains the specific location of each salient feature, but also forms a complete feature framework with spatial consistency, providing a basis for self-attention calculation.

[0206] Perform explanatory self-attention map calculation on the bundle salient feature data and the bundle structure aggregation data to obtain the bundle weighted explanatory feature map data;

[0207] In one embodiment, after high-level structure aggregation, important feature areas are further strengthened through explanatory self-attention map calculation. Self-attention map calculation is based on significant feature data and structural aggregation data, and gives higher explanatory weights to key areas in the form of attention weights. In specific operations, feature data is input into the self-attention model, the contribution of each area to the overall structure is calculated, and higher weights are given to high-contribution areas. Through this weight distribution, the bundle-weighted explanatory feature map data is obtained, so that areas with important features are more prominent in the image, providing intuitive explanatory information for morphological mapping.

[0208] Global morphological mapping is performed according to the line bundle weighted explanatory feature map data to obtain second line bundle image feature data.

[0209] In one embodiment, after the self-attention map is calculated, the wire harness weighted explanatory feature map data is used for global morphological mapping to generate an image representation containing complete features. Global morphological mapping forms a global mapping map containing each key feature by applying the self-attention weight to the feature distribution of the original image. First, the weighted explanatory feature map is fused with the basic feature map to make the key area more prominent, and then a variety of visual effects such as color and depth are applied to enhance the mapping effect to ensure that each significant feature is clearly presented in the image. The output second wire harness image feature data records the global morphology and key features of the wire harness, forming an integrated feature map with clear visual effects and prominent features, which provides detailed image support for further defect detection and analysis.

[0210] Preferably, the three-dimensional reconstruction is specifically:

[0211] Step S27: performing morphological classification according to the wire harness morphological feature data to obtain wire harness morphological classification data;

[0212] In one embodiment, at the initial stage of three-dimensional reconstruction, morphological classification based on the morphological feature data of the wire beam is key. First, various types of feature information in the wire beam image are collected, including edge features, texture features, and surface features. The extraction of these features is achieved through basic convolution operations to generate a feature map. After the feature map is extracted, it is input into the clustering model, and the image features are classified by an unsupervised learning method (e.g., based on K-means or density clustering). Specifically, edge features are manifested as areas with significant grayscale changes in the image, while texture features are obtained by analyzing small grayscale changes between pixels. By using methods such as gray level co-occurrence matrix (GLCM) or local binary pattern (LBP), the changes in the surface and detail texture of the wire beam are quantified. After each feature is classified, it is further marked in the feature map, such as marking each feature area with a color or label, so that the classification of edges, surfaces, and textures is clear and discernible.

[0213] Step S28: performing feature map fusion according to the wire harness morphology classification data to obtain wire harness morphology feature map fusion data;

[0214] In one embodiment, feature map fusion is to integrate each type of morphological features into a single feature map, thereby retaining all features in one view. Feature maps of different categories are superimposed together. When superimposing, different weights need to be assigned to each type of feature. For example, edge features will be assigned higher weights because they play a major role in shape, while texture features will be assigned lower weights. To achieve this process, different feature maps are synthesized into a complete morphological feature map using methods such as weighted averaging or feature splicing. In specific operations, weighted convolution or layer overlay technology is used to ensure that the features of each layer can be clearly presented. Multi-scale information can also be combined in the feature fusion process. For example, edge features can be more clearly displayed at low scales, while fine texture features are more obvious at high scales. Therefore, different convolution kernels can be applied to feature maps of different scales to ensure that multi-scale information is still complete after fusion. The fused feature map is a complete feature map of the line bundle, which provides an image containing multiple types of feature information for depth estimation and perspective matching.

[0215] Step S29: performing depth estimation on the wire harness morphological feature map fusion data to obtain wire harness feature depth data;

[0216] In one embodiment, the depth information of each feature point in the image relative to the observation angle is inferred based on the fused feature map data. Typically, depth estimation is achieved through parallax-based analysis. First, multi-view image data (such as left and right views) is acquired, and the images of the same object from different views are aligned. After the alignment is completed, the depth of each point can be calculated by comparing the displacement difference (parallax) of the same feature point in each view. Combined with the "stereo matching" method in computer vision, the displacement of each point in the image at different viewpoints is determined. In order to calculate an accurate depth map, a stereo matching algorithm is applied to the fused feature map to ensure that different types of features (such as edges and textures) are involved in the depth estimation. When calculating the depth, a method based on pixel grayscale differences is used to match the depth information of the feature points.

[0217] Step S210: performing perspective matching according to the line bundle feature depth data to obtain line bundle perspective matching data;

[0218] In one embodiment, perspective matching is to pair feature points under different perspectives to generate three-dimensional coordinates. When performing perspective matching, the position and depth information of each feature point in the multi-perspective image must first be identified. Next, the images of each perspective are mapped to the same coordinate system through spatial mapping based on the intrinsic and extrinsic parameters of the camera, thereby ensuring the coordinate consistency between the multi-perspective images. Then, the feature point matching algorithm is applied to find the position of the same feature point in different perspectives by matching pixel features or descriptors to ensure that the same feature point can be matched in the multi-perspective image. Mismatching is avoided by setting a reasonable matching threshold. In addition, in order to reduce matching errors, matching point pairs can be screened, for example, using the random sampling consensus (RANSAC) method to remove abnormal matching points, thereby ensuring the accuracy of perspective matching. The perspective matching data will eventually record the three-dimensional position of each matching point, so that each matching point pair determines a position in the three-dimensional space, providing accurate coordinates for the generation of three-dimensional point clouds.

[0219] Step S211: generating a three-dimensional point cloud according to the line bundle view matching data to obtain line bundle point cloud data;

[0220] In one embodiment, a three-dimensional point cloud is generated by triangulation based on the perspective matching data. First, the three-dimensional coordinates of each feature point are determined. Using the known perspective matching data and parallax information, the coordinates in the two-dimensional image are converted into three-dimensional space coordinates through the triangulation formula. At this time, it is necessary to ensure the accuracy of the perspective matching, because the coordinate error of the perspective matching point will directly affect the generation accuracy of the point cloud. In the process of generating the point cloud, the three-dimensional coordinates of all feature points are collected into a point cloud data structure. This point cloud image contains information such as the external contour and surface details of the harness.

[0221] Step S212: Perform three-dimensional grid reconstruction based on the wire harness point cloud data to obtain three-dimensional reconstruction data of the wire harness.

[0222] In one embodiment, meshing reconstruction is to convert the generated point cloud data into a complete three-dimensional surface model. In the meshing process, the point cloud data is first connected to generate a continuous triangular face network. The neighbor connection algorithm or the Delaunay triangulation algorithm is used to connect adjacent points to form a mesh containing facets. When generating a triangular mesh, ensure that each point is only connected to the points in its neighborhood to ensure the smoothness of the mesh. After the mesh connection is completed, the mesh model needs to be optimized, such as applying a smoothing filter to remove sharp edges or discontinuous surfaces between polygons. In actual processing, Laplace smoothing and other technologies can be used to make the model surface more natural. In addition, by adjusting the face size and face distribution density, the detail expression ability of the model can be improved. The generated three-dimensional reconstruction data contains complete mesh information and coordinate records, forming a three-dimensional wire harness model with a coherent surface and clear structure.

[0223] Preferably, the microstructure defect detection is specifically:

[0224] Step S31: dividing the wire harness morphology feature data into feature blocks to obtain the wire harness morphology local feature data;

[0225] In one embodiment, after the morphological features of the wire harness are preliminarily identified, the image is divided into feature blocks according to the morphological feature data of the wire harness, and the wire harness image is divided into several local areas to facilitate the analysis of defects at a fine scale. The first step of feature block division is to preliminarily locate the morphological features of the wire harness (such as edges, surface textures, and corners). For each area, block division is performed based on similar morphological features or textures. For example, the surface area of ​​the entire wire harness can be divided into several local blocks based on the texture and grayscale changes of the image, so as to detect the fine structural features of each block separately. The block division process is completed by grid or regional growth, where grid division ensures the consistency of regional division, and regional growth helps to better adapt to complex shapes. The obtained local feature data of the wire harness morphology records the feature information of each block, providing data support for candidate defect detection.

[0226] Step S32: identifying defect candidate regions according to the wire harness morphology local feature data to obtain wire harness defect candidate region data;

[0227] In one embodiment, after the feature blocks are divided, defect candidate areas are identified based on the feature information of each block. Defect candidate area identification is usually based on the abnormal performance of local features, such as abnormal morphological changes, prominent texture differences, or abnormal grayscale gradients. In a specific implementation, the standard texture features and grayscale values ​​of each feature block are first calculated and compared with the overall standard reference data (i.e., conventional defect-free wire harness feature data). If an area shows significant differences in texture or morphology, it can be preliminarily identified as a defect candidate area. The output of this step is the wire harness defect candidate area data, which records the coordinates and basic information of each defective area.

[0228] Step S33: performing texture feature matching according to the wire harness defect candidate area data and the preset conventional wire harness area data to obtain wire harness texture abnormality data;

[0229] In one embodiment, after the defect candidate area is determined, the next step is to match the texture features of the candidate area with the standard wire harness area. The core of texture feature matching is to compare the texture difference between the candidate area and the normal area. First, the texture features (such as grayscale value, texture direction, texture thickness, etc.) of the defect candidate area are matched with the preset standard area. Specifically, texture descriptors (such as LBP, GLCM, etc.) can be used to extract the texture features of the candidate area, and then the similarity is calculated with the features of the standard area. If the similarity is low, it means that there is abnormal texture in the area. Obtain the abnormal data of the wire harness texture, record the texture difference features of each abnormal area, and provide a basis for further deformation analysis.

[0230] Step S34: performing local deformation similarity calculation and relative deformation measurement processing according to the abnormal data of wire harness texture, and distributing and obtaining wire harness local deformation similarity data and wire harness relative deformation measurement data;

[0231] In one embodiment, for the identified texture abnormal area, the degree of morphological change of the abnormal area is further analyzed by deformation similarity calculation and relative deformation measurement. First, the local deformation similarity is calculated to determine the similarity between the deformation of the abnormal area and the normal area. The key points of the candidate area and the standard area can be extracted by a key point-based matching algorithm, and their similarity can be calculated. Next, a relative deformation measurement is performed to quantify the actual magnitude of the deformation and calculate the amount of change in size, shape or direction of the candidate area. For example, changes in the area, perimeter, etc. of the candidate area can be reflected in the relative deformation measurement. The output wire harness local deformation similarity data and wire harness relative deformation measurement data will provide accurate deformation parameters for further analysis of defects.

[0232] Step S35: performing morphological comparison according to the wire harness local deformation similarity data and the wire harness relative deformation measurement data to obtain wire harness morphological comparison data;

[0233] In one embodiment, after obtaining the local deformation similarity and relative deformation data, the abnormal area is morphologically compared to further confirm the significant features of the candidate defect. By comparing the morphological features (such as contour, area, texture distribution, etc.) of the candidate area with the normal area, the change in the morphological features of the area can be identified. In specific implementation, by comparing the edge contour shape, texture density, and grayscale value of the candidate area and the standard area, the difference between the two is quantified. For example, in the morphological comparison, the similarity of the edge contour and the change in texture density will be recorded to form the wire harness morphological comparison data. The comparison data will help to classify the defects in the next step.

[0234] Step S36: Defect classification and confidence estimation are performed based on the wire harness morphology comparison data to obtain first wire harness defect detection data.

[0235] In one embodiment, based on the morphological comparison data, candidate defects are classified and the confidence of the defects is estimated. Defect classification is implemented based on the analysis results of the comparison data to identify different defect types (such as cracks, dents, or wear). During the classification process, the defect features can be discriminated by a machine learning model or rule set. For example, if the edge contour is discontinuous and the similarity is low, it is a crack; if the density of the local area increases, it is a dent, etc. Then, the reliability of the defect is estimated by calculating the confidence of the classification result. The confidence calculation is performed based on the significance of the defect type (such as deformation amplitude, area, comparison feature difference, etc.), so as to assign a confidence value to each defect, and obtain the first harness defect detection data, including defect type, location and confidence.

[0236] Preferably, the micro-morphological defect detection is specifically:

[0237] Step S37: extracting surface features from the wire harness 3D reconstruction data to obtain wire harness surface feature data;

[0238] In one embodiment, surface geometry information is first extracted from the three-dimensional reconstruction data, including surface normal vectors, curvature distribution, and surface normal direction. By calculating the curvature change of each mesh patch, relatively flat areas, protruding edges, or sharp corners can be identified. Surface roughness can be estimated by calculating the height change between adjacent points, and a larger change usually indicates a rougher surface. During the extraction process, a local curvature estimation and roughness calculation method based on differential is used to ensure the accuracy of the feature data, and the output is the wire harness surface feature data, which provides the necessary feature description for local area division.

[0239] Step S38: performing local area division and mesh refinement on the wire harness surface feature data to obtain local refined mesh data;

[0240] In one embodiment, after the surface feature data is obtained, it is divided into local regions, and mesh refinement is performed in each region. The specific operation includes dividing the surface feature data into several regions based on the changes in curvature and roughness. When dividing the regions, the regional growing method or the adaptive mesh refinement method can be used so that the regions with large changes in curvature are refined. For each local region, mesh refinement is further performed to improve the resolution, for example, the facets in the region are divided into smaller triangular facets so as to present the local details more clearly. During the mesh refinement process, it is necessary to ensure that the generated facets can increase the surface details while maintaining the original shape, so that the local tiny defects can be clearly identified in the subsequent morphological anomaly detection. The obtained local refined mesh data records the coordinates and features of each refined region, providing a high-resolution mesh structure for the detection of subsequent steps.

[0241] Step S39: performing morphological anomaly detection according to the local refined grid data to obtain wire harness morphological anomaly data;

[0242] In one embodiment, based on the local refined mesh data, abnormal features on the structure are identified through morphological anomaly detection. The specific implementation includes detecting those areas on the refined mesh that are different from the normal mesh patches, which appear as irregular protrusions, depressions or non-smooth areas. The local curvature-based anomaly detection method can be used to calculate the curvature value of each mesh patch and compare it with the curvature of the adjacent patches. If the curvature of a certain area changes abnormally, it is marked as a morphologically abnormal area. In addition, the shape mutation points can be identified by calculating the angle or normal vector difference between adjacent patches, and the output is the line bundle morphological anomaly data, which records the location and characteristics of all abnormal areas and provides a basis for structural integrity analysis.

[0243] Step S310: performing structural integrity analysis based on the wire harness 3D reconstruction data and the wire harness morphology abnormality data to obtain wire harness structural integrity data;

[0244] In one embodiment, after obtaining the morphological anomaly data, the overall structure of the harness is evaluated for defects through structural integrity analysis. Specifically, the morphological anomaly data is comprehensively analyzed with the three-dimensional reconstruction data to check whether there are large structural defects. For example, for long harnesses or complex three-dimensional structures, the impact on the overall structure can be determined by calculating the distribution density, area and shape of the abnormal area. In addition, the stability and integrity of the structure can be determined based on the size, number and position of the abnormal area. Through neighborhood analysis and continuity checks, it is verified whether the harness structure remains coherent in the abnormal area, or whether there are serious fractures, deformations and other problems. The output harness structure integrity data contains an assessment of the impact of each abnormal area on the overall structure, providing more detailed data support for texture detection and defect classification.

[0245] Step S311: performing fine texture detection according to the abnormal data of the wire harness morphology to obtain abnormal data of the wire harness fine texture;

[0246] In one embodiment, after the structural integrity analysis, fine texture detection is performed on the abnormal area to identify more subtle surface defects (such as cracks, tiny protrusions, etc.). The core of fine texture detection is to analyze the texture features of the abnormal area, which is completed through high-resolution texture analysis methods. For example, through texture detection techniques such as gray level co-occurrence matrix (GLCM) or local binary pattern (LBP), the texture features of the abnormal area are extracted and compared with the texture features of the normal area. If there is a significant difference between the area with abnormal texture and the normal area, it is marked as a fine texture abnormal area. In addition, in order to more accurately identify texture anomalies, it is also possible to combine wavelet transform, Fourier transform and other methods to convert spatial texture features into frequency domain features to detect high-frequency texture changes. The obtained wire harness fine texture abnormality data records the coordinates and characteristics of the texture abnormal area in detail, providing accurate texture information for defect classification.

[0247] Step S312: performing defect classification and positioning processing according to the wire harness fine texture abnormality data to obtain second wire harness defect detection data.

[0248] In one embodiment, after obtaining the fine texture abnormal data, each abnormal area is defect-classified and its specific location is determined. The classification process divides the abnormal area into specific defect categories, such as cracks, dents, wear, etc., based on the texture, morphological characteristics and structural integrity analysis results of the abnormal data. During classification, different abnormal features are matched with corresponding defect types through machine learning or rule-based classifiers. For example, areas where texture features show periodic fractures can be classified as cracks, while areas with large surface height deviations are protrusions or dents. Through confidence estimation, a confidence value is assigned to each defect type to indicate the reliability of the classification, and the generated second harness defect detection data contains the type, location and confidence of each defect. This information not only provides a detailed description of the defect, but also provides a reliable basis for subsequent repair and quality assessment.

[0249] Preferably, step S4 is specifically:

[0250] Step S41: Initializing the marking position according to the first wire harness defect detection data and the second wire harness defect detection data to obtain defect position marking data;

[0251] In one embodiment, the first wire harness defect detection data (based on two-dimensional images) and the second wire harness defect detection data (based on three-dimensional reconstruction) provide defect information of different dimensions respectively. By integrating these data, the initial position coordinates of each defect point are generated, including the pixel position in the two-dimensional image and the spatial coordinates in the three-dimensional reconstruction. In specific implementation, the two-dimensional image data and the three-dimensional reconstruction data are unified through coordinate mapping, and a unique tag ID is assigned to each defect position.

[0252] Step S42: plane marking the wire harness image data according to the first wire harness defect detection data to obtain first wire harness image defect marking data;

[0253] In one embodiment, after the defect position is initialized, the first wire harness defect detection data is used to mark the defect in a plane on the two-dimensional image. In actual operation, the two-dimensional image data is first loaded into the marking tool, and a mark is generated at the defect position provided by the detection data. These marks can be circular, square or other shaped symbols, and can be distinguished by color or symbol according to different defect types (such as a red circle for a crack, a blue square for a dent, etc.). When marking, each defect is accurately located and sized according to the defect coordinates and size information provided in the first wire harness defect detection data to ensure that the mark matches the actual defect position.

[0254] Step S43: performing three-dimensional superposition on the wire harness image data according to the defect position mark data and the wire harness three-dimensional reconstruction data to obtain second wire harness image defect mark data;

[0255] In one embodiment, the defect position is superimposed on the three-dimensional reconstruction data of the wire harness according to the defect position marking data to achieve three-dimensional marking. In the specific operation, the defect position marking data is first aligned with the three-dimensional reconstruction data so that the marking position of each defect is accurately located in the three-dimensional model. Then, the first wire harness image defect marking data is mapped to the three-dimensional space so that the two-dimensional marking data corresponds to the defect position in the three-dimensional reconstruction. During the superposition process, the coordinate transformation and scaling operations are performed so that the mark can be correctly displayed in the three-dimensional space. Finally, the second wire harness image defect marking data is generated, which not only records the specific position of each defect in the three-dimensional space, but also retains the characteristics of the two-dimensional mark.

[0256] Step S44: performing defect visualization rendering according to the first wire harness image defect marking data and the second wire harness image defect marking data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0257] In one embodiment, after completing the three-dimensional overlay, the last step is to perform visual rendering of the defect, and to overlay and render the first and second wire beam image defect mark data to generate a complete defect mark image. First, the first wire beam image defect mark data (plane mark) and the second wire beam image defect mark data (three-dimensional overlay mark) are loaded simultaneously in the visualization interface, and transparency or depth rendering is used to enable the two-dimensional and three-dimensional marks to be displayed simultaneously. Secondly, each marked area is rendered to ensure that the color, size and transparency of the mark are coordinated with the original image, so that the mark does not obstruct the actual image content, but can also clearly show the specific location and type of the defect. The rendering result will generate a complete defect mark image that integrates two-dimensional plane and three-dimensional stereo information, so that the spatial position, size, type and other information of the defect are clear at a glance.

[0258] Preferably, the present application also provides a wire harness defect detection system based on visual recognition, which is used to execute the wire harness defect detection method based on visual recognition as described above. The wire harness defect detection system based on visual recognition includes:

[0259] A wire harness image segmentation module, used for acquiring wire harness image data, and performing image segmentation according to the wire harness image data to obtain wire harness image segmentation data;

[0260] A wire harness morphological feature extraction and three-dimensional reconstruction module is used to extract morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and to perform three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data;

[0261] A wire harness defect detection module is used to perform microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and perform micromorphological defect detection on the wire harness three-dimensional reconstruction data to obtain second wire harness defect detection data;

[0262] The wire harness image defect marking module is used to mark the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection.

[0263] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0264] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A wire harness defect detection method based on visual recognition, characterized in that: The following steps are involved: Step S1: acquiring wire beam image data, and performing image segmentation according to the wire beam image data to obtain wire beam image segmentation data; Step S2: extracting morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and performing three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data; Step S3: performing microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and performing micromorphological defect detection on the wire harness 3D reconstruction data to obtain second wire harness defect detection data; Step S4: marking the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection; The three-dimensional reconstruction is as follows: Performing morphological classification according to the wire harness morphological feature data to obtain wire harness morphological classification data; Perform feature map fusion according to the wire harness morphology classification data to obtain wire harness morphology feature map fusion data; Performing depth estimation on the fusion data of the wire harness morphological feature map to obtain the wire harness feature depth data; Performing perspective matching according to the wire harness feature depth data to obtain wire harness perspective matching data; Generate three-dimensional point cloud according to the wire harness view matching data to obtain wire harness point cloud data; Three-dimensional mesh reconstruction is performed based on the wire harness point cloud data to obtain three-dimensional reconstruction data of the wire harness.

2. The method according to claim 1, characterized in that Step S1 is specifically as follows: Ambient light data is obtained by collecting ambient light through an ambient light sensor; Get harness material data; Evaluate the material surface characteristics according to the wire harness material data to obtain the wire harness surface characteristics data; Setting visual parameters according to ambient light data and harness surface characteristic data to obtain visual parameter data; According to the visual parameter data, the wire harness multi-view image acquisition is performed through a preset visual device to obtain the wire harness image data; Perform geometric calibration according to the wire harness image data to obtain wire harness geometric calibration data; Perform edge detection according to the wire harness geometry calibration data to obtain wire harness edge data; Perform region segmentation and marking according to the wire harness edge data to obtain wire harness region segmentation data; Smooth detail reconstruction is performed based on the wire harness region segmentation data to obtain wire harness image segmentation data.

3. The method according to claim 2, characterized in that The smooth detail reconstruction is as follows: Perform multi-resolution analysis based on the wire harness area segmentation data to obtain wire harness multi-resolution level data; Perform edge enhancement according to the multi-resolution level data of the wire bundle to obtain multi-level edge enhancement data of the wire bundle; Performing texture restoration on the wire harness multi-level edge enhancement data to obtain the wire harness multi-level texture restoration data; Perform super-resolution restoration based on the multi-level texture restoration data of the wire harness to obtain super-resolution data of the wire harness; Performing structure-preserving mapping on the wire bundle super-resolution data to obtain wire bundle structure mapping data; Performing feature smoothing processing on the wire harness structure mapping data to obtain wire harness feature smoothing data; Perform local detail alignment based on the wire harness feature smoothing data to obtain multi-level detail alignment data of the wire harness; The wire harness multi-level detail alignment data is reconstructed and segmented and outputted to obtain wire harness image segmentation data.

4. The method according to claim 3, characterized in that Features The specific smoothing process is: Performing multi-scale similar block search on the wire harness structure mapping data to obtain multi-scale similar block data of the wire harness; A lightweight convolutional network similarity weight calculation is performed based on the multi-scale similarity block data of the wire bundle to obtain the wire bundle similarity weight data; The wire harness structure mapping data is weighted averaged according to the wire harness similarity weight data to obtain the wire harness similarity weighted data; Perform pixel update according to the wire bundle similarity weighted data to obtain wire bundle pixel update data; An edge preservation check is performed based on the wire harness pixel update data to obtain the wire harness feature smoothing data.

5. The method according to claim 1, characterized in that The morphological feature extraction is specifically as follows: Capturing the area contour according to the wire harness image segmentation data to obtain the wire harness area contour data; Performing contour reconstruction according to the wire harness area contour data to obtain wire harness contour reconstruction data; Performing coupled convolution feature extraction and interpreted convolution feature extraction on the wire beam contour reconstruction data to obtain first wire beam image feature data and second wire beam image feature data respectively; Performing wire harness shape mapping on the first wire harness image feature data and the second wire harness image feature data to obtain first wire harness shape mapping data and second wire harness shape mapping data; Perform detail structure difference mapping according to the first wire harness shape mapping data and the second wire harness shape mapping data to obtain wire harness shape detail structure difference data; Generate a shape feature map based on the wire harness morphology detail structure difference data to obtain wire harness morphology feature data; The coupled convolution feature extraction is specifically as follows: Performing initial convolution layer processing on the wire harness contour reconstruction data to obtain wire harness initial convolution layer data; Separate the coupled feature channels of the initial convolutional layer data of the wire harness to obtain the wire harness geometric structure feature data and the wire harness texture feature data respectively; Perform residual connection convolution layer processing on the wire harness geometric structure feature data to obtain high-order feature data of the wire harness geometric structure, and perform dense connection convolution layer processing on the wire harness texture feature data to obtain high-order feature data of the wire harness texture; Performing feature fusion on the high-order feature data of the wire harness geometry structure and the high-order feature data of the wire harness texture to obtain coupled fusion feature map data; The coupled fusion feature map data will be subjected to local receptive field enhancement to obtain coupled local enhanced feature map data, wherein the local receptive field enhancement includes calculating the response weights of pixels in the coupled fusion feature map data through an attention mechanism or other weight calculation method, determining the importance of details in each local area, and obtaining a response weight matrix. The coupled fusion feature map data is weighted and merged according to the response weight matrix, and the eigenvalue of each pixel position is adjusted according to the weight matrix to form coupled local enhanced feature map data; Performing feature compression on the coupled local enhanced feature map data to obtain first line beam image feature data; The explanation of convolution feature extraction is as follows: Calculate the confidence of the wire harness contour reconstruction data according to the coupled fusion feature map data to obtain the wire harness area confidence data; Performing feature selective convolution layer processing on the wire harness contour reconstruction data according to the wire harness region confidence data to obtain wire harness selective convolution layer data; Perform feature comparison on the wire bundle selective convolution layer data and the coupled fusion feature map data to obtain wire bundle comparison feature data; Performing a significance analysis on the wire harness contrast feature data to obtain the wire harness significance feature data; Perform high-level structural aggregation on the wire harness significant feature data to obtain wire harness structural aggregation data; Perform explanatory self-attention map calculation on the bundle salient feature data and the bundle structure aggregation data to obtain the bundle weighted explanatory feature map data; Global morphological mapping is performed according to the line bundle weighted explanatory feature map data to obtain second line bundle image feature data.

6. The method according to claim 1, characterized in that The microstructure defect detection is specifically as follows: Perform feature block division according to the wire harness morphology feature data to obtain the wire harness morphology local feature data; Identify defect candidate areas according to the local feature data of the wire harness morphology to obtain wire harness defect candidate area data; Perform texture feature matching based on the wire harness defect candidate area data and the preset conventional wire harness area data to obtain the wire harness texture abnormality data; According to the abnormal data of wire harness texture, local deformation similarity calculation and relative deformation measurement processing are performed to obtain the wire harness local deformation similarity data and the wire harness relative deformation measurement data; Performing shape comparison based on the wire harness local deformation similarity data and the wire harness relative deformation measurement data to obtain wire harness shape comparison data; Defect classification and confidence estimation are performed based on the wire harness morphology comparison data to obtain first wire harness defect detection data.

7. The method according to claim 1, characterized in that The micro-morphological defect detection is specifically: Extracting surface features from the wire harness three-dimensional reconstruction data to obtain wire harness surface feature data; Performing local area division and mesh refinement on the surface feature data of the harness to obtain local refined mesh data; Perform morphological anomaly detection based on local refined grid data to obtain harness morphological anomaly data; Perform structural integrity analysis based on the wire harness 3D reconstruction data and wire harness morphology abnormality data to obtain wire harness structural integrity data; Perform fine texture detection according to the abnormal data of wire harness morphology to obtain abnormal data of wire harness fine texture; Defect category classification and positioning processing are performed based on the wire harness fine texture abnormality data to obtain second wire harness defect detection data.

8. The method according to claim 1, characterized in that Step S4 is specifically as follows: Initialize the marking position according to the first wire harness defect detection data and the second wire harness defect detection data to obtain defect position marking data; Performing plane marking on the wire harness image data according to the first wire harness defect detection data to obtain first wire harness image defect marking data; Performing three-dimensional superposition on the wire harness image data according to the defect position marking data and the wire harness three-dimensional reconstruction data to obtain second wire harness image defect marking data; Defect visualization rendering is performed according to the first wire harness image defect marking data and the second wire harness image defect marking data to obtain wire harness image defect marking data to assist in wire harness defect detection.

9. A wire harness defect detection system based on visual recognition, characterized in that: For executing the wire harness defect detection method based on visual recognition as claimed in claim 1, the wire harness defect detection system based on visual recognition comprises: A wire harness image segmentation module, used for acquiring wire harness image data, and performing image segmentation according to the wire harness image data to obtain wire harness image segmentation data; A wire harness morphological feature extraction and three-dimensional reconstruction module is used to extract morphological features according to the wire harness image segmentation data to obtain wire harness morphological feature data, and to perform three-dimensional reconstruction according to the wire harness morphological feature data to obtain wire harness three-dimensional reconstruction data; A wire harness defect detection module is used to perform microstructure defect detection according to the wire harness morphological feature data to obtain first wire harness defect detection data, and perform micromorphological defect detection on the wire harness three-dimensional reconstruction data to obtain second wire harness defect detection data; The wire harness image defect marking module is used to mark the wire harness image data according to the first wire harness defect detection data and the second wire harness defect detection data to obtain wire harness image defect marking data to assist in wire harness defect detection.

Images