Object surface defect detection method, device, equipment and medium
By extracting and fusing texture and depth features on the surface of the object, the problem of low detection accuracy of complex surfaces in the prior art is solved, and high-precision defect detection is achieved, suitable for curved surfaces and irregular surfaces, simplifying the detection process.
Patent Information
- Application Number
- CN202410985469.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
The existing object surface defect detection algorithm has limited accuracy and poor ease of use in complex surface scenes, cannot be applied to curved and irregular surfaces, and has complex pre-processing and low robustness.
By extracting texture surface features and depth defect features and fusing them in preset dimensions, high-precision fusion features are generated and pre-trained defect detection models are input to improve detection accuracy, which is suitable for complex surface scenarios.
It realizes high-precision defect detection in complex surface scenes, simplifies the pre-processing process, improves ease of use and robustness, makes full use of texture and depth information, and improves detection accuracy.
Smart Images

Figure CN118941524A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for detecting surface defects of an object. Background Art
[0002] During the product production process, even tiny defects on the surface of the corresponding object of the corresponding product may cause the product's performance to deteriorate or malfunction during use, thus threatening the safety of users. Therefore, it is of great significance to ensure the safety of the product by conducting 3D defect detection on the surface of the corresponding object and timely discovering and eliminating these potential safety hazards.
[0003] Existing defect detection algorithms usually require complex pre-processing, such as multiple complex operations such as plane fitting, line fitting, coordinate transformation, curvature calculation, etc. The process involves a large number of parameter adjustments, poor usability, and low robustness, resulting in limited accuracy of the corresponding defect detection results. The application scenarios are also limited to objects with regular surfaces or high consistency, and cannot be applied to complex surface scenes such as curved surfaces and irregular surfaces. Summary of the invention
[0004] The present invention provides a method, device, equipment and medium for detecting surface defects of an object, so as to realize defect detection of complex surface scenes and further improve the corresponding defect detection accuracy.
[0005] According to one aspect of the present invention, a method for detecting surface defects of an object is provided, the method comprising:
[0006] Based on the image information of the object to be detected, extracting the texture surface features and depth defect features of each preset dimension corresponding to the object to be detected; wherein the image information includes texture image information and depth image information;
[0007] The texture surface features of each preset dimension are respectively fused with the depth defect features of the corresponding dimension to obtain fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features;
[0008] Connect the fusion features of each preset dimension to obtain the fusion features of the target object;
[0009] The target object fusion features are input into a pre-trained defect detection model to obtain the corresponding target defect detection results.
[0010] According to another aspect of the present invention, there is provided a device for detecting surface defects of an object, the device comprising:
[0011] A texture depth feature extraction module, used to extract texture surface features and depth defect features of each preset dimension corresponding to the object to be detected based on the image information of the object to be detected; wherein the image information includes texture image information and depth image information;
[0012] A fusion feature acquisition module is used to fuse the texture surface features of each preset dimension with the depth defect features of the corresponding dimension to obtain the fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features;
[0013] A target object fusion feature acquisition module is used to connect the fusion features of each preset dimension to obtain the target object fusion feature;
[0014] The target defect detection result acquisition module is used to input the target object fusion features into a pre-trained defect detection model to obtain the corresponding target defect detection results.
[0015] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for detecting surface defects of an object described in any embodiment of the present invention.
[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for detecting surface defects of an object described in any embodiment of the present invention when executed.
[0020] According to another aspect of the present invention, a computer program product is provided, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the computer program / instruction implements the method for detecting surface defects of an object as described in any embodiment of the present invention.
[0021] The technical solution of the embodiment of the present invention presets different dimensions and extracts the texture surface features and depth defect features of the surface of the object to be detected based on each preset dimension, so as to further fuse the texture surface features and depth defect features of each preset dimension, and then connect the fused features of each preset dimension to obtain high-precision fusion features of the surface of the object to be detected including corresponding texture information and depth information. It can output higher-precision defect detection results using the corresponding trained defect detection model, thereby avoiding the complicated pre-processing process in the existing defect detection process, and can be applicable to the detection of corresponding defects in complex surface scenes such as curved surfaces and irregular surfaces. It has high ease of use and robustness, and at the same time, it also realizes the full utilization of the texture information and depth information of the corresponding defects, further improving the accuracy of the corresponding defect detection.
[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1 is a flow chart of a method for detecting surface defects of an object provided according to Embodiment 1 of the present invention;
[0025] Figure 2A is a flow chart of another method for detecting surface defects of an object provided according to Embodiment 2 of the present invention;
[0026] Figure 2B A schematic diagram of generating texture depth fusion features based on a texture depth fusion module provided in Embodiment 2 of the present invention.
[0027] Figure 2C A schematic diagram of generating boundary fusion features based on a boundary fusion module provided in Embodiment 2 of the present invention.
[0028] Figure 2D A schematic diagram of generating local fusion features based on a cross-modal fusion module according to the second embodiment of the present invention.
[0029] Figure 2E A schematic diagram of generating overall fusion features based on a multi-scale feature fusion module provided in Embodiment 2 of the present invention.
[0030] Figure 3A is a flow chart of another method for detecting surface defects of an object provided according to Embodiment 3 of the present invention;
[0031] Figure 3B This is a schematic diagram of an algorithm for a method for detecting surface defects of an object provided in Embodiment 3 of the present invention.
[0032] Figure 4 is a schematic structural diagram of a device for detecting surface defects of an object provided according to a fourth embodiment of the present invention;
[0033] Figure 5 It is a structural schematic diagram of an electronic device for implementing the method for detecting surface defects of an object according to an embodiment of the present invention; DETAILED DESCRIPTION
[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0036] Figure 1 The present invention provides a method for detecting surface defects of an object. This embodiment is applicable to the case of defect detection on complex surface scenes such as curved surfaces or irregular surfaces. The method can be performed by a device for detecting surface defects of an object. The device can be implemented in the form of hardware and / or software, and the device can be configured in an electronic device. Figure 1 Said method comprises:
[0037] S110. Extracting texture surface features and depth defect features of preset dimensions corresponding to the object to be detected based on image information of the object to be detected; wherein the image information includes texture image information and depth image information.
[0038] Among them, the object to be detected can be an object with a regular surface or a high surface consistency, or it can be an object with a curved surface or an irregular surface. The curved surface or irregular surface can be, for example, the arc weld surface formed during the welding process of the corresponding battery pack shell and battery holder in the lithium battery industry. The texture surface feature can be the texture feature information corresponding to the surface to be detected, and the depth defect feature can be the depth feature information of the defects existing in the surface to be detected. The preset dimension can be used to characterize the granularity of feature extraction of the texture information and depth information of the surface to be detected. The surface to be detected can be the surface area of the object to be detected where defect detection is required.
[0039] Specifically, the texture image information corresponding to the surface to be detected can be input into a pre-trained texture surface feature extraction model, and the texture surface features of each preset dimension corresponding to the surface to be detected can be output. Correspondingly, the depth image information corresponding to the surface to be detected can be input into a pre-trained depth surface feature extraction model, and the depth defect features of each preset dimension corresponding to the surface to be detected can be output. The corresponding texture surface feature extraction model and the depth surface feature extraction model can be a pre-trained deep learning model, a neural network model, or a machine learning model, etc., which is not specifically limited here.
[0040] S120, fusing the texture surface features of each preset dimension with the depth defect features of the corresponding dimension respectively to obtain fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features.
[0041] Specifically, the texture surface features of a preset dimension and the depth defect features of a preset dimension uniquely corresponding to the preset dimension can be input into a pre-trained feature fusion network model to obtain the fusion features corresponding to the preset dimension. The feature fusion network model can be a pre-trained deep learning model, etc., which is not specifically limited here.
[0042] S130, connecting the fusion features of each preset dimension to obtain the fusion features of the target object.
[0043] The target object fusion feature may include feature information of each defect in the surface to be detected corresponding to the object to be detected. The process of connecting the fusion features of each preset dimension may be implemented in a related image processing technology, which is not specifically limited here.
[0044] S140, inputting the target object fusion features into a pre-trained defect detection model to obtain a corresponding target defect detection result.
[0045] The defect detection model may be a network model that takes the corresponding target fusion features as input and outputs the corresponding defect information of the surface to be detected. The network model may be a deep learning model, a neural network model, or a machine learning model, etc., which is not specifically limited here.
[0046] In an embodiment of the present invention, by presetting different dimensions, and extracting the texture surface features and depth defect features of the surface of the object to be inspected based on each preset dimension, the texture surface features and depth defect features of the preset dimensions are further fused, and then the fused features of the preset dimensions are connected again to obtain high-precision fusion features of the surface of the object to be inspected including corresponding texture information and depth information. This enables the use of a corresponding trained defect detection model to output a higher-precision defect detection result, thereby avoiding the complicated pre-processing process in the existing defect detection process, and can be applicable to the detection of corresponding defects in complex surface scenes such as curved surfaces and irregular surfaces, with high ease of use and robustness, and at the same time, it also realizes the full utilization of the texture information and depth information of the corresponding defects, further improving the accuracy of the corresponding defect detection.
[0047] Figure 2A This is a flow chart of a method for detecting surface defects of an object provided in the second embodiment of the present invention. This embodiment is further optimized based on the above embodiments. It should be noted that for the parts not described in detail in this embodiment, please refer to the relevant descriptions in other embodiments.
[0048] Further, "based on the image information of the object to be detected, extracting the texture surface features of each preset dimension corresponding to the object to be detected" is refined into "according to the texture image information of the object to be detected and the preset first texture extraction model, obtaining the shallow texture features corresponding to the object to be detected; and according to the depth image information of the object to be detected and the preset first depth extraction model, obtaining the shallow depth features corresponding to the object to be detected; according to the preset second texture extraction model and the shallow texture features, obtaining the local texture features corresponding to the object to be detected; and according to the preset second depth extraction model and the shallow depth features, obtaining the local depth features corresponding to the object to be detected; according to the preset third texture extraction model and the local texture features, obtaining the global texture features corresponding to the object to be detected; and according to the preset third depth extraction model and the local depth features, obtaining the global depth features corresponding to the object to be detected", so as to improve the extraction mechanism of the corresponding texture surface features and depth defect features.
[0049] refer to Figure 2A, the method specifically comprises the following steps:
[0050] S210. Acquire shallow texture features corresponding to the object to be detected based on texture image information of the object to be detected and a preset first texture extraction model; and acquire shallow depth features corresponding to the object to be detected based on depth image information of the object to be detected and a preset first depth extraction model.
[0051] Among them, the first texture extraction model is a pre-trained network model for extracting shallow texture features corresponding to corresponding texture image information. The first depth extraction model is a pre-trained network model for extracting shallow depth features corresponding to corresponding depth image information. The first texture extraction model can take the corresponding texture image information as input and the corresponding shallow texture features as output. The first depth extraction model can take the corresponding depth image information as input and the corresponding shallow depth features as output.
[0052] S220, obtaining local texture features corresponding to the object to be detected according to a preset second texture extraction model and the shallow texture features; and obtaining local depth features corresponding to the object to be detected according to a preset second depth extraction model and the shallow depth features.
[0053] The second texture extraction model is a pre-trained network model for extracting local texture features corresponding to the corresponding texture image information. The second depth extraction model is a pre-trained network model for extracting local depth features corresponding to the corresponding depth image information. The second texture extraction model can take the corresponding shallow texture features as input and the corresponding local texture features as output. The second depth extraction model can take the corresponding shallow depth features as input and the corresponding local depth features as output.
[0054] S230. Acquire global texture features corresponding to the object to be detected according to a preset third texture extraction model and the local texture features; and acquire global depth features corresponding to the object to be detected according to a preset third depth extraction model and the local depth features.
[0055] The third texture extraction model is a pre-trained network model for extracting global texture features corresponding to the corresponding texture image information. The third depth extraction model is a pre-trained network model for extracting global depth features corresponding to the corresponding depth image information. The third texture extraction model can take the corresponding local texture features as input and the corresponding global texture features as output. The third depth extraction model can take the corresponding local depth features as input and the corresponding global depth features as output.
[0056] S240, fusing the texture surface features of each preset dimension with the depth defect features of the corresponding dimension respectively to obtain fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features;
[0057] Exemplarily, the fusion features of each preset dimension may include a boundary fusion feature, a local fusion feature, and a global fusion feature; accordingly, the texture surface features of each preset dimension are fused with the depth defect features of the corresponding dimension to obtain the fusion features of each preset dimension, which may include:
[0058] According to a preset first feature fusion model, the first texture feature and the first depth feature are fused to obtain a first fused feature; wherein the first texture feature is a shallow texture feature, the first depth feature is a shallow depth feature, and the first fused feature is a boundary fusion feature; according to a preset second feature fusion model, the second texture feature and the second depth feature are fused to obtain a second fused feature; wherein the second texture feature includes a local texture feature and a global texture feature, the second depth feature includes a local depth feature and a global depth feature, and the second fused feature includes a local fusion feature and a global fusion feature.
[0059] Among them, the first feature fusion model and the second feature model can be a pre-trained network model for fusing texture surface features of preset dimensions and corresponding depth defect features. The preset dimensions can include shallow, local and global. Texture surface features can include shallow texture features, local texture features and global texture features. Depth defect features can include shallow depth features, local depth features and global depth features. Boundary fusion features can be boundary feature information corresponding to each defect on the surface to be detected corresponding to the object to be detected.
[0060] It can be understood that by setting corresponding fusion models for texture surface features and corresponding depth defect features of different preset dimensions, the corresponding fusion process can be set in a targeted manner based on the granularity corresponding to the corresponding texture surface features and depth defect features themselves under each preset dimension, thereby making full use of the texture information and depth information in each texture surface feature and depth defect feature of different preset dimensions, which helps to avoid the situation where there is a lot of information missing in the target object fusion features obtained by subsequently connecting each fusion feature, and thus helps to improve the accuracy of corresponding defect detection.
[0061] Exemplarily, according to a preset first feature fusion model, fusing the first texture feature and the first depth feature to obtain a first fusion feature may include:
[0062] Based on a preset boundary feature fusion model, the shallow texture feature and the shallow depth feature are fused to obtain a first boundary fusion feature that includes both the shallow texture feature and the shallow depth feature; the first boundary feature is subjected to a first enhancement process to obtain a first boundary fusion enhancement feature; the first boundary fusion enhancement feature is input into a preset multi-scale boundary feature extraction model to obtain a multi-scale second boundary fusion feature and a target boundary contour trajectory; the second boundary fusion feature and the target boundary contour trajectory are subjected to a second enhancement process to obtain the boundary fusion feature.
[0063] The boundary feature fusion model may be a pre-trained network model for performing boundary fusion processing on shallow texture features and shallow depth features. The multi-scale boundary feature extraction model may be a pre-trained network model for performing multi-scale processing on boundary features, and is used to output multi-scale boundary features. The boundary feature may be the boundary information of defects existing on the surface to be detected corresponding to the object to be detected. The target boundary contour trajectory may be the boundary contour trajectory of each defect on the surface to be detected corresponding to the object to be detected.
[0064] It can be understood that by fusing the corresponding shallow texture features and shallow depth features to generate the corresponding boundary fusion features, multiple enhancement processing and multi-scale processing of the corresponding boundary features are introduced, so that the corresponding boundary fusion features obtained contain finer-grained boundary feature information, which helps to improve the accuracy of the corresponding defect detection results generated subsequently.
[0065] Exemplarily, according to a preset second feature fusion model, fusing the second texture feature and the second depth feature to obtain the second fusion feature may include:
[0066] According to a preset common feature fusion model, the second texture feature and the second depth feature are fused to obtain a common fusion feature; based on a preset first weighted feature fusion model and a second weighted feature fusion model, the texture feature and the depth feature are weightedly fused to obtain a first weighted fusion feature and a second weighted fusion feature; according to a first feature fusion sub-model, the first weighted fusion feature and the common fusion feature are fused to obtain a first candidate fusion feature; and according to a second feature fusion sub-model, the second weighted fusion feature and the common fusion feature are fused to obtain a second candidate fusion feature; the first candidate fusion feature and the second candidate fusion feature are connected to obtain the second fusion feature.
[0067] Among them, the common feature fusion model can be a network model that fuses the texture information and depth information of the surface to be detected corresponding to the corresponding object to be detected in the second texture feature and the second depth feature. The common fusion feature may include the texture information of the surface to be detected and the depth information at the corresponding position of the texture information. The first weighted fusion model and the second weighted fusion model can be network models that perform cross-modal weighted feature fusion on texture information and depth information. The weighting mechanisms corresponding to the first weighted fusion model and the second weighted fusion model can be different, and the weighting mechanism can be set as needed, which is not specifically limited here. For example, the weighting mechanism of the first weighted fusion model can be an addition weight, and the weighting mechanism of the second weighted fusion model can be a multiplication weight, etc. The first weighted fusion sub-model can be a network model that fuses the first weighted fusion feature and the common fusion feature, and the second weighted fusion sub-model can be a network model that fuses the second weighted fusion feature and the common fusion feature.
[0068] It can be understood that, by fusing the corresponding second texture features and the second depth features to generate the corresponding second fusion features, cross-modal weighted processing for the corresponding texture information and depth information is introduced, and the weighted fusion features obtained are fused with the corresponding common fusion features respectively, and then the fusion results are connected to achieve the fusion of the corresponding second texture features and the second depth features, thereby avoiding the situation where the fused features due to the absence of corresponding depth information or texture information or the small amount of information appear in the missing information, which helps to improve the accuracy of the generated corresponding fusion features.
[0069] S250, connecting the fusion features of each preset dimension to obtain the fusion features of the target object;
[0070] Exemplarily, connecting the candidate fusion features to obtain the target object fusion feature may include:
[0071] The boundary fusion feature is subjected to multi-scale processing to obtain the target boundary fusion feature; the local fusion feature is subjected to multi-scale processing to obtain the target local fusion feature; based on a preset feature integration model, the target local fusion feature and the global fusion feature are integrated, and redundant information is filtered out to obtain a comprehensive fusion feature; the target boundary fusion feature and the comprehensive fusion feature are connected to obtain the target object fusion feature.
[0072] The feature integration model may be a pre-trained network model for integrating the target local fusion features and the global fusion features. Redundant information may be feature information corresponding to information types that are not required during defect detection, such as color feature information such as marker pen marks.
[0073] It can be understood that by performing multi-scale processing on the boundary fusion features and the local fusion features respectively, it is possible to extract texture information and depth information of different scales, and complete the automatic fusion of texture features and depth features, thereby avoiding the information loss caused by only referring to the texture information and depth information of a single dimension, which in turn helps to improve the information completeness of the target object fusion features corresponding to the corresponding object to be detected, and thus helps to improve the accuracy of the corresponding defect detection results.
[0074] S260: Input the target object fusion features into a pre-trained defect detection model to obtain a corresponding target defect detection result.
[0075] In an embodiment of the present invention, by respectively extracting shallow texture features, local texture features and global texture features corresponding to the surface of the corresponding object to be inspected, as well as the corresponding shallow depth features, local depth features and global depth features, texture information and depth information of different granularities corresponding to the surface of the corresponding object to be inspected can be obtained, thereby helping to avoid information missing when the corresponding texture information and depth information are subsequently fused, and further helping to improve the accuracy and reliability of the corresponding defect detection results.
[0076] For ease of understanding, specific examples of generating texture depth fusion features, boundary fusion features, local fusion features, and overall fusion features are provided with reference to Figures 2b-2e. It should be noted that the sizes of the features involved in the figures are only for illustrative purposes and should not be construed as specific limitations of the present invention.
[0077] Figure 2B This is a schematic diagram of a method for generating texture depth fusion features based on a texture depth fusion module. Figure 2BAs shown in , the texture deep fusion module mainly includes a texture feature extraction branch, a depth feature extraction branch and a feature fusion branch. In the texture feature extraction branch, a texture image with a size of 1*3*640*640 can be input into the texture extraction module 1 to obtain a shallow texture feature R1 with a size of 1*16*320*320. The R1 is passed through the texture feature module 2 to obtain a local texture feature R2 with a size of 1*24*160*160. The R2 is passed through the texture feature module 3 to obtain a global texture feature R3 with a size of 1*32*80*80. Among them, the texture extraction module 1 can be the above-mentioned first texture extraction model, the texture extraction module 2 can be the above-mentioned second texture extraction model, and the texture extraction module 3 can be the above-mentioned third texture extraction model. Similarly, a depth image with a size of 1*3*640*640 can be input into the depth feature extraction branch to obtain a shallow depth feature D1 with a size of 1*16*320*320, a local depth feature D2 with a size of 1*24*160*160, and a global depth feature D3 with a size of 1*32*80*80. Among them, the depth extraction module 1 can be the first depth extraction model mentioned above, the depth extraction module 2 can be the second depth extraction model mentioned above, and the depth extraction module 3 can be the third depth extraction model mentioned above. The above R1 and D1 can be input into the feature fusion module 1, the above R2 and D2 can be input into the feature fusion module 2, and the above R3 and D3 can be input into the feature fusion module 3. The feature fusion module 1 can be a boundary feature fusion module, and the boundary feature fusion module can be the first feature fusion model mentioned above, and the output result can be a boundary fusion feature F1 with a size of 1*16*320*320. Feature fusion module 2 and feature fusion module 3 are cross-modal fusion modules, which can be the above-mentioned second feature fusion model, and the output results can be local fusion features F2 with a size of 1*72*160*160 and global fusion features F3 with a size of 1*96*80*80. The above F1, F2 and F3 can be input into feature fusion module 4. Feature fusion module 4 is a multi-scale fusion module, and the output can be an overall fusion feature F4 with a size of 1*256*80*80, which can be used as a texture depth fusion feature C3 with a size of 1*256*80*80. The texture depth fusion feature can be the above-mentioned target object fusion feature.
[0078] Figure 2C A schematic diagram of generating boundary fusion features based on the boundary fusion module. Figure 2CAs shown, the size of the shallow texture feature R1 and the shallow depth feature D1 can both be 1*16*320*320, and the features of the shallow texture feature R1 and the shallow depth feature D1 can be extracted in the boundary feature fusion block. Subsequently, the combined features, R1 and D1 can be input into the feature enhancement block to complete the enhancement of the boundary information, and the enhanced feature information can be input into the multi-scale boundary feature extraction block to obtain multi-scale features and boundary contour trajectories. Finally, the multi-scale features and boundary contour trajectories, as well as the above R1 and D1 can be input into the feature enhancement block to obtain a boundary fusion feature F1 with a size of 1*16*320*320. Among them, the boundary feature fusion block can be the above-mentioned boundary feature fusion model, and the corresponding feature enhancement block can perform the above-mentioned first enhancement processing and the above-mentioned second enhancement processing accordingly. The multi-scale boundary feature extraction block can be the above-mentioned multi-scale boundary feature extraction model. The multi-scale feature can be the above-mentioned second boundary fusion feature of the multi-scale, and the boundary contour trajectory can be the above-mentioned target boundary contour trajectory.
[0079] Figure 2D This is a schematic diagram of generating local fusion features based on a cross-modal fusion module. Figure 2D As shown, taking the local texture feature R2 and the local depth feature D2 as examples, the sizes of the local texture feature R2 and the local depth feature D2 are both 1*24*160*160. After being input into the common fusion block, the common fusion feature can be obtained by using the common feature fusion block. At the same time, after being input into the weighted feature fusion block, the weighted feature fusion block can be used to perform cross-modal weighted feature fusion on the texture information and the depth information to obtain the weighted fusion feature. Subsequently, the obtained common fusion feature and the weighted fusion feature are respectively input into the feature fusion block to obtain the fusion feature. The local fusion feature F2 with a size of 1*72*160*160 is obtained by connecting and integrating the fusion features. Among them, the common feature fusion block can be the above-mentioned common feature fusion model, the weighted feature fusion block can be the above-mentioned first weighted feature fusion model and the above-mentioned second weighted feature fusion model, the weighted fusion feature can be the above-mentioned first weighted fusion feature and the second weighted fusion feature, and the feature fusion block can be the above-mentioned first feature fusion sub-model and the second feature fusion sub-model. The fusion feature can be the above-mentioned first candidate fusion feature and the above-mentioned second candidate fusion feature.
[0080] Figure 2E This is a schematic diagram of a method for generating overall fusion features based on a multi-scale feature fusion module. Figure 2EAs shown, the input of the multi-scale feature fusion module includes the outputs of the feature fusion module 1, the feature fusion module 2 and the feature fusion module 3, which correspond to the boundary fusion feature F1, the local fusion feature F2 and the global fusion feature F3. Specifically, the boundary fusion feature F1 is first processed at multiple scales to ensure the consistency of the data structure. Then, the local fusion feature F2 is processed at multiple scales, and the processed local fusion features and the global fusion features F3 are input into the feature integration module to eliminate the redundant information in the features, complement each other, and obtain the comprehensive fusion features. The processed boundary fusion features and the comprehensive fusion features are connected, and the boundary details are supplemented to the comprehensive fusion features to obtain the overall fusion feature F4 with a size of 1*256*80*80. Among them, the feature integration block can be the above-mentioned feature integration model.
[0081] Figure 3A This is a flow chart of a method for detecting surface defects of an object provided in the third embodiment of the present invention. This embodiment is further optimized based on the above embodiments. It should be noted that for the parts not described in detail in this embodiment, please refer to the relevant descriptions in other embodiments.
[0082] Furthermore, “inputting the target object fusion features into a pre-trained defect detection model to obtain the corresponding target defect detection results” is refined into “inputting the target object fusion features into a preset first feature extraction model to obtain target object fusion features of a first size; inputting the target object fusion features of the first size into a preset second feature extraction model to obtain target object fusion features of the second size; inputting the target object fusion features, the target object fusion features of the first size and the target object fusion features of the second size into the defect detection model to obtain the target defect detection results”, so as to help improve the accuracy of the output defect detection results.
[0083] refer to Figure 3A , the method specifically comprises the following steps:
[0084] S310. Extracting texture surface features and depth defect features of preset dimensions corresponding to the object to be detected based on image information of the object to be detected; wherein the image information includes texture image information and depth image information.
[0085] S320, fusing the texture surface features of each preset dimension with the depth defect features of the corresponding dimension respectively to obtain fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features.
[0086] S330, connecting the fusion features of each preset dimension to obtain the fusion features of the target object.
[0087] S340: Input the target object fusion feature into a preset first feature extraction model to obtain a target object fusion feature of a first size.
[0088] S350: Input the target object fusion features of the first size into a preset second feature extraction model to obtain the target object fusion features of the second size.
[0089] S360: Input the target object fusion feature, the target object fusion feature of the first size, and the target object fusion feature of the second size into the defect detection model to obtain the target defect detection result.
[0090] The first feature extraction model may be a trained network model for obtaining fusion features of a target object of a first size. The second feature extraction model may be a trained network model for obtaining fusion features of a target object of a second size. The original size, second size, and third size corresponding to the fusion features of the corresponding target object may be different sizes. The defect detection model may detect defects on the surface of the object to be detected and generate a network model corresponding to the defect detection results.
[0091] In an embodiment of the present invention, by generating new target object fusion features of the first size and the second size corresponding to the corresponding target object fusion features, and using the target object fusion features of the original size, the target object fusion features of the first size, and the target object fusion features of the second size as inputs of the corresponding defect detection model, it is possible to fully pay attention to the feature information of different granularities on the surface of the corresponding object to be detected during the corresponding defect detection process, thereby helping to improve the accuracy of the corresponding defect detection.
[0092] For ease of understanding, refer to Figure 3B A specific example of an algorithm for a method for detecting surface defects of an object is provided.
[0093] Figure 3B This is an algorithm schematic diagram of a surface defect method. Figure 3BAs shown, based on the input image information, the texture depth fusion module can be used to obtain the three outputs C3, C4, and C5 of the backbone network BackBone using the feature extraction block in the backbone network BackBone. Among them, C3 can be the output of the texture depth fusion module, that is, it can be the target object fusion feature of the original size. C4 can be the target object fusion feature of the first size, and C5 can be the target object fusion feature of the second size. C4 can be obtained based on C3, and C5 can be obtained based on C4. The original size can be 1*256*80*80, the first size can be 1*512*40*40, and the second size can be 1*1024*20*20. The neck network Neck and the head network Head can be included in the above-mentioned defect detection model, wherein Neck can obtain fused features O3, O4, O5 at different scales based on the feature fusion processing of the input C3, C4 and C5, wherein the size of O3 can be 1*256*80*80, the size of O4 can be 1*256*40*40, and the size of O5 can be 1*256*20*20. The above-mentioned O3, O4 and O5 are input into the Head, and the Head can select the defect location task or the defect segmentation task according to the task requirements for the output O3, O4, O5 of the different scales, and perform regression to output the corresponding detection results. It should be noted that the sizes of the various features involved above are only for exemplary purposes and should not be understood as specific limitations of the present invention.
[0094] Figure 4 The fourth embodiment of the present invention provides a device for detecting surface defects of an object. This embodiment is applicable to the case of defect detection on complex surface scenes such as curved surfaces or irregular surfaces. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 4 As shown, the device comprises:
[0095] The texture depth feature extraction module 410 is used to extract the texture surface features and depth defect features of each preset dimension corresponding to the object to be detected based on the image information of the object to be detected; wherein the image information includes texture image information and depth image information;
[0096] The fusion feature acquisition module 420 is used to fuse the texture surface features of each preset dimension with the depth defect features of the corresponding dimension to obtain the fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features;
[0097] The target object fusion feature acquisition module 430 is used to connect the fusion features of each preset dimension to obtain the target object fusion feature;
[0098] The target defect detection result acquisition module 440 is used to input the target object fusion features into a pre-trained defect detection model to obtain the corresponding target defect detection results.
[0099] The technical solution of the embodiment of the present invention presets different dimensions and extracts the texture surface features and depth defect features of the surface of the object to be detected based on each preset dimension, so as to further fuse the texture surface features and depth defect features of each preset dimension, and then connect the fused features of each preset dimension to obtain high-precision fusion features of the surface of the object to be detected including corresponding texture information and depth information. It can output higher-precision defect detection results using the corresponding trained defect detection model, thereby avoiding the complicated pre-processing process in the existing defect detection process, and can be applicable to the detection of corresponding defects in complex surface scenes such as curved surfaces and irregular surfaces. It has high ease of use and robustness, and at the same time, it also realizes the full utilization of the texture information and depth information of the corresponding defects, further improving the accuracy of the corresponding defect detection.
[0100] Optionally, the texture depth feature extraction module 410 may include:
[0101] A shallow feature extraction unit, configured to obtain shallow texture features corresponding to the object to be detected based on texture image information of the object to be detected and a preset first texture extraction model; and to obtain shallow depth features corresponding to the object to be detected based on depth image information of the object to be detected and a preset first depth extraction model;
[0102] A local feature extraction unit, used to obtain the local texture feature corresponding to the object to be detected according to the preset second texture extraction model and the shallow texture feature; and to obtain the local depth feature corresponding to the object to be detected according to the preset second depth extraction model and the boundary depth feature;
[0103] The global feature extraction unit is used to obtain the global texture feature corresponding to the object to be detected according to the preset third texture extraction model and the local texture feature; and to obtain the global depth feature corresponding to the object to be detected according to the preset third depth extraction model and the local depth feature.
[0104] Optionally, the fusion features of each preset dimension may include boundary fusion features, local fusion features and global fusion features; accordingly, the fusion feature acquisition module 420 may include:
[0105] A first fusion feature acquisition unit is used to fuse the first texture feature and the first depth feature according to a preset first feature fusion model to obtain a first fusion feature; wherein the first texture feature is a shallow texture feature, the first depth feature is a shallow depth feature, and the first fusion feature is a boundary fusion feature;
[0106] The second fusion feature acquisition unit is used to fuse the second texture feature and the second depth feature according to a preset second feature fusion model to obtain a second fusion feature; wherein the second texture feature includes a local texture feature and a global texture feature, the second depth feature includes a local depth feature and a global depth feature, and the second fusion feature includes a local fusion feature and a global fusion feature.
[0107] Optionally, the target object fusion feature acquisition module 430 may include:
[0108] A target boundary fusion feature acquisition unit, used for performing multi-scale processing on the boundary fusion feature to obtain a target boundary fusion feature;
[0109] A target local fusion feature acquisition unit, used for performing multi-scale processing on the local fusion feature to obtain a target local fusion feature;
[0110] A comprehensive fusion feature acquisition unit, used to integrate the target local fusion feature and the global fusion feature based on a preset feature integration model, and filter out redundant information to obtain a comprehensive fusion feature;
[0111] The target object fusion feature acquisition unit is used to connect the target boundary fusion feature and the comprehensive fusion feature to obtain the target object fusion feature.
[0112] Optionally, the first fusion feature acquisition unit may include:
[0113] A first boundary fusion feature acquisition subunit is used to fuse the shallow texture feature and the shallow depth feature based on a preset boundary feature fusion model to obtain a first boundary fusion feature including both the shallow texture feature and the shallow depth feature;
[0114] A first boundary fusion enhancement feature acquisition subunit is used to perform a first enhancement process on the first boundary feature to obtain a first boundary fusion enhancement feature;
[0115] A multi-scale boundary feature extraction subunit, used for inputting the first boundary fusion enhancement feature into a preset multi-scale boundary feature extraction model to obtain a multi-scale second boundary fusion feature and a target boundary contour trajectory;
[0116] The boundary fusion feature acquisition subunit is used to perform a second enhancement process on the second boundary fusion feature and the target boundary contour trajectory to obtain the boundary fusion feature.
[0117] Optionally, the second fusion feature acquisition unit may include:
[0118] A common fusion feature acquisition subunit, used to fuse the second texture feature and the second depth feature according to a preset common feature fusion model to obtain a common fusion feature;
[0119] A weighted fusion feature acquisition subunit, configured to perform weighted fusion processing on the texture feature and the depth feature based on a preset first weighted feature fusion model and a second weighted feature fusion model, respectively, to obtain a first weighted fusion feature and a second weighted fusion feature;
[0120] a candidate fusion feature acquisition subunit, configured to fuse the first weighted fusion feature and the shared fusion feature according to a first feature fusion submodel to obtain a first candidate fusion feature; and to fuse the second weighted fusion feature and the shared fusion feature according to a second feature fusion submodel to obtain a second candidate fusion feature;
[0121] The second fusion feature acquisition subunit is used to connect the first candidate fusion feature and the second candidate fusion feature to obtain the second fusion feature.
[0122] Optionally, the target defect detection result acquisition module 440 may include:
[0123] A first size acquisition unit, used for inputting the target object fusion feature into a preset first feature extraction model to obtain the target object fusion feature of a first size;
[0124] A second size acquisition unit, used for inputting the target object fusion feature of the first size into a preset second feature extraction model to obtain the target object fusion feature of the second size;
[0125] The target defect detection result acquisition unit is used to input the target object fusion feature, the target object fusion feature of the first size and the target object fusion feature of the second size into the defect detection model to obtain the target defect detection result.
[0126] The object surface defect detection device provided in the embodiment of the present invention can execute any object surface defect detection method provided in the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing each object surface defect detection method. For the contents not described in detail in the embodiment of the present invention, reference can be made to the description in the embodiment of any object surface defect detection method in the embodiment of the present invention.
[0127] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.
[0128] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0129] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0130] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0131] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for detecting surface defects of an object.
[0132] In some embodiments, the method for detecting surface defects of an object may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for detecting surface defects of an object described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for detecting surface defects of an object in any other appropriate manner (e.g., by means of firmware).
[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0134] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0135] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0139] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.
[0140] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.
[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.
[0142] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for detecting surface defects of an object, characterized in that: include: Based on the image information of the object to be detected, extracting the texture surface features and depth defect features of each preset dimension corresponding to the object to be detected; wherein the image information includes texture image information and depth image information; The texture surface features of each preset dimension are respectively fused with the depth defect features of the corresponding dimension to obtain fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features; Connect the fusion features of each preset dimension to obtain the fusion features of the target object; The target object fusion features are input into a pre-trained defect detection model to obtain the corresponding target defect detection results.
2. The method according to claim 1, characterized in that Based on the image information of the object to be detected, extracting the texture surface features of each preset dimension corresponding to the object to be detected, including: Acquire shallow texture features corresponding to the object to be detected based on the texture image information of the object to be detected and a preset first texture extraction model; and acquire shallow depth features corresponding to the object to be detected based on the depth image information of the object to be detected and a preset first depth extraction model; According to the preset second texture extraction model and the shallow texture features, the local texture features corresponding to the object to be detected are obtained; and according to the preset second depth extraction model and the shallow depth features, the local depth features corresponding to the object to be detected are obtained; According to the preset third texture extraction model and the local texture features, the global texture features corresponding to the object to be detected are obtained; and according to the preset third depth extraction model and the local depth features, the global depth features corresponding to the object to be detected are obtained.
3. The method according to claim 2, characterized in that The fusion features of each preset dimension include boundary fusion features, local fusion features and global fusion features; accordingly, the texture surface features of each preset dimension are fused with the depth defect features of the corresponding dimension to obtain the fusion features of each preset dimension, including: According to a preset first feature fusion model, the first texture feature and the first depth feature are fused to obtain a first fused feature; wherein the first texture feature is a shallow texture feature, the first depth feature is a shallow depth feature, and the first fused feature is a boundary fusion feature; According to a preset second feature fusion model, the second texture feature and the second depth feature are fused to obtain a second fusion feature; wherein the second texture feature includes a local texture feature and a global texture feature, the second depth feature includes a local depth feature and a global depth feature, and the second fusion feature includes a local fusion feature and a global fusion feature.
4. The method according to claim 3, characterized in that The step of connecting the candidate fusion features to obtain the target object fusion features includes: Performing multi-scale processing on the boundary fusion features to obtain target boundary fusion features; Performing multi-scale processing on the local fusion features to obtain target local fusion features; Based on a preset feature integration model, the target local fusion feature and the global fusion feature are integrated, and redundant information is filtered out to obtain a comprehensive fusion feature; The target boundary fusion feature and the comprehensive fusion feature are connected to obtain the target object fusion feature.
5. The method according to claim 3, characterized in that: The step of fusing the first texture feature and the first depth feature according to the preset first feature fusion model to obtain the first fusion feature includes: Based on a preset boundary feature fusion model, the shallow texture feature and the shallow depth feature are fused to obtain a first boundary fusion feature including both the shallow texture feature and the shallow depth feature; Performing a first enhancement process on the first boundary feature to obtain a first boundary fusion enhancement feature; Inputting the first boundary fusion enhancement feature into a preset multi-scale boundary feature extraction model to obtain a multi-scale second boundary fusion feature and a target boundary contour trajectory; A second enhancement process is performed on the second boundary fusion feature and the target boundary contour trajectory to obtain the boundary fusion feature.
6. The method according to claim 3, characterized in that The step of fusing the second texture feature and the second depth feature according to a preset second feature fusion model to obtain a second fusion feature includes: According to a preset common feature fusion model, the second texture feature and the second depth feature are fused to obtain a common fusion feature; Based on a preset first weighted feature fusion model and a second weighted feature fusion model, weighted fusion processing is performed on the texture feature and the depth feature respectively to obtain a first weighted fusion feature and a second weighted fusion feature; According to the first feature fusion sub-model, the first weighted fusion feature and the shared fusion feature are fused to obtain a first candidate fusion feature; and according to the second feature fusion sub-model, the second weighted fusion feature and the shared fusion feature are fused to obtain a second candidate fusion feature; The first candidate fusion feature and the second candidate fusion feature are connected to obtain the second fusion feature.
7. The method according to any one of claims 1 to 6, characterized in that The step of inputting the target object fusion features into a pre-trained defect detection model to obtain a corresponding target defect detection result includes: Inputting the target object fusion feature into a preset first feature extraction model to obtain a target object fusion feature of a first size; Inputting the fused features of the target object of the first size into a preset second feature extraction model to obtain the fused features of the target object of the second size; The target object fusion feature, the target object fusion feature of the first size and the target object fusion feature of the second size are input into the defect detection model to obtain the target defect detection result.
8. A device for detecting surface defects of an object, characterized in that: include: A texture depth feature extraction module, used to extract texture surface features and depth defect features of each preset dimension corresponding to the object to be detected based on the image information of the object to be detected; wherein the image information includes texture image information and depth image information; A fusion feature acquisition module is used to fuse the texture surface features of each preset dimension with the depth defect features of the corresponding dimension to obtain the fusion features of each preset dimension; wherein there is a one-to-one correspondence between the texture surface features and the preset dimensions of the depth defect features; A target object fusion feature acquisition module is used to connect the fusion features of each preset dimension to obtain the target object fusion feature; The target defect detection result acquisition module is used to input the target object fusion features into a pre-trained defect detection model to obtain the corresponding target defect detection results.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for detecting surface defects of an object according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for detecting surface defects of an object according to any one of claims 1 to 7 when executed.
Citation Information
Patent Citations
Intelligent fan blade defect detection method based on bimodal fusion
CN114429457A
Surface defect detection method and equipment based on feature fusion
CN114550021A
Machine vision steel wire rope surface defect detection method based on multi-feature fusion
CN116843626A
Glass detection method, device, equipment and medium
CN117252845A
Image processing method and device, intelligent terminal and computer readable storage medium
CN117291811A
