Object surface defect detection method and device, computer readable storage medium and electronic equipment

Abstract

The application provides an object surface defect detection method and device, a computer readable storage medium and an electronic device. The method comprises: acquiring image data of an object surface to be detected, grouping the image data to obtain a plurality of groups of image data; constructing a grouped convolutional network model, wherein the grouped convolutional network model is obtained by training a plurality of groups of data, each group of data in the plurality of groups of data comprising historical image data and labels corresponding to defect information in the historical image data; and applying the grouped convolutional network model to process the plurality of groups of image data to obtain defect information, wherein the defect information is information representing defects in the image data. The application solves the problem that defects on the outer surface of an object cannot be accurately identified in the prior art.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method, apparatus, computer-readable storage medium, and electronic device for detecting defects on the surface of an object. Background Technology

[0002] In modern industrial production, product quality control is a crucial step in ensuring that the final product meets high standards. Surface defect detection, as an important component of quality control, is widely used in various fields such as automobile manufacturing, electronic equipment, metal processing, plastic products, and textiles. For example, during the production of air conditioner outdoor unit covers, various defects such as scratches, dents, pores, and cracks may occur, affecting the product's appearance and functional stability. Traditional detection methods, such as manual inspection and rule-based image processing techniques, suffer from low efficiency, poor adaptability, and susceptibility to subjective factors, making it difficult to meet the high-efficiency and high-precision demands of modern industrial production.

[0003] With the development of deep learning technology, convolutional neural networks (CNNs) have been applied to defect detection due to their powerful feature extraction capabilities. However, the fixed shape and size of the convolutional kernels in traditional CNNs limit their adaptability to multi-scale and multi-directional defects. Especially when processing large-scale images, the computational complexity is high, which may not be suitable for the real-time requirements of industrial sites, resulting in the inability to identify surface defects of objects in a timely manner. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, computer-readable storage medium, and electronic device for detecting defects on the surface of an object, so as to at least solve the problem that the prior art cannot accurately identify defects on the outer surface of an object.

[0005] To achieve the above objectives, according to one aspect of this application, a defect detection method is provided, comprising: acquiring image data of the surface of an object to be detected; grouping the image data to obtain multiple sets of image data; constructing a grouped convolutional network model, wherein the grouped convolutional network model is trained using multiple sets of data, each set of data including: historical image data and labels corresponding to defect information in the historical image data; and applying the grouped convolutional network model to process the multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0006] Optionally, the grouped convolutional network model includes at least a grouped convolutional network and a serpentine convolutional network. Applying the grouped convolutional network model to process multiple sets of image data includes: determining the computational cost of convolutional features for each set of image data, obtaining model parameters of an initial convolutional network model, and determining a feature threshold based on the model parameters. The initial convolutional network model includes a regular convolutional network. When the computational cost of convolutional features is greater than or equal to the feature threshold, convolutional processing is performed on the set of image data using the grouped convolutional network of the grouped convolutional network model, and the processing direction of the convolutional kernels in the grouped convolution is determined using the serpentine convolutional network of the grouped convolutional network model.

[0007] Optionally, obtaining the model parameters of the initial convolutional network model includes: acquiring historical real image data of the surface of the object to be detected; simulating the historical real image data under different lighting conditions to obtain historical virtual image data; training the initial convolutional network model based on the historical real image data and the historical virtual image data, so that the initial convolutional network model can identify defect information in the historical real image data and the historical virtual image data; and obtaining the number of input channels, the number of output channels, the size of the convolutional kernel of the ordinary convolutional network, and the height and width of the input image of the initial convolutional network model after training, to obtain the model parameters of the initial convolutional network model.

[0008] Optionally, determining the feature threshold based on the model parameters includes: obtaining the number of groups of the image data of the surface of the object to be detected; calculating the product of the number of input channels, the size of the convolution kernel, and the height and width of the input image to obtain a preset product; and calculating the ratio of the preset product to the number of groups to obtain the feature threshold.

[0009] Optionally, the grouped convolutional network model further includes a regular convolutional network. Applying the grouped convolutional network model to process multiple sets of image data further includes: when the computational cost of the convolutional features is less than the feature threshold, performing convolution processing on the set of image data through the regular convolutional network of the grouped convolutional network model.

[0010] Optionally, determining the processing direction of the convolution kernel in the grouped convolution through the serpentine convolutional network of the grouped convolutional network model includes: calculating the offset of each preset direction through the convolution kernel of the serpentine convolutional network; and determining the direction of the grouped convolution as the preset direction corresponding to the largest offset.

[0011] Optionally, the image data is grouped to obtain multiple sets of image data, including: obtaining the number of input channels of the grouped convolutional network model; and uniformly grouping the image data according to the number of input channels of the grouped convolutional network model to obtain multiple sets of image data.

[0012] According to another aspect of this application, a defect detection device is provided, comprising: a grouping unit for acquiring image data of the surface of an object to be detected, and grouping the image data to obtain multiple sets of image data; a construction unit for constructing a grouped convolutional network model, wherein the grouped convolutional network model is trained using multiple sets of data, each set of data including: historical image data and labels corresponding to defect information in the historical image data; and a processing unit for applying the grouped convolutional network model to process the multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0013] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the aforementioned defect detection methods.

[0014] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the defect detection methods described above.

[0015] By applying the technical solution of this application, image data of the surface of an object to be detected is acquired, and the image data is grouped to obtain multiple sets of image data. A grouped convolutional network model is constructed, wherein the grouped convolutional network model is trained on multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data. The grouped convolutional network model is applied to process the multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data. Compared with the prior art, which cannot accurately identify defects on the outer surface of an object, this application improves the convolution efficiency by performing grouped convolution on the image data through the grouped convolutional network in the grouped convolutional network model, and determines the direction of grouped convolution through a serpentine convolutional network to perform directional convolution processing on the image data features. Therefore, it can solve the problem of not being able to accurately identify defects on the outer surface of an object in the prior art, and achieve the effect of accurately identifying defects on the outer surface of an air conditioner. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A schematic flowchart of a defect detection method according to an embodiment of this application is shown;

[0018] Figure 2 A specific method for detecting surface defects of an object is shown according to an embodiment of this application;

[0019] Figure 3 A schematic diagram of a grouped convolutional network according to an embodiment of this application is shown;

[0020] Figure 4 A schematic diagram of a thresholded grouped convolution effect provided according to an embodiment of this application is shown;

[0021] Figure 5 A schematic diagram of a directional snake convolution operation provided according to an embodiment of this application is shown;

[0022] Figure 6 A structural block diagram of a defect detection device provided according to an embodiment of this application is shown. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] As described in the background section, existing technologies cannot accurately identify defects on the outer surface of objects. To address the problem of inaccurate identification of defects on the outer surface of objects, embodiments of this application provide a method, apparatus, computer-readable storage medium, and electronic device for detecting defects on the outer surface of objects.

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0028] This embodiment provides a defect detection method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 1 This is a flowchart of a defect detection method according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0030] Step S201: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data;

[0031] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0032] Step S202: Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0033] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0034] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the processing direction of the convolutional kernels in the grouped convolutions. By dynamically adjusting the direction and shape of the kernels, the convolution operation can be performed along the direction of feature density, thereby more effectively extracting feature information from defective regions.

[0035] Step S203: Apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0036] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0037] This embodiment acquires image data of the surface of an object to be detected, groups the image data to obtain multiple sets of image data, and constructs a grouped convolutional network model. This model is trained using multiple sets of data, each set including historical image data and labels corresponding to defect information within those historical data. The grouped convolutional network model is then applied to process the multiple sets of image data to obtain defect information, which characterizes defects in the image data. Compared to existing technologies that cannot accurately identify defects on the outer surface of objects, this application improves convolution efficiency by using a grouped convolutional network within the grouped convolutional network model to perform grouped convolution on the image data. Furthermore, a serpentine convolutional network is used to determine the direction of the grouped convolutions, enabling directional convolution processing of the image data's features. Therefore, this approach solves the problem of inaccurate identification of defects on the outer surface of objects in existing technologies, achieving accurate identification of defects on the outer surface of air conditioners.

[0038] In specific implementation, the grouped convolutional network model includes at least a grouped convolutional network and a serpentine convolutional network. Step S202 can be implemented through the following steps: Step S2021: Determine the computational cost of the convolutional features for each group of image data and obtain the model parameters of the initial convolutional network model. Determine the feature threshold based on the model parameters, wherein the initial convolutional network model includes a regular convolutional network; Step S2022: When the computational cost of the convolutional features is greater than or equal to the feature threshold, perform convolution processing on the group of image data using the grouped convolutional network of the grouped convolutional network model, and determine the direction of the grouped convolution using the serpentine convolutional network. This method improves the computational efficiency of the model by reducing unnecessary feature interactions through the grouped convolutional network, while the serpentine convolutional network enhances the model's ability to capture features, especially complex and variable defect features.

[0039] Specifically, the computational cost of convolutional features for each group is determined, and a feature threshold is calculated based on the model parameters of the initial convolutional network model. The application of the feature threshold allows the grouped convolutional network to dynamically adjust its processing flow. For feature maps that exceed the feature threshold, the model will assign them to specific groups for grouped convolution, while for feature maps that do not reach the threshold, the model may choose not to perform grouped convolution, directly maintaining their original state, or processing them through a simple linear transformation to save computational resources.

[0040] In some optional implementations, step S2021 can be achieved through the following steps: Step S20211: Obtain historical real image data of the surface of the object to be detected, and simulate the historical real image data under different lighting conditions to obtain historical virtual image data; Step S20212: Train the initial convolutional network model based on the historical real image data and the historical virtual image data, so that the initial convolutional network model can identify defect information in the historical real image data and the historical virtual image data; Step S20213: Obtain the number of input channels, the number of output channels, the size of the convolution kernel of the ordinary convolutional network, and the height and width of the input image of the initial convolutional network model after training, to obtain the model parameters of the initial convolutional network model. This method obtains the model parameters of the initial convolutional network model through the above steps, so that the convolution parameter threshold of the feature map can be accurately determined based on the model parameters, and further determines whether to perform grouped convolution.

[0041] Specifically, grouped convolution is determined by a threshold and can also be called thresholded grouped convolution. The principle is to decompose ordinary convolution into a set of identity mapping convolutions and multiple accelerated grouped convolutions. First, assume the number of output channels is n, and a total of s groups are needed. The number of channels after the first part of convolution is m, where m = n / s. The output channels obtained through the second part are m × (s-1). Then, in the third step, the feature maps obtained from the previous two steps are added together according to the number of channels (dim = 1), thus obtaining the output feature map with the following number of channels: Calculations show that the number of output channels is still n. Therefore, the grouped convolution module has a significant speedup effect compared to traditional convolution kernels. The specific speedup ratio is calculated as follows: assuming the number of output feature maps is n, c represents the number of input channels, h and w represent the height and width of the input image, respectively, the size of the ordinary convolution kernel is r×r, and the size of the convolution kernel performing linear transformation is d×d, r s For the theoretical speedup ratio, r c Let r be the compression ratio, assuming r = d, s << c (the second term in the denominator is for accelerating convolution generation, which does not need to use all input feature channels, so it does not need to be multiplied by c). The calculation process is as follows:

[0042] The computational cost of averaging feature map parameters is shown in the formula: Params Average =(n in *n out *r*r*h*w) / s,n in n represents the number of input channels of the feature map. out This represents the number of output channels in the feature map. It is calculated by grouping the total number of feature map parameters and then applying a threshold to each group. If the number of features is less than the average value (Params), then... AverageIf no grouped convolution is performed, the subsequent directional snake convolution will not be handled in any special way.

[0043] In some optional implementations, step S2021 can be achieved through the following steps: obtaining the number of groups of the image data of the surface of the object to be detected; calculating the product of the number of input channels, the size of the convolution kernel, and the height and width of the input image to obtain a preset product; calculating the ratio of the preset product to the number of groups to obtain the feature threshold. This method, through the above steps, ensures both the effective extraction of key features by the model and avoids over-processing of background or redundant features.

[0044] Specifically, the choice of the number of groups directly affects the processing efficiency of feature maps and the allocation of computational resources. For example, the more groups there are, the fewer feature map channels each group processes, which theoretically reduces computation and improves model speed, but may also affect the comprehensive extraction of features. The preset product refers to the product of the number of input channels, the number of output channels, the size of the convolution kernel, and the height and width of the input image, n. in *n out *r*r*h*w. Calculate feature thresholds: Params Average =(n in *n out The feature threshold (T) is calculated by dividing the preset product by the number of groups (s). The feature threshold (T) essentially defines the processing "value" or "cost" of each feature map group, used to determine which feature maps should undergo more complex convolution processing, and which feature maps can be simplified, such as being skipped or processed by a simple linear transformation.

[0045] In some optional implementations, the grouped convolutional network model further includes a regular convolutional network. Step S2022 can be implemented through the following steps: when the computational cost of the convolutional features is less than the feature threshold, the group of image data is convolved using the regular convolutional network of the grouped convolutional network model. This method performs regular convolution processing on image data with fewer feature convolutional parameters than the feature convolutional parameter threshold through the above steps, which can further reduce the computational cost.

[0046] Specifically, a convolutional feature computation cost less than the feature threshold indicates a small number of features in the group, potentially corresponding to the background portion of the image—meaning this portion is not a defect. Therefore, this portion can be treated with only ordinary convolution. Ordinary convolutional networks use a single convolutional kernel sliding across the entire input. While not as efficient as grouped convolutions, this is sufficient to capture necessary information for simple or open feature regions while avoiding unnecessary computational overhead. Whether features are processed by grouped convolutional networks or ordinary convolutional networks, they ultimately need to be integrated to form a complete image feature description. In practical applications, the feature threshold can be dynamically adjusted to better adapt to different input images and application scenarios. For example, when processing images with richer textures, the threshold can be appropriately lowered to ensure that no small but important features are missed.

[0047] In some alternative implementations, step S2022 can be achieved through the following steps: calculating the offset of each preset direction using the convolution kernel of the serpentine convolutional network; determining the direction of the grouped convolution as the preset direction corresponding to the largest offset. This method, through the above steps, maintains computational efficiency while dynamically adjusting the convolution direction according to the actual distribution of features, thereby improving the detection capability for complex surface defects such as cracks, scratches, and dents.

[0048] Specifically, the directional snake convolution operation incorporates continuity constraints into the design of the convolution kernel. Each convolution position uses its previous position as a reference, freely choosing its swing direction, thus ensuring continuity of perception while allowing for free choice. The convolution operation of features moves based on the previous result (the direction shown by the straight line in the figure). However, the orientation and perspective of defect features are never unique, so fusing multi-view features in the design is an inevitable choice. However, fusing more features results in a larger network load, and so many features inevitably lead to redundancy. Therefore, the original snake convolution operation uses a grouping and random dropout strategy during the feature fusion training process, which alleviates the pressure on network memory to some extent and avoids the model falling into overfitting. However, this patent uses a thresholded grouping convolution strategy to alleviate this pressure for efficiency, achieving a balance between speed and accuracy. Before processing begins, a series of preset directions are determined for the snake convolution network. These directions can be based on the four basic directions of the Cartesian coordinate system (up, down, left, right) and their combinations, or more detailed angular directions, depending on the application requirements and the expected distribution of defect features. The offset is calculated in each preset direction by the convolution kernel of the snake convolution network. The offset here reflects the feature density or importance in a specific direction of the feature map. It can be the response strength of the convolution kernel in that direction, the gradient change of the feature map, or some form of attention score. Among all preset directions, the direction with the largest offset is selected as the direction for the next group convolution. This means that the model will prioritize convolution operations along the direction where the feature distribution is densest or the changes are most significant, thereby more effectively capturing and analyzing defect information. Based on the selected direction, the shape and orientation of the convolution kernel for group convolution are dynamically adjusted to ensure that the convolution operation can be performed along the feature flow direction, thereby improving the accuracy and efficiency of feature extraction. After completing the convolution processing in a specific direction, the feature map is updated and fed back to the serpentine convolutional network, and the offset calculation and direction selection are performed again until the features are fully extracted or the predetermined number of iterations is reached. The feature maps in all directions are integrated into a comprehensive representation for use by subsequent classification or localization modules.

[0049] In some alternative implementations, step S201 can be achieved through the following steps: Step S2011: Obtain the number of input channels of the grouped convolutional network model; Step S2012: Divide the image data into multiple groups according to the number of input channels of the grouped convolutional network model. This method groups the images through the above steps, ensuring that the image data can be effectively grouped and processed according to the number of input channels of the model. This step not only optimizes computational efficiency but also promotes the performance improvement of the model when processing high-dimensional data.

[0050] Specifically, the image data is uniformly grouped according to the number of input channels in the grouped convolutional network model, resulting in multiple groups of image data. Based on the model design requirements and the computing power of the target hardware, the number of groups for the input channels is determined. For example, if the model has 96 input channels, it can be divided into 32 groups, each containing 3 channels, allowing the model to process features in parallel across different groups. The input image data is uniformly divided according to the determined number of groups. For each group, the corresponding channels of the image data are separated to form independent data groups for subsequent processing. After grouping the image data, each group of data is convolved using the convolution kernel corresponding to that group in the model. This grouping processing method can significantly reduce the consumption of computing resources, especially for networks with a large number of input channels, because the convolution operation of each group can be executed in parallel. After completing the convolution processing of all groups, the resulting feature maps are recombined to form a complete feature output.

[0051] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the object surface defect detection method of this application will be described in detail below with reference to specific embodiments.

[0052] This embodiment relates to a specific method for detecting defects on the surface of an object, such as... Figure 2 As shown, it includes the following steps:

[0053] Step S1: Defect dataset (image data of the surface of the object to be detected), extract the defect model;

[0054] Step S2: Construct a virtual dataset;

[0055] Step S3: Traditional feature extraction network;

[0056] Step S4: Pre-train weights;

[0057] Step S5: The improved feature extraction network, i.e., the grouped convolutional network, is as follows: Figure 3 As shown, P1, P2, P3, and P4 are abbreviations for different feature layers of the network. P1 to P4 are thresholded grouped convolutions of 3×3 Conv, respectively. The third column shows the size of the image after being passed through layers in the feature network, which are 640×640, 320×320, 160×160, and 80×80, respectively. A schematic diagram of the thresholded grouped convolution effect is shown below. Figure 4 As shown, ordinary convolution is the process of processing an image input to obtain an image output; thresholded group convolution is the process of processing an image input through a convolution operation from group 1 to group s to obtain the convolution results of group 1 to group s, thus obtaining the final image output.

[0058] Step S6: Train the defect inference model using real defect image data and real image data;

[0059] Step S7: Defect detection, directional snake convolution operation as follows Figure 5 As shown, this application uses only directional convolution in the x and y directions to facilitate rapid matching and implementation of defect localization. The coordinate calculation method in the x direction is K. i+c =(x i+c ,y i+c )=(x i +c,y i+c +∑Δy i+c The method for calculating the coordinates in the y-direction is K. j+c =(x j+c ,y i+c )=(x j +∑Δx j+c ,y i +c).

[0060] This application also provides a surface defect detection device. It should be noted that the surface defect detection device of this application can be used to execute the surface defect detection method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0061] The following describes the object surface defect detection device provided in the embodiments of this application.

[0062] Figure 6 This is a schematic diagram of a surface defect detection device according to an embodiment of this application. Figure 6 As shown, the device includes:

[0063] Grouping unit 10 is used to acquire image data of the surface of the object to be detected, and to group the image data to obtain multiple groups of image data.

[0064] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0065] The construction unit 20 is used to construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0066] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0067] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the direction of the grouped convolutions. By dynamically adjusting the direction and shape of the convolution kernels, the convolution operation is performed along the direction of feature density, thereby more effectively extracting feature information from defect regions.

[0068] The processing unit 30 is used to apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information that characterizes defects in the image data.

[0069] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0070] This embodiment acquires image data of the surface of an object to be detected, groups the image data to obtain multiple sets of image data, and constructs a grouped convolutional network model. This model is trained using multiple sets of data, each set including historical image data and labels corresponding to defect information within those historical data. The grouped convolutional network model is then applied to process the multiple sets of image data to obtain defect information, which characterizes defects in the image data. Compared to existing technologies that cannot accurately identify defects on the outer surface of objects, this application improves convolution efficiency by using a grouped convolutional network within the grouped convolutional network model to perform grouped convolution on the image data. Furthermore, a serpentine convolutional network is used to determine the direction of the grouped convolutions, enabling directional convolution processing of the image data's features. Therefore, this approach solves the problem of inaccurate identification of defects on the outer surface of objects in existing technologies, achieving accurate identification of defects on the outer surface of air conditioners.

[0071] In specific implementation, the grouped convolutional network model includes at least a grouped convolutional network and a serpentine convolutional network. Step S202 can be implemented through the following steps: Step S2021: Determine the computational cost of the convolutional features for each group of image data and obtain the model parameters of the initial convolutional network model. Determine the feature threshold based on the model parameters, wherein the initial convolutional network model includes a regular convolutional network; Step S2022: When the computational cost of the convolutional features is greater than or equal to the feature threshold, perform convolution processing on the group of image data using the grouped convolutional network of the grouped convolutional network model, and determine the direction of the grouped convolution using the serpentine convolutional network. This device improves the computational efficiency of the model by reducing unnecessary feature interactions through the grouped convolutional network, while the serpentine convolutional network enhances the model's ability to capture features, especially complex and variable defect features.

[0072] Specifically, the number of features in each group is determined, and a threshold for the feature convolution parameters is calculated based on the model parameters of the initial convolutional network model. The application of the feature convolution parameter threshold allows the grouped convolutional network to dynamically adjust its processing flow. For feature convolution groups that exceed the feature convolution parameter threshold, and for feature maps that do not reach the threshold, the model may choose not to perform grouped convolution and directly maintain their original state, or process them through a simple linear transformation to save computational resources.

[0073] In some optional implementations, step S2021 can be achieved through the following steps: Step S20211: Obtain historical real image data of the surface of the object to be detected, and simulate the historical real image data under different lighting conditions to obtain historical virtual image data; Step S20212: Train the initial convolutional network model based on the historical real image data and the historical virtual image data, so that the initial convolutional network model can identify defect information in the historical real image data and the historical virtual image data; Step S20213: Obtain the number of input channels, the number of output channels, the size of the convolution kernel of the ordinary convolutional network, and the height and width of the input image of the initial convolutional network model after training, to obtain the model parameters of the initial convolutional network model. This device obtains the model parameters of the initial convolutional network model through the above steps, which allows for accurate determination of feature thresholds based on the model parameters, and further determination of whether to perform grouped convolution.

[0074] Specifically, grouped convolution is determined by a threshold and can also be called thresholded grouped convolution. The principle is to decompose ordinary convolution into a set of identity mapping convolution and multiple accelerated grouped convolutions. First, we can assume that the number of output channels is n, and a total of s groups are needed. The number of channels after the first part of convolution is m, where m = n / s. The output channels obtained in the second part are m×(s-1). Then, in the third step, the feature maps obtained in the previous two steps are added together according to the number of channels (dim=1). This gives us the output feature map with the number of channels: The calculation shows that the number of output channels is still n. Therefore, the grouped convolution module has a significant speedup effect compared to traditional convolution kernels. The specific speedup ratio is calculated as follows: Assuming the number of output feature maps is n, c represents the number of input channels, h and w represent the height and width of the input image, the size of the ordinary convolution kernel is r×r, the size of the convolution kernel for linear transformation is d×d, rs is the theoretical speedup ratio, rc is the parameter compression ratio, assuming r=d, s<<c (the second term in the denominator is for accelerating convolution generation, which does not need to use all input feature channels, so it does not need to be multiplied by c). The calculation process is as follows: The computational cost of the average value of the feature map convolution parameters is shown in the formula: Params Average =(n in *n out *r*r*h*w) / s,n in n represents the number of input channels. out This indicates the number of output channels. It is calculated by grouping the total number of convolutional parameters in the feature map and then dividing each group according to a threshold. If the number of convolutional parameters in the feature map is less than the average value (Params), then... Average If no grouped convolution is performed, the subsequent directional snake convolution will not be handled in any special way.

[0075] In some optional implementations, step S2021 can be achieved through the following steps: obtaining the number of groups of the image data of the surface of the object to be detected; calculating the product of the number of input channels, the size of the convolution kernel, and the height and width of the input image to obtain a preset product; calculating the ratio of the preset product to the number of groups to obtain the feature threshold. This device, through the above steps, ensures both the effective extraction of key features by the model and avoids over-processing of background or redundant features.

[0076] Specifically, the choice of the number of groups directly affects the processing efficiency of feature maps and the allocation of computational resources. For example, the more groups there are, the fewer feature map channels each group processes, which theoretically reduces computation and improves model speed, but may also affect the comprehensive extraction of features. The preset product refers to the product of the number of input channels, the number of output channels, the size of the convolution kernel, and the height and width of the input image, n. in *n out *r*r*h*w. Calculate feature thresholds: Params Average =(n in *n out The feature threshold (T) is calculated by dividing the preset product by the number of groups (s). The feature threshold (T) essentially defines the processing "value" or "cost" of each feature map group, used to determine which feature maps should undergo more complex convolution processing, and which feature maps can be simplified, such as being skipped or processed by a simple linear transformation.

[0077] In some optional implementations, the grouped convolutional network model further includes a regular convolutional network. Step S2022 can be achieved by the following steps: when the computational cost of the convolutional features is less than the feature threshold, the group of image data is convolved using the regular convolutional network of the grouped convolutional network model. This device performs regular convolution processing on image data where the computational cost of the convolutional features is less than the feature threshold using the above steps, thus further reducing the computational cost.

[0078] Specifically, a convolutional feature computation cost less than the feature threshold indicates that the computational cost of that group of convolutional features is relatively low, potentially corresponding to the background portion of the image, i.e., not a defective part. Therefore, this portion can be treated with only ordinary convolution. Ordinary convolutional networks use a single convolutional kernel sliding across the entire input. While not as efficient as grouped convolutions, it is sufficient to capture necessary information for simple or open feature regions while avoiding unnecessary computational overhead. Whether features are processed by grouped convolutional networks or ordinary convolutional networks, they ultimately need to be integrated to form a complete image feature description. In practical applications, the feature threshold can be dynamically adjusted to better adapt to different input images and application scenarios. For example, when processing images with richer textures, the threshold can be appropriately lowered to ensure that no small but important features are missed.

[0079] In some alternative implementations, step S2022 can be achieved through the following steps: calculating the offset of each preset direction using the convolution kernel of the serpentine convolutional network; determining the direction of the grouped convolution as the preset direction corresponding to the largest offset. This device, through the above steps, maintains computational efficiency while dynamically adjusting the convolution direction according to the actual distribution of features, thereby improving the detection capability of complex surface defects such as cracks, scratches, and dents.

[0080] Specifically, the directional snake convolution operation incorporates continuity constraints into the design of the convolution kernel. Each convolution position uses its previous position as a reference, freely choosing its swing direction, thus ensuring continuity of perception while allowing for free choice. The convolution operation of features moves based on the previous result (the direction shown by the straight line in the figure). However, the orientation and perspective of defect features are never unique, so fusing multi-view features in the design is an inevitable choice. However, fusing more features results in a larger network load, and so many features inevitably lead to redundancy. Therefore, the original snake convolution operation uses a grouping and random dropout strategy during the feature fusion training process, which alleviates the pressure on network memory to some extent and avoids the model falling into overfitting. However, this patent uses a thresholded grouping convolution strategy to alleviate this pressure for efficiency, achieving a balance between speed and accuracy. Before processing begins, a series of preset directions are determined for the snake convolution network. These directions can be based on the four basic directions of the Cartesian coordinate system (up, down, left, right) and their combinations, or more detailed angular directions, depending on the application requirements and the expected distribution of defect features. The offset is calculated in each preset direction by the convolution kernel of the snake convolution network. The offset here reflects the feature density or importance in a specific direction of the feature map. It can be the response strength of the convolution kernel in that direction, the gradient change of the feature map, or some form of attention score. Among all preset directions, the direction with the largest offset is selected as the direction for the next group convolution. This means that the model will prioritize convolution operations along the direction where the feature distribution is densest or the changes are most significant, thereby more effectively capturing and analyzing defect information. Based on the selected direction, the shape and orientation of the convolution kernel for group convolution are dynamically adjusted to ensure that the convolution operation can be performed along the feature flow direction, thereby improving the accuracy and efficiency of feature extraction. After completing the convolution processing in a specific direction, the feature map is updated and fed back to the serpentine convolutional network, and the offset calculation and direction selection are performed again until the features are fully extracted or the predetermined number of iterations is reached. The feature maps in all directions are integrated into a comprehensive representation for use by subsequent classification or localization modules.

[0081] In some alternative implementations, step S201 can be achieved through the following steps: step S2011: obtaining the number of input channels of the grouped convolutional network model; step S2012: uniformly grouping the image data according to the number of input channels of the grouped convolutional network model to obtain multiple groups of image data. This device groups the images through the above steps, ensuring that the image data can be effectively grouped and processed according to the number of input channels of the model. This step not only optimizes computational efficiency but also promotes the performance improvement of the model when processing high-dimensional data.

[0082] Specifically, the image data is uniformly grouped according to the number of input channels in the grouped convolutional network model, resulting in multiple groups of image data. Based on the model design requirements and the computing power of the target hardware, the number of groups for the input channels is determined. For example, if the model has 96 input channels, it can be divided into 32 groups, each containing 3 channels, allowing the model to process features in parallel across different groups. The input image data is uniformly divided according to the determined number of groups. For each group, the corresponding channels of the image data are separated to form independent data groups for subsequent processing. After grouping the image data, each group of data is convolved using the convolution kernel corresponding to that group in the model. This grouping processing method can significantly reduce the consumption of computing resources, especially for networks with a large number of input channels, because the convolution operation of each group can be executed in parallel. After completing the convolution processing of all groups, the resulting feature maps are recombined to form a complete feature output.

[0083] The defect detection device includes a processor and a memory. The aforementioned grouping units, construction units, processing units, etc., are all stored as program units in the memory. The processor executes the aforementioned program units stored in the memory to achieve the corresponding functions. All of the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0084] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and by adjusting kernel parameters, defects on the outer surface of an object can be accurately identified.

[0085] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0086] This invention provides a computer-readable storage medium including a stored program, wherein the program, when running, controls the device containing the computer-readable storage medium to execute the object surface defect detection method.

[0087] Specifically, methods for detecting defects on object surfaces include:

[0088] Step S201: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data;

[0089] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0090] Step S202: Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0091] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0092] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the direction of the grouped convolutions. By dynamically adjusting the direction and shape of the convolution kernels, the convolution operation is performed along the direction of feature density, thereby more effectively extracting feature information from defect regions.

[0093] Step S203: Apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0094] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0095] This invention provides a processor for running a program, wherein the program executes the defect detection method during runtime.

[0096] Specifically, defect detection methods include:

[0097] Step S201: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data;

[0098] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0099] Step S202: Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0100] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0101] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the direction of the grouped convolutions. By dynamically adjusting the direction and shape of the convolution kernels, the convolution operation is performed along the direction of feature density, thereby more effectively extracting feature information from defect regions.

[0102] Step S203: Apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0103] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0104] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0105] Step S201: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data;

[0106] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0107] Step S202: Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0108] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0109] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the direction of the grouped convolutions. By dynamically adjusting the direction and shape of the convolution kernels, the convolution operation is performed along the direction of feature density, thereby more effectively extracting feature information from defect regions.

[0110] Step S203: Apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0111] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0112] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0113] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0114] Step S201: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data;

[0115] Specifically, taking the top cover of an air conditioner outdoor unit as an example, image data of the surface of the top cover can be acquired using an industrial camera or other image capture equipment. This image data contains various information about the surface of the top cover, such as defect features like scratches, dents, and pores. After acquiring the image data, it is grouped. The feature maps of the image data can be divided according to the number of channels, forming multiple independent groups. After the grouping operation, the originally single image data is decomposed into multiple groups, each containing a portion of feature information. This decomposition allows the network to process features from different groups in parallel in subsequent convolution operations, and the number of convolutions for each group is smaller, thereby improving computational efficiency.

[0116] Step S202: Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data;

[0117] Specifically, after acquiring and grouping image data, a grouped convolutional network model is constructed. Historical image data is collected by gathering a large amount of historical image data containing product surface defects. This data should cover various types of defects, such as cracks, scratches, and dents. Detailed annotations are made for each historical image, including the type, location, and severity of the defect. This annotation information will serve as the label data for training the model. Graphics rendering technology is used to simulate product surfaces under different lighting conditions and background environments, generating a large amount of virtual image data to further enrich the training set and enhance the model's generalization ability. Real image data and virtual image data are integrated to form multiple training datasets containing defect information and their corresponding labels. Each dataset represents product surface features under different conditions.

[0118] This model comprises at least two key components: a grouped convolutional network and a serpentine convolutional network. The grouped convolutional network is primarily used to perform grouped convolutions on image data. Its purpose is to reduce computational complexity by minimizing interactions between feature maps while maintaining the model's ability to represent features. The serpentine convolutional network determines the direction of the grouped convolutions. By dynamically adjusting the direction and shape of the convolution kernels, the convolution operation is performed along the direction of feature density, thereby more effectively extracting feature information from defect regions.

[0119] Step S203: Apply the grouped convolutional network model to process multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data.

[0120] Specifically, a grouped convolutional network model is used for image processing to obtain defect information, such as scratches, dents, and pores. Local feature analysis: The output of each grouped convolutional layer is further analyzed to identify possible defect features. The model learns which feature combinations are associated with specific defects. Global information integration: Deeper layers of the network integrate feature information from different groups to form a comprehensive understanding of the entire image, which is particularly important for identifying large-scale defects spanning multiple local regions. Thresholding: Using pre-set feature thresholds, highly active feature regions associated with defects are selected, further improving detection accuracy.

[0121] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0122] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0131] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0132] 1) In the defect detection method of this application, image data of the surface of the object to be detected is acquired, and the image data is grouped to obtain multiple sets of image data; a grouped convolutional network model is constructed, wherein the grouped convolutional network model is trained through multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data; the grouped convolutional network model is applied to process the multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data. Compared with the prior art, which cannot accurately identify defects on the outer surface of an object, this application improves the convolution efficiency by performing grouped convolution on the image data through the grouped convolutional network in the grouped convolutional network model, and determines the direction of grouped convolution through a serpentine convolutional network to perform directional convolution processing on the image data features. Therefore, it can solve the problem of not being able to accurately identify defects on the outer surface of an object in the prior art, and achieve the effect of accurately identifying defects on the outer surface of an air conditioner.

[0133] 2) In the defect detection device of this application, image data of the surface of the object to be detected is acquired, and the image data is grouped to obtain multiple sets of image data; a grouped convolutional network model is constructed, wherein the grouped convolutional network model is trained through multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data; the grouped convolutional network model is applied to process the multiple sets of image data to obtain defect information, wherein the defect information is information characterizing defects in the image data. Compared with the prior art, which cannot accurately identify defects on the outer surface of an object, this application improves the convolution efficiency by performing grouped convolution on the image data through the grouped convolutional network in the grouped convolutional network model, and determines the direction of grouped convolution through a serpentine convolutional network to perform directional convolution processing on the image data features. Therefore, it can solve the problem of not being able to accurately identify defects on the outer surface of an object in the prior art, and achieve the effect of accurately identifying defects on the outer surface of an air conditioner.

[0134] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting defects on the surface of an object, characterized in that, include: Acquire image data of the surface of the object to be detected, and group the image data to obtain multiple groups of image data; Construct a grouped convolutional network model, wherein the grouped convolutional network model is trained by multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data; The computational cost of convolutional features for each group of image data is determined, and the model parameters of the initial convolutional network model are obtained. A feature threshold is determined based on the model parameters. The initial convolutional network model includes a standard convolutional network. Obtaining the model parameters of the initial convolutional network model includes: acquiring historical real image data of the surface of the object to be detected; simulating the historical real image data under different lighting conditions to obtain historical virtual image data; training the initial convolutional network model based on the historical real image data and the historical virtual image data, enabling the initial convolutional network model to recognize defect information in the historical real image data and the historical virtual image data; and obtaining the number of input channels, the number of output channels, and the size of the convolutional kernel of the standard convolutional network after training. The height and width of the input image are used to obtain the model parameters of the initial convolutional network model. Determining the feature threshold based on the model parameters includes: obtaining the number of groups of image data representing the surface of the object to be detected; calculating the product of the number of input channels, the size of the convolutional kernel, and the height and width of the input image to obtain a preset product; calculating the ratio of the preset product to the number of groups to obtain the feature threshold; when the computational cost of the convolutional feature is greater than or equal to the feature threshold, performing convolution processing on the group of image data through the group convolutional network of the group convolutional network model, and determining the processing direction of the convolutional kernel in the group convolution through the serpentine convolutional network of the group convolutional network model to obtain defect information, wherein the defect information is information characterizing defects in the image data.

2. The method according to claim 1, characterized in that, The grouped convolutional network model also includes a regular convolutional network. Applying the grouped convolutional network model to process multiple sets of image data further includes: If the computational cost of the convolutional features is less than the feature threshold, the group of image data is convolved by the ordinary convolutional network of the grouped convolutional network model.

3. The method according to claim 1, characterized in that, The processing direction of the convolution kernels in the grouped convolutions is determined by the serpentine convolutional network of the grouped convolutional network model, including: The offset in each preset direction is calculated using the convolution kernel of the serpentine convolutional network; The direction of the grouped convolution is determined to be the preset direction corresponding to the largest offset.

4. The method according to claim 1, characterized in that, The image data is grouped to obtain multiple groups of image data, including: Obtain the number of input channels of the grouped convolutional network model; The image data is evenly grouped according to the number of input channels of the grouped convolutional network model to obtain multiple groups of image data.

5. A device for detecting surface defects of an object, characterized in that, include: A grouping unit is used to acquire image data of the surface of the object to be detected, and to group the image data to obtain multiple groups of image data. A construction unit is used to construct a grouped convolutional network model, wherein the grouped convolutional network model is trained through multiple sets of data, and each set of data includes: historical image data and labels corresponding to defect information in the historical image data; The processing unit is configured to determine the computational cost of convolutional features for each group of image data, obtain the model parameters of the initial convolutional network model, and determine the feature threshold based on the model parameters. The initial convolutional network model includes a standard convolutional network. Obtaining the model parameters of the initial convolutional network model includes: acquiring historical real image data of the surface of the object to be detected; simulating the historical real image data under different lighting conditions to obtain historical virtual image data; training the initial convolutional network model based on the historical real image data and the historical virtual image data, enabling the initial convolutional network model to identify defect information in the historical real image data and the historical virtual image data; and acquiring the number of input channels, the number of output channels, and the size of the convolutional kernel of the standard convolutional network after training. The model parameters of the initial convolutional network model are obtained by taking the height and width of the input image as an example. Determining the feature threshold based on the model parameters includes: obtaining the number of groups of image data on the surface of the object to be detected; calculating the product of the number of input channels, the size of the convolutional kernel, and the height and width of the input image to obtain a preset product; calculating the ratio of the preset product to the number of groups to obtain the feature threshold; when the computational cost of the convolutional feature is greater than or equal to the feature threshold, performing convolution processing on the group of image data through the group convolutional network of the group convolutional network model, and determining the processing direction of the convolutional kernel in the group convolution through the serpentine convolutional network of the group convolutional network model to obtain defect information, wherein the defect information is information characterizing defects in the image data.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the object surface defect detection method according to any one of claims 1 to 4.

7. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing the object surface defect detection method according to any one of claims 1 to 4.

Images