Object defect detection method, system and equipment, medium and product

Abstract

The invention relates to the technical field of data processing, and particularly provides an object defect detection method, system and device, a medium and a product.The method comprises the steps that target feature information of target objects of the same type and various models is determined; wherein the target feature information is used for indicating object features of the target object under the condition of no defect; obtaining a target image of a to-be-recognized object, and extracting image feature information of the target image; wherein the type of the to-be-recognized object is the same as that of the target object; performing defect detection on the to-be-recognized object based on the image feature information and the target feature information to obtain a defect detection result; wherein the defect detection result is used for indicating defect information of the to-be-identified object under the condition that the to-be-identified object has defects.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method, system, device, medium and product for detecting object defects. Background Art

[0002] At present, the defect inspection link of the surface of objects has always been a difficult point in the quality inspection of objects. Especially for the defect inspection of objects with coating layers, the traditional quality inspection method is that the quality inspectors use the naked eye to observe the surface of the object, and then identify the defective objects based on the observation results of the quality inspectors. However, the output of objects in the production process is usually large, and the work of manually observing the defects of objects is labor-intensive and time-inefficient. Summary of the invention

[0003] In view of the above problems, the present disclosure is proposed. The present disclosure provides a method, system, device, medium and product for detecting object defects.

[0004] According to one aspect of the present disclosure, a method for detecting defects of an object is provided, comprising:

[0005] Determining target feature information of target objects of various models of the same type; wherein the target feature information is used to indicate the object features of the target object in a defect-free state;

[0006] Acquire a target image of an object to be identified, and extract image feature information of the target image; wherein the object to be identified is of the same type as the target object;

[0007] Based on the image feature information and the target feature information, defect detection is performed on the object to be identified to obtain a defect detection result; wherein the defect detection result is used to indicate defect information of the object to be identified when a defect exists.

[0008] In addition, according to another embodiment of one aspect of the present disclosure, the performing defect detection on the object to be identified based on the image feature information and the target feature information to obtain a defect detection result includes:

[0009] Performing block processing on the target image to obtain a plurality of image blocks, and determining sub-image feature information of each of the image blocks based on the image feature information;

[0010] Calculating the similarity between the sub-image feature information and the target feature information to obtain multiple similarities;

[0011] A defect detection result of the object to be identified is determined based on the multiple similarities.

[0012] In addition, according to another embodiment of one aspect of the present disclosure, the determining the defect detection result of the object to be identified based on the multiple similarities includes:

[0013] When it is detected that there is a target similarity less than a preset similarity threshold among the multiple similarities, a defect detection result of the object to be identified is determined based on position information of the image block corresponding to the target similarity.

[0014] In addition, according to another embodiment of one aspect of the present disclosure, determining target feature information of target objects of various models of the same type includes:

[0015] The target feature alignment model is used to extract features from the object images of target objects of the same type and various models to obtain the target feature information.

[0016] In addition, according to another embodiment of one aspect of the present disclosure, the target feature alignment model is obtained by training in the following manner:

[0017] Acquire a training sample image and defect description information of the training sample image;

[0018] Acquire standard sample features of a standard sample image; wherein the objects in the training sample image and the objects in the standard sample image are of the same type of object;

[0019] Determine the target detection index between the training sample image, the defect description information and the standard sample image, and train the feature alignment model to be trained based on the target detection index to obtain the target feature alignment model; wherein the target detection index includes: similarity and difference.

[0020] In addition, according to another embodiment of one aspect of the present disclosure, the determining of the target detection index between the training sample image, the defect description information and the standard sample image includes:

[0021] Extracting features from the training sample images to obtain the training sample features;

[0022] Calculating the similarities between the defect description information, the training sample features, and the standard sample features to obtain multiple similarities, and calculating the differences between the defect description information, the training sample features, and the standard sample features to obtain multiple differences;

[0023] The multiple similarities and the multiple differences are determined as the target detection indicators.

[0024] In addition, according to another embodiment of one aspect of the present disclosure, the training of the feature alignment model to be trained based on the target detection index includes:

[0025] Adding the multiple similarities to obtain a target similarity;

[0026] Adding the multiple differences to obtain a target difference;

[0027] Determining a target loss function based on the target similarity and the target difference;

[0028] Based on the target loss function, the feature alignment model to be trained is trained to obtain the target feature alignment model.

[0029] In addition, in another embodiment according to one aspect of the present disclosure, the training sample images include a first type of images with defects and a second type of images without defects;

[0030] The training of the feature alignment model to be trained based on the target detection index includes:

[0031] Determining target detection indicators corresponding to the first category of images and the second category of images in the target detection indicators, and obtaining a first target detection indicator and a second target detection indicator respectively;

[0032] Determining a first loss function based on a difference between similarity and difference in the first target detection indicator, and determining a second loss function based on a difference between difference and similarity in the second target detection indicator;

[0033] The target loss function is determined based on the first loss function and the second loss function.

[0034] According to another aspect of the present disclosure, there is provided a system for detecting object defects, comprising: a multi-angle image sampling device, an adjustment device, and a processor; wherein the adjustment device is arranged on a detection device body of an object to be identified, and the adjustment device is arranged in an upper area of ​​the object to be identified, and the multi-angle image sampling device is arranged on the adjustment device and on the detection device body;

[0035] The regulating device is used to control the corresponding image sampling device to reach a specified position based on the model of the object to be identified;

[0036] The image sampling device is used to collect images of the object to be identified that enters the collection area of ​​the image sampling device after reaching the designated position, so as to obtain a target image;

[0037] The processor is used to acquire the target image, and perform defect detection on the target image according to the above method to obtain a defect detection result of the object to be identified.

[0038] In addition, in another embodiment according to one aspect of the present disclosure, the system further includes: a lighting device; wherein the lighting device is arranged adjacent to the image sampling device, and the lighting device is used to provide illumination to a collection area of ​​the image sampling device.

[0039] In addition, according to another embodiment of one aspect of the present disclosure, the system further includes: a defect display device and a display device; wherein the defect display device includes a plurality of running lights, and a running light is arranged corresponding to each detection position, and the detection position is a position where the object to be identified is placed;

[0040] The processor is configured to send a first display signal to a target running light corresponding to the object to be identified, and send a second display signal to the display device when it is determined that the object to be identified has a defect based on a defect detection result of the object to be identified;

[0041] The target running light is used to turn on after receiving the first display signal;

[0042] The display device is used to display the defect information of the object to be identified after receiving the second display signal.

[0043] In addition, according to another embodiment of one aspect of the present disclosure, the multi-angle image sampling device includes: a bottom image sampling device and a side image sampling device, wherein the lighting device arranged adjacent to the bottom image sampling device is a dome light source, and the lighting device arranged adjacent to the side image sampling device is a striped surface light source.

[0044] According to another aspect of the present disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement steps of a method for determining a log template.

[0045] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of a method for determining a log template are implemented.

[0046] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, which implements the steps of a method for determining a log template when the computer program / instruction is executed by a processor.

[0047] As will be described in detail below, a method, system, device, medium and product for detecting object defects according to an embodiment of the present disclosure are provided.

[0048] First, target feature information of target objects of various models of the same type is determined; wherein the target feature information is used to indicate the object features of the target object in a defect-free state; secondly, a target image of the object to be identified is acquired, and image feature information of the target image is extracted; wherein the object to be identified is of the same type as the target object; finally, defect detection is performed on the object to be identified based on the image feature information and the target feature information to obtain a defect detection result; wherein the defect detection result is used to indicate the defect information of the object to be identified in a defect-presence state.

[0049] In the embodiment of the present disclosure, by comparing the image feature information of the target image containing the object to be identified with the target feature information of the target object, defect detection of the object to be identified can be quickly performed, thereby improving the defect detection efficiency of the object and accelerating the defect detection speed; at the same time, the technical solution of the present disclosure can be used to perform defect detection on objects of the same type and various models of objects to be identified, thereby simplifying the defect detection process and further improving the defect detection efficiency.

[0050] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0052] Figure 1 A flowchart of a method for detecting object defects provided in an embodiment of the present disclosure.

[0053] Figure 2 A model architecture diagram for determining target feature information in the object defect detection method provided in the embodiment of the present disclosure

[0054] Figure 3 A specific flow chart of a method for detecting object defects provided in an embodiment of the present disclosure.

[0055] Figure 4 A schematic diagram of a system for detecting object defects provided in an embodiment of the present disclosure.

[0056] Figure 5A schematic diagram of a device for detecting object defects provided in an embodiment of the present disclosure.

[0057] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0059] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0060] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0061] According to research, the quality inspection of the paint surface on the surface of an object has always been a difficult point in the quality inspection of the object. Especially in the field of thermos cups, the traditional quality inspection method is for quality inspectors to observe the body of the thermos cup with the naked eye. Due to the huge output of thermos cups, visual quality inspection is mostly labor-intensive and time-inefficient.

[0062] In the prior art, objects can also be inspected offline by automatic detection. Quality inspectors are required to place objects on the detection equipment, which increases labor costs and is difficult to meet the production line rhythm, resulting in low time efficiency. In addition, when the detection equipment needs objects of different models of the same type, it needs to be adjusted to the current model, which also leads to the problem of low time efficiency.

[0063] Based on the above research, the present disclosure provides a method for detecting object defects, which determines the target feature information of various models of target objects of the same type, and performs defect detection based on the image feature information and target feature information of the object to be identified. There is no need to change the defect detection model or method according to the model of the object to be identified. It can be adapted to the defect detection of various models of objects of the same type. In addition, this solution does not require manual cooperation, which saves labor costs and improves time efficiency.

[0064] To facilitate understanding of this embodiment, a method for detecting object defects disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the method for detecting object defects provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities. In some possible implementations, the method for detecting object defects in the server can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0065] Reference Figure 1 FIG. 1 is a flow chart of a method for detecting object defects provided by an embodiment of the present disclosure, wherein the method comprises steps S101 to S103, wherein:

[0066] S101. Determine target feature information of target objects of the same type and of various models; wherein the target feature information is used to indicate object features of the target object in a defect-free state.

[0067] In the embodiments of the present disclosure, the same type is used to indicate products of the same category. For example, a thermos cup, a computer, and a table belong to different categories, respectively.

[0068] Here, the same type of object may have multiple different models, and the sizes of objects of different models may be different. For example, when the target object is a thermos cup, the heights and diameters of thermos cups of different models may be different.

[0069] The target feature information is feature information obtained after feature extraction of an image containing the target object. Here, the target object contained in the image is required to be an object without defects. In other words, the appearance of the target object is in an ideal state. For example, when the target object is a thermos cup, the target object is defect-free, which means that the reflective paint surface of the thermos cup has no defects such as blemishes, bumps, and dents.

[0070] S102, obtaining a target image of the object to be identified, and extracting image feature information of the target image; wherein the object to be identified and the target object are of the same type.

[0071] Before acquiring the target image of the object to be identified, it is necessary to identify the model of the object to be identified, so as to determine the acquisition position of the image acquisition device according to the model; after controlling the image acquisition device to move to the acquisition position (i.e., the designated position in the following embodiment), the image of the object to be identified can be acquired to obtain the target image of the object to be identified.

[0072] By means of a mobile image acquisition device, the technical solution of the present disclosure can be applied to various types of objects to be identified, thereby expanding the scope of application of the technical solution of the present disclosure.

[0073] In the embodiments of the present disclosure, after acquiring the target image, the feature extraction (Vision Transformer, VIT) encoder can be used to extract the features of the target image to obtain the image feature information of the target image. In addition, other feature extraction networks can also be selected, which are not specifically limited here, subject to what can be implemented.

[0074] In the embodiment of the present disclosure, after the image acquisition device acquires the object to be identified, it can obtain an original image; wherein the original image can contain at least one object to be identified. At this time, the original image can be segmented to obtain multiple segmented images, wherein each segmented image contains an image of the object to be identified. Here, each segmented image can be used as a target image.

[0075] In another optional embodiment, after the original image is acquired, an object to be identified whose integrity and size meet the requirements can be determined in the original image; then, the original image is cropped to obtain an image containing the object to be identified, thereby obtaining a target image.

[0076] S103. Based on the image feature information and the target feature information, defect detection is performed on the object to be identified to obtain a defect detection result; wherein the defect detection result is used to indicate defect information of the object to be identified when there are defects.

[0077] In the embodiment of the present disclosure, the defect detection step is specifically as follows: each image feature information can be compared with the target feature information to obtain a distance vector between each image feature information and each target feature information. Then, based on the distance vector between each image feature information and each target feature information, the defect detection result of the image to be detected can be determined.

[0078] In another optional implementation, the model of the object to be identified may be determined first, and then the target feature information matching the model may be determined in the target feature information, and then the image feature information may be compared with the target feature information matching the model to obtain the corresponding distance vector.

[0079] In an embodiment of the present disclosure, first, target feature information of various models of target objects of the same type is determined; wherein the target feature information is used to indicate the object features of the target object in a defect-free state; secondly, a target image of the object to be identified is obtained, and image feature information of the target image is extracted; wherein the object to be identified is of the same type as the target object; finally, based on the image feature information and the target feature information, defect detection is performed on the object to be identified to obtain a defect detection result; wherein the defect detection result is used to indicate the defect information of the object to be identified in the presence of defects.

[0080] In the above-mentioned implementation mode, by comparing the image feature information of the target image containing the object to be identified with the target feature information of the target object, the defect detection of the object to be identified can be quickly performed, thereby improving the defect detection efficiency of the object and accelerating the defect detection speed; at the same time, the technical solution disclosed in the present invention can be used to perform defect detection on objects of various models of the same type to be identified, thereby simplifying the defect detection process and further improving the defect detection efficiency.

[0081] In an optional implementation, the above steps perform defect detection on the object to be identified based on the image feature information and the target feature information to obtain a defect detection result, which specifically includes the following steps:

[0082] First, the target image is divided into blocks to obtain multiple image blocks, and based on the image feature information, the sub-image feature information of each image block is determined;

[0083] Secondly, the similarity between the sub-image feature information and the target feature information is calculated to obtain multiple similarities;

[0084] Finally, the defect detection result of the object to be identified is determined based on multiple similarities.

[0085] In the embodiment of the present disclosure, the target image can be divided into blocks according to a preset rule to obtain a plurality of image blocks, wherein the image blocks are all square image blocks.

[0086] The preset rules include at least one of the following: (1) determining the shortest side of the object to be identified in the target image, and dividing the target image into blocks based on the shortest side to obtain multiple image blocks.

[0087] (2) A block length is preset based on the size of the target image, and the target image is divided into blocks based on the block length to obtain a plurality of image blocks.

[0088] Here, after a plurality of image blocks are determined, a feature extraction encoder may be used to perform feature extraction on each image block to obtain sub-image feature information corresponding to each image block.

[0089] Here, the distance between the sub-image feature information and the target feature information may be used as the similarity between the sub-image feature information and the target feature information.

[0090] Here, the maximum similarity may be determined among the multiple similarities corresponding to each sub-image feature information, and then the sub-defect detection result of the image block corresponding to the sub-image feature information is determined based on the maximum similarity.

[0091] Afterwards, the sub-defect detection result corresponding to each image block is determined as the defect detection result of the object to be identified.

[0092] In an optional implementation, the above step of determining the defect detection result of the object to be identified based on multiple similarities specifically includes the following steps:

[0093] When a target similarity smaller than a preset similarity threshold value is detected among the multiple similarities, a defect detection result of the object to be identified is determined based on position information of the image block corresponding to the target similarity.

[0094] In the embodiments of the present disclosure, firstly, the target similarity of each image block (ie, the maximum similarity in the above description) may be determined. Secondly, it may be detected whether the target similarity is less than a preset similarity threshold.

[0095] Secondly, when it is determined that there is a target similarity less than a preset similarity threshold, the position information of the image block corresponding to the target similarity is determined.

[0096] Finally, the position information of the target similarity and the defect information of the image block corresponding to the target similarity are determined as the defect detection result of the object to be identified.

[0097] Here, in the case where the target feature information includes the defect type, the defect type of the image block corresponding to the target similarity can also be determined when there is a target similarity less than a preset similarity threshold. Afterwards, the defect type, location information, and defect information of the image block corresponding to the target similarity are determined as the defect detection result.

[0098] In the above implementation, a plurality of image blocks are obtained by dividing the target image into blocks. It is possible to detect whether each image block has a defect. In the case where a defect is detected in the object to be identified based on any image block, the location of the defect can be determined based on the position of the image block. This allows the user to find the location of the defect more conveniently, saving time and efficiency.

[0099] In an optional implementation, the above step of determining target feature information of target objects of the same type and of various models specifically includes the following steps:

[0100] The target feature alignment model is used to extract features from the object images of various models of target objects of the same type to obtain target feature information.

[0101] In the disclosed embodiment, the target feature alignment model is a trained feature alignment model, and object images of target objects of various models of the same type can be used as inputs of the target feature alignment model to obtain target feature information.

[0102] In an optional implementation, the target feature alignment model is trained in the following manner:

[0103] First, obtain the training sample image and the defect description information of the training sample image;

[0104] Then, a standard sample feature of the standard sample image is obtained; wherein the object in the training sample image and the object in the standard sample image are objects of the same type;

[0105] Finally, the target detection indexes between the training sample images, defect description information and standard sample images are determined, and the feature alignment model to be trained is trained based on the target detection indexes to obtain a target feature alignment model; wherein the target detection indexes include: similarity and difference.

[0106] In an embodiment of the present disclosure, the training sample images include a first type of images with defects and a second type of images without defects.

[0107] The defect description information of the first type of image can be an empty set or "the object corresponding to the image has no defects".

[0108] The defect description information of the second type of image includes, but is not limited to: the specific location of the defect of the object corresponding to the second type of image and the type of the defect, etc.

[0109] Here, the standard sample image is an image captured when the corresponding target object of the training image has no defects. That is, there is no defect information of the target object in the standard sample image.

[0110] Here, the first feature extraction encoder may be used to extract features from the training sample image to obtain training sample features. The second feature extraction encoder may be used to extract features from the standard sample image to obtain standard sample features. The first feature extraction encoder and the second feature extraction encoder are exactly the same.

[0111] In the disclosed embodiment, the defect description information may be processed by a text preprocessing (Bidirectional Encoder Representations from Transformers, Bert) model to obtain an information processing result.

[0112] Afterwards, the training sample features, standard sample features and information processing results can be used as inputs of the feature alignment model to be trained, and the feature alignment model to be trained is trained based on the target detection index to obtain the target feature alignment model. Here, the feature alignment model to be trained can be a Bert random initialization model.

[0113] Among them, the similarities and differences between the training sample features, the standard sample features and the information processing results can be determined to obtain the target detection index.

[0114] In an optional implementation, the above steps of determining the target detection index between the training sample image, the defect description information and the standard sample image specifically include the following steps:

[0115] First, feature extraction is performed on the training sample image to obtain the training sample features;

[0116] Then, the similarities between the defect description information, the training sample features and the standard sample features are calculated to obtain multiple similarities, and the differences between the defect description information, the training sample features and the standard sample features are calculated to obtain multiple differences;

[0117] Finally, multiple similarities and multiple differences are determined as target detection indicators.

[0118] In an embodiment of the present disclosure, a first feature extraction encoder may be used to perform feature extraction on a training sample image to obtain training sample features.

[0119] The processing result of the second feature encoder on the standard sample image can be determined as the standard sample feature. Then, the processing result of the text preprocessing on the defect description information can be determined as the information processing result.

[0120] By calculating the similarity between the information processing results, training sample features and standard sample features, we obtain the first similarity ITS(i1, t) between the defect description information and the training sample features, the second similarity ITS(i2, t) between the defect description information and the standard sample features, and the third similarity IIS(i1, i2) between the training sample features and the standard sample features.

[0121] By calculating the differences between the information processing results, the training sample features and the standard sample features, we can obtain the first difference ITD(i1, t) between the defect description information and the training sample features, the second difference ITD(i2, t) between the defect description information and the standard sample features, and the third difference IID(i1, i2) between the training sample features and the standard sample features.

[0122] Here, the similarity thresholds between the defect description information, the training sample features and the standard sample features, as well as the difference thresholds between the defect description information, the training sample features and the standard sample features can be determined. Afterwards, the similarity between the defect description information, the training sample features and the standard sample features can be determined by the Mahalanobis distance.

[0123] Based on the similarity between the defect description information, training sample features and standard sample features, the first similarity ITS(i1, t), the second similarity ITS(i2, t), the third similarity IIS(i1, i2), the first difference ITD(i1, t), the second difference ITD(i2, t) and the third difference IID(i1, i2) are determined.

[0124] Here, the first similarity ITS(i1, t), the second similarity ITS(i2, t), the third similarity IIS(i1, i2), the first difference ITD(i1, t), the second difference ITD(i2, t) and the third difference IID(i1, i2) can be determined as target detection indicators.

[0125] For example, when the similarity between the defect description information and the training sample features is less than the similarity threshold between the defect description information and the training sample features, the first preset result is determined as the first similarity ITS(i1, t); otherwise, the second preset result is determined as the first similarity ITS(i1, t).

[0126] When the similarity between the defect description information and the training sample features is greater than the difference threshold between the defect description information and the training sample features, the third preset result is determined as the first difference ITD(i1, t); otherwise, the fourth preset result is determined as the first difference ITD(i1, t).

[0127] In an optional implementation, the above step trains the feature alignment model to be trained based on the target detection index, specifically including the following steps:

[0128] First, multiple similarities are added to obtain the target similarity;

[0129] Secondly, multiple differences are added together to obtain the target difference;

[0130] Secondly, based on the target similarity and target difference, the target loss function is determined;

[0131] Finally, based on the target loss function, the feature alignment model to be trained is trained to obtain the target feature alignment model.

[0132] In the embodiment of the present disclosure, the first similarity ITS(i1, t), the second similarity ITS(i2, t) and the third similarity IIS(i1, i2) can be added to obtain the target similarity S(i1, i2, t). That is, the target similarity meets the following conditions:

[0133] S(i1,i2,t)=ITS(i1,t)+ITS(i2,t)+IIS(i1,i2).

[0134] In the embodiment of the present disclosure, the first difference ITD(i1, t), the second difference ITD(i2, t) and the third difference IID(i1, i2) can be added to obtain the target difference D(i1, i2, t). That is, the target difference D(i1, i2, t) meets the following conditions:

[0135] D(i1,i2,t)=ITD(i1,t)+ITD(i2,t)+IID(i1,i2).

[0136] Here, the target similarity S(i1, i2, t) and the target difference D(i1, i2, t) can be subtracted to obtain the target loss function.

[0137] When the target loss function does not meet the training conditions, the parameters of the feature alignment model to be trained are adjusted until the target loss function meets the training conditions, and the current feature alignment model to be trained is determined as the target feature alignment model. The training condition is that the target loss function shows a trend of gradually decreasing and stabilizing.

[0138] Here, the feature information output by the target feature alignment model may be determined as the target feature information and saved in a memory bank.

[0139] Reference Figure 2 As shown, it is a model architecture diagram for determining target feature information in the object defect detection method provided by the embodiment of the present disclosure, including: a first feature extraction encoder, a second feature extraction encoder, a text preprocessing model, a target feature alignment model and a memory bank. Among them:

[0140] The training sample image is input into the first feature code extractor, the standard sample image is input into the second feature code extractor, and the defect description information is input into the text preprocessing model; the training sample features, standard sample features and information processing results are obtained respectively. The training sample features, standard sample features and information processing results are input into the target feature alignment model for processing. The target feature information output by the target feature alignment model is stored in the memory bank.

[0141] In an optional implementation, the above step trains the feature alignment model to be trained based on the target detection index, specifically including the following steps:

[0142] First, determine the target detection indicators corresponding to the first type of image and the second type of image in the target detection indicators, and obtain the first target detection indicator and the second target detection indicator respectively;

[0143] Then, a first loss function is determined based on a difference between similarity and difference in the first target detection index, and a second loss function is determined based on a difference between difference and similarity in the second target detection index;

[0144] Finally, the target loss function is determined based on the first loss function and the second loss function.

[0145] In an embodiment of the present disclosure, after determining the first target detection index, the first target similarity S1(i1, i2, t) and the first target difference D1(i1, i2, t) of the first type of image can be determined based on the first target detection index.

[0146] After the second target detection index is determined, the second target similarity S2(i1, i2, t) and the second target difference D2(i1, i2, t) of the second type of image may be determined based on the second target detection index.

[0147] Afterwards, the first loss function Fs(i1, i2, t) can be determined based on the first target similarity S1(i1, i2, t) and the first target difference D1(i1, i2, t). The first loss function Fs(i1, i2, t) meets the following conditions:

[0148] Fs(i1,i2,t)=S1(i1,i2,t)-D1(i1,i2,t).

[0149] Afterwards, the second loss function Fd(i1, i2, t) can be determined based on the second target similarity S2(i1, i2, t) and the second target difference D2(i1, i2, t). The second loss function Fd(i1, i2, t) meets the following conditions:

[0150] Fd(i1,i2,t)=D2(i1,i2,t)-S2(i1,i2,t).

[0151] Here, the first loss function Fs(i1, i2, t) and the second loss function Fd(i1, i2, t) may be used as target loss functions.

[0152] See also Figure 3 As shown, it is a specific flow chart of a method for detecting object defects provided by an embodiment of the present disclosure, wherein:

[0153] S10, acquiring a captured image of the object to be identified; wherein the captured image includes a bottom image and a side image of the object to be identified.

[0154] S20: When the captured image contains multiple objects to be identified, perform image segmentation processing on the target image to obtain multiple target images containing one object to be identified.

[0155] S30, determining target feature information of target objects of various models of the same type; wherein the target feature information is used to indicate the object features of the target object in a defect-free state, and the type of the target object is the same as the type of the object to be identified.

[0156] S40. Based on the image feature information of multiple target images and the target feature information, defect detection is performed on the object to be identified to obtain defect detection results of all the objects to be identified contained in the target images; wherein the defect detection results are used to indicate defect information of the object to be identified when defects exist.

[0157] In the above-mentioned implementation mode, by comparing the image feature information of the target image containing the object to be identified with the target feature information of the target object, the defect detection of the object to be identified can be quickly performed, thereby improving the defect detection efficiency of the object and accelerating the defect detection speed; at the same time, the technical solution disclosed in the present invention can be used to perform defect detection on objects of various models of the same type to be identified, thereby simplifying the defect detection process and further improving the defect detection efficiency.

[0158] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0159] Based on the same inventive concept, an object defect detection system corresponding to the object defect detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the system in the embodiment of the present disclosure is similar to the above-mentioned object defect detection method in the embodiment of the present disclosure, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0160] See also Figure 4 , which is a schematic diagram of an object defect detection system provided by an embodiment of the present disclosure, the system comprises: a multi-angle image sampling device 1, an adjustment device 2 and a processor (not shown); wherein the adjustment device is arranged on a detection device body of an object to be identified, and the adjustment device is arranged in an upper area of ​​the object to be identified, and the multi-angle image sampling device is arranged on the adjustment device and the detection device body;

[0161] The regulating device is used to control the corresponding image sampling device to reach a specified position based on the model of the object to be identified;

[0162] The image sampling device is used to collect images of the object to be identified that enters the collection area of ​​the image sampling device after reaching the designated position, so as to obtain a target image;

[0163] The processor is used to acquire the target image and perform defect detection on the target image to obtain a defect detection result of the object to be identified.

[0164] Here, the adjusting device includes at least one up-and-down adjusting device, wherein each image sampling device in the multi-angle image sampling device is respectively mounted on a different up-and-down adjusting device; wherein the image acquisition device can move with the movement of the up-and-down adjusting device.

[0165] In an embodiment of the present disclosure, the above system further includes: a detection device body (ie, a rack frame). Figure 3 As shown, two up and down adjustment devices can be provided, and each up and down adjustment device is installed on the rack frame. Among them, one up and down adjustment device is installed at the top of the rack frame, and the other is installed in front of the designated position.

[0166] Here, the system further includes a stabilizing device and an assembly line. The stabilizing device is located at the bottom of the rack frame, and an assembly line is arranged below the stabilizing device. The stabilizing device can be used to place objects to be identified; wherein, objects to be identified of different models can be placed on the stabilizing device.

[0167] The assembly line is used to drive the stabilizing device to move to a specified position so that the multi-angle image sampling device can collect images of each object to be identified placed on the stabilizing device.

[0168] Here, multiple production lines may be provided, and the models of the objects to be identified on the stabilizing devices installed on each production line may be different or the same.

[0169] In an optional embodiment, the system further includes: a lighting device; wherein the lighting device is disposed adjacent to the image sampling device, and the lighting device is used to provide illumination to a collection area of ​​the image sampling device.

[0170] Here, when multiple production lines are arranged on the rack frame, the lighting device can provide illumination for multiple objects to be identified on the multiple production lines. Similarly, the image sampling device can collect images for multiple objects to be identified on the multiple production lines.

[0171] In an optional embodiment, the system further includes: a defect display device and a display device; wherein the defect display device includes a plurality of running lights, a running light is arranged corresponding to each detection position, and the detection position is the position where the object to be identified is placed.

[0172] The processor is used to send a first display signal to a target flow light corresponding to the object to be identified, and send a second display signal to the display device when it is determined that the object to be identified has a defect based on a defect detection result of the object to be identified.

[0173] In the disclosed embodiment, a position marker can be set at each placement position, and a corresponding relationship between the running light and the position marker can be established. When it is determined that any object to be identified has a defect based on the defect detection result, the position marker can be extracted from the target image corresponding to the object to be identified, and then the target running light corresponding to the object to be identified is determined based on the corresponding relationship between the position marker and the running light, and then the first display signal is sent to the target running light.

[0174] The target flowing light is used to turn on after receiving the first display signal.

[0175] The display device is used to display the defect information of the object to be identified after receiving the second display signal.

[0176] In an embodiment of the present disclosure, at least one running light may be provided. When one running light is provided, the running light may be turned on when the object to be identified meets a preset display condition.

[0177] The preset display condition is that the defective object to be identified reaches the location of the running light and the defect display device receives a first display signal.

[0178] Here, a running light may be provided at each position of the object placed by the stabilizing device, and when the object to be identified meets the preset display conditions, the target running light corresponding to the object to be identified is turned on.

[0179] In the above implementation, the running light can light up along with the defective object to be identified, so as to assist the user in locating the defective object to be identified and improve the efficiency of picking up the object to be identified.

[0180] In an optional embodiment, the multi-angle image sampling device includes: a bottom image sampling device and a side image sampling device, wherein the lighting device arranged adjacent to the bottom image sampling device is a dome light source, and the lighting device arranged adjacent to the side image sampling device is a stripe-type surface light source.

[0181] In an embodiment of the present disclosure, the lighting device comprises: a bottom lighting device and a side lighting device, wherein the bottom lighting device and the bottom image sampling device are arranged adjacent to each other, and the side lighting device and the side image sampling device are arranged adjacent to each other.

[0182] Here, the bottom lighting device uses a dome light source, which can achieve a uniform lighting environment, reduce shadows and insufficiently lit areas, and provide a stable lighting environment. The side lighting device uses a bar-type surface light source, which can effectively reduce the problem of blurred highlights caused by paint surface reflection. At the same time, it can sensitively display defects such as dents that are difficult to display with ordinary light sources.

[0183] Based on the same inventive concept, an object defect detection device corresponding to the object defect detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned object defect detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0184] Reference Figure 5 FIG. 1 is a schematic diagram of an object defect detection device provided by an embodiment of the present disclosure, wherein the device comprises: a determination module 11, an extraction module 12, and a detection module 13; wherein:

[0185] A determination module, used to determine target feature information of target objects of the same type and of various models; wherein the target feature information is used to indicate the object features of the target object in a defect-free state;

[0186] An extraction module, used to obtain a target image of an object to be identified and extract image feature information of the target image; wherein the object to be identified is of the same type as the target object;

[0187] A detection module is used to perform defect detection on the object to be identified based on the image feature information and the target feature information to obtain a defect detection result; wherein the defect detection result is used to indicate defect information of the object to be identified when a defect exists.

[0188] The disclosed embodiment determines the target feature information of various models of target objects of the same type, and performs defect detection based on the image feature information and target feature information of the object to be identified. There is no need to change the defect detection model or method according to the model of the object to be identified. It can adapt to the defect detection of various models of objects of the same type. In addition, this solution does not require manual cooperation, which saves labor costs and improves time efficiency.

[0189] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0190] Corresponds to Figure 1 The security monitoring method of the edge computing service in the present disclosure also provides an electronic device 600, such as Figure 6FIG. 6 is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present disclosure, including:

[0191] Processor 61, memory 62, and bus 63; memory 62 is used to store execution instructions, including internal memory 621 and external memory 622; the internal memory 621 is also called internal memory, which is used to temporarily store the operation data in the processor 61 and the data exchanged with the external memory 622 such as a hard disk. The processor 61 exchanges data with the external memory 622 through the internal memory 621. When the electronic device 600 is running, the processor 61 communicates with the memory 62 through the bus 63, so that the processor 61 executes the following instructions:

[0192] Determining target feature information of target objects of various models of the same type; wherein the target feature information is used to indicate the object features of the target object in a defect-free state;

[0193] Acquire a target image of an object to be identified, and extract image feature information of the target image; wherein the object to be identified is of the same type as the target object;

[0194] Based on the image feature information and the target feature information, defect detection is performed on the object to be identified to obtain a defect detection result; wherein the defect detection result is used to indicate defect information of the object to be identified when a defect exists.

[0195] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0196] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0197] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0198] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0199] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0200] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0201] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for detecting object defects, characterized in that: include: Determining target feature information of target objects of various models of the same type; wherein the target feature information is used to indicate the object features of the target object in a defect-free state; Acquire a target image of an object to be identified, and extract image feature information of the target image; wherein the object to be identified is of the same type as the target object; Based on the image feature information and the target feature information, defect detection is performed on the object to be identified to obtain a defect detection result; wherein the defect detection result is used to indicate defect information of the object to be identified when a defect exists.

2. The method according to claim 1, characterized in that The performing defect detection on the object to be identified based on the image feature information and the target feature information to obtain a defect detection result includes: Performing block processing on the target image to obtain a plurality of image blocks, and determining sub-image feature information of each of the image blocks based on the image feature information; Calculating the similarity between the sub-image feature information and the target feature information to obtain multiple similarities; A defect detection result of the object to be identified is determined based on the multiple similarities.

3. The method according to claim 2, characterized in that The step of determining the defect detection result of the object to be identified based on the multiple similarities includes: When it is detected that there is a target similarity less than a preset similarity threshold among the multiple similarities, a defect detection result of the object to be identified is determined based on position information of the image block corresponding to the target similarity.

4. The method according to claim 1, characterized in that The determining of target feature information of target objects of the same type and of various models includes: The target feature alignment model is used to extract features from the object images of target objects of the same type and various models to obtain the target feature information.

5. The method according to claim 4, characterized in that The target feature alignment model is obtained by training in the following way: Acquire a training sample image and defect description information of the training sample image; Acquire standard sample features of a standard sample image; wherein the objects in the training sample image and the objects in the standard sample image are of the same type of object; Determine the target detection index between the training sample image, the defect description information and the standard sample image, and train the feature alignment model to be trained based on the target detection index to obtain the target feature alignment model; wherein the target detection index includes: similarity and difference.

6. The method according to claim 5, characterized in that The determining of the target detection index between the training sample image, the defect description information and the standard sample image includes: Extracting features from the training sample images to obtain training sample features; Calculating the similarities between the defect description information, the training sample features, and the standard sample features to obtain multiple similarities, and calculating the differences between the defect description information, the training sample features, and the standard sample features to obtain multiple differences; The multiple similarities and the multiple differences are determined as the target detection indicators.

7. The method according to claim 6, characterized in that The training of the feature alignment model to be trained based on the target detection index includes: Adding the multiple similarities to obtain a target similarity; Performing addition processing on the multiple differences to obtain a target difference; Determining a target loss function based on the target similarity and the target difference; Based on the target loss function, the feature alignment model to be trained is trained to obtain the target feature alignment model.

8. The method according to claim 5, characterized in that The training sample images include a first type of images with defects and a second type of images without defects; The training of the feature alignment model to be trained based on the target detection index includes: Determining target detection indicators corresponding to the first category of images and the second category of images in the target detection indicators, and obtaining a first target detection indicator and a second target detection indicator respectively; Determining a first loss function based on a difference between similarity and difference in the first target detection indicator, and determining a second loss function based on a difference between difference and similarity in the second target detection indicator; A target loss function is determined based on the first loss function and the second loss function.

9. A system for detecting defects in an object, characterized in that: include: A multi-angle image sampling device, an adjustment device and a processor; wherein the adjustment device is arranged on the detection device body of the object to be identified, and the adjustment device is arranged in the upper area of ​​the object to be identified, and the multi-angle image sampling device is arranged on the adjustment device and the detection device body; The regulating device is used to control the corresponding image sampling device to reach a specified position based on the model of the object to be identified; The image sampling device is used to collect images of the object to be identified that enters the collection area of ​​the image sampling device after reaching the designated position, so as to obtain a target image; The processor is used to acquire the target image and perform defect detection on the target image according to the method described in any one of claims 1 to 8 to obtain a defect detection result of the object to be identified.

10. The system according to claim 9, characterized in that The system further includes: an illumination device; wherein the illumination device is disposed adjacent to the image sampling device, and the illumination device is used to provide illumination to a collection area of ​​the image sampling device.

11. The system according to claim 9, characterized in that The system further comprises: a defect display device and a display device; wherein the defect display device comprises a plurality of running lights, and a running light is arranged corresponding to each detection position, and the detection position is the position where the object to be identified is placed; The processor is configured to send a first display signal to a target running light corresponding to the object to be identified, and send a second display signal to the display device when it is determined that the object to be identified has a defect based on a defect detection result of the object to be identified; The target running light is used to turn on after receiving the first display signal; The display device is used to display the defect information of the object to be identified after receiving the second display signal.

12. The system according to claim 10, characterized in that The multi-angle image sampling device includes: a bottom image sampling device and a side image sampling device, wherein the lighting device arranged adjacent to the bottom image sampling device is a dome light source, and the lighting device arranged adjacent to the side image sampling device is a stripe-type surface light source.

13. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claims 1 to 8.

14. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images