Abnormity determination method and device of to-be-detected image

Abstract

The embodiment of the invention provides a to-be-detected image abnormity determination method and device, and relates to the technical field of machine vision automatic detection. The method comprises the steps of obtaining a to-be-detected image, inputting the to-be-detected image into an image segmentation model, and determining a segmentation result; wherein the image segmentation model is used for identifying and segmenting a plurality of target objects in the to-be-detected image; determining a category information image according to a segmentation result; wherein each category information image corresponds to one target object; the number of the category information images is at least two; and inputting the category information image into a preset image recognition model, and based on the preset image recognition model, determining whether the positions among the plurality of target objects in the to-be-detected image are abnormal. By adopting the technical scheme, the logic abnormity of the product can be detected, so that the condition that the product breaks down or the performance is reduced in the subsequent use process is avoided.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of automatic machine vision detection, and in particular to a method and device for determining abnormalities of an image to be detected. Background Art

[0002] In modern industrial production, the requirements for product quality are getting higher and higher. Traditional industrial defect detection mainly focuses on the identification of surface defects, such as cracks, dents or uneven colors. These defects often directly affect the appearance and basic functions of the product, so they are easily taken seriously. However, in addition to surface defects, there is a more hidden type of defect - logical anomalies. Logical anomalies refer to problems that do not conform to design logic or production rules during the assembly or installation process of the product.

[0003] For example, the screws of an electronic device are not tightened, or the components are placed in the wrong position. These problems may not immediately affect the appearance of the product, but may cause the product to malfunction or perform poorly in subsequent use. Since logical anomalies usually involve the relative position or combination of components, they are often difficult to detect using traditional image processing technology.

[0004] Therefore, there is an urgent need for a method for determining anomalies in images to be detected, which can detect logical anomalies of products and thus avoid product failures or performance degradation during subsequent use. Summary of the invention

[0005] In order to solve the above technical problem or at least partially solve the above technical problem, an embodiment of the present disclosure provides a method and device for determining an abnormality of an image to be detected.

[0006] A first aspect of an embodiment of the present disclosure provides a method for determining an abnormality of an image to be detected, the method comprising:

[0007] Acquire an image to be detected, input the image to be detected into an image segmentation model, and determine a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected;

[0008] Determine a category information image according to the segmentation result; wherein each category information image corresponds to one target object; and the number of the category information images is at least two;

[0009] The category information image is input into a preset image recognition model, and based on the preset image recognition model, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal; wherein the preset image recognition model is obtained by training the original image recognition model.

[0010] In one example, determining whether positions between the plurality of target objects in the image to be detected are abnormal based on the preset image recognition model includes:

[0011] Based on the preset image recognition model, determining a target feature vector of the image to be detected; wherein the target feature vector is used to characterize the global image features of the image to be detected;

[0012] Retrieve the mean eigenvector;

[0013] According to the relationship between the mean feature vector and the target feature vector, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal; wherein the mean feature vector is determined by the initial feature vectors of multiple positive samples.

[0014] In one example, determining whether positions between the plurality of target objects in the image to be detected are abnormal according to the relationship between the mean feature vector and the target feature vector includes:

[0015] Calculating the distance value between the mean feature vector and the target feature vector;

[0016] It is determined whether positions between the plurality of target objects in the image to be detected are abnormal according to the distance value.

[0017] In one example, determining whether positions between the plurality of target objects in the image to be detected are abnormal according to the distance value includes:

[0018] If the distance value is greater than a threshold, determining that the positions of the multiple target objects in the image to be detected are abnormal; wherein the threshold is determined by the current feature vectors of the multiple positive samples;

[0019] If the distance value is less than or equal to the threshold, it is determined that the positions of the multiple target objects in the image to be detected are normal.

[0020] In one example, determining the category information image according to the segmentation result includes:

[0021] Based on the image segmentation model, extracting the target object in the segmentation result;

[0022] Generating target object images from the target object; wherein the number of the target object images is equal to the number of the target objects;

[0023] The category information image is generated by using the plurality of target object images.

[0024] In one example, extracting the target object in the segmentation result based on the image segmentation model includes:

[0025] Based on the image segmentation model, the target object in the segmentation result is extracted sequentially according to the pixels.

[0026] In one example, after generating the category information image from the plurality of target object images, the method further includes:

[0027] The category information image is scaled according to a first preset size to obtain a scaled category information image.

[0028] In one example, after obtaining the zoomed category information image, the method further includes:

[0029] After pooling the scaled category information image to a second preset size, a pooled category information image is obtained;

[0030] The pooled category information image is converted into a one-dimensional feature vector; wherein the one-dimensional feature vector is used to be input into the preset image recognition model.

[0031] In one example, the image segmentation model is obtained by training with multiple labeled positive samples.

[0032] In one example, the image segmentation model is obtained by training with a plurality of positive samples with annotations, including:

[0033] Retrieving a preset number of positive samples from a preset database; wherein the positive samples represent images of the same product that meet preset conditions;

[0034] Performing labeling processing on multiple target objects in the positive sample to obtain the positive sample with labels;

[0035] The labeled positive samples are input into the original image segmentation model, and the original image segmentation model is trained to obtain an image segmentation model.

[0036] In one example, the labeling of the multiple target objects in the positive sample to obtain the positive sample with labels includes:

[0037] Pixel points in the positive sample are obtained, and a plurality of target objects in the positive sample are labeled according to the pixel points to obtain the positive sample with the label.

[0038] In one example, the target object includes at least one of the following: background, substrate, laser engraving line, and auxiliary material.

[0039] In one example, the mean feature vector is determined by initial feature vectors of multiple positive samples, including:

[0040] Obtaining an initial feature vector of each of the positive samples and the number of the positive samples;

[0041] The mean eigenvector is determined according to the initial eigenvector of each of the positive samples and the number of the positive samples.

[0042] In one example, obtaining an initial feature vector of each positive sample includes:

[0043] The category information image of the positive sample is input into the original image recognition model to obtain the initial feature vector of the positive sample.

[0044] In one example, determining the mean feature vector according to the initial feature vector of each of the positive samples and the number of the positive samples includes:

[0045] Calculate a sum feature vector according to the initial feature vector of each positive sample;

[0046] The sum feature vector is divided by the number of positive samples to determine the mean feature vector.

[0047] In one example, calculating the distance value between the mean feature vector and the target feature vector includes at least one of the following methods:

[0048] Euclidean distance, cosine distance, Chebyshev distance.

[0049] In one example, the threshold is determined by current feature vectors of multiple positive samples, including:

[0050] The threshold is determined according to the current feature vector and the mean feature vector of each positive sample.

[0051] In one example, determining the threshold according to the current feature vector and the mean feature vector of each of the positive samples includes:

[0052] Calculating the distance value between the current feature vector and the mean feature vector of each of the positive samples, and determining the maximum distance value among the distance values;

[0053] The maximum distance value is determined as the threshold.

[0054] A second aspect of an embodiment of the present disclosure provides a device for determining an abnormality of an image to be detected, the device comprising:

[0055] An acquisition module is used to acquire an image to be detected, input the image to be detected into an image segmentation model, and determine a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected;

[0056] A first determination module is used to determine a category information image according to the segmentation result; wherein each category information image corresponds to one target object; and the number of the category information images is at least two;

[0057] The second determination module is used to input the category information image into a preset image recognition model, and determine whether the positions between the multiple target objects in the image to be detected are abnormal based on the preset image recognition model; wherein the preset image recognition model is obtained by training the original image recognition model.

[0058] In one example, the second determining module is specifically configured to:

[0059] Based on the preset image recognition model, determining a target feature vector of the image to be detected; wherein the target feature vector is used to characterize the global image features of the image to be detected;

[0060] Retrieve the mean eigenvector;

[0061] According to the relationship between the mean feature vector and the target feature vector, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal; wherein the mean feature vector is determined by the initial feature vectors of multiple positive samples.

[0062] In one example, the second determining module is specifically configured to:

[0063] Calculating the distance value between the mean feature vector and the target feature vector;

[0064] It is determined whether positions between the plurality of target objects in the image to be detected are abnormal according to the distance value.

[0065] In one example, the second determining module is specifically configured to:

[0066] If the distance value is greater than a threshold, determining that the positions of the multiple target objects in the image to be detected are abnormal; wherein the threshold is determined by the current feature vectors of the multiple positive samples;

[0067] If the distance value is less than or equal to the threshold, it is determined that the positions of the multiple target objects in the image to be detected are normal.

[0068] In one example, the first determining module is used to:

[0069] Based on the image segmentation model, extracting the target object in the segmentation result;

[0070] Generating target object images from the target object; wherein the number of the target object images is equal to the number of the target objects;

[0071] The category information image is generated by using the plurality of target object images.

[0072] In one example, the first determining module is specifically configured to:

[0073] Based on the image segmentation model, the target object in the segmentation result is extracted sequentially according to the pixels.

[0074] In one example, after generating the category information image from the plurality of target object images, the apparatus further includes:

[0075] The scaling processing module is used to scale the category information image according to a first preset size to obtain a scaled category information image.

[0076] In one example, after obtaining the zoomed category information image, the device further includes:

[0077] A pooling processing module, used for pooling the scaled category information image to a second preset size to obtain a pooled category information image;

[0078] A conversion processing module is used to convert the category information image after pooling processing into a one-dimensional feature vector; wherein the one-dimensional feature vector is used to be input into the preset image recognition model.

[0079] In one example, the image segmentation model is obtained by training with multiple labeled positive samples.

[0080] In one example, the image segmentation model is obtained by training with a plurality of positive samples with annotations, including:

[0081] Retrieving a preset number of positive samples from a preset database; wherein the positive samples represent images of the same product that meet preset conditions;

[0082] Performing labeling processing on multiple target objects in the positive sample to obtain the positive sample with labels;

[0083] The labeled positive samples are input into the original image segmentation model, and the original image segmentation model is trained to obtain an image segmentation model.

[0084] In one example, the labeling of the multiple target objects in the positive sample to obtain the positive sample with labels includes:

[0085] Pixel points in the positive sample are obtained, and a plurality of target objects in the positive sample are labeled according to the pixel points to obtain the positive sample with the label.

[0086] In one example, the target object includes at least one of the following: background, substrate, laser engraving line, and auxiliary material.

[0087] In one example, the mean feature vector is determined by initial feature vectors of multiple positive samples, including:

[0088] Obtaining an initial feature vector of each of the positive samples and the number of the positive samples;

[0089] The mean eigenvector is determined according to the initial eigenvector of each of the positive samples and the number of the positive samples.

[0090] In one example, obtaining an initial feature vector of each positive sample includes:

[0091] The category information image of the positive sample is input into the original image recognition model to obtain the initial feature vector of the positive sample.

[0092] In one example, determining the mean feature vector according to the initial feature vector of each of the positive samples and the number of the positive samples includes:

[0093] Calculate a sum feature vector according to the initial feature vector of each positive sample;

[0094] The sum feature vector is divided by the number of positive samples to determine the mean feature vector.

[0095] In one example, calculating the distance value between the mean feature vector and the target feature vector includes at least one of the following methods:

[0096] Euclidean distance, cosine distance, Chebyshev distance.

[0097] In one example, the threshold is determined by current feature vectors of multiple positive samples, including:

[0098] The threshold is determined according to the current feature vector and the mean feature vector of each positive sample.

[0099] In one example, determining the threshold according to the current feature vector and the mean feature vector of each of the positive samples includes:

[0100] Calculating the distance value between the current feature vector and the mean feature vector of each of the positive samples, and determining the maximum distance value among the distance values;

[0101] The maximum distance value is determined as the threshold.

[0102] A third aspect of an embodiment of the present disclosure provides an electronic device, the server comprising: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method of the first aspect above.

[0103] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect described above can be implemented.

[0104] The disclosed embodiment provides a method and device for determining abnormality of an image to be detected, the method comprising: acquiring the image to be detected, inputting the image to be detected into an image segmentation model, and determining the segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected; determining a category information image based on the segmentation result; wherein each category information image corresponds to one target object; the number of category information images is at least two; inputting the category information image into a preset image recognition model, and determining whether the positions between the multiple target objects in the image to be detected are abnormal based on the preset image recognition model. The use of this technical solution can detect logical abnormalities of the product, thereby avoiding product failures or performance degradation during subsequent use. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0106] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0107] Figure 1 It is a flowchart of a method for determining abnormality of an image to be detected provided by an embodiment of the present disclosure;

[0108] Figure 2 It is a flowchart of a method for determining abnormality of an image to be detected provided by an embodiment of the present disclosure;

[0109] Figure 3 is a schematic diagram of an abnormality determination process of an image to be detected provided by an embodiment of the present disclosure;

[0110] Figure 4 is a structural schematic diagram of a device for determining abnormality of an image to be detected provided by an embodiment of the present disclosure;

[0111] Figure 5 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0112] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0113] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0114] Figure 1 1 is a flow chart of a method for determining abnormality of an image to be detected provided by an embodiment of the present disclosure, and the method can be performed by an electronic device. The electronic device can be exemplarily understood as a device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, etc. Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0115] S101, obtaining an image to be detected, inputting the image to be detected into an image segmentation model, and determining a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected.

[0116] In one example, the image to be detected is an image of a product to be detected. There may be multiple images to be detected. The image to be detected includes multiple target objects, and the target objects include at least one of the following: background, substrate, laser engraving line, and auxiliary materials. The number of auxiliary materials is at least one. The image segmentation model may be a U-net model.

[0117] In this embodiment, the segmentation result is image information including multiple target objects.

[0118] S102. Determine category information images according to the segmentation result; wherein each category information image corresponds to a target object; and the number of category information images is at least two.

[0119] In one example, the number of category information images is consistent with the number of target objects, that is, if the number of target objects is 3, then the number of category information images is also 3.

[0120] S103: input the category information image into a preset image recognition model, and determine whether positions between multiple target objects in the image to be detected are abnormal based on the preset image recognition model.

[0121] In one example, the preset image recognition model may be a Vi T model. The preset image recognition model is used to identify the position between multiple target objects in the image to be detected, and then it is determined whether the position is abnormal. Specifically, the abnormal situation may be an abnormal distance between the target objects, or an abnormal stacking relationship between the target objects. The preset image recognition model is obtained by training the original image recognition model.

[0122] The disclosed embodiment provides a method for determining abnormality of an image to be detected, the method comprising: obtaining the image to be detected, inputting the image to be detected into an image segmentation model, and determining the segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected; determining a category information image according to the segmentation result; wherein each category information image corresponds to a target object; the number of category information images is at least two; inputting the category information image into a preset image recognition model, and determining whether the positions between the multiple target objects in the image to be detected are abnormal based on the preset image recognition model. The use of this technical solution can detect logical abnormalities of the product, thereby avoiding the occurrence of product failures or performance degradation during subsequent use.

[0123] Figure 2 The embodiment of the present disclosure is optimized on the basis of the above embodiment, and the embodiment of the present disclosure can be combined with various optional solutions in one or more of the above embodiments.

[0124] like Figure 2 As shown, the method for determining abnormality of the image to be detected may include the following steps:

[0125] S201, obtaining an image to be detected, inputting the image to be detected into an image segmentation model, and determining a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected.

[0126] In one example, an image segmentation model is trained with multiple labeled positive samples.

[0127] In one example, an image segmentation model is trained with multiple positive examples with annotations, including:

[0128] Retrieving a preset number of positive samples from a preset database; wherein the positive samples represent images of the same product that meet preset conditions;

[0129] Label multiple target objects in the positive sample to obtain positive samples with labels;

[0130] The labeled positive samples are input into the original image segmentation model, and the original image segmentation model is trained to obtain an image segmentation model.

[0131] In one example, the preset database is used to store positive samples, and the preset number may be 80. The positive samples represent images of the same product that meet preset conditions, that is, images of qualified products.

[0132] In one example, a target object in a positive sample is annotated, for example, the annotation process can be pixel-level annotation, and then a positive sample with annotations is obtained, wherein the number of annotations in the positive sample with annotations is the number of target objects. After obtaining the positive sample with annotations, the positive sample with annotations is input into the original image segmentation model, and the original image segmentation model is trained by the annotations and the positive sample to obtain the image segmentation model. Specifically, the training is performed based on the first loss function, wherein the first loss function is the mean square error, and specifically, the first loss function is the mean of the difference between the segmentation result of each pixel of the positive sample and the annotation result of the positive sample. Then the first loss function is minimized to obtain the image segmentation model.

[0133] In one example, multiple target objects in a positive sample are labeled to obtain positive samples with labels, including:

[0134] The pixels in the positive sample are obtained, and multiple target objects in the positive sample are labeled according to the pixels to obtain the positive sample with labels.

[0135] In one example, the process of labeling multiple target objects in the positive sample according to pixel points is the same as the process of the panoptic segmentation task.

[0136] S202: Extract the target object in the segmentation result based on the image segmentation model.

[0137] In one example, based on the image segmentation model, extracting the target object in the segmentation result includes:

[0138] Based on the image segmentation model, the target objects in the segmentation results are extracted one by one according to the pixels.

[0139] In an example, the target objects in the segmentation result may be extracted sequentially according to the pixel points in the rows, or the target objects in the segmentation result may be extracted sequentially according to the pixel points in the columns.

[0140] S203, generating target object images from the target object; wherein the number of target object images is equal to the number of target objects.

[0141] In one example, a target object generates a target object image, wherein the generated target object image contains other target objects, but the colors of the other target objects are different from the color of the target object. The benefit of such a setting is to display the target object.

[0142] S204, generating category information images from the multiple target object images; wherein each category information image corresponds to one target object; and the number of category information images is at least two.

[0143] In one example, after generating a category information image from a plurality of target object images, the method further includes:

[0144] The category information image is scaled according to a first preset size to obtain a scaled category information image.

[0145] In one example, since the size of the image that can be processed by the preset image recognition model is limited, the category information image needs to be scaled according to a first preset size, and then the scaled category information image can be input into the preset image recognition model. The first preset size can be 128x128 pixels.

[0146] In one example, after obtaining the zoomed category information image, the method further includes:

[0147] After pooling the scaled category information image to a second preset size, a pooled category information image is obtained;

[0148] The category information image after pooling processing is converted into a one-dimensional feature vector; wherein the one-dimensional feature vector is used to input into a preset image recognition model.

[0149] In one example, the second preset size may be 8x8 pixels. In this embodiment, the scaled category information image is first pooled and then converted into a one-dimensional feature vector. The benefit of this setting is to increase the accuracy of the preset image recognition model.

[0150] S205 , inputting the category information image into a preset image recognition model, and determining a target feature vector of the image to be detected based on the preset image recognition model; wherein the target feature vector is used to characterize the global image features of the image to be detected.

[0151] In an example, the target feature vector of the image to be detected may be F. The target feature vector F is used to characterize the global image features of the image to be detected. Specifically, it may include multiple target objects and also include the positional relationship between the target objects.

[0152] S206, retrieve the mean feature vector.

[0153] In one example, the mean feature vector is determined by initial feature vectors of multiple positive samples, including:

[0154] Get the initial feature vector of each positive sample and the number of positive samples;

[0155] The mean eigenvector is determined based on the initial eigenvector of each positive sample and the number of positive samples.

[0156] In one example, obtaining an initial feature vector of each positive sample includes:

[0157] The category information image of the positive sample is input into the original image recognition model to obtain the initial feature vector of the positive sample.

[0158] In this embodiment, the category information image of the positive sample is obtained based on the positive sample, and the category information image of the positive sample is input into the original image recognition model to obtain the initial feature vector of the positive sample. For example, the initial feature vector of the positive sample A is FA, the initial feature vector of the positive sample B is FB, and the initial feature vector of the positive sample C is FC, then the number of positive samples is 3.

[0159] In one example, determining the mean feature vector according to the initial feature vector of each positive sample and the number of positive samples includes:

[0160] According to the initial feature vector of each positive sample, calculate the sum feature vector;

[0161] Divide the sum eigenvector by the number of positive samples to determine the mean eigenvector.

[0162] Further, the mean eigenvector FM=(FA+FB+FC) divided by 3. Wherein, FA+FB+FC is the sum eigenvector.

[0163] S207. Determine whether positions between multiple target objects in the image to be detected are abnormal based on the relationship between the mean feature vector and the target feature vector; wherein the mean feature vector is determined by initial feature vectors of multiple positive samples.

[0164] In one example, determining whether positions between multiple target objects in an image to be detected are abnormal based on a relationship between a mean feature vector and a target feature vector includes:

[0165] Calculate the distance value between the mean eigenvector and the target eigenvector;

[0166] According to the distance value, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal.

[0167] In one example, the distance value is used to represent the similarity between the mean feature vector and the target feature vector.

[0168] In one example, calculating the distance value between the mean feature vector and the target feature vector includes at least one of the following methods:

[0169] Euclidean distance, cosine distance, Chebyshev distance.

[0170] In one example, determining whether positions between multiple target objects in an image to be detected are abnormal according to the distance value includes:

[0171] If the distance value is greater than a threshold, it is determined that the positions of the multiple target objects in the image to be detected are abnormal; wherein the threshold is determined by the current feature vectors of the multiple positive samples;

[0172] If the distance value is less than or equal to the threshold, it is determined that the positions of the multiple target objects in the image to be detected are normal.

[0173] In one example, if the distance value is greater than a threshold, it means that the difference between the mean feature vector and the target feature vector is large, and it is determined that the positions of the multiple target objects in the image to be detected are abnormal. If the distance value is less than or equal to the threshold, it means that the difference between the mean feature vector and the target feature vector is small, and it is determined that the positions of the multiple target objects in the image to be detected are normal.

[0174] In one example, the threshold is determined by the current feature vectors of multiple positive samples, including:

[0175] The threshold is determined based on the current feature vector and the mean feature vector of each positive sample.

[0176] In one example, determining a threshold according to a current feature vector and a mean feature vector of each positive sample includes:

[0177] Calculate the distance between the current feature vector and the mean feature vector of each positive sample, and determine the maximum distance value among the distance values;

[0178] The maximum distance value is determined as the threshold.

[0179] In one example, the current feature vector of the positive sample is obtained by inputting the positive sample into a preset image recognition model. Specifically, the distance values ​​between the current feature vectors of multiple positive samples and the mean feature vector can be calculated, and the maximum distance value can be determined from the distance values, and the maximum distance value can be determined as the threshold. For a clearer explanation, please refer to Figure 3 A schematic diagram of an abnormality determination process of an image to be detected is shown.

[0180] In one example, the preset image recognition model is obtained by training the original image recognition model, specifically, by training based on the second loss function. The second loss function is the mean of the distance values ​​between the initial feature vector and the mean feature vector of each positive sample. During the training process, the weight information of the original image recognition model is optimized with the goal of minimizing the second loss function, thereby obtaining the preset image recognition model.

[0181] The disclosed embodiment provides a method for determining abnormality of an image to be detected, the method comprising: determining whether the positions between multiple target objects in the image to be detected are abnormal according to the relationship between the mean feature vector and the target feature vector; wherein the mean feature vector is determined by the initial feature vectors of multiple positive samples. The technical solution reduces the degree of dependence of the preset image recognition model on negative samples.

[0182] Figure 4 is a schematic diagram of the structure of a device for determining abnormality of an image to be detected provided by an embodiment of the present disclosure. The device for determining abnormality of an image to be detected can be understood as the above-mentioned electronic device or a part of the functional modules in the above-mentioned electronic device. Figure 4 As shown, the abnormality determination device 40 of the image to be detected includes:

[0183] The acquisition module 401 is used to acquire the image to be detected, input the image to be detected into the image segmentation model, and determine the segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected;

[0184] A first determination module 402 is used to determine a category information image according to the segmentation result; wherein each category information image corresponds to a target object; and the number of category information images is at least two;

[0185] The second determination module 403 is used to input the category information image into a preset image recognition model, and determine whether the positions between multiple target objects in the image to be detected are abnormal based on the preset image recognition model; wherein the preset image recognition model is obtained by training the original image recognition model.

[0186] In one example, the second determining module 403 is specifically configured to:

[0187] Based on a preset image recognition model, a target feature vector of the image to be detected is determined; wherein the target feature vector is used to characterize the global image features of the image to be detected;

[0188] Retrieve the mean eigenvector;

[0189] According to the relationship between the mean feature vector and the target feature vector, it is determined whether the positions of multiple target objects in the image to be detected are abnormal; wherein the mean feature vector is determined by the initial feature vectors of multiple positive samples.

[0190] In one example, the second determining module 403 is specifically configured to:

[0191] Calculate the distance value between the mean eigenvector and the target eigenvector;

[0192] According to the distance value, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal.

[0193] In one example, the second determining module 403 is specifically configured to:

[0194] If the distance value is greater than a threshold, it is determined that the positions of the multiple target objects in the image to be detected are abnormal; wherein the threshold is determined by the current feature vectors of the multiple positive samples;

[0195] If the distance value is less than or equal to the threshold, it is determined that the positions of the multiple target objects in the image to be detected are normal.

[0196] In one example, the first determining module 402 is configured to:

[0197] Based on the image segmentation model, extract the target object in the segmentation result;

[0198] Generating target object images from the target object; wherein the number of the target object images is equal to the number of the target objects;

[0199] A category information image is generated from a plurality of target object images.

[0200] In one example, the first determining module 402 is specifically configured to:

[0201] Based on the image segmentation model, the target objects in the segmentation results are extracted one by one according to the pixels.

[0202] In one example, after generating a category information image from a plurality of target object images, the apparatus 40 further includes:

[0203] The scaling processing module 404 is used to scale the category information image according to a first preset size to obtain a scaled category information image.

[0204] In one example, after obtaining the category information image after scaling, the device 40 further includes:

[0205] A pooling processing module 405 is used to pool the scaled category information image to a second preset size to obtain a pooled category information image;

[0206] The conversion processing module 406 is used to convert the category information image after the pooling process into a one-dimensional feature vector; wherein the one-dimensional feature vector is used to be input into a preset image recognition model.

[0207] In one example, an image segmentation model is trained with multiple labeled positive samples.

[0208] In one example, an image segmentation model is trained with multiple positive examples with annotations, including:

[0209] Retrieving a preset number of positive samples from a preset database; wherein the positive samples represent images of the same product that meet preset conditions;

[0210] Label multiple target objects in the positive sample to obtain positive samples with labels;

[0211] The labeled positive samples are input into the original image segmentation model, and the original image segmentation model is trained to obtain an image segmentation model.

[0212] In one example, multiple target objects in a positive sample are labeled to obtain positive samples with labels, including:

[0213] The pixels in the positive sample are obtained, and multiple target objects in the positive sample are labeled according to the pixels to obtain the positive sample with labels.

[0214] In one example, the target object includes at least one of the following: background, substrate, laser engraving line, and auxiliary material.

[0215] In one example, the mean feature vector is determined by initial feature vectors of multiple positive samples, including:

[0216] Get the initial feature vector of each positive sample and the number of positive samples;

[0217] The mean eigenvector is determined based on the initial eigenvector of each positive sample and the number of positive samples.

[0218] In one example, obtaining an initial feature vector of each positive sample includes:

[0219] The category information image of the positive sample is input into the original image recognition model to obtain the initial feature vector of the positive sample.

[0220] In one example, determining the mean feature vector according to the initial feature vector of each positive sample and the number of positive samples includes:

[0221] According to the initial feature vector of each positive sample, calculate the sum feature vector;

[0222] Divide the sum eigenvector by the number of positive samples to determine the mean eigenvector.

[0223] In one example, calculating the distance value between the mean feature vector and the target feature vector includes at least one of the following methods:

[0224] Euclidean distance, cosine distance, Chebyshev distance.

[0225] In one example, the threshold is determined by the current feature vectors of multiple positive samples, including:

[0226] The threshold is determined based on the current feature vector and the mean feature vector of each positive sample.

[0227] In one example, determining a threshold according to a current feature vector and a mean feature vector of each positive sample includes:

[0228] Calculate the distance between the current feature vector and the mean feature vector of each positive sample, and determine the maximum distance value among the distance values;

[0229] The maximum distance value is determined as the threshold.

[0230] The device provided in this embodiment can execute the method of any of the above embodiments, and its execution method and beneficial effects are similar, which will not be repeated here.

[0231] An embodiment of the present disclosure further provides an electronic device, which includes: a memory, in which a computer program is stored; and a processor, which is used to execute the computer program. When the computer program is executed by the processor, the method of any of the above embodiments can be implemented.

[0232] For example, Figure 5Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 5 , which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0233] like Figure 5 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0234] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0235] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0236] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0237] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0238] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0239] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the image to be detected, inputs the image to be detected into an image segmentation model, and determines the segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected; determines a category information image based on the segmentation result; wherein each category information image corresponds to a target object; the number of category information images is at least two; inputs the category information image into a preset image recognition model, and determines whether the positions between the multiple target objects in the image to be detected are abnormal based on the preset image recognition model.

[0240] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0241] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0242] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0243] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0244] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0245] The embodiments of the present disclosure further provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The execution method and beneficial effects are similar and will not be repeated here.

[0246] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0247] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining abnormality of an image to be detected, characterized in that: include: Acquire an image to be detected, input the image to be detected into an image segmentation model, and determine a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected; Determine a category information image according to the segmentation result; wherein each category information image corresponds to one target object; and the number of the category information images is at least two; The category information image is input into a preset image recognition model, and based on the preset image recognition model, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal; wherein the preset image recognition model is obtained by training the original image recognition model.

2. The method according to claim 1, characterized in that The determining whether positions between the plurality of target objects in the image to be detected are abnormal based on the preset image recognition model includes: Based on the preset image recognition model, determining a target feature vector of the image to be detected; wherein the target feature vector is used to characterize the global image features of the image to be detected; Retrieve the mean eigenvector; According to the relationship between the mean feature vector and the target feature vector, it is determined whether the positions between the multiple target objects in the image to be detected are abnormal; wherein the mean feature vector is determined by the initial feature vectors of multiple positive samples.

3. The method according to claim 2, characterized in that The step of determining whether positions between the plurality of target objects in the image to be detected are abnormal according to the relationship between the mean feature vector and the target feature vector comprises: Calculating the distance value between the mean feature vector and the target feature vector; It is determined whether positions between the plurality of target objects in the image to be detected are abnormal according to the distance value.

4. The method according to claim 3, characterized in that The step of determining whether positions between the plurality of target objects in the image to be detected are abnormal according to the distance value comprises: If the distance value is greater than a threshold, determining that the positions of the multiple target objects in the image to be detected are abnormal; wherein the threshold is determined by the current feature vectors of the multiple positive samples; If the distance value is less than or equal to the threshold, it is determined that the positions of the multiple target objects in the image to be detected are normal.

5. The method according to claim 1, characterized in that Determining the category information image according to the segmentation result includes: Based on the image segmentation model, extracting the target object in the segmentation result; Generating target object images from the target object; wherein the number of the target object images is equal to the number of the target objects; The category information image is generated by using the plurality of target object images.

6. The method according to claim 5, characterized in that The step of extracting the target object from the segmentation result based on the image segmentation model includes: Based on the image segmentation model, the target object in the segmentation result is extracted sequentially according to the pixels.

7. The method according to claim 5, characterized in that After generating the category information image from the plurality of target object images, the method further comprises: The category information image is scaled according to a first preset size to obtain a scaled category information image.

8. A device for determining abnormality of an image to be detected, characterized in that: The device comprises: An acquisition module is used to acquire an image to be detected, input the image to be detected into an image segmentation model, and determine a segmentation result; wherein the image segmentation model is used to identify and segment multiple target objects in the image to be detected; A first determination module is used to determine a category information image according to the segmentation result; wherein each category information image corresponds to one target object; and the number of the category information images is at least two; The second determination module is used to input the category information image into a preset image recognition model, and determine whether the positions between the multiple target objects in the image to be detected are abnormal based on the preset image recognition model; wherein the preset image recognition model is obtained by training the original image recognition model.

9. An electronic device, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images