Information processing method and apparatus, and electronic device and storage medium

Abstract

Embodiments of the present application disclose an information processing method and device, electronic equipment and a storage medium. The method comprises: an image of a calibration object is collected by a photographing device based on different values of a photographing parameter; a part of first candidate values with better photographing quality are selected based on evaluation indexes of the collected images; an image of a product to be detected is collected by the photographing device based on each of the first candidate values; and second candidate values with better photographing quality are selected based on semantic features of the collected images, so as to determine a target value of the photographing parameter according to the second candidate values.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and more specifically, to an information processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence, defect detection models are increasingly being applied in actual production and daily life. These models detect defects by processing product images, and their accuracy heavily relies on the image quality of the photographic equipment. However, many factories currently use photographic equipment designed for only one type of product or defect. Changes in production lines or the environment can affect the imaging quality of these devices, consequently impacting the accuracy of the detection model.

[0003] Therefore, when production lines change or the environment changes, the imaging parameters of the photography equipment need to be reset. Currently, these parameters are manually set by professional photographers based on various conditions such as lighting and product size, which is time-consuming and labor-intensive. Summary of the Invention

[0004] This application provides an information processing method, apparatus, electronic device, and storage medium, including the following technical solutions:

[0005] An information processing method, the method comprising:

[0006] The image of the calibration object is acquired by the camera based on different values ​​of the camera parameters.

[0007] The first candidate value of the photographing parameter is determined based on the evaluation index of each frame image of the calibrated object, wherein the evaluation index of the image acquired based on the first candidate value is better than the evaluation index of the image acquired based on non-first candidate values.

[0008] The imaging device acquires images of the product under inspection based on each first candidate value;

[0009] Based on the semantic features of each frame of the product being inspected, a second candidate value is determined from the first candidate value. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0010] The target value of the image capture parameter is determined based on the second candidate value.

[0011] In the above method, preferably, determining the second candidate value from the first candidate values ​​based on the semantic features of each frame image of the detected product includes:

[0012] The feature extractor in the pre-trained detection model is used to extract features from each frame of the image of the product being detected.

[0013] For each frame of the inspected product, features of pre-labeled defective and non-defective regions are extracted from the feature map of each frame.

[0014] A second candidate value is determined from the first candidate values ​​based on the features of the defective region and the features of the non-defective region extracted from each frame image of the inspected product.

[0015] The above method, preferably, includes determining the second candidate value from the first candidate values ​​based on the features of the defective region and the features of the non-defective region extracted from each frame image of the inspected product, comprising:

[0016] Obtain the feature difference values ​​of defective and non-defective regions in each frame image of the inspected product;

[0017] The first candidate value corresponding to the image of the detected product whose feature difference value satisfies the first condition is determined as the second candidate value; wherein, the feature difference value that satisfies the first condition is greater than the feature difference value that does not satisfy the first condition.

[0018] In the above method, preferably, determining the first candidate value of the photographing parameter based on each frame of the acquired image of the calibration object includes:

[0019] Obtain at least one evaluation index for each frame of the acquired calibration object;

[0020] Based on the evaluation metrics of each frame of the calibration object, a comprehensive metric for each frame of the calibration object is obtained.

[0021] The values ​​of the photographing parameters corresponding to the images whose comprehensive indicators meet the second condition are determined as the first candidate values; the comprehensive indicators that meet the second condition are better than the comprehensive indicators that do not meet the second condition.

[0022] In the above method, preferably, determining the target value of the photographing parameter based on the second candidate value includes: repeatedly executing the following process to obtain a new second candidate value until the termination condition is met:

[0023] A third candidate value is generated based on the latest obtained second candidate value; each third candidate value is obtained by combining some values ​​from at least two second candidate values;

[0024] The imaging device captures images of the product under test based on each third candidate value. Based on the semantic features of each frame of the captured images of the product under test, a new second candidate value is determined from the third candidate values.

[0025] Select one of the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value.

[0026] In the above method, preferably, the termination condition includes:

[0027] The number of iterations in the process of obtaining the new second candidate value reaches the target number; and / or,

[0028] In the image of the product being inspected acquired by the imaging device based on the latest second candidate value, the feature difference between the pre-marked defect area and the non-defect area is greater than the target difference threshold, or the rate of increase of the feature difference between the pre-marked defect area and the non-defect area is less than the rate threshold.

[0029] Preferably, the above method yields better detection results when images acquired based on the target value are used in a defect detection model compared to images acquired based on non-target values.

[0030] In the above method, preferably, selecting a second candidate value from the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value includes:

[0031] If the termination condition is met, the imaging device will capture images of the detected product based on each of the new second candidate values ​​obtained.

[0032] The image is displayed as a series of frames captured by the imaging device based on the obtained new second candidate values.

[0033] In response to a selection command, a target image frame is determined from the displayed images;

[0034] The second candidate value corresponding to the target image is determined as the target value.

[0035] In the above method, preferably, the calibration material includes one of a standard calibration plate and the product being tested.

[0036] An information processing apparatus, the apparatus comprising:

[0037] The first acquisition module is used to acquire images of the calibration object by means of a camera device based on different values ​​of the camera parameters;

[0038] The first determining module is used to determine a first candidate value of the photographing parameter based on the evaluation index of each frame image of the collected calibration object, wherein the evaluation index of the image collected based on the first candidate value is better than the evaluation index of the image collected based on non-first candidate values.

[0039] The second acquisition module is used to acquire images of the product being inspected based on each first candidate value using the imaging device.

[0040] The second determining module is used to determine a second candidate value from the first candidate values ​​based on the semantic features of each frame image of the product being detected. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0041] The third determining module is used to determine the target value of the photographing parameters based on the second candidate value.

[0042] An electronic device, comprising:

[0043] Memory, used to store programs;

[0044] A processor for calling and executing the program in the memory, thereby implementing the steps of the information processing method as described in any of the preceding claims.

[0045] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the information processing method as described in any of the preceding claims.

[0046] As can be seen from the above scheme, the information processing method, apparatus, electronic device, and storage medium provided in this application acquire images of a calibration object based on different values ​​of photographing parameters using a photographing device. First candidate values ​​with better image quality are selected based on evaluation indicators of the acquired images. Then, the photographing device acquires images of the product under test based on each first candidate value. The first candidate values ​​are further filtered based on semantic features of the acquired images to select second candidate values ​​with better image quality. Finally, the target values ​​of the photographing parameters are determined based on the second candidate values. This application incorporates an automatic filtering process with different dimensions during the adjustment of the photographing parameters of the photographing device, thereby reducing the workload of manual operation and improving the efficiency of adjusting the photographing parameters of the photographing device. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart illustrating an implementation of the information processing method provided in this application embodiment;

[0049] Figure 2 A flowchart illustrating an implementation of this application for determining a second candidate value from a first candidate value based on the semantic features of each frame of the collected images of the product under test;

[0050] Figure 3a A flowchart illustrating an implementation of this application for determining a second candidate value from a first candidate value based on features of defective and non-defective regions extracted from each frame of an image of the inspected product;

[0051] Figure 3b A flowchart illustrating an implementation of obtaining the feature difference values ​​between defective and non-defective regions in each frame of an image of a product under inspection, as provided in this application embodiment;

[0052] Figure 4 A flowchart illustrating an implementation of determining a first candidate value for photographing parameters based on each frame of the collected calibration object, as provided in this application embodiment;

[0053] Figure 5 A flowchart illustrating one implementation of obtaining a new second candidate value provided in this application embodiment;

[0054] Figure 6 A schematic diagram of the structure of an information processing apparatus provided in an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0056] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar parts and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated herein. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0058] A flowchart of an implementation of the information processing method provided in this application is shown below. Figure 1 As shown, it may include:

[0059] Step S101: The image of the calibration object is acquired by the camera based on different values ​​of the camera parameters.

[0060] The photographic equipment here is used for defect detection in actual production and daily life; that is, the photographic equipment is set up in the actual production / living environment.

[0061] In this application, the photographing parameters of the photographing device are a set of parameters related to taking a picture. As an example, this set of parameters may include, but is not limited to: aperture size, focal length, exposure time, light source color, lighting angle, lighting intensity, etc. For ease of description, the photographing parameters are denoted as θ, where θ is an N-dimensional vector. Each element in this vector is a value of a parameter, and different elements represent different parameter values. Here, N is the number of parameters in the aforementioned set of parameters.

[0062] For any two values ​​of the photography parameter θ, denoted as vector θ1 and vector θ2, if θ1 and θ2 are different, then θ1 and θ2 have at least one different element; if θ1 and θ2 are the same, then all elements of θ1 and θ2 are the same.

[0063] In this application, the different values ​​of the imaging parameter θ can be multiple different values ​​randomly sampled from several possible values ​​of the imaging parameter θ. Or,

[0064] Different values ​​of the image capture parameter θ can be obtained by sampling from a set of possible values ​​of θ according to a certain sampling rule. For example, different values ​​of the image capture parameter may have different numbers. The different values ​​of the image capture parameter can be arranged in descending or ascending order of their numbers. The sampling rule could be to select one value from every M values ​​(where M is an integer greater than 1). Or,

[0065] Different values ​​of the image parameter θ can be obtained by sampling a certain number of values ​​from several possible values ​​of the image parameter based on random sampling or according to sampling rules, and then determining multiple different values ​​based on the iterative process of a genetic algorithm on the sampled values.

[0066] In this application, an image of the calibration object can be acquired separately based on each value of the imaging parameters using an imaging device. That is, assuming there are b different values ​​for the imaging parameters, b frames of images can be obtained.

[0067] Step S102: Determine the first candidate value of the shooting parameters based on the evaluation index of each frame image of the collected calibration object, wherein the evaluation index of the image collected based on the first candidate value is better than the evaluation index of the image collected based on non-first candidate values.

[0068] This application selects images with better evaluation indicators from each frame of the collected calibration object, and determines the values ​​of the imaging parameters used in the selected images by the imaging device as the first candidate values.

[0069] Step S103: The above-mentioned imaging device is used to capture images of the product being inspected based on each first candidate value.

[0070] The product being inspected is a defective product. There can be only one product or multiple products being inspected.

[0071] When there are multiple products to be inspected, for each product, images of the inspected product can be captured separately using the aforementioned imaging device based on each first candidate value. That is, if there are 'a' products to be inspected and 'b' first candidate values, then a*b frames of images of the inspected products can be obtained.

[0072] Step S104: Based on the semantic features of each frame of the product being inspected, determine the second candidate value from the first candidate value. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0073] The semantic features of an image refer to the features used to detect defects in the product being inspected.

[0074] This application selects images with better semantic features from each frame of the product being tested, and determines the values ​​of the photographing parameters used by the photographing device to collect the selected images as the second candidate values.

[0075] Step S105: Determine the target value of the image taking parameters based on the second candidate value.

[0076] The target value of the photography parameters is the final value of the photography parameters when the photography device is used in actual production / life.

[0077] The information processing method provided in this application involves acquiring images of a calibration object using an imaging device based on different values ​​of imaging parameters. Firstly, a subset of candidate values ​​with better image quality are selected based on evaluation metrics of the acquired images. Then, the imaging device acquires images of the product under test based on each of these first candidate values. Further filtering of the first candidate values ​​is performed based on the semantic features of the acquired images, resulting in secondly, higher-quality candidate values. Finally, the target values ​​of the imaging parameters are determined based on these secondly candidate values. This application incorporates an automatic filtering process with different dimensions during the adjustment of the imaging parameters (an initial screening process based on evaluation metrics of the calibration object images, and a finer screening process based on the semantic features of the product under test), thereby reducing the workload of manual operation and improving the efficiency of adjusting the imaging parameters of the imaging device.

[0078] In an optional embodiment, the flowchart of one method for determining a second candidate value from a first candidate value based on the semantic features of each frame of the collected images of the product under test is shown below. Figure 2 As shown, it may include:

[0079] Step S201: Use the feature extractor in the pre-trained detection model to extract features from each frame of the image of the product being detected.

[0080] In this application, the pre-trained detection model can be a pre-trained defect detection model. Optionally, the defect detection model can be implemented using a Convolutional Neural Network (CNN), or other networks, such as a Recurrent Neural Network (RNN), or a Deep Neural Network (DNN), etc.

[0081] As an example, a defect detection model can be a model used to detect defective regions in a product under inspection. This defect detection model includes a feature extractor and a defect region identifier; the feature extractor extracts features from an image of the product under inspection, and the defect region identifier processes the features extracted by the feature extractor to output a detection result that characterizes the defective region in the image of the product under inspection. This defect detection model can be trained through the following process: inputting a sample image into the defect detection model, obtaining the detection result output by the model, and updating the parameters of the defect detection model with the goal of ensuring that the defective region characterized by the detection result output by the model approximates the defective region (i.e., the label) in the sample image.

[0082] As an example, a defect detection model can be a model used to detect whether a product contains defective regions; it can also be called a classification model. This defect detection model includes a feature extractor and a defect identifier. The feature extractor extracts features from the image of the product being inspected, and the defect identifier processes the features extracted by the feature extractor to output a detection result. This detection result indicates whether the image of the product being inspected contains defective regions. This defect detection model can be trained through the following process: inputting a sample image into the defect detection model, obtaining the detection result output by the defect detection model, and updating the parameters of the defect detection model with the goal of approximating the presence of defective regions (i.e., labels) in the sample image as indicated by the detection result output by the defect detection model.

[0083] For each frame of the product being inspected, the feature extractor extracts features from that frame of the product being inspected, resulting in a feature map. This feature map is the semantic feature of that frame of the image. Each element in the feature map is a multi-dimensional vector that represents the semantic information of a region in that frame of the image in multiple dimensions (e.g., including but not limited to color, brightness, texture, sharpness, etc.).

[0084] Step S202: Extract the features of pre-labeled defective and non-defective regions from the feature map of each frame image.

[0085] In this application, the defective regions in each frame of the image being detected are already labeled, and the regions outside the defective regions are the non-defective regions. Therefore, the non-defective regions are also equivalent to being labeled.

[0086] It should be noted that in this application, it is not necessary to manually annotate the defect area for every frame of the inspected product. For the i-th (i = 1, 2, ..., I, where I is the number of inspected products) inspected product, it is only necessary to manually annotate the defect area for any frame of the i-th inspected product. Optionally, the annotator can paint over the defect area of ​​any frame of the i-th inspected product, and then the pixels in any frame of the i-th inspected product located in the painted area are identified as defective pixels (denoted as defective pixels), and the pixels located outside the painted area are identified as non-defective pixels (denoted as non-defective pixels). After determining the defective and non-defective areas, any frame of the i-th inspected product can be divided into multiple rectangular regions. The rectangular region containing the defective pixels in any frame is identified as the defective region in that frame, and the other regions are the non-defective regions in that frame.

[0087] Based on the position of the defective region in any frame of the i-th inspected product, the defective and non-defective regions in other frames of the i-th inspected product can be determined.

[0088] For the j-th frame image of the i-th inspected product (j = 1, 2, ..., J, where J is the number of images acquired for the i-th inspected product), the size of the feature map extracted from the j-th frame image is scaled proportionally to the size of the j-th frame image. For example, if the size of the j-th frame image is 2048*1536, the size of the feature map extracted from the j-th frame image might be 512*384; or, if the size of the j-th frame image is 1280*960, the size of the feature map extracted from the j-th frame image might be 640*480. Thus, each element in the feature map extracted from the j-th frame image corresponds to a region in the j-th frame image. Based on this, the defective and non-defective regions in the feature map of the j-th frame image can be determined according to the defective regions in the j-th frame image and the correspondence between the elements in the feature map extracted from the j-th frame image and the regions in the j-th frame image. If the k-th element (k = 1, 2, ..., K, where K is the number of elements in the feature map extracted from the j-th frame image) corresponds to a defect region in the j-th frame image, then the k-th element in the feature map extracted from the j-th frame image can be determined as a semantic feature of a defect region in the j-th frame image.

[0089] Step S203: Determine the second candidate value from the first candidate value based on the features of the defective region and the features of the non-defective region extracted from each frame image of the inspected product.

[0090] In an optional embodiment, a second candidate value can be determined from a first candidate value by analyzing the feature differences between defective and non-defective regions extracted from each frame of the product under inspection. Specifically, a flowchart illustrating an implementation of determining a second candidate value from a first candidate value based on the features of defective and non-defective regions extracted from each frame of the product under inspection is shown below. Figure 3a As shown, it may include:

[0091] Step S301: Obtain the feature difference values ​​of defective and non-defective areas in each frame of the image of the product being inspected.

[0092] For each frame of the inspected product image, calculate the feature difference value between the defective region and the non-defective region in that frame image. Optionally, each element in the feature map of that frame image may correspond to at least one region in that frame image. For any element in the feature map of that frame image, if the region in that frame image corresponding to the element contains a defective region, determine that the element belongs to the feature of a defective region; otherwise, determine that the element does not belong to the feature of a defective region.

[0093] As an example, for each frame of an image, for each element (i.e., feature) in the defective region, calculate the distance between that element and each element in the non-defective region. Assuming there are *e* elements in the defective region and *f* elements in the non-defective region, then for that frame of the image, *e*f* distance values ​​can be calculated. The minimum distance among these calculated distances can be used as the feature difference value between the defective and non-defective regions in that frame of the image. Alternatively, the average of the calculated distances can be used as the feature difference value between the defective and non-defective regions in that frame of the image. Or, the median of the calculated distances can be used as the feature difference value between the defective and non-defective regions in that frame of the image.

[0094] like Figure 3b The diagram illustrates a flowchart of an implementation for obtaining the feature difference values ​​of defective and non-defective regions in each frame of an image of a product under inspection, according to an embodiment of this application. In this example diagram, each combination of photographing parameters represents a value of the photographing parameter, with a total of n values, i.e., photographing parameter combination 1 to photographing parameter combination n. For each combination of photographing parameters, the photographing device acquires an image of the item under inspection (i.e., the product under inspection) based on that combination, obtaining one frame of image (i.e.,... Figure 3b In the context of image capture, taking image capture parameter combination 1 as an example, after acquiring an image based on image capture parameter combination 1 (referred to as image 1 for ease of description and distinction), a pre-trained feature extractor is used to extract features from image 1, resulting in a feature map of image 1 (referred to as feature map). Figure 1 Then, based on the pre-labeled abnormal areas (i.e., defective areas) and normal areas (i.e., non-defective areas), features are used to... Figure 1 The features of abnormal and normal regions are extracted, and then the feature difference between the normal and abnormal regions is calculated (denoted as diff1 for ease of description) to evaluate the image quality of image 1. After calculating the feature difference diff1 between the normal and abnormal regions of image 1, the imaging device acquires an image (denoted as image 2 for ease of description and distinction) based on imaging parameter combination 2. Then, the pre-trained feature extractor is used to extract features from image 2 to obtain the feature map of image 2 (denoted as feature map). Figure 2 Then, based on the pre-labeled abnormal areas (i.e., defective areas) and normal areas (i.e., non-defective areas), features are used to... Figure 2 The features of abnormal and normal regions are extracted, and then the feature difference value between the normal and abnormal regions (denoted as diff2 for ease of explanation) is calculated to evaluate the image quality of image 2. This process is repeated until the feature difference value between the normal and abnormal regions of image n acquired based on the image parameter combination n is obtained (denoted as diff2 for ease of explanation). n ).

[0095] It should be noted that, Figure 3b The rectangles and ellipses in the image are only to illustrate that normal and abnormal areas can be identified in the acquired image. They do not mean that normal areas are marked with rectangles and abnormal areas are marked with ellipses. In practical applications, only abnormal areas need to be marked, and the marking of abnormal areas is done at the pixel level, which can be achieved through the aforementioned drawing method.

[0096] Step S302: Determine the first candidate value corresponding to the image of the detected product whose feature difference value satisfies the first condition as the second candidate value; wherein, the feature difference value that satisfies the first condition is greater than the feature difference value that does not satisfy the first condition.

[0097] As an example, images with feature difference values ​​greater than a difference threshold can be selected, and the first candidate value used by the camera to capture the selected image (with feature difference values ​​greater than the difference threshold) can be determined as the second candidate value. Alternatively,

[0098] The images of the product under inspection can be sorted in descending order of feature difference values. The top G images in the sorting are selected, and the first candidate value used by the selected images (top G in the feature difference value sorting) captured by the camera is determined as the second candidate value.

[0099] by Figure 3b For example, Figure 3b Among the three frames of images displayed, acquired based on image parameter combination 1, image parameter combination 2, and image parameter combination n, only the feature difference value diff2 of the image acquired based on image parameter combination 2 satisfies the first condition. Therefore, image parameter combination 2 can be considered as a second candidate value.

[0100] Since the feature difference value that satisfies the first condition is greater than the feature difference value that does not satisfy the first condition, the image corresponding to the feature difference value that satisfies the first condition can obtain the defect detection result more quickly and accurately when used in the defect detection model. Therefore, the detection effect of the image corresponding to the feature difference value that satisfies the first condition when used in the defect detection model is better than the detection effect of the image corresponding to the feature difference value that does not satisfy the first condition when used in the defect detection model.

[0101] In an optional embodiment, the flowchart of one method for determining the first candidate values ​​of the imaging parameters based on each frame of the acquired calibration object is shown below. Figure 4 As shown, it may include:

[0102] Step S401: Obtain at least one evaluation index for each frame of the acquired calibration object.

[0103] In this application, for each frame of the corresponding calibration object, only one evaluation index may be obtained, or multiple evaluation indexes may be obtained for each frame of the corresponding calibration object.

[0104] As an example, the evaluation metrics may include at least one of the following metrics: spatial frequency response (SFR), noise, and color.

[0105] Spatial frequency response is used to characterize the sharpness of an image; noise and color are used to characterize the color quality of an image.

[0106] The same evaluation metrics were obtained for different frames of images.

[0107] Step S402: Based on the evaluation metrics of each frame of the calibration object, obtain the comprehensive metric of each frame of the calibration object.

[0108] If each frame of the calibration object obtains an evaluation index, that evaluation index can be used as the comprehensive evaluation index for that frame. If each frame of the calibration object obtains multiple evaluation indices, for each frame, the multiple evaluation indices of that frame can be weighted and summed to obtain the comprehensive index for that frame.

[0109] Step S403: Determine the values ​​of the image capture parameters corresponding to the image whose comprehensive index meets the second condition as the first candidate values; the comprehensive index that meets the second condition is better than the comprehensive index that does not meet the second condition.

[0110] As an example, if a smaller comprehensive index indicates better image quality, then a comprehensive index satisfying the second condition is smaller than a comprehensive index not satisfying the second condition. In this case, the images of the calibration object can be sorted in ascending order of comprehensive index, and the top H images in the sorting can be selected as images whose comprehensive index satisfies the second condition. The values ​​of the imaging parameters used by the imaging device to capture the top H images in the sorting can be determined as the first candidate values; or, the values ​​of the imaging parameters used by the imaging device to capture images whose comprehensive index is less than the first index threshold can be determined as the first candidate values.

[0111] If a higher comprehensive index indicates better image quality, then the comprehensive index satisfying the second condition is greater than the comprehensive index not satisfying the second condition. In this case, the images of the calibration object can be sorted in descending order of comprehensive index, and the top H images in the sorting can be selected as the images whose comprehensive index satisfies the second condition. The values ​​of the imaging parameters used by the imaging device to capture the top H images in the sorting can be determined as the first candidate values; or, the values ​​of the imaging parameters used by the imaging device to capture images whose comprehensive index is greater than the second index threshold can be determined as the first candidate values.

[0112] In an optional embodiment, one way to implement the above-described determination of the target value of the imaging parameters based on the second candidate value can be by repeatedly executing... Figure 5 The process of obtaining a new second candidate value is shown until the termination condition is met. (As shown) Figure 5 The diagram shown is a flowchart of an implementation for obtaining a new second candidate value according to an embodiment of this application, which may include:

[0113] Step S501: Generate a third candidate value based on the latest obtained second candidate value; each third candidate value is obtained by combining some values ​​from at least two second candidate values.

[0114] When the process of obtaining a new second candidate value is executed for the first time, the latest obtained second candidate value is the second candidate value selected from the first candidate value; when the process of obtaining a new second candidate value is executed for the second time, the latest obtained second candidate value is the second candidate value obtained based on step S502.

[0115] As an example, based on the latest obtained second candidate value, a genetic algorithm or bird flocking algorithm can be used to generate a new batch of values ​​as the third candidate value. For example, for any two recently obtained second candidate values, θ1 = (θ 11 θ 12 ,...θ 1N ) and θ2=(θ 21 θ 22 ,...θ 2N ), where an example of obtaining the third candidate values ​​β1 and / or β2 based on these two second candidate values ​​can be:

[0116] β1=(θ 21 θ 22 θ 13 θ 14 ,...θ 1N )

[0117] And / or,

[0118] β2=(θ 11 θ 12 θ 23 θ 24 ,...θ 2N )

[0119] In other words, the elements in θ1 and θ2 can be recombine to obtain new candidate values, namely the third candidate value. The example here is based on two second candidate values ​​to obtain a third candidate value. In actual implementation, it is also possible to obtain a third candidate value based on three or more second candidate values, and so on.

[0120] Step S502: The image of the product to be tested is acquired by the imaging device based on each third candidate value.

[0121] Step S503: Based on the semantic features of each frame of the collected images of the product being tested, determine a new second candidate value from the third candidate values.

[0122] The process of determining a new second candidate value from the third candidate value can be referred to the process of determining a second candidate value from the first candidate value, which will not be repeated here.

[0123] Optionally, the termination condition may include:

[0124] The number of iterations in the process of obtaining a new second candidate value (i.e., the number of times the above process of obtaining a new second candidate value is executed) reaches the target number; and / or,

[0125] In the image of the inspected product acquired by the imaging device based on the latest second candidate value, the feature difference between the pre-labeled defective region and the non-defective region is greater than the target difference threshold, or the rate of increase of the feature difference between the pre-labeled defective region and the non-defective region is less than the rate threshold. The rate of increase corresponding to the nth iteration can be characterized by the increase ratio of the feature difference corresponding to the nth iteration to the feature difference corresponding to the (-1)th iteration. Optionally, if the increase ratio of the feature difference corresponding to the nth iteration to the feature difference corresponding to the (n-1)th iteration is less than the ratio threshold, it is determined that the rate of increase of the feature difference between the pre-labeled defective region and the non-defective region is less than the rate threshold.

[0126] Optionally, if both of the above conditions are met, the termination condition can be determined when at least one condition is satisfied.

[0127] The feature difference corresponding to the nth iteration can be obtained as follows:

[0128] After obtaining a new second candidate value through the nth iteration, the inspected product is captured by the imaging device based on the latest second candidate value. The feature difference between the defective and non-defective regions in each frame of the captured image is extracted (see the aforementioned embodiment for the specific implementation method, which will not be repeated here). The mean of the feature difference corresponding to each frame of the image is taken as the feature difference corresponding to the nth iteration, or the maximum value of the feature difference corresponding to each frame of the image is taken as the feature difference corresponding to the nth iteration, or the median of the feature difference corresponding to each frame of the image is taken as the feature difference corresponding to the nth iteration.

[0129] The process of obtaining the feature differences corresponding to the (n-1)th iteration is the same as the process of obtaining the feature differences corresponding to the nth iteration, and will not be repeated here.

[0130] Select one of the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value.

[0131] Among them, the detection performance of images acquired based on target values ​​is better than that of images acquired based on non-target values ​​when used in the defect detection model.

[0132] As an example, if the termination condition is met, the imaging device can acquire images of the product being inspected based on each of the new second candidate values ​​obtained.

[0133] The image displayed is a series of frames captured by the camera based on the newly acquired second candidate values.

[0134] In response to a selection command, a frame of the displayed image is selected as the target image;

[0135] The second candidate value corresponding to the target image is determined as the target value.

[0136] In this application, if the termination condition is met, the image captured by the imaging device based on the latest second candidate value can be displayed through an interactive interface so that the user can select the frame image with the best quality (the best effect for defect detection). Then, the second candidate value corresponding to the image selected by the user is determined as the target value, which is the final value of the imaging parameters of the imaging device.

[0137] Based on this application, users only need to select the final value from a few selected candidate values, which reduces the user's workload while improving the accuracy and efficiency of selecting the values ​​of the photo parameters.

[0138] In an optional embodiment, the calibration material may be a standard calibration plate or the product being tested; that is, the calibration material includes either a standard calibration plate or the product being tested.

[0139] A calibration board can be used in fields such as machine vision, image measurement, photogrammetry, and 3D reconstruction. It can be used to correct lens distortion, determine the conversion relationship between physical dimensions and pixels, determine the relationship between the 3D geometric position of a point on the surface of a spatial object and its corresponding point in the image, and establish the geometric model of camera imaging, etc.

[0140] Corresponding to the method embodiments, this application also provides an information processing apparatus. A schematic diagram of the structure of the information processing apparatus provided in the embodiments of this application is shown below. Figure 6 As shown, it may include:

[0141] The system comprises a first acquisition module 601, a first determination module 602, a second acquisition module 603, a second determination module 604, and a third determination module 605; wherein...

[0142] The first acquisition module 601 is used to acquire images of the calibration object by means of a camera device based on different values ​​of the camera parameters;

[0143] The first determining module 602 is used to determine a first candidate value of the photographing parameter based on the evaluation index of each frame image of the collected calibration object, wherein the evaluation index of the image collected based on the first candidate value is better than the evaluation index of the image collected based on non-first candidate values.

[0144] The second acquisition module 603 is used to acquire images of the product being inspected based on each first candidate value using the imaging device.

[0145] The second determining module 604 is used to determine a second candidate value from the first candidate values ​​based on the semantic features of each frame image of the product under test collected. The semantic features of the image collected based on the second candidate value are used to improve the detection effect of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0146] The third determining module 605 is used to determine the target value of the photographing parameters based on the second candidate value.

[0147] The information processing device provided in this application incorporates an automatic screening process of different dimensions (an initial screening process for the values ​​of the photographing parameters based on the evaluation indicators of the calibration object image, and a fine screening process for the values ​​of the photographing parameters based on the semantic features of the product being detected) during the process of adjusting the photographing parameters of the photographing device, thereby reducing the workload of manual operation and improving the adjustment efficiency of the photographing parameters of the photographing device.

[0148] In an optional embodiment, the second determining module 604 includes:

[0149] The first feature extraction unit is used to extract features from each frame of the detected product using the feature extractor in the pre-trained detection model.

[0150] The second feature extraction unit is used to extract features of pre-labeled defect areas and non-defect areas from the feature map of each frame of the inspected product for each frame of the image.

[0151] The first determining unit is configured to determine a second candidate value from the first candidate values ​​based on the features of the defective region and the features of the non-defective region extracted from each frame image of the inspected product.

[0152] In an optional embodiment, the first determining unit is configured to:

[0153] Obtain the feature difference values ​​of defective and non-defective regions in each frame image of the inspected product;

[0154] The first candidate value corresponding to the image of the detected product whose feature difference value satisfies the first condition is determined as the second candidate value; wherein, the feature difference value that satisfies the first condition is greater than the feature difference value that does not satisfy the first condition.

[0155] In an optional embodiment, the first determining module 602 includes:

[0156] An evaluation index acquisition unit is used to acquire at least one evaluation index for each frame of the collected calibration object;

[0157] The comprehensive index acquisition unit is used to obtain the comprehensive index of each frame of the calibrator based on each evaluation index of each frame of the calibrator.

[0158] The second determining unit is used to determine the value of the photographing parameter corresponding to the image whose comprehensive index meets the second condition as the first candidate value; the comprehensive index that meets the second condition is better than the comprehensive index that does not meet the second condition.

[0159] In an optional embodiment, the third determining module 605 is configured to: repeatedly execute the process of obtaining a new second candidate value until the termination condition is met:

[0160] A third candidate value is generated based on the latest obtained second candidate value; each third candidate value is obtained by combining some values ​​from at least two second candidate values;

[0161] The imaging device captures images of the product under test based on each third candidate value. Based on the semantic features of each frame of the captured images of the product under test, a new second candidate value is determined from the third candidate values.

[0162] Select one of the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value.

[0163] In an optional embodiment, the satisfaction of the termination condition includes:

[0164] The number of iterations in the process of obtaining the new second candidate value reaches the target number; and / or,

[0165] In the image of the product being inspected acquired by the imaging device based on the latest second candidate value, the feature difference between the pre-marked defect area and the non-defect area is greater than the target difference threshold, or the rate of increase of the feature difference between the pre-marked defect area and the non-defect area is less than the rate threshold.

[0166] In an optional embodiment, the detection performance of the defect detection model is better when the image acquired based on the target value is used for the defect detection model than when the image acquired based on the non-target value is used for the defect detection model.

[0167] In an optional embodiment, when the third determining module 605 selects a second candidate value from the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value, it is used to:

[0168] If the termination condition is met, the imaging device will capture images of the detected product based on each of the new second candidate values ​​obtained.

[0169] The image is displayed as a series of frames captured by the imaging device based on the obtained new second candidate values.

[0170] In response to a selection command, a target image frame is determined from the displayed images;

[0171] The second candidate value corresponding to the target image is determined as the target value.

[0172] In an optional embodiment, the calibration material includes one of a standard calibration plate and the product being tested.

[0173] Corresponding to the method embodiments, this application also provides an electronic device, a schematic diagram of which is shown below. Figure 7 As shown, it may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0174] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4.

[0175] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0176] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0177] The memory 3 stores a program, and the processor 1 can call the program stored in the memory 3. The program is used for:

[0178] The image of the calibration object is acquired by the camera based on different values ​​of the camera parameters.

[0179] The first candidate value of the photographing parameter is determined based on the evaluation index of each frame image of the calibrated object, wherein the evaluation index of the image acquired based on the first candidate value is better than the evaluation index of the image acquired based on non-first candidate values.

[0180] The imaging device acquires images of the product under inspection based on each first candidate value;

[0181] Based on the semantic features of each frame of the product being inspected, a second candidate value is determined from the first candidate value. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0182] The target value of the image capture parameter is determined based on the second candidate value.

[0183] Optionally, the refined and extended functions of the program can be found in the description above.

[0184] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:

[0185] The image of the calibration object is acquired by the camera based on different values ​​of the camera parameters.

[0186] The first candidate value of the photographing parameter is determined based on the evaluation index of each frame image of the calibrated object, wherein the evaluation index of the image acquired based on the first candidate value is better than the evaluation index of the image acquired based on non-first candidate values.

[0187] The imaging device acquires images of the product under inspection based on each first candidate value;

[0188] Based on the semantic features of each frame of the product being inspected, a second candidate value is determined from the first candidate value. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values.

[0189] The target value of the image capture parameter is determined based on the second candidate value.

[0190] Optionally, the refined and extended functions of the program can be found in the description above.

[0191] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0193] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0195] It should be understood that in the embodiments of this application, the claims, various embodiments, and features can be combined with each other to solve the aforementioned technical problems.

[0196] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information processing method, the method comprising: The image of the calibration object is acquired by the camera based on different values ​​of the camera parameters. The first candidate value of the photographing parameter is determined based on the evaluation index of each frame image of the calibrated object, wherein the evaluation index of the image acquired based on the first candidate value is better than the evaluation index of the image acquired based on non-first candidate values. The imaging device acquires images of the product under inspection based on each first candidate value; Based on the semantic features of each frame of the product being inspected, a second candidate value is determined from the first candidate value. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values. The target value of the image capture parameter is determined based on the second candidate value.

2. The method according to claim 1, wherein determining a second candidate value from the first candidate values ​​based on the semantic features of each frame image of the detected product includes: The feature extractor in the pre-trained detection model is used to extract features from each frame of the image of the product being detected. For each frame of the inspected product, features of pre-labeled defective and non-defective regions are extracted from the feature map of each frame. A second candidate value is determined from the first candidate values ​​based on the features of the defective region and the non-defective region extracted from each frame image of the inspected product.

3. The method according to claim 2, wherein determining a second candidate value from the first candidate values ​​based on the features of the defective region and the features of the non-defective region extracted from each frame image of the inspected product comprises: Obtain the feature difference values ​​of defective and non-defective regions in each frame of the image of the inspected product; The first candidate value corresponding to the image of the detected product whose feature difference value satisfies the first condition is determined as the second candidate value; wherein, the feature difference value that satisfies the first condition is greater than the feature difference value that does not satisfy the first condition.

4. The method according to claim 1, wherein determining the first candidate value of the photographing parameter based on the evaluation index of each frame image of the acquired calibration object includes: Obtain at least one evaluation index for each frame of the acquired calibration object; Based on the evaluation metrics of each frame of the calibrated object, a comprehensive metric for each frame of the calibrated object is obtained. The values ​​of the photographing parameters corresponding to the images whose comprehensive indicators meet the second condition are determined as the first candidate values; the comprehensive indicators that meet the second condition are better than the comprehensive indicators that do not meet the second condition.

5. The method of claim 1, wherein determining the target value of the photographing parameter based on the second candidate value comprises: Repeat the following process to obtain a new second candidate value until the termination condition is met: A third candidate value is generated based on the latest obtained second candidate value; each third candidate value is obtained by combining some values ​​from at least two second candidate values; The imaging device captures images of the product under test based on each third candidate value. Based on the semantic features of each frame of the captured images of the product under test, a new second candidate value is determined from the third candidate values. Select one of the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value.

6. The method according to claim 5, wherein satisfying the termination condition includes: The number of iterations in the process of obtaining a new second candidate value reaches the target number. And / or, In the image of the product being inspected acquired by the imaging device based on the latest second candidate value, the feature difference between the pre-marked defect area and the non-defect area is greater than the target difference threshold, or the rate of increase of the feature difference between the pre-marked defect area and the non-defect area is less than the rate threshold.

7. The method of claim 5, wherein, When images acquired based on the target value are used in a defect detection model, the detection performance is better than that of images acquired based on non-target values.

8. The method according to claim 7, wherein selecting a second candidate value from the new second candidate values ​​obtained under the condition of satisfying the termination condition as the target value comprises: If the termination condition is met, the imaging device will capture images of the detected product based on each of the new second candidate values ​​obtained. The image is displayed as a series of frames captured by the imaging device based on the obtained new second candidate values. In response to a selection command, a target image frame is determined from the displayed images; The second candidate value corresponding to the target image is determined as the target value.

9. The method according to claim 1, wherein the calibrator comprises: One of the standard calibration plate and the product being tested.

10. An information processing apparatus, the apparatus comprising: The first acquisition module is used to acquire images of the calibration object by means of a camera device based on different values ​​of the camera parameters; The first determining module is used to determine a first candidate value of the photographing parameter based on the evaluation index of each frame image of the collected calibration object, wherein the evaluation index of the image collected based on the first candidate value is better than the evaluation index of the image collected based on non-first candidate values. The second acquisition module is used to acquire images of the product being inspected based on each first candidate value using the imaging device. The second determining module is used to determine a second candidate value from the first candidate values ​​based on the semantic features of each frame image of the product being detected. The semantic features of the image collected based on the second candidate value are used to improve the detection performance of the defect detection model compared to the semantic features of the image collected based on non-second candidate values. The third determining module is used to determine the target value of the photographing parameters based on the second candidate value.

Images