A method, device and storage medium for generating correlation positioning calibration data set

Abstract

The present application discloses a method, device and storage medium for generating a correlation positioning calibration data set, which is used to reduce the workload and error of manual calibration processing and improve the quality and generation efficiency of the data set. The original image data of the target object is collected in an automated manner; the collected original image data is preprocessed according to a preprocessing method; at least one type of image feature is extracted from the original image data according to a feature extraction method to generate correlation image features; the original image data is subjected to correlation analysis according to a correlation analysis method and combined with correlation image features to generate feature correlation degree data; the original image data is screened and optimized according to the feature correlation degree data, and image data that does not meet the correlation degree conditions is screened out to generate an optimized image set; a data set is organized in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, and the related calibration parameters are a set of parameters generated during the calibration process.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of data calibration, and in particular, to a method, device, and storage medium for generating a correlation positioning calibration data set. Background Art

[0002] With the rapid development of machine vision and image processing, more and more models for machine vision and image processing are widely used in people's daily lives. In the fields of video-based target detection, various face recognition and target tracking, the core is machine vision and image processing.

[0003] Machine vision and image processing models need to be pre-trained with large amounts of training data to continuously enhance the recognition capabilities of the models. Although data collection is relatively convenient at present, the complexity of data labeling cannot be ignored.

[0004] In the field of machine vision and image processing, correlation localization is a very important technology. In correlation localization, high-quality calibration data sets are highly relied on to ensure the accuracy and robustness of the algorithm model.

[0005] Traditional manual data annotation methods are not only inefficient, but also prone to annotation errors, and can only annotate small amounts of high-quality data. In this context, it is particularly important to seek efficient and accurate annotation methods. Therefore, there is an urgent need for an efficient and automated calibration dataset generation method to improve the quality and generation efficiency of the dataset. Summary of the invention

[0006] The present application discloses a method, device and storage medium for generating a correlation positioning calibration data set, which are used to reduce the workload and errors of manual calibration processing and improve the quality and generation efficiency of the data set.

[0007] The first aspect of the present application discloses a method for generating a correlation positioning calibration data set, comprising:

[0008] Collecting raw image data of the target object in an automated manner;

[0009] Preprocessing the collected raw image data according to a preprocessing method;

[0010] Extract at least one type of image features from the original image data according to the feature extraction method to generate relevant image features;

[0011] According to the correlation analysis method and combined with the correlation image features, the original image data is subjected to correlation analysis to generate feature correlation degree data;

[0012] The original image data is screened and optimized according to the feature correlation degree data, image data that does not meet the correlation degree conditions are screened out, and an optimized image set is generated;

[0013] A data set is organized in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, wherein the related calibration parameters are a set of parameters generated during the calibration process.

[0014] Optionally, raw image data of the target object is collected in an automated manner, including:

[0015] Light up the screen so that it displays a normal layer detection picture;

[0016] Set the initial position of the camera during the acquisition process and the physical distance between images;

[0017] The screen is captured according to the initial position and physical spacing distance to generate original image data.

[0018] Optionally, the relevant calibration parameters include physical spacing distance, coarse extraction interval, fine extraction interval and pixel layer index data of the optimized image set;

[0019] The dataset is organized in a standard format based on the optimized image set and related calibration parameters to generate the target calibration dataset, including:

[0020] Performing rough extraction and fine extraction from the optimized image set according to the rough extraction interval and the fine extraction interval to generate a rough search image set and a fine search image set;

[0021] The physical interval distance, the coarse search image set, the fine search image set, the pixel layer index data of the optimized image set and the pixel layer image are organized in the form of a tree data structure to obtain a target calibration data set.

[0022] Optionally, the preprocessing method includes image denoising processing, image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing;

[0023] The collected raw image data is preprocessed according to the preprocessing method, including:

[0024] Get the center coordinate position of the original image data;

[0025] Obtain the spatial domain standard deviation and color domain standard deviation of the original image data;

[0026] Calculate the dual-domain filter kernel according to the center coordinate position, the standard deviation in the spatial domain and the preset Laplace RBF function;

[0027] De-noising the original image data according to the dual-domain filter kernel;

[0028] The original image data is subjected to image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing.

[0029] Optionally, extracting at least one type of image features from the original image data according to a feature extraction method to generate relevant image features includes:

[0030] Determine at least one inspection item after the original image data becomes a calibration data set, the inspection item including display screen area positioning inspection, defect inspection, and foreign body inspection;

[0031] Determine N associated data types and respective detection weights of the associated data types according to the detection items, where N is an integer greater than 0;

[0032] According to N associated data types and in combination with a feature extraction method, data is collected for M original images in the original image data to generate N data groups, each data group corresponds to an associated data type, each original image has corresponding data in the data group, and M is an integer greater than 1;

[0033] Determine at least one original image from the original image data according to the detection item and the data in the N data sets, and generate a set of associated image features;

[0034] generating association weights according to the same images in the associated image feature set;

[0035] At least one type of image feature is determined from the associated image feature set according to the detection weight and the associated weight.

[0036] Optionally, the feature extraction method includes an edge detection extraction method, a corner detection extraction method, a scale-invariant feature transformation extraction method, and / or a directional gradient histogram extraction method.

[0037] Optionally, the correlation analysis method includes a PHash perceptual hashing method, a SSIM structural similarity method, a histogram distance measurement method, and an Ncc normalized cross-correlation calculation method.

[0038] The second aspect of the present application discloses a device for generating a correlation positioning calibration data set, comprising:

[0039] An acquisition unit, used for acquiring raw image data of a target object in an automated manner;

[0040] A preprocessing unit, used for preprocessing the collected raw image data according to a preprocessing method;

[0041] A first generating unit, configured to extract at least one type of image features from the original image data according to a feature extraction method, and generate correlation image features;

[0042] A second generating unit is used to perform correlation analysis on the original image data according to the correlation analysis method and in combination with the correlation image features to generate feature correlation degree data;

[0043] The third generating unit is used to screen and optimize the original image data according to the feature correlation degree data, screen out the image data that does not meet the correlation degree condition, and generate an optimized image set;

[0044] The fourth generating unit is used to organize the data set in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, wherein the related calibration parameters are a parameter set generated during the calibration process.

[0045] Optional, acquisition unit, including:

[0046] Light up the screen so that it displays a normal layer detection picture;

[0047] Set the initial position of the camera during the acquisition process and the physical distance between images;

[0048] The screen is captured according to the initial position and physical spacing distance to generate original image data.

[0049] Optionally, the relevant calibration parameters include physical spacing distance, coarse extraction interval, fine extraction interval and pixel layer index data of the optimized image set;

[0050] The fourth generation unit comprises:

[0051] Performing rough extraction and fine extraction from the optimized image set according to the rough extraction interval and the fine extraction interval to generate a rough search image set and a fine search image set;

[0052] The physical interval distance, the coarse search image set, the fine search image set, the pixel layer index data of the optimized image set and the pixel layer image are organized in the form of a tree data structure to obtain a target calibration data set.

[0053] Optionally, the preprocessing method includes image denoising processing, image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing;

[0054] Pre-processing unit, including:

[0055] The collected raw image data is preprocessed according to the preprocessing method, including:

[0056] Get the center coordinate position of the original image data;

[0057] Obtain the spatial domain standard deviation and color domain standard deviation of the original image data;

[0058] Calculate the dual-domain filter kernel according to the center coordinate position, the standard deviation in the spatial domain and the preset Laplace RBF function;

[0059] De-noising the original image data according to the dual-domain filter kernel;

[0060] The original image data is subjected to image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing.

[0061] Optionally, the first generating unit includes:

[0062] Determine at least one inspection item after the original image data becomes a calibration data set, the inspection item including display screen area positioning inspection, defect inspection, and foreign body inspection;

[0063] Determine N associated data types and respective detection weights of the associated data types according to the detection items, where N is an integer greater than 0;

[0064] According to N associated data types and in combination with a feature extraction method, data is collected for M original images in the original image data to generate N data groups, each data group corresponds to an associated data type, each original image has corresponding data in the data group, and M is an integer greater than 1;

[0065] Determine at least one original image from the original image data according to the detection item and the data in the N data sets, and generate a set of associated image features;

[0066] generating association weights according to the same images in the associated image feature set;

[0067] At least one type of image feature is determined from the associated image feature set according to the detection weight and the associated weight to generate a correlation image feature.

[0068] Optionally, the feature extraction method includes an edge detection extraction method, a corner detection extraction method, a scale-invariant feature transformation extraction method, and / or a directional gradient histogram extraction method.

[0069] Optionally, the correlation analysis method includes one of a PHash perceptual hashing method, a SSIM structural similarity method, a histogram distance measurement method, and an Ncc normalized cross-correlation calculation method.

[0070] A third aspect of the present application provides a device for generating a correlation positioning calibration data set, comprising:

[0071] Processor, memory, input-output unit, and bus;

[0072] The processor is connected to the memory, the input and output unit, and the bus;

[0073] The memory stores a program, and the processor calls the program to execute the first aspect and any optional generation method of the first aspect.

[0074] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional generation method of the first aspect.

[0075] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0076] In the present application, the original image data of the target object is first collected in an automated manner, with the purpose of generating a large amount of image data in a short time. Next, a large amount of image data needs to be processed in a unified manner so that the various specification parameters of a large number of image features can be relatively unified, and the collected original image data needs to be preprocessed according to the preprocessing method. Next, the detection activity to be performed after the original image data generates the target calibration data set is determined, and the features to be extracted are analyzed according to the detection activities, and the feature extraction method is selected, and then at least one type of image feature is extracted from the original image data according to the feature extraction method to generate correlation image features. Next, the original image data is subjected to correlation analysis according to the correlation analysis method and combined with the correlation image features to generate feature correlation degree data. After the correlation degree data is obtained, the original image data is screened and optimized according to the feature correlation degree data, and the image data that does not meet the correlation degree conditions is screened out to generate an optimized image set. That is, the original image data with a weak correlation degree with the correlation image feature is screened out, and the original image data with a strong correlation degree is retained, so as to improve the quality of the data. Finally, the data set is organized in a standard format according to the optimized image set and the relevant calibration parameters to generate the target calibration data set, and the relevant calibration parameters are the parameter set generated during the calibration process.

[0077] By determining the correlation image features of the original image data and performing correlation comparative analysis on the original image data based on the correlation image features, the original image data that does not meet the requirements is screened out, and the original image data whose quality meets the correlation requirements is retained. Finally, the retained optimized image set and multiple calibration parameters used in the calibration process are organized into a standard format and data set. No manual annotation is required, and it is automatic and efficient. The original image data is automatically calibrated to greatly improve the quality and generation efficiency of the data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0079] Figure 1 A schematic diagram of an embodiment of a method for generating a correlation positioning and calibration data set of the present application;

[0080] Figure 2 A schematic diagram of an embodiment of the first stage of the method for generating a correlation positioning and calibration data set of the present application;

[0081] Figure 3 A schematic diagram of an embodiment of the second stage of the method for generating a correlation positioning and calibration data set of the present application;

[0082] Figure 4 A schematic diagram of an embodiment of a device for generating a correlation positioning and calibration data set of the present application;

[0083] Figure 5 A schematic diagram of another embodiment of a device for generating a correlation positioning and calibration data set according to the present application. DETAILED DESCRIPTION

[0084] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0085] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0086] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0087] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0088] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0089] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0090] In the prior art, correlation positioning is a very important technology in the field of machine vision and image processing. In correlation positioning, high-quality calibration data sets are highly relied on to ensure the accuracy and robustness of the algorithm model.

[0091] Traditional manual data annotation methods are not only inefficient, but also prone to annotation errors, and can only annotate small amounts of high-quality data. In this context, it is particularly important to seek efficient and accurate annotation methods. Therefore, there is an urgent need for an efficient and automated calibration dataset generation method to improve the quality and generation efficiency of the dataset.

[0092] Based on this, the present application discloses a method, device and storage medium for generating a correlation positioning calibration data set, which are used to reduce the workload and errors of manual calibration processing and improve the quality and generation efficiency of the data set.

[0093] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0094] The method of the present application can be applied to a server, a device, a terminal or other devices with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is made by taking the execution subject as an example of a terminal.

[0095] See also Figure 1 The present application provides an embodiment of a method for generating a correlation positioning calibration data set, comprising:

[0096] 101. Collecting original image data of the target object in an automated manner;

[0097] The terminal first needs to collect a large amount of raw image data in an automated way. In this embodiment, the image acquisition is mainly performed on a target object such as a display screen through an industrial camera. In addition to images, it can also be video and sensor data, etc., which are not limited here. This automatic acquisition method can greatly reduce the workload of manual acquisition and the workload of calibration processing, and improve the efficiency of generating calibration data sets.

[0098] 102. Preprocessing the collected original image data according to a preprocessing method;

[0099] The terminal preprocesses the collected raw image data according to the preprocessing method. In this embodiment, the preprocessing method includes but is not limited to a variety of denoising, enhancement, segmentation, registration, scaling and other processing methods. The preprocessing is used to improve the quality and availability of the data, and the image also needs to be processed in a unified manner. This preprocessing method can reduce the influence of subjective factors, improve the accuracy of the data and the efficiency of subsequent processing. It should be noted that the choice of preprocessing method is determined by the data set that needs to be constructed. Some raw images need to be denoised and segmented, and some need to be preprocessed by denoising and scaling.

[0100] 103. Extract at least one type of image features from the original image data according to a feature extraction method to generate relevant image features;

[0101] The terminal determines the image features to be extracted based on the inspection items required by the calibration data set, selects the feature extraction method based on the image features, obtains the image features with high correlation, and the image features with high correlation are used for subsequent correlation calculation. Among them, the inspection items include display area positioning inspection, defect detection, foreign body detection and other inspections for display screens.

[0102] It should be noted that there are many ways to extract features, including but not limited to edge detection, corner detection, scale-invariant feature transform (SIFT), histogram of oriented gradients (HOG), etc.

[0103] For example, in the area positioning detection of display screens, a calibrated data set is required to perform edge detection, corner point detection, and calibration frame pixel layered detection. In the original image data, the feature extraction that needs to be performed also includes edge detection, corner point detection, and calibration frame pixel layered detection. Feature extraction is performed according to these three feature extraction methods to obtain three different groups of features. Based on the three different groups of features, at least one image with the highest correlation with the detection item is determined from the original image data as the correlation image feature.

[0104] 104. Perform correlation analysis on the original image data according to the correlation analysis method and in combination with the correlation image features to generate feature correlation degree data;

[0105] After the terminal obtains the image with the highest correlation with the detection item as the correlation image feature, it can perform correlation analysis on the original image data using the correlation image feature. Specifically, different correlation analysis methods need to be used for correlation analysis, and the correlation analysis method needs to be selected according to different detection items to obtain feature correlation degree data. The feature correlation degree data can be a set of correlation thresholds, each of which corresponds to an original image data, or a set of parameter sets, each of which corresponds to an original image.

[0106] In this embodiment, the correlation between the extracted image features is calculated to determine the degree of correlation with the original image data (the degree of correlation is determined by the calculated feature correlation degree data). There are many correlation analysis methods. In the field of display screens, the PHash perceptual hashing method, the SSIM structural similarity method, the histogram distance measurement method, and the Ncc normalized cross-correlation calculation method are mainly used.

[0107] 105. Screen and optimize the original image data according to the feature correlation degree data, screen out the image data that does not meet the correlation degree conditions, and generate an optimized image set;

[0108] After the terminal obtains the feature correlation degree data of each original image data, it can filter out the original image data that does not meet the preset conditions and generate an optimized image set. The terminal optimizes and filters the original image data based on the results of the correlation calculation, retains the data with high correlation, and removes the data with low correlation. This method can further improve the quality and accuracy of the data set.

[0109] 106. Organizing a data set in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, where the related calibration parameters are a set of parameters generated during the calibration process.

[0110] After the terminal obtains the optimized image set, it also needs to obtain the calibration parameters that are highly correlated with the detection items during the entire calibration process, associate the calibration parameters with the optimized image set, and organize the data set in a standard format to generate a target calibration data set.

[0111] In this embodiment, the original image data of the target object is first collected in an automated manner, with the purpose of generating a large amount of image data in a short time. Next, a large amount of image data needs to be processed in a unified manner so that the various specification parameters of a large number of image features can be relatively unified. The collected original image data needs to be preprocessed according to the preprocessing method. Next, the detection activity to be performed after the original image data generates the target calibration data set is determined, and the detection activity is analyzed to obtain the features to be extracted, so as to select the feature extraction method, and then at least one type of image feature extraction is performed on the original image data according to the feature extraction method to generate correlation image features. Next, the original image data is subjected to correlation analysis according to the correlation analysis method and combined with the correlation image features to generate feature correlation degree data. After the correlation degree data is obtained, the original image data is screened and optimized according to the feature correlation degree data, and the image data that does not meet the correlation degree conditions is screened out to generate an optimized image set. That is, the original image data with a weak correlation degree with the correlation image feature is screened out, and the original image data with a strong correlation degree is retained, so as to improve the quality of the data. Finally, the data set is organized in a standard format according to the optimized image set and the relevant calibration parameters to generate the target calibration data set.

[0112] By determining the correlation image features of the original image data and performing correlation comparative analysis on the original image data based on the correlation image features, the original image data that does not meet the requirements is screened out, and the original image data whose quality meets the correlation requirements is retained. Finally, the retained optimized image set and multiple calibration parameters used in the calibration process are organized into a standard format and data set. No manual annotation is required, and it is automatic and efficient. The original image data is automatically calibrated to greatly improve the quality and generation efficiency of the data set.

[0113] See also Figure 2 and Figure 3 The present application provides an embodiment of a method for generating a correlation positioning calibration data set, comprising:

[0114] 201. Light up the screen so that the screen displays a normal layer detection picture;

[0115] 202. Setting the initial position of the camera during the acquisition process of the acquisition device and the physical interval distance for taking images;

[0116] 203. Capture the screen according to the initial position and the physical spacing distance to generate original image data;

[0117] In this embodiment, the terminal automatically collects image data, mainly for display screen image collection. For example: for the calibration data set used for pixel layer positioning in the display screen foreign body stratification, for the foreign body detection test project, it is necessary to first light up the target screen to display a normal stratification detection screen, first set the initial position of the z-axis when taking the image to Zs, then determine the total stroke L, and determine the physical spacing distance space of the image according to the parameters such as the number of layers and thickness of the display screen (such as taking the stroke L = 4mm, the spacing space = 0.01mm can be determined according to the actual screen thickness and positioning accuracy, and is not specifically limited here). The terminal controls the z-axis to drive the acquisition imaging module to move downward from the initial position Zs, and collects an image at each spacing space distance until the z-axis movement stroke reaches the set stroke L. In this way, during the z-axis movement process, the camera will take a large number of images, which will be used as calibration original image data, recorded as {In | n = 1, 2, 3, ..., 399, 400}.

[0118] It should be noted that if the inspection item corresponding to the calibration data set is defect detection, it is necessary to take grayscale as a variable and the calibration screen as a variable to capture the image. At this time, a reference position can be selected for fixed shooting.

[0119] 204. Obtain the center coordinate position of the original image data;

[0120] 205. Obtaining a spatial domain standard deviation and a color domain standard deviation of the original image data;

[0121] 206. Calculate a dual-domain filter kernel according to the center coordinate position, the spatial domain standard deviation, and a preset Laplace RBF function;

[0122] 207. De-noising the original image data according to the dual-domain filter kernel;

[0123] 208. Perform image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing on the original image data;

[0124] In this embodiment, the terminal first determines the valid area (display area) in the original image data, and then determines the center coordinate position according to the display area. This embodiment proposes a Laplace RBF kernel filtering method, which requires converting the original image data into a spatial domain and a color domain, and then obtaining the spatial domain standard deviation and the color domain standard deviation. Through the center coordinate position, the spatial domain standard deviation and the preset Laplace RBF function Calculate the dual-domain filter kernel. The formula is as follows:

[0125]

[0126] Among them, the center coordinate position , spatial domain standard deviation , color domain standard deviation , is the pixel position on the image to be detected that is not less than the center threshold. is the gray value of the center pixel, is the gray value of the pixel position that is not less than the center threshold, The above noise reduction method can obtain the Laplace RBF kernel by designing a new Laplace RBF function, and can effectively process the noise in the display area by performing noise reduction on the original image data according to the dual-domain filter kernel.

[0127] It should be noted that, in addition to the noise reduction process, the remaining image enhancement process, image segmentation process, image registration process, and / or image scaling process can be processed selectively.

[0128] For example, for the calibration data set used for pixel layer positioning in the stratification of foreign matter on display screens, after the noise reduction process is completed, the image preprocessing method is mainly as follows: In order to improve the processing efficiency of the original image data, in the preprocessing stage, the calibration original image data is mainly reduced by a certain magnification s (generally 0.25, 0.125, etc., without special limitation) by using image scaling to obtain the reduced image data {Isn | sn=1, 2, 3, ... , 399, 400}.

[0129] 209. Determine at least one detection item after the original image data becomes a calibration data set, where the detection items include display screen area positioning detection, defect detection, and foreign matter detection;

[0130] 210. Determine N associated data types and respective detection weights of the associated data types according to the detection items, where N is an integer greater than 0;

[0131] 211. Collect data for M original images in the original image data according to the N associated data types and in combination with the feature extraction method to generate N data groups, each data group corresponds to an associated data type, each original image has corresponding data in the data group, and M is an integer greater than 1;

[0132] 212. Determine at least one original image from the original image data according to the detection item and the data in the N data sets, and generate a set of associated image features;

[0133] 213. Generate association weights according to the same images in the associated image feature set;

[0134] 214. Determine at least one type of image feature from the associated image feature set according to the detection weight and the associated weight, and generate a correlation image feature;

[0135] In this embodiment, the terminal determines at least one detection item after the original image data becomes the calibration data set. The display area positioning detection, defect detection, and foreign body detection here are only general directions. If a convolutional neural network is used to identify defects, or blob analysis is used to analyze the extracted image features, these are two detection items using different data types.

[0136] According to the detection project, N associated data types and the detection weights of each associated data type are determined, where N is an integer greater than 0. For example: the detection project is foreign body detection, and the specific method used is to perform a detailed analysis through the focus value. At this time, the focus value is used as one of the associated data types for detection. Secondly, in addition to the focus value, brightness, as a calculation parameter of the focus value, is also determined as one of the associated data types. At this time, the detection weight of the focus value and the detection weight of the brightness are determined. At this time, two associated data types can be obtained from the original image data, one is the focus value and the other is the brightness. Assume that the detection weight of the focus value is 0.75 and the detection weight of the brightness is 0.25. Next, the terminal collects data (collection of focus value and brightness parameters) for multiple original images in the original image data according to the two associated data types and in combination with the feature extraction method, and generates two data groups.

[0137] According to the detection items and the data in the two data sets, at least one original image is determined from the original image data to generate a set of associated image features. That is, at least one reference image is selected as a reference according to the focus value of each original image data (the focus value detection weight of this part of the reference image is 0.75), and then at least one reference image is selected according to the brightness of each original image data (the brightness detection weight of this part of the reference image is 0.25). The reference images selected by the two methods may be the same.

[0138] The association weight is generated according to the same images in the associated image feature set, that is, if there is a same reference image in the images of two associated image feature sets, the association weight is increased for this reference image, that is, the overall weight of the image is increased.

[0139] The terminal determines at least one type of image feature from the associated image feature set according to the detection weight and the associated weight, and generates a correlation image feature, that is, selects at least one reference image as the correlation image feature according to the weight.

[0140] For example: for the calibration data set used for pixel layer positioning in the display screen foreign body layering, the focus value calculation method calculates and analyzes the original image data {Isn | sn=1, 2, 3, ... , 399, 400}. First, the original image data is mean filtered to generate a mean filtered image. The mean and standard deviation of the original image data, the mean and standard deviation of the mean filtered image, and the covariance between the original image data and the mean filtered image are obtained. And the corresponding brightness, contrast and structure functions are obtained according to the detection item. The mean, standard deviation and covariance are substituted into the function to calculate the focus value and generate a focus value set. Finally, a set of focus values ​​{FVn | n=1, 2, 3, ... , 399, 400} is obtained, and the image corresponding to the maximum focus value is taken as the reference position pixel layer image, recorded as Ip, and this image is the correlation image feature of the calibration image set.

[0141] 215. Perform correlation analysis on the original image data according to the correlation analysis method and in combination with the correlation image features to generate feature correlation degree data;

[0142] For example, for the calibration data set used for pixel layer positioning in display screen foreign body delamination, the normalized cross-correlation calculation method Ncc is used here.

[0143] By using the normalized cross-correlation calculation method Ncc, each original image data in the image data {Isn | sn=1, 2, 3, ... , 399,400} is correlated with the reference position pixel layer image Ip to obtain a set of correlation coefficients {fn | n=1, 2, 3, ... , 399, 400} (feature correlation degree data). Each original image data has a correlation coefficient, wherein the value range of the Ncc correlation coefficient in this embodiment is [-1, 1], -1 indicates complete negative correlation, 1 indicates complete positive correlation, and 0 indicates no correlation. The larger the value, the stronger the correlation.

[0144] 216. Screen and optimize the original image data according to the feature correlation degree data, screen out the image data that does not meet the correlation degree conditions, and generate an optimized image set;

[0145] The following is a case analysis. For example, for the calibration data set used for pixel layer positioning in display screen foreign body delamination, noise interference needs to be suppressed before optimization and screening. First, the data {fn | n=1, 2, 3, ... , 399, 400} is smoothed to generate denoised feature correlation degree data {fnm | nm=1, 2, 3, ... , 399, 400}. Then, {fnm | nm=1, 2, 3, ... , 399, 400} is screened according to the preset screening threshold fth. All data with fnm values ​​less than the threshold fth at both ends are removed to obtain the screened data {fnm | nm=Ind_S, Ind_S+1, Ind_S+2, ... , Ind_E-1, Ind_E}, where Ind_S and Ind_E represent the starting index and ending index of the screened data, respectively. Take {fnm | nm=Ind_S, Ind_S+1, The index corresponding to the maximum value among Ind_S+2, ... , Ind_E-1, Ind_E} is the pixel layer index n p .

[0146] 217. Performing rough extraction and fine extraction from the optimized image set according to the rough extraction interval and the fine extraction interval to generate a rough search image set and a fine search image set;

[0147] 218. Organize the physical interval distance, the coarse search image set, the fine search image set, the pixel layer index data of the optimized image set, and the pixel layer image in a tree data structure to obtain a target calibration data set.

[0148] In this embodiment, the optimized image needs to be sorted, specifically, coarse sampling and fine sampling need to be performed, and the corresponding calibration parameters are associated, and finally adjusted according to the form of the calibration data set.

[0149] For example, for the calibration data set used for pixel layer positioning in the display screen foreign body stratification, we first need to set the image extraction intervals step to be the coarse extraction interval step1 and the fine extraction interval step2, and extract an image from the filtered image data {Isn| sn=Ind_S, Ind_S+1, Ind_S+2, ... , Ind_E-1, Ind_E} at each coarse extraction interval step1 and fine extraction interval step2, respectively, to obtain the coarse search image set I CS and fine search image set I JS(Step 1 is the number of image extraction intervals when making a rough search image set, usually 5 or 10, and step 2 is the number of image extraction intervals when making a fine search image set, usually fixed to 1.) Finally, the calibration parameters are obtained, and the image physical spacing space and the index number n of the pixel layer image are taken. p etc., the calibration parameters are combined with the pixel layer image Ip, the coarse search image set I CS and fine search image set I JS The final calibration data set is organized and generated in the form of a tree data structure.

[0150] In this embodiment, the original image data of the target object is first collected in an automated manner, with the purpose of generating a large amount of image data in a short time. Specifically, the screen can be lit up so that the screen displays a normal layered detection screen. The initial position of the camera during the collection process of the collection device and the physical interval distance of the image are set. The screen is collected according to the initial position and the physical interval distance to generate original image data.

[0151] Next, a large amount of image data needs to be processed in a unified manner so that the various specification parameters of a large number of image features can be relatively unified. The collected original image data needs to be preprocessed according to the preprocessing method. Among them, the denoising process can be: obtaining the center coordinate position of the original image data. Obtaining the spatial domain standard deviation and color domain standard deviation of the original image data. Calculating the dual-domain filter kernel based on the center coordinate position, the spatial domain standard deviation and the preset Laplace RBF function. Denoising the original image data according to the dual-domain filter kernel.

[0152] Next, determine the detection activities to be performed after the original image data generates the target calibration data set, analyze according to the detection activities, obtain the features to be extracted, select the feature extraction method, and then extract at least one type of image features from the original image data according to the feature extraction method to generate correlation image features. Specifically, it can be: determine at least one detection item after the original image data becomes the calibration data set, and the detection items include display area positioning detection, defect detection, and foreign body detection. Determine N associated data types and their respective detection weights according to the detection items, where N is an integer greater than 0. According to the N associated data types and in combination with the feature extraction method, data is collected for M original images in the original image data to generate N data groups, each data group corresponds to an associated data type, and each original image has corresponding data in the data group, where M is an integer greater than 1. Determine at least one original image from the original image data according to the detection items and the data in the N data sets, and generate a set of associated image features. Generate association weights according to the same images in the associated image feature set. Determine at least one type of image features from the associated image feature set according to the detection weights and the association weights, and generate correlation image features.

[0153] Next, the original image data is subjected to correlation analysis according to the correlation analysis method and combined with the correlation image features to generate feature correlation degree data. After obtaining the correlation degree data, the original image data is screened and optimized according to the feature correlation degree data, and the image data that does not meet the correlation degree conditions is screened out to generate an optimized image set. That is, the original image data with a weak correlation degree with the correlation image features is screened out, and the original image data with a strong correlation degree is retained, thereby improving the quality of the data. Finally, the data set is organized in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set. Specifically, it can be: according to the rough extraction interval and the fine extraction interval, rough extraction and fine extraction are performed from the optimized image set to generate a rough search image set and a fine search image set. The physical interval distance, the rough search image set, the fine search image set, the pixel layer index data of the optimized image set, and the pixel layer image are organized in the form of a tree data structure to generate a target calibration data set.

[0154] By determining the correlation image features of the original image data and performing correlation comparative analysis on the original image data based on the correlation image features, the original image data that does not meet the requirements is screened out, and the original image data whose quality meets the correlation requirements is retained. Finally, the retained optimized image set and multiple calibration parameters used in the calibration process are organized into a standard format and data set. No manual annotation is required, and it is automatic and efficient. The original image data is automatically calibrated to greatly improve the quality and generation efficiency of the data set.

[0155] See also Figure 4 The present application provides an embodiment of a device for generating a correlation positioning calibration data set, comprising:

[0156] The acquisition unit 401 is used to acquire the original image data of the target object in an automated manner;

[0157] Optionally, the acquisition unit 401 includes:

[0158] Light up the screen so that it displays a normal layer detection picture;

[0159] Set the initial position of the camera during the acquisition process and the physical distance between images;

[0160] The screen is captured according to the initial position and physical spacing distance to generate original image data.

[0161] A preprocessing unit 402, configured to preprocess the collected raw image data according to a preprocessing method;

[0162] Optionally, the preprocessing method includes image denoising processing, image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing;

[0163] The pre-processing unit 402 includes:

[0164] The collected raw image data is preprocessed according to the preprocessing method, including:

[0165] Get the center coordinate position of the original image data;

[0166] Obtain the spatial domain standard deviation and color domain standard deviation of the original image data;

[0167] Calculate the dual-domain filter kernel according to the center coordinate position, the standard deviation in the spatial domain and the preset Laplace RBF function;

[0168] De-noising the original image data according to the dual-domain filter kernel;

[0169] The original image data is subjected to image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing.

[0170] The first generating unit 403 is used to extract at least one type of image features from the original image data according to the feature extraction method to generate correlation image features;

[0171] Optionally, the first generating unit 403 includes:

[0172] Determine at least one inspection item after the original image data becomes a calibration data set, the inspection item including display screen area positioning inspection, defect inspection, and foreign body inspection;

[0173] Determine N associated data types and respective detection weights of the associated data types according to the detection items, where N is an integer greater than 0;

[0174] According to N associated data types and in combination with a feature extraction method, data is collected for M original images in the original image data to generate N data groups, each data group corresponds to an associated data type, each original image has corresponding data in the data group, and M is an integer greater than 1;

[0175] Determine at least one original image from the original image data according to the detection item and the data in the N data sets, and generate a set of associated image features;

[0176] generating association weights according to the same images in the associated image feature set;

[0177] At least one type of image feature is determined from the associated image feature set according to the detection weight and the associated weight to generate a correlation image feature.

[0178] Optionally, the feature extraction method includes an edge detection extraction method, a corner detection extraction method, a scale-invariant feature transformation extraction method, and / or a directional gradient histogram extraction method.

[0179] The second generating unit 404 is used to perform correlation analysis on the original image data according to the correlation analysis method and in combination with the correlation image features to generate feature correlation degree data;

[0180] Optionally, the correlation analysis method includes a PHash perceptual hashing method, a SSIM structural similarity method, a histogram distance measurement method, and an Ncc normalized cross-correlation calculation method.

[0181] The third generating unit 405 is used to screen and optimize the original image data according to the feature correlation degree data, screen out the image data that does not meet the correlation degree condition, and generate an optimized image set;

[0182] The fourth generating unit 406 is used to organize the data set in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set.

[0183] Optionally, the relevant calibration parameters include physical spacing distance, coarse extraction interval, fine extraction interval and pixel layer index data of the optimized image set;

[0184] The fourth generating unit 406 includes:

[0185] Performing rough extraction and fine extraction from the optimized image set according to the rough extraction interval and the fine extraction interval to generate a rough search image set and a fine search image set;

[0186] The physical interval distance, the coarse search image set, the fine search image set, the pixel layer index data of the optimized image set and the pixel layer image are organized in the form of a tree data structure to obtain a target calibration data set.

[0187] See also Figure 5 The present application provides a device for generating a correlation positioning calibration data set, comprising:

[0188] Processor 501 , memory 502 , input / output unit 503 , and bus 504 .

[0189] The processor 501 is connected to the memory 502 , the input and output unit 503 , and the bus 504 .

[0190] The memory 502 stores a program, and the processor 501 calls the program to execute the following steps: Figure 1 , Figure 2 and Figure 3 , the generation method in .

[0191] The present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the program performs the following steps: Figure 1 , Figure 2 and Figure 3 The generation method in .

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0193] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for generating a correlation positioning calibration data set, characterized in that: include: Collecting raw image data of the target object in an automated manner; Preprocessing the collected original image data according to a preprocessing method; Extract at least one type of image features from the original image data according to a feature extraction method to generate relevant image features; Extracting at least one type of image features from the original image data according to a feature extraction method to generate relevant image features includes: Determine at least one detection item after the original image data becomes a calibration data set, the detection items include display screen area positioning detection, defect detection, and foreign body detection; determine N associated data types and their respective detection weights according to the detection items, where N is an integer greater than 0; perform data collection for M original images in the original image data according to the N associated data types and in combination with a feature extraction method, and generate N data groups, each data group corresponds to an associated data type, and each original image has corresponding data in the data group, where M is an integer greater than 1; determine at least one original image from the original image data according to the detection items and the data in the N data sets, and generate an associated image feature set; generate an associated weight according to the same image in the associated image feature set; determine at least one type of image feature from the associated image feature set according to the detection weight and the associated weight, and generate a correlation image feature; Performing correlation analysis on the original image data according to a correlation analysis method and in combination with the correlation image features to generate feature correlation degree data; Screening and optimizing the original image data according to the feature association degree data, filtering out image data that does not meet the association degree condition, and generating an optimized image set; A data set is organized in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, wherein the related calibration parameters are a set of parameters generated during the calibration process.

2. The generation method according to claim 1, characterized in that: Collect raw image data of the target object in an automated manner, including: Lighting up the screen so that the screen displays a normal layer detection picture; Set the initial position of the camera during the acquisition process and the physical distance between images; The screen is captured according to the initial position and the physical spacing distance to generate original image data.

3. The generation method according to claim 2, characterized in that: The relevant calibration parameters include the physical spacing distance, the coarse extraction interval, the fine extraction interval and the pixel layer index data of the optimized image set; Organizing a data set in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set includes: Performing rough extraction and fine extraction from the optimized image set according to the rough extraction interval and the fine extraction interval to generate a rough search image set and a fine search image set; The physical interval distance, the coarse search image set, the fine search image set, the pixel layer index data of the optimized image set and the pixel layer image are organized in a tree data structure to obtain a target calibration data set.

4. The generation method according to any one of claims 1 to 3, characterized in that: The preprocessing method includes image denoising processing, image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing; Preprocessing the collected original image data according to a preprocessing method includes: Obtaining the center coordinate position of the original image data; Obtaining a spatial domain standard deviation and a color domain standard deviation of the original image data; Calculate a dual-domain filter kernel according to the center coordinate position, the spatial domain standard deviation and a preset Laplace RBF function; Performing noise reduction on the original image data according to the dual-domain filter kernel; The original image data is subjected to image enhancement processing, image segmentation processing, image registration processing, and / or image scaling processing.

5. The generation method according to claim 1, characterized in that: The feature extraction method includes an edge detection extraction method, a corner point detection extraction method, a scale-invariant feature transformation extraction method, and / or a directional gradient histogram extraction method.

6. The generation method according to any one of claims 1 to 3, characterized in that: The correlation analysis method includes at least one of a PHash perceptual hashing method, a SSIM structural similarity method, a histogram distance measurement method, and an Ncc normalized cross-correlation calculation method.

7. A device for generating a correlation positioning calibration data set, characterized in that: include: An acquisition unit, used for acquiring raw image data of a target object in an automated manner; A preprocessing unit, used for preprocessing the collected original image data according to a preprocessing method; A first generating unit, configured to extract at least one type of image features from the original image data according to a feature extraction method, and generate correlation image features; The first generation unit comprises: Determine at least one inspection item after the original image data becomes a calibration data set, the inspection item including display screen area positioning inspection, defect inspection, and foreign body inspection; Determine N associated data types and respective detection weights of the associated data types according to the detection items, where N is an integer greater than 0; According to N associated data types and in combination with a feature extraction method, data is collected for M original images in the original image data to generate N data groups, each data group corresponds to an associated data type, each original image has corresponding data in the data group, and M is an integer greater than 1; Determine at least one original image from the original image data according to the detection item and the data in the N data sets, and generate a set of associated image features; generating association weights according to the same images in the associated image feature set; Determine at least one type of image feature from the associated image feature set according to the detection weight and the associated weight, and generate a correlation image feature; A second generating unit, configured to perform a correlation analysis on the original image data according to a correlation analysis method and in combination with the correlation image features, to generate feature correlation degree data; A third generating unit is used to screen and optimize the original image data according to the feature association degree data, screen out image data that does not meet the association degree condition, and generate an optimized image set; The fourth generating unit is used to organize the data set in a standard format according to the optimized image set and related calibration parameters to generate a target calibration data set, wherein the related calibration parameters are a parameter set generated during the calibration process.

8. A device for generating a correlation positioning calibration data set, characterized in that: include: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the generation method according to any one of claims 1 to 6.

Images