Image calibration method and device, storage medium, and electronic device

Abstract

The present disclosure provides an image calibration method and apparatus, an electronic device, and a storage medium; relating to the technical field of image processing. The image calibration method includes: obtaining an original structure image to be calibrated; performing line segment detection on the original structure image to extract line segment data in the original structure image; determining calibration data corresponding to the original structure image through the line segment data; and performing a standardization process on the original structure image according to the calibration data to obtain a standardized structure image corresponding to the original structure image. The present disclosure can perform a standardization process on pictures with similar and periodically repeating patterns taken in the same production stage of the same panel product, improving the efficiency and accuracy of calibration.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image calibration method, an image calibration device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of science and technology, the accuracy of most industrial products has received increasing attention. Standardizing the processing of pictures helps improve the accuracy of defect analysis and the quality of industrial products.

[0003] Currently, in the related art, when calibrating pictures of products with a large number of identical or similar structures, such as liquid crystal panel products, due to the few image features in the pictures, it is difficult to directly calibrate through feature matching, and the accuracy of the obtained calibration results is low; while the method of gradually attempting calibration by setting a step size has a large amount of calculation, and when the product area is large, the calibration efficiency is low.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present disclosure is to provide an image calibration method, an image calibration device, an electronic device, and a computer-readable storage medium, so as to overcome to a certain extent the problems of low calibration accuracy, large calculation amount, and low calibration efficiency when calibrating product pictures caused by the limitations and defects of the related art.

[0006] According to a first aspect of the present disclosure, there is provided an image calibration method, including:

[0007] Obtain an original structure image to be calibrated;

[0008] Perform line segment detection on the original structure image, and extract line segment data in the original structure image;

[0009] Determine calibration data corresponding to the original structure image through the line segment data;

[0010] Perform standardization processing on the original structure image according to the calibration data to obtain a standardized structure image corresponding to the original structure image.

[0011] In an exemplary embodiment of the present disclosure, the calibration data includes an inclination rotation angle and a scaling ratio, and the determining the calibration data corresponding to the original structure image through the line segment data includes:

[0012] Determine the inclination rotation angle corresponding to the original structure image through the line segment data; and

[0013] The scaling ratio corresponding to the original structure image is determined by the line segment data.

[0014] In an exemplary embodiment of the present disclosure, determining the tilt rotation angle corresponding to the original structure image using the line segment data includes:

[0015] Obtaining a preset first length threshold;

[0016] Filtering the line segment data by the first length threshold and retaining the first line segment data;

[0017] The tilt rotation angle corresponding to the original structure image is determined according to the first line segment data.

[0018] In an exemplary embodiment of the present disclosure, determining the tilt rotation angle corresponding to the original structure image according to the first line segment data includes:

[0019] Calculating a line segment inclination angle corresponding to each line segment in the first line segment data;

[0020] The average value of the line segment tilt angles is used as the tilt rotation angle corresponding to the original structure image.

[0021] In an exemplary embodiment of the present disclosure, calculating the line segment inclination angle corresponding to each line segment in the first line segment data includes:

[0022] Calculating the inclination angle of the target line segment in the first line segment data corresponding to any positive direction;

[0023] The associated angle corresponding to the inclination angle is calculated according to the inclination angle, and the angle with the smallest absolute value between the inclination angle and the associated angle is used as the segment inclination angle of the target segment.

[0024] In an exemplary embodiment of the present disclosure, before taking the angle with the smallest absolute value between the inclination angle and the associated angle as the segment inclination angle of the target segment, the method further includes:

[0025] Get the preset tilt angle threshold;

[0026] Filter out line segments whose angle, between the inclination angle and the associated angle, having the smallest absolute value is greater than an inclination angle threshold.

[0027] In an exemplary embodiment of the present disclosure, the normalizing the original structural image according to the calibration data includes:

[0028] Performing tilt correction on the original structural image according to the tilt rotation angle to obtain a tilt-corrected original structural image; and

[0029] Scale the original structural image after skew correction according to the scaling ratio to obtain a standardized structural image corresponding to the original structural image.

[0030] In an exemplary embodiment of the present disclosure, before determining the scaling ratio corresponding to the original structural image through the line segment data, the method further includes:

[0031] Arbitrarily select a target structural image in the original structural image;

[0032] Perform skew correction on the target structural image to obtain a reference image corresponding to the original structural image.

[0033] In an exemplary embodiment of the present disclosure, determining the scaling ratio corresponding to the original structural image through the line segment data includes:

[0034] Obtain a preset second length threshold;

[0035] Filter the line segment data through the second length threshold to retain second line segment data;

[0036] Determine the scaling ratio corresponding to the original structural image according to the second line segment data.

[0037] In an exemplary embodiment of the present disclosure, determining the scaling ratio corresponding to the original structural image according to the second line segment data includes:

[0038] Estimate a short side line segment corresponding to the original structural image according to the midpoint coordinates corresponding to the second line segment data;

[0039] Obtain a standard short side line segment in the reference image;

[0040] Determine the scaling ratio corresponding to the original structural image according to the short side line segment and the standard short side line segment.

[0041] In an exemplary embodiment of the present disclosure, estimating the short side line segment corresponding to the original structural image according to the midpoint coordinates corresponding to the second line segment data includes:

[0042] Obtain a preset clustering threshold;

[0043] Cluster the second line segment data according to the clustering threshold and the midpoint coordinates to obtain a line segment set;

[0044] Calculate the short side line segment corresponding to the original structural image through the line segment set obtained by clustering.

[0045] In an exemplary embodiment of the present disclosure, calculating the short-side line segments corresponding to the original structure image from the set of line segments obtained by clustering includes:

[0046] Determining the minimum coordinate value and the maximum coordinate value of the midpoint coordinates in each of the sets of line segments;

[0047] Sorting the set of line segments, and calculating the short-side line segments corresponding to the original structure image according to the minimum coordinate value and the maximum coordinate value in the sorted set of line segments.

[0048] According to a second aspect of the present disclosure, there is provided an image calibration apparatus, including:

[0049] An original structure image acquisition module, configured to acquire an original structure image to be calibrated;

[0050] A line segment data extraction module, configured to perform line segment detection on the original structure image and extract line segment data in the original structure image;

[0051] A calibration data determination module, configured to determine calibration data corresponding to the original structure image through the line segment data;

[0052] An original structure image calibration module, configured to perform normalization processing on the original structure image according to the calibration data to obtain a normalized structure image corresponding to the original structure image.

[0053] In an exemplary embodiment of the present disclosure, the calibration data determination module 2230 includes:

[0054] An inclination rotation angle determination unit, configured to determine the inclination rotation angle corresponding to the original structure image through the line segment data; and

[0055] A scaling ratio determination unit, configured to determine the scaling ratio corresponding to the original structure image through the line segment data.

[0056] In an exemplary embodiment of the present disclosure, the inclination rotation angle determination unit further includes:

[0057] A first length threshold acquisition subunit, configured to acquire a preset first length threshold;

[0058] A line segment data filtering unit, configured to filter the line segment data through the first length threshold and retain first line segment data;

[0059] An inclination rotation angle determination subunit, configured to determine the inclination rotation angle corresponding to the original structure image according to the first line segment data.

[0060] In an exemplary embodiment of the present disclosure, the tilt rotation angle determination unit includes:

[0061] A line segment tilt angle calculation subunit, configured to calculate the line segment tilt angle corresponding to each line segment in the first line segment data;

[0062] A tilt rotation angle calculation subunit, configured to use the average value of the line segment tilt angles as the tilt rotation angle corresponding to the original structure image.

[0063] In an exemplary embodiment of the present disclosure, the line segment tilt angle calculation unit is further configured to:

[0064] Calculate the tilt angle corresponding to the target line segment in the first line segment data with respect to any positive direction;

[0065] Calculate the associated angle corresponding to the tilt angle, and use the angle with the smallest absolute value among the tilt angle and the associated angle as the line segment tilt angle of the target line segment.

[0066] In an exemplary embodiment of the present disclosure, the image calibration device 2200 further includes a line segment screening unit, and the line segment screening unit is configured to:

[0067] Obtain a preset tilt angle threshold;

[0068] Filter out the line segments for which the angle with the smallest absolute value among the tilt angle and the associated angle is greater than the tilt angle threshold.

[0069] In an exemplary embodiment of the present disclosure, the original structure image calibration module 2240 further includes:

[0070] A tilt correction subunit, configured to perform tilt correction on the original structure image according to the tilt rotation angle to obtain an original structure image after tilt correction; and

[0071] An image scaling subunit, configured to scale the original structure image after tilt correction according to the scaling ratio to obtain a standardized structure image corresponding to the original structure image.

[0072] In an exemplary embodiment of the present disclosure, the image calibration device 2200 further includes a reference image generation unit, and the reference image generation unit is configured to:

[0073] Arbitrarily select a target structure image in the original structure image;

[0074] Perform tilt correction on the target structure image to obtain a reference image corresponding to the original structure image.

[0075] In an exemplary embodiment of the present disclosure, the scaling ratio determination unit includes:

[0076] A second length threshold acquisition subunit, configured to acquire a preset second length threshold;

[0077] A second line segment data generation subunit, configured to filter the line segment data by using the second length threshold, and retain the second line segment data;

[0078] A scaling ratio determination subunit, configured to determine the scaling ratio corresponding to the original structure image according to the second line segment data.

[0079] In an exemplary embodiment of the present disclosure, the scaling ratio determination subunit further includes:

[0080] A short side line segment estimation subunit, configured to estimate the short side line segment corresponding to the original structure image according to the midpoint coordinates corresponding to the second line segment data;

[0081] A standard short side line segment acquisition subunit, configured to acquire the standard short side line segment in the reference image;

[0082] A scaling ratio calculation subunit, configured to determine the scaling ratio corresponding to the original structure image according to the short side line segment and the standard short side line segment.

[0083] In an exemplary embodiment of the present disclosure, the short side line segment estimation unit includes:

[0084] A clustering threshold acquisition subunit, configured to acquire a preset clustering threshold;

[0085] A line segment set clustering subunit, configured to cluster the second line segment data according to the clustering threshold and the midpoint coordinates, to obtain a line segment set;

[0086] A short side line segment calculation subunit, configured to calculate the short side line segment corresponding to the original structure image through the line segment set obtained by clustering.

[0087] In an exemplary embodiment of the present disclosure, the short side line segment calculation subunit is further configured to:

[0088] Determine the coordinate minimum value and the coordinate maximum value of the midpoint coordinates in each of the line segment sets;

[0089] Sort the line segment sets, and calculate the short side line segment corresponding to the original structure image according to the coordinate minimum value and the coordinate maximum value in the sorted line segment sets.

[0090] According to a third aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method according to any one of the above by executing the executable instructions.

[0091] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method according to any one of the above is implemented.

[0092] The exemplary embodiments of the present disclosure may have the following partial or all beneficial effects:

[0093] In the image calibration method provided by an exemplary embodiment of the present disclosure, line segment detection is performed on an original structure image to be calibrated, and line segment data in the original structure image is extracted; calibration data corresponding to the original structure image is determined through the line segment data; and the original structure image is standardized according to the calibration data to obtain a standardized structure image corresponding to the original structure image. On the one hand, calibration data is calculated through a large amount of line segment data in the original structure image, and the original structure image is standardized according to the calibration data to obtain a standardized structure image. Since only the line segment data is involved in the calculation, the calculation amount is small, and the calibration efficiency of the original structure image can be effectively improved; on the other hand, the calibration data is determined through a large amount of line segment data in the original structure image, which can effectively improve the accuracy of the calibration data, and further improve the calibration accuracy of the original structure image.

[0094] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0096] Figure 1 A schematic diagram showing an exemplary system architecture of an image calibration method and apparatus to which embodiments of the present disclosure can be applied;

[0097] Figure 2 A schematic diagram showing the structure of a computer system of an electronic device suitable for implementing embodiments of the present disclosure;

[0098] Figure 3 A schematic flowchart showing an image calibration method according to an embodiment of the present disclosure;

[0099] Figure 4 Schematically shows the structural schematic diagram when the picture of the liquid crystal panel according to an embodiment of the present disclosure is in a horizontal view;

[0100] Figure 5 Schematically shows the structural schematic diagram when the picture of the liquid crystal panel according to an embodiment of the present disclosure is in a vertical view;

[0101] Figure 6 Schematically shows the flow schematic diagram for determining the tilt rotation angle corresponding to the original structure image according to an embodiment of the present disclosure;

[0102] Figure 7 Schematically shows the flow schematic diagram for re-filtering line segments according to an embodiment of the present disclosure;

[0103] Figure 8 Schematically shows the flow schematic diagram for determining the scaling ratio corresponding to the original structure image according to an embodiment of the present disclosure;

[0104] Figure 9 Schematically shows the flow schematic diagram for calculating the scaling ratio according to an embodiment of the present disclosure;

[0105] Figure 10 Schematically shows the flow schematic diagram for estimating the side length of the short-side line segment according to an embodiment of the present disclosure;

[0106] Figure 11 Schematically shows the flow schematic diagram for calculating the side length of the short-side line segment according to an embodiment of the present disclosure;

[0107] Figure 12 Schematically shows the flow schematic diagram for calibrating the original structure image according to an embodiment of the present disclosure;

[0108] Figure 13 Schematically shows the flow schematic diagram for generating a reference image according to an embodiment of the present disclosure;

[0109] Figure 14 Schematically shows the flow schematic diagram for normalizing the original structure image according to an embodiment of the present disclosure;

[0110] Figure 15 Schematically shows the schematic block diagram of an image calibration device according to an embodiment of the present disclosure. Detailed implementation manners

[0111] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will recognize that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or may be implemented using other methods, components, devices, steps, etc. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0112] In addition, the drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0113] Figure 1 A schematic diagram of a system architecture of an exemplary application environment of an image calibration method and apparatus to which embodiments of the present disclosure can be applied is shown.

[0114] As Figure 1 shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The terminal devices 101, 102, 103 may be various electronic devices having a display screen, including but not limited to desktop computers, portable computers, smartphones, and tablet computers, etc. It should be understood that Figure 1 the number of terminal devices, networks, and servers in

[0115] The image calibration method provided by the embodiments of the present disclosure is generally executed by the server 105. Correspondingly, the image calibration device is generally disposed in the server 105. However, those skilled in the art can easily understand that the image calibration method provided by the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, and 103. Correspondingly, the image calibration device can also be disposed in the terminal devices 101, 102, and 103. No special limitation is made in this exemplary embodiment. For example, in an exemplary embodiment, the user may upload the original structure image to the server 105 through the terminal devices 101, 102, and 103, and the server transmits the standardized structure image to the terminal devices 101, 102, and 103 through the image calibration method provided by the embodiments of the present disclosure.

[0116] Figure 2 FIG. shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure.

[0117] It should be noted that Figure 2 the computer system 200 of the electronic device shown is only an example, and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0118] As Figure 2 shown, the computer system 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage section 208 into the random access memory (RAM) 203. In the RAM 203, various programs and data required for system operation are also stored. The CPU 201, ROM 202, and RAM 203 are connected to each other through a bus 204. The input / output (I / O) interface 205 is also connected to the bus 204.

[0119] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. The drive 210 is also connected to the I / O interface 205 as required. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as required, so that the computer program read from it can be installed into the storage section 208 as required.

[0120] In particular, according to embodiments of the present disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 209, and / or installed from the removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, various functions defined in the methods and apparatuses of the present application are executed. In some embodiments, the computer system 200 may further include an AI (Artificial Intelligence) processor, which is used to process computational operations related to machine learning.

[0121] It should be noted that the computer-readable medium shown in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination of the above.

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0123] The units described in the embodiments of the present disclosure can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0124] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device is caused to implement the methods described in the following embodiments. For example, the electronic device may implement the steps shown in Figure 3 , Figures 6 to 14 and so on.

[0125] The technical solutions of the embodiments of the present disclosure will be elaborated in detail below:

[0126] The inventors found that in the related art when performing skew correction on a picture, the common picture skew correction methods include the following two:

[0127] The first one is to first detect the text in the picture, enclose each character with a rectangle, calculate the center point of the rectangle, then connect the center points of the rectangles into a line segment, estimate the inclination angle of the line segment, thereby determining the inclination angle of the picture, and then perform skew correction; the second method is to pre-estimate a possible inclination range of an image, let the picture rotate within the range at a certain step size, calculate once every rotation, and perform a global change degree estimation once. Eventually, a rotation angle can always be found such that the global change is minimized, thereby completing skew correction.

[0128] However, there is little text in pictures of products with a large number of identical or similar structures such as liquid crystal panel products, and the method of estimating the tilt angle by rotating the picture multiple times has a relatively large amount of computation. Therefore, both of the above two tilt correction methods are not very suitable for tilt correction of pictures of products with a large number of identical or similar structures such as liquid crystal panel products.

[0129] In addition, the inventors also found that the scale scaling of pictures is essentially image registration. Common image registration includes the following two types:

[0130] The first is the feature point-based method: first detect feature points on two pictures, calculate feature descriptors, and then use the feature point descriptors to perform feature point matching to estimate the transformation matrix between the two pictures. At this time, one of the pictures can be inversely mapped back to the other picture so that the scales of the two pictures are consistent; the second method is to preset the transformation relationship between the two pictures, such as similarity transformation, affine transformation, etc., and use the optimization method to perform iteration to estimate the transformation relationship parameters.

[0131] However, pictures of products with a large number of identical or similar structures such as liquid crystal panel products have obvious periodic repeated graphic features, which makes many of the detected feature points have very similar descriptors, making it difficult to estimate the transformation matrix. Simply put, the registration method based on feature point matching is only suitable for conventional natural scenes, at least scenes without a large number of periodic repetitions. After testing, presetting the transformation relationship and then calculating the transformation relationship parameters through optimization iteration is inaccurate and has a large amount of computation, and has no obvious practical value.

[0132] Based on the above one or more problems, the present exemplary embodiment provides an image calibration method. This image calibration method can be applied to the above-mentioned server 105, or to one or more of the above-mentioned terminal devices 101, 102, 103. Of course, it can also be used in a system constructed by one or more of the terminal devices 101, 102, 103 and the server 105. No special limitation is made in this exemplary embodiment. The following takes the terminal device executing this method as an example for illustration. Refer to Figure 3 As shown, this image calibration method may include the following steps S310 to S340:

[0133] Step S310: Obtain the original structure image to be calibrated.

[0134] Step S320: Perform line segment detection on the original structure image and extract the line segment data in the original structure image.

[0135] Step S330: Determine the calibration data corresponding to the original structure image through the line segment data.

[0136] Step S340: Standardize the original structure image according to the calibration data to obtain the standardized structure image corresponding to the original structure image.

[0137] In the image calibration method provided by this exemplary embodiment, line segment detection is performed on the original structure image to be calibrated, and the line segment data in the original structure image is extracted; the calibration data corresponding to the original structure image is determined through the line segment data; the original structure image is standardized according to the calibration data to obtain the standardized structure image corresponding to the original structure image. On the one hand, the calibration data is calculated through the line segment data existing in large quantities in the original structure image, and the original structure image is standardized according to the calibration data to obtain the standardized structure image. Since only the line segment data is involved in the calculation, the calculation amount is small, and the calibration efficiency of the original structure image can be effectively improved; on the other hand, the calibration data is determined through the line segment data existing in large quantities in the original structure image, which can effectively improve the accuracy of the calibration data, and further improve the calibration accuracy of the original structure image.

[0138] Next, the above steps of this exemplary embodiment will be described in more detail.

[0139] In step S310, the original structure image to be calibrated is obtained.

[0140] In an exemplary embodiment of the present disclosure, the original structure image may refer to a picture of a product to be calibrated with a large number of identical or similar structures. For example, the original structure image may be a picture corresponding to a liquid crystal panel product, or a circuit board product with a large number of identical or similar structures and a repetitive periodic distribution. Of course, it may also be other to-be-calibrated images with a large number of identical or similar structures and a repetitive periodic distribution. This exemplary embodiment does not make special limitations on this.

[0141] The original structure image may be obtained from an offline database, or directly obtained from an image acquisition device through real-time network communication. This exemplary embodiment does not make special limitations on the source of the original structure image.

[0142] In step S320, line segment detection is performed on the original structure image, and the line segment data in the original structure image is extracted.

[0143] In an exemplary embodiment of the present disclosure, line segment detection may refer to a processing process that can identify line segments from the original structure image. For example, line segment detection may be a processing process that identifies line segments from the original structure image through a line segment detection algorithm (Line Segment Detector, LSD), or a processing process that identifies line segments from the original structure image through an image edge detection algorithm. Of course, it may also be other processing processes that can identify line segments from the original structure image. This exemplary embodiment does not make special limitations on this.

[0144] The line segment data may refer to the line segments recognized from the original structural image, and may include the length data of the line segments and the coordinate point data corresponding to the line segments. For example, the line segment data may be a horizontal line segment with a length of 2 cm in the original structural image, or a vertical line segment with a length of 3 cm in the original structural image. Of course, it may also be other line segments in the original structural image. This exemplary embodiment does not make special limitations on this.

[0145] In step S330, calibration data corresponding to the original structural image is determined through the line segment data.

[0146] In an exemplary embodiment of the present disclosure, the calibration data may refer to the data that can calibrate the original structural image. For example, the calibration data may be the tilt rotation angle corresponding to the original structural image calculated through the line segment data, or the scaling ratio corresponding to the original structural image calculated through the line segment data. Of course, it may also be other calibration data corresponding to the original structural image that can be calculated through the line segment data. This exemplary embodiment is not limited thereto.

[0147] In step S340, the original structural image is standardized according to the calibration data to obtain the standardized structural image corresponding to the original structural image.

[0148] In an exemplary embodiment of the present disclosure, the standardization process may refer to a process that can calibrate the original structural image. For example, the standardization process may be a process of correcting the tilt of the original structural image through the tilt rotation angle in the calibration data, or a process of scaling the original structural image through the scaling ratio in the calibration data. Of course, it may also be other processes of calibrating the original structural image through the calibration data. This exemplary embodiment does not make special limitations on this.

[0149] The standardized structural image may be an image obtained by the original structural image through the standardization process. By performing defect analysis on the product using the standardized structural image corresponding to the product, the accuracy of defect analysis can be effectively improved.

[0150] Next, a detailed description will be given taking the calibration image of a liquid crystal panel product with a large number of identical or similar structures as an example.

[0151] First, for the same production stage of the same liquid crystal panel product, the distance from the image acquisition unit to the liquid crystal panel and the orientation of the image acquisition unit are fixed. Therefore, for the images taken, the width is the same, the height is the same, the number of repetition periods of the metal and non-metal regions is the same, and the orientation is the same, either all are horizontal images or all are vertical images. And because the image acquisition unit moves on the liquid crystal panel to locate the defective area, the incomplete metal and non-metal regions at the edges of the pictures are not exactly the same, and there is a translation relationship between different pictures.

[0152] In addition, due to the jitter of the image acquisition unit itself, there may be slight tilting, as well as slight magnification or reduction between the pictures taken at the same production stage of the same liquid crystal panel product. The calibration of the liquid crystal panel pictures is to perform tilt correction on the pictures taken at the same production stage of the same liquid crystal panel product and make the scales of all pictures, or rather, the short side length and the long side length of the largest non-metal region in all pictures, exactly the same respectively.

[0153] The liquid crystal panel pictures may include the largest non-metal square, the long side line segment and the short side line segment of the largest non-metal square, horizontal pictures, vertical pictures, and the long side spacing of the largest non-metal square. The specific definitions are as follows:

[0154] For the liquid crystal panel pictures to be analyzed, there are a large number of metal lines in the horizontal and vertical directions. The metal lines have a certain width, and the widths of the lines are different at different positions.

[0155] The non-metal regions are surrounded by metal lines into a grid shape, called non-metal squares, and the metal lines and non-metal squares are repeated periodically. The square with the long side line segment and the short side line segment both being the maximum value is called the largest non-metal square.

[0156] If the long side line segment of the largest non-metal square is in the horizontal direction, the liquid crystal panel picture is called a horizontal picture, and the distance between the lower boundary of one largest non-metal square and the upper boundary of the adjacent largest non-metal square below it is called the long side spacing of the largest non-metal square.

[0157] If the long side line segment of the largest non-metal square is in the vertical direction, the liquid crystal panel picture is called a vertical picture, and the distance between the right boundary of one largest non-metal square and the left boundary of the adjacent largest non-metal square on its right is called the long side spacing of the largest non-metal square.

[0158] Figure 4 Schematically shows a structural schematic diagram of a liquid crystal panel picture as a horizontal picture according to an embodiment of the present disclosure.

[0159] Reference Figure 4As shown, the original structural image 401 is a horizontal view of the liquid crystal panel picture. Among them, the rectangular area 402 represents the largest non-metallic grid square horizontally in the liquid crystal panel picture. The line segment 403 represents the long side line segment of the largest non-metallic grid square 402 horizontally, the line segment 404 represents the short side line segment of the largest non-metallic grid square 402 horizontally, and the spacing 405 represents the long side spacing of the largest non-metallic grid square horizontally.

[0160] Figure 5 Schematically shows a schematic structural diagram when the liquid crystal panel picture according to an embodiment of the present disclosure is a vertical view.

[0161] Reference Figure 5 As shown, the original structural image 501 is a vertical view of the liquid crystal panel picture. Among them, the rectangular area 502 represents the largest non-metallic grid square vertically in the liquid crystal panel picture. The line segment 503 represents the long side line segment of the largest non-metallic grid square 502 vertically, the line segment 504 represents the short side line segment of the largest non-metallic grid square 502 vertically, and the spacing 505 represents the long side spacing of the largest non-metallic grid square vertically.

[0162] In an exemplary embodiment of the present disclosure, the calibration data may include the tilt rotation angle and the scaling ratio. The tilt rotation angle corresponding to the original structural image can be determined through the line segment data; and the scaling ratio corresponding to the original structural image can be determined through the line segment data.

[0163] Among them, the tilt rotation angle may be the angle by which the original structural image is deflected determined through the line segment data in the original structural image. The tilt rotation angle corresponding to the original structural image can be determined through the line segment data so as to reverse-rotate the original structural image according to the tilt rotation angle to achieve tilt correction of the original structural image.

[0164] The scaling ratio may be the ratio of the line segment data in the original structural image compared to the determined standard line segment data. Since when the original structural image is scaled, the scaling ratio of all line segment data is the same, the scaling ratio between the line segment data is the scaling ratio of the original structural image. The scaling ratio corresponding to the original structural image can be determined through the line segment data so as to scale the original structural image according to the scaling ratio to achieve consistent (standardized) scale of the original structural image.

[0165] In an exemplary embodiment of the present disclosure, it is possible to Figure 6 determine the tilt rotation angle corresponding to the original structural image through the steps in:

[0166] Reference Figure 6 As shown, in step S610, obtain a preset first length threshold;

[0167] Step S620, filter the line segment data by the first length threshold, and retain the first line segment data;

[0168] Step S630, determine the tilt rotation angle corresponding to the original structure image according to the first line segment data.

[0169] Among them, the first length threshold may refer to filtering the line segments in the original structure image that may affect the calculation of the tilt rotation angle. For example, the first length threshold may be 0.75 mm (the side length of the short side of the largest non-metallic square in the liquid crystal panel picture is 1 mm), or 0.75 cm (the side length of the short side of the largest non-metallic square in the liquid crystal panel picture is 1 cm). Of course, the first length threshold can be custom-set according to the size of the line segment data in the original structure image, as long as the line segments that may affect the calculation of the tilt rotation angle in the original structure image can be filtered by the first length threshold. For example, the first length threshold can be calculated by the relational expression (1):

[0170] ThreshL = k × L short (1)

[0171] Among them, ThreshL can represent the first length threshold, k can represent the threshold coefficient (the value is a constant, custom-set according to the actual situation. For example, the threshold coefficient can be 0.75 or 0.6, and this example embodiment does not make special limitations on this), and L short can represent the side length of the short side of the largest non-metallic square in the liquid crystal panel picture. Retain the line segments with a line segment length greater than the first length threshold ThreshL as the first line segment data.

[0172] The first line segment data can be the line segments retained after filtering the line segment data by the first length threshold. The first line segment data can be the line segments with a line segment length greater than the first length threshold. Determining the tilt rotation angle corresponding to the original structure image by the first line segment data effectively improves the accuracy of the tilt rotation angle calculation because the line segments that may affect the tilt rotation angle calculation are filtered.

[0173] Specifically, the tilt rotation angle corresponding to the original structure image can be determined through the following steps:

[0174] Calculate the line segment tilt angle corresponding to each line segment in the first line segment data;

[0175] Take the average value of the line segment tilt angles as the tilt rotation angle corresponding to the original structure image.

[0176] Among them, the line segment inclination angle may refer to the inclination angle corresponding to the line segment in the first line segment data. For example, the line segment inclination angle corresponding to line segment A may be 11°, the line segment inclination angle corresponding to line segment B may be 12°, and the line segment inclination angle corresponding to line segment C may be 13°. Then, the inclination rotation angle corresponding to the original structure image is (11° + 12° + 13°) / 3 = 12°.

[0177] Further, the line segment inclination angle corresponding to each line segment in the first line segment data can be calculated through the following steps:

[0178] Calculate the inclination angle corresponding to the target line segment in the first line segment data and any positive direction;

[0179] Calculate the associated angle corresponding to the inclination angle, and use the angle with the smallest absolute value among the inclination angle and the associated angle as the line segment inclination angle of the target line segment.

[0180] Among them, the target line segment may be any line segment in the first line segment data, and any positive direction may be the horizontal direction or the vertical direction. This exemplary embodiment does not make special limitations on this. The inclination angle may be the acute angle between the target line segment and any positive direction. For example, the inclination angle may be the acute angle of 10° between the target line segment and the horizontal direction, or the acute angle of 80° between the target line segment and the vertical direction. Of course, this is only an illustrative example here, and this exemplary embodiment does not make special limitations on this.

[0181] The associated angle may be the angle including all directions of the target line segment calculated from the inclination angle corresponding to the target line segment and any positive direction. For example, the associated angle may include the angle obtained by rotating the inclination angle corresponding to the target line segment and any positive direction by 90° in the positive or negative direction, or may include the angle obtained by rotating the inclination angle corresponding to the target line segment and any positive direction by 180° in the positive or negative direction. Of course, if the original structure image contains line segments with other angles (non-horizontal or non-vertical directions), the associated angle may also include the angle obtained by rotating the inclination angle by other angles in the positive or negative direction. This exemplary embodiment does not make special limitations on this.

[0182] In an exemplary embodiment of the present disclosure, the angle with the smallest absolute value among the inclination angle and the associated angle can be used as the line segment inclination angle of the target line segment.

[0183] For example, assume that the inclination angle of the target line segment is A = alpha, where alpha ∈ [-180°, 180°]. Then the associated angles corresponding to the inclination angle A are B = alpha – 90°, C = alpha + 90°, D = alpha – 180°, and E = alpha + 180°. Next, calculate the absolute values absA, absB, absC, absD, and absE corresponding to the inclination angle and the associated angles, determine the minimum value among absA, absB, absC, absD, and absE, and use the original angle value corresponding to the angle with the smallest absolute value as the line segment inclination angle. For example, if the value of absA is the smallest, then the inclination angle A is the line segment inclination angle of the target line segment; if the value of absB is the smallest, then the associated angle B is the line segment inclination angle of the target line segment, and so on. Of course, this is only an illustrative example, and the present exemplary embodiment is not limited thereto.

[0184] By repeatedly executing the above steps for determining the line segment inclination angle of the target line segment, determine the line segment inclination angle corresponding to each line segment in the first line segment data. Since the line segment inclination angle can be directly determined by the inclination angle of the target line segment and the associated angles without detecting the approximate state (such as the horizontal direction or the vertical direction) in which the target line segment is currently located, the calculation efficiency of the line segment inclination angle is effectively improved.

[0185] Optionally, it is also possible to detect the approximate state (such as the horizontal direction or the vertical direction) in which the target line segment is currently located, and then determine the line segment inclination angle of the target line segment by the included angle between the target line segment and the normal line in the horizontal direction or the included angle between the target line segment and the normal line in the vertical direction. The present exemplary embodiment is not limited thereto.

[0186] In an exemplary embodiment of the present disclosure, before using the angle with the smallest absolute value among the inclination angle and the associated angles as the line segment inclination angle of the target line segment, the line segment can also be Figure 7 re-filtered through the steps in to further ensure the accuracy of the line segment inclination angle:

[0187] Refer to Figure 7 shown, step S710, obtain a preset inclination angle threshold;

[0188] Step S720, filter the line segments for which the angle with the smallest absolute value among the inclination angle and the associated angles is greater than the inclination angle threshold.

[0189] Among them, the inclination angle threshold may refer to the angle threshold for screening and filtering line segments with too large inclination angles. For example, the inclination angle threshold may be 10°. When the inclination angle of a line segment is greater than 10°, it can be considered that the line segment has a large deviation and inclination, and it can be considered that the image captured corresponding to the line segment is taken by mistake and needs to be retaken, or it is a line segment that has a significant error impact on the calculation of the inclination rotation angle of the original structure image and needs to be excluded; the inclination angle threshold may be 5°. Of course, it can also be other angle thresholds, and specific custom settings need to be made according to the actual situation. This exemplary embodiment does not make special limitations on this.

[0190] By setting the inclination angle threshold, while calculating the inclination angle of the line segment, filtering the line segments that have an impact on the calculation of the calibration data can effectively ensure the calculation accuracy of the inclination rotation angle corresponding to the original structure image and improve the calibration accuracy of the original structure image.

[0191] In an exemplary embodiment of the present disclosure, before determining the scaling ratio corresponding to the original structure image through the line segment data, a reference image can be generated through the following steps:

[0192] Arbitrarily select a target structure image in the original structure image;

[0193] Perform inclination correction on the target structure image to obtain the reference image corresponding to the original structure image.

[0194] Among them, the target structure image may refer to an image in which the line segments are clearly displayed (in focus) selected from the original structure image. Perform the above inclination correction steps on the target structure image to obtain the reference image corresponding to the original structure image. The reference image may refer to a specified image used to unify the scale specifications of other original structure images.

[0195] Unify the size specifications of other original structure images according to the scale specifications of the reference image to standardize all original structure images. By setting the reference image and performing unified calculations only according to the standard of the reference image, the calculation amount in the calibration process can be effectively reduced and the calculation efficiency can be improved.

[0196] In an exemplary embodiment of the present disclosure, the scaling ratio corresponding to the original structure image can be determined through the steps in Figure 8 :

[0197] Refer to Figure 8 As shown, in step S810, obtain a preset second length threshold;

[0198] In step S820, filter the line segment data through the second length threshold to retain the second line segment data;

[0199] Step S830, determine the scaling ratio corresponding to the original structure image according to the second line segment data.

[0200] Among them, the second length threshold may refer to filtering line segments with a long enough length. For example, the second length threshold can be used to filter long enough line segments among the long side line segments of the largest non-metal squares in the liquid crystal panel picture (such as the long side line segments of the largest non-metal squares at the edge of the original structure image are not of complete length). For example, the second length threshold can be 0.75 mm (such as the side length of the long side line segment of the largest non-metal square in the liquid crystal panel picture is 1 mm), or 0.75 cm (such as the side length of the long side line segment of the largest non-metal square in the liquid crystal panel picture is 1 cm). Of course, the second length threshold can be custom-set according to the size of the long side line segments in the original structure image, as long as the long side line segments that may affect the calculation of the short side line segment side length can be filtered by the second length threshold. For example, the second length threshold can be calculated by the relational expression (2):

[0201] ThreshLTest = q × L long (2)

[0202] Among them, ThreshLTest can represent the second length threshold, q can represent the threshold coefficient (the value is a constant, custom-set according to the actual situation. For example, the threshold coefficient can be 0.75 or 0.6, and this exemplary embodiment does not make special limitations on this), and L long can represent the side length of the long side line segment of the largest non-metal square in the liquid crystal panel picture. Retain the long side line segments with a line segment length greater than the second length threshold ThreshLTest as the second line segment data.

[0203] The second line segment data can be the long side line segments retained after filtering the line segment data through the second length threshold. The second line segment data can be the long side line segments with a line segment length greater than the second length threshold. Determining the scaling ratio corresponding to the original structure image through the second line segment data effectively improves the accuracy of the scaling ratio calculation because the long side line segments that may affect the scaling ratio calculation are filtered.

[0204] After filtering to obtain the second line segment data with a long enough length, the second line segment data can be further filtered according to a preset inclination angle screening threshold to obtain line segments that are approximately horizontal or approximately vertical in the second line segment data, further improving the accuracy of the scaling ratio calculated according to the second line segment data.

[0205] For example, for a horizontal diagram, retain the approximately horizontal line segments in the second line segment data. Assume the inclination angle is beta, where beta ∈ [-180°, 180°]. Retain the line segments for which the minimum value of the absolute values of X = beta, Y = beta – 180°, and Z = beta + 180° is less than the inclination angle screening threshold (for example, the inclination angle screening threshold can be 0.5° or 1.0°. The specific value can be customized according to the actual situation, and this example embodiment does not make special limitations on this).

[0206] For a vertical diagram, retain the approximately vertical line segments in the second line segment data. Assume the inclination angle is beta, where beta ∈ [-180°, 180°]. Retain the line segments for which the minimum value of the absolute values of X = beta, Y = beta – 90°, and Z = beta + 90° is less than the inclination angle screening threshold (for example, the inclination angle screening threshold can be 0.5° or 1.0°. The specific value can be customized according to the actual situation, and this example embodiment does not make special limitations on this).

[0207] Specifically, the scaling ratio corresponding to the original structure image can be determined through Figure 9 the steps in:

[0208] Refer to Figure 9 As shown, in step S910, estimate the short side line segment corresponding to the original structure image based on the midpoint coordinates corresponding to the second line segment data;

[0209] In step S920, obtain the standard short side line segment in the reference image;

[0210] In step S930, determine the scaling ratio corresponding to the original structure image based on the short side line segment and the standard short side line segment.

[0211] Among them, the midpoint coordinates can refer to the coordinates of the midpoint of the line segment corresponding to the second line segment data (such as the long side line segment). Estimate the short side line segment corresponding to the original structure image through the coordinates of the midpoint of the line segment corresponding to the second line segment data (such as the long side line segment). For example, if the original structure image is a horizontal diagram (that is, when the original structure image is a liquid crystal panel picture, the long side line segment of the largest non - metal grid is horizontal, and the short side line segment of the largest non - metal grid is vertical), then estimate the side length of the short side line segment corresponding to the original structure image according to the ordinate of the midpoint coordinates; if the original structure image is a vertical diagram (that is, when the original structure image is a liquid crystal panel picture, the long side line segment of the largest non - metal grid is vertical, and the short side line segment of the largest non - metal grid is horizontal), then estimate the side length of the short side line segment corresponding to the original structure image according to the abscissa of the midpoint coordinates.

[0212] The length of the short-side line segment can refer to the length of the key line segment in the original structure image that can be estimated from the second line segment data. After obtaining the length of the short-side line segment, obtain the length of the short-side line segment in the reference image, i.e., the standard short-side line segment, and calculate the ratio of the short-side line segment to the standard short-side line segment, which is the scaling ratio corresponding to the original structure image.

[0213] Further, the length of the short-side line segment corresponding to the original structure image can be estimated through the steps Figure 10 as follows:

[0214] Refer to Figure 10 shown in Figure S1010, obtain the preset clustering threshold;

[0215] Step S1020, cluster the second line segment data according to the clustering threshold and the midpoint coordinates to obtain a line segment set;

[0216] Step S1030, calculate the short-side line segment corresponding to the original structure image through the line segment set obtained by clustering.

[0217] Among them, the clustering threshold can refer to the threshold that can cluster line segments within a certain range. For example, the clustering threshold can be 1.25 mm (such as the interval between the long-side line segments of the largest non-metallic squares in the liquid crystal panel picture is 1 mm), or it can be 1.25 cm (such as the interval between the long-side line segments of the largest non-metallic squares in the liquid crystal panel picture is 1 cm). Of course, the clustering threshold can be custom-set according to the size of the interval between the long-side line segments of the largest non-metallic squares in the original structure image, as long as the long-side line segments with an interval distance within a certain range can be clustered together by the clustering threshold. For example, the clustering threshold can be calculated through the relational expression (3):

[0218] ThreshLCluster = p × L adj (3)

[0219] Among them, ThreshLCluster can represent the clustering threshold, p can represent the threshold coefficient (the value is a constant, custom-set according to the actual situation. For example, the threshold coefficient can be 1.25, or it can be 1.50. This exemplary embodiment does not make special limitations on this), and L adj can represent the interval between the long-side line segments of the largest non-metallic squares in the liquid crystal panel picture. Cluster two long-side line segments with the difference in midpoint coordinates (abscissa or ordinate) less than the clustering threshold ThreshLCluster together to obtain a line segment set.

[0220] The line segment set can be a set of line segments belonging to one cluster, and the length of the short-side line segment corresponding to the original structure image can be estimated and calculated through the line segment set obtained by clustering.

[0221] For example, the second line segment data can be clustered by constructing an undirected graph:

[0222] Suppose there are M line segments. First, determine M vertices 1, 2, 3, ……, M of the undirected graph G, and each vertex corresponds to a line segment. Use a two-layer loop to traverse and fill the edges of the undirected graph G: the outer loop takes the subscript i to traverse the vertices from 1 to M, and the inner loop takes the subscript j to traverse the vertices from i + 1 to M. If the distance between line segment i and line segment j is less than the clustering threshold, then add an edge between vertex i and vertex j of G; if it is a horizontal graph, the distance between line segment i and line segment j is the absolute value of the difference in the ordinates of the midpoints of the two line segments. If it is a vertical graph, the distance between line segment i and line segment j is the absolute value of the difference in the abscissas of the midpoints of the two line segments. Perform connected component analysis on the undirected graph G, and the line segments belonging to the same connected component are used as a cluster, that is, a line segment set.

[0223] Specifically, it can be done through Figure 11 the steps in to estimate and calculate the side length of the short side line segment corresponding to the original structure image:

[0224] Refer to Figure 11 shown, step S1110, determine the minimum coordinate value and the maximum coordinate value of the midpoint coordinates in each of the line segment sets;

[0225] Step S1120, sort the line segment sets, and calculate the short side line segment corresponding to the original structure image according to the minimum coordinate value and the maximum coordinate value in the sorted line segment sets.

[0226] For example, when the liquid crystal panel picture is a horizontal graph, calculate the ordinate of the midpoint of the second line segment data. Cluster according to the ordinates of these midpoints, so that for any line segment in each cluster, there is at least one line segment in the clustered line segment set, and the absolute value of the difference in the ordinates of the midpoints of the two is less than or equal to the clustering threshold. Then calculate the minimum value and the maximum value of the ordinates of the midpoints of the line segments in each line segment set, and sort the multiple line segment sets from smallest to largest according to the minimum value of the ordinates of the midpoints of the line segments in the line segment sets. After the sorting is completed, the minimum value and the maximum value of the ordinates of the midpoints of each line segment set are y 1min ,y 1max ,y 2min ,y 2max ,y 3min ,y 3max ……y Nmin ,y Nmax ,and then subtract the maximum value of the ordinate of the midpoint of the previous line segment set from the minimum value of the ordinate of the midpoint of the next line segment set in the sorting to obtain a series of estimated values of the short side line segment side lengths of the largest non-metal grid squares, that is, it can be expressed as the relational expression group (4):

[0227]

[0228] Then, take the average value of the estimated values to obtain the estimated value of the side length of the short side of the final maximum non-metal grid square, which can be expressed by the relational expression (5):

[0229] LShortE = (LShortE1 + LShortE2 + …… + LShortE(N - 1)) / (N - 1) (5)

[0230] Among them, LshortE can represent the estimated value of the side length of the short side of the maximum non-metal grid square.

[0231] For the case where the liquid crystal panel picture is a vertical picture, the calculation process is the same as that when the liquid crystal panel picture is a horizontal picture, except that during the calculation, it is through the abscissa of the midpoint of the second line segment data, which will not be elaborated here.

[0232] Furthermore, after calculating the estimated value of the side length of the short side of the maximum non-metal grid square, the scaling ratio corresponding to the original structure image can be determined according to the estimated short side line segment and the standard short side line segment corresponding to the reference picture.

[0233] For example, assume that the estimated value of the side length of the short side of the maximum non-metal grid square can be expressed as LShortE, and the side length of the standard short side line segment corresponding to the maximum non-metal grid in the reference picture can be expressed as LShort. Then, the scaling ratio can be expressed by the relational expression (6):

[0234] S = LShort / LShortE (6)

[0235] Among them, S can represent the scaling ratio corresponding to the original structure image.

[0236] In an exemplary embodiment of the present disclosure, the original structure image can be standardized through the steps in Figure 12 :

[0237] Refer to Figure 12 shown, step S1210, perform skew correction on the original structure image according to the skew rotation angle to obtain the skew-corrected original structure image; and

[0238] step S1220, scale the skew-corrected original structure image according to the scaling ratio to obtain the standardized structure image corresponding to the original structure image.

[0239] For example, assume that the width corresponding to the original structure image can be expressed as W, and the height can be expressed as H. Then, scaling the skew-corrected original structure image according to the scaling ratio can be expressed by the relational expression group (7):

[0240] W'=W×S,

[0241] H'=H×S (7)

[0242] Wherein, W' may represent the width of the standardized structural image corresponding to the original structural image, H' may represent the height of the standardized structural image corresponding to the original structural image, and S may represent the scaling ratio corresponding to the original structural image.

[0243] Figure 13 The figure schematically shows a flow chart of generating a reference image according to an embodiment of the present disclosure.

[0244] refer to Figure 13 As shown, in step S1310, a clearer candidate reference image is selected from the original structure image;

[0245] Step S1320, performing line segment detection on the candidate reference image to determine line segment data in the candidate reference image;

[0246] Step S1330, retaining line segments in the line segment data that are long enough and approximately horizontal;

[0247] Step S1340, calculating a first deviation angle of the line segment in step S1330 from the horizontal direction;

[0248] Step S1350, retaining line segments in the line segment data that are long enough and approximately vertical;

[0249] Step S1360, calculating a second deviation angle of the line segment in step S1350 from the vertical direction;

[0250] Step S1370, calculating an average of the first deviation angle and the second deviation angle, and using the average as the tilt rotation angle of the candidate reference image;

[0251] Step S1380, reversely rotating the candidate reference image according to the tilt rotation angle to obtain a reference image;

[0252] Step S1390 , measuring the length of the long side segment, the length of the short side segment of the largest non-metallic square in the reference image, and the distance between the long side segments of the largest non-metallic squares.

[0253] Figure 14 The following schematically shows a flow chart of normalizing an original structural image according to an embodiment of the present disclosure.

[0254] refer to Figure 14 As shown, step S1401, obtaining the original structure image;

[0255] Step S1402, performing line segment detection on the original structure image to determine line segment data in the original structure image;

[0256] Step S1403: Retain the line segments in the line segment data that are long enough and approximately horizontal.

[0257] Step S1404: Calculate the first deviation angle of the line segments in Step S1403 from the horizontal direction.

[0258] Step S1405: Retain the line segments in the line segment data that are long enough and approximately vertical.

[0259] Step S1406: Calculate the second deviation angle of the line segments in Step S1405 from the vertical direction.

[0260] Step S1407: Calculate the average value of the first deviation angle and the second deviation angle, and use the average value as the tilt rotation angle of the original structure image.

[0261] Step S1408: Perform reverse rotation on the original structure image according to the tilt rotation angle.

[0262] Step S1409: Obtain the original structure image after tilt correction.

[0263] Step S1410: Perform line segment detection on the original structure image after tilt correction obtained in Step S1409.

[0264] Step S1411: Retain the line segments in the original structure image after tilt correction whose long sides of the largest non-metal squares are long enough.

[0265] Step S1412: Determine whether the original structure image is a horizontal image. If it is, execute Step S1413; otherwise, consider it a vertical image and execute Step S1416.

[0266] Step S1413: Cluster the line segments according to the midpoint ordinates of the line segments obtained in Step S1411 to obtain a line segment set.

[0267] Step S1414: Sort the obtained line segment set in ascending order according to the minimum value of the ordinates in the line segment set.

[0268] Step S1415: Estimate the side length of the short side line segment of the largest non-metal square using the vertical spacing between adjacent line segment sets.

[0269] Step S1416: Cluster the line segments according to the midpoint abscissas of the line segments obtained in Step S1411 to obtain a line segment set.

[0270] Step S1417: Sort the obtained line segment set in ascending order according to the minimum value of the abscissas in the line segment set.

[0271] Step S1418, estimate the side length of the short side line segment of the largest non-metal square using the horizontal spacing of the adjacent line segment set;

[0272] Step S1419, use the ratio of the side length of the short side line segment obtained in Step S1415 or Step S1418 to the side length of the short side line segment of the reference image as the scaling ratio corresponding to the original structure image;

[0273] Step S1420, scale the tilt-corrected original structure image obtained in Step S1409 according to the scaling ratio obtained in Step S1419;

[0274] Step S1421, obtain the standardized structure image corresponding to the original structure image.

[0275] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the shown steps must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0276] Furthermore, in the present exemplary embodiment, an image calibration device is also provided. This image calibration device can be applied to a server or a terminal device. Refer to Figure 15 As shown, the image calibration device 1500 may include an original structure image acquisition module 1510, a line segment data extraction module 1520, a calibration data determination module 1530, and an original structure image calibration module 1540. Among them:

[0277] The original structure image acquisition module 1510 is configured to acquire the original structure image to be calibrated;

[0278] The line segment data extraction module 1520 is configured to perform line segment detection on the original structure image and extract the line segment data in the original structure image;

[0279] The calibration data determination module 1530 is configured to determine the calibration data corresponding to the original structure image through the line segment data;

[0280] The original structure image calibration module 1540 is configured to perform standardization processing on the original structure image according to the calibration data to obtain the standardized structure image corresponding to the original structure image.

[0281] In an exemplary embodiment of the present disclosure, the calibration data determination module 1530 includes:

[0282] An inclination rotation angle determination unit for determining an inclination rotation angle corresponding to the original structure image through the line segment data; and

[0283] A scaling ratio determination unit for determining a scaling ratio corresponding to the original structure image through the line segment data.

[0284] In an exemplary embodiment of the present disclosure, the inclination rotation angle determination unit further includes:

[0285] A first length threshold acquisition subunit for acquiring a preset first length threshold;

[0286] A line segment data filtering unit for filtering the line segment data through the first length threshold to retain first line segment data;

[0287] An inclination rotation angle determination subunit for determining an inclination rotation angle corresponding to the original structure image according to the first line segment data.

[0288] In an exemplary embodiment of the present disclosure, the inclination rotation angle determination unit includes:

[0289] A line segment inclination angle calculation subunit for calculating a line segment inclination angle corresponding to each line segment in the first line segment data;

[0290] An inclination rotation angle calculation subunit for taking the average value of the line segment inclination angles as the inclination rotation angle corresponding to the original structure image.

[0291] \>In an exemplary embodiment of the present disclosure, the line segment inclination angle calculation unit is further configured to:

[0292] Calculate an inclination angle corresponding to a target line segment in the first line segment data and any positive direction;

[0293] Calculate an associated angle corresponding to the inclination angle, and take the angle with the smallest absolute value among the inclination angle and the associated angle as the line segment inclination angle of the target line segment.

[0294] In an exemplary embodiment of the present disclosure, the image calibration device 1500 further includes a line segment screening unit, and the line segment screening unit is configured to:

[0295] Acquire a preset inclination angle threshold;

[0296] Filter the line segments with the angle having the smallest absolute value among the inclination angle and the associated angle greater than the inclination angle threshold.

[0297] In an exemplary embodiment of the present disclosure, the original structure image calibration module 1540 further includes:

[0298] An inclination correction subunit, configured to perform inclination correction on the original structure image according to the inclination rotation angle to obtain an inclination-corrected original structure image; and

[0299] An image scaling subunit, configured to scale the inclination-corrected original structure image according to the scaling ratio to obtain a standardized structure image corresponding to the original structure image.

[0300] In an exemplary embodiment of the present disclosure, the image calibration device 1500 further includes a reference image generation unit, and the reference image generation unit is configured to:

[0301] Arbitrarily select a target structure image in the original structure image;

[0302] Perform inclination correction on the target structure image to obtain a reference image corresponding to the original structure image.

[0303] In an exemplary embodiment of the present disclosure, the scaling ratio determination unit includes:

[0304] A second length threshold acquisition subunit, configured to acquire a preset second length threshold;

[0305] A second line segment data generation subunit, configured to filter the line segment data through the second length threshold to retain second line segment data;

[0306] A scaling ratio determination subunit, configured to determine the scaling ratio corresponding to the original structure image according to the second line segment data.

[0307] In an exemplary embodiment of the present disclosure, the scaling ratio determination subunit further includes:

[0308] A short side line segment estimation subunit, configured to estimate a short side line segment corresponding to the original structure image according to the midpoint coordinates corresponding to the second line segment data;

[0309] A standard short side line segment acquisition subunit, configured to acquire a standard short side line segment in the reference image;

[0310] A scaling ratio calculation subunit, configured to determine the scaling ratio corresponding to the original structure image according to the short side line segment and the standard short side line segment.

[0311] In an exemplary embodiment of the present disclosure, the short side line segment estimation unit includes:

[0312] A clustering threshold acquisition subunit, configured to acquire a preset clustering threshold;

[0313] A line segment set clustering subunit, configured to cluster the second line segment data according to the clustering threshold and the midpoint coordinates to obtain a line segment set;

[0314] A short side line segment calculation subunit, configured to calculate the short side line segments corresponding to the original structure image through the line segment set obtained by clustering.

[0315] In an exemplary embodiment of the present disclosure, the short side line segment calculation subunit is further configured to:

[0316] Determine the minimum coordinate value and the maximum coordinate value of the midpoint coordinates in each of the line segment sets;

[0317] Sort the line segment sets, and calculate the short side line segments corresponding to the original structure image according to the minimum coordinate value and the maximum coordinate value in the sorted line segment sets.

[0318] The specific details of each module or unit in the above image calibration device have been described in detail in the corresponding image calibration method, and thus will not be elaborated here.

[0319] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules or units may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by a plurality of modules or units.

[0320] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0321] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image calibration method, characterized in that, Including: Obtain the original structure image to be calibrated; Perform line segment detection on the original structure image, and extract line segment data in the original structure image; Determine the tilt rotation angle corresponding to the original structure image through the line segment data; Arbitrarily select a target structure image in the original structure image, perform tilt correction on the target structure image, and obtain a reference image corresponding to the original structure image; Obtain a preset second length threshold, filter the line segment data through the second length threshold, and retain the second line segment data; Estimate the short side line segment corresponding to the original structure image according to the midpoint coordinates corresponding to the second line segment data; Obtain the standard short side line segment in the reference image; determine the scaling ratio corresponding to the original structure image according to the short side line segment and the standard short side line segment; Perform standardization processing on the original structure image according to the tilt rotation angle and the scaling ratio to obtain a standardized structure image corresponding to the original structure image.

2. The image calibration method according to claim 1, wherein Determining the tilt rotation angle corresponding to the original structure image through the line segment data includes: Obtain a preset first length threshold; Filter the line segment data through the first length threshold, and retain the first line segment data; Determine the tilt rotation angle corresponding to the original structure image according to the first line segment data.

3. The image calibration method according to claim 2, wherein Determining the tilt rotation angle corresponding to the original structure image according to the first line segment data includes: Calculate the line segment tilt angle corresponding to each line segment in the first line segment data; Take the average value of the line segment tilt angles as the tilt rotation angle corresponding to the original structure image.

4. The image calibration method according to claim 3, wherein Calculating the line segment tilt angle corresponding to each line segment in the first line segment data includes: Calculate the tilt angle corresponding to the target line segment in the first line segment data and any positive direction; Calculate the associated angle corresponding to the tilt angle according to the tilt angle, and take the angle with the smallest absolute value among the tilt angle and the associated angle as the line segment tilt angle of the target line segment.

5. The image calibration method according to claim 4, wherein Before taking the angle with the smallest absolute value among the tilt angle and the associated angle as the line segment tilt angle of the target line segment, the method further includes: Obtain a preset tilt angle threshold; Filter the line segments whose absolute value of the smallest angle among the tilt angle and the associated angle is greater than the tilt angle threshold.

6. The image calibration method according to claim 1, wherein The performing standardization processing on the original structure image according to the tilt rotation angle and the scaling ratio includes: Perform tilt correction on the original structure image according to the tilt rotation angle to obtain the original structure image after tilt correction; and Scale the original structure image after tilt correction according to the scaling ratio to obtain a standardized structure image corresponding to the original structure image.

7. The image calibration method according to claim 1, wherein Estimating the short side line segment corresponding to the original structure image according to the midpoint coordinates corresponding to the second line segment data includes: Obtain a preset clustering threshold; Cluster the second line segment data according to the clustering threshold and the midpoint coordinates to obtain a line segment set; Calculate the short side line segment corresponding to the original structure image through the line segment set obtained by clustering.

8. The image calibration method according to claim 7, wherein Calculating the short-side line segments corresponding to the original structure image from the set of line segments obtained by clustering includes: Determining the minimum coordinate value and the maximum coordinate value of the midpoint coordinates in each of the sets of line segments; Sorting the sets of line segments, and calculating the short-side line segments corresponding to the original structure image according to the minimum coordinate value and the maximum coordinate value in the sorted sets of line segments.

9. An image calibration device, characterized in that, Including: An original structure image acquisition module, configured to acquire an original structure image to be calibrated; A line segment data extraction module, configured to perform line segment detection on the original structure image and extract the line segment data in the original structure image; A calibration data determination module, configured to determine the calibration data corresponding to the original structure image through the line segment data; An original structure image calibration module, configured to perform normalization processing on the original structure image according to the calibration data to obtain a normalized structure image corresponding to the original structure image; The image calibration device further includes a reference image generation unit, and the reference image generation unit is configured to: arbitrarily select a target structure image in the original structure image; Perform skew correction on the target structure image to obtain a reference image corresponding to the original structure image; The calibration data determination module includes: a tilt rotation angle determination unit, configured to determine the tilt rotation angle corresponding to the original structure image through the line segment data; A scaling ratio determination unit, including: a second length threshold acquisition subunit, configured to acquire a preset second length threshold; a second line segment data generation subunit, configured to filter the line segment data through the second length threshold to retain second line segment data; a scaling ratio determination subunit, configured to determine the scaling ratio corresponding to the original structure image according to the second line segment data; The scaling ratio determination subunit includes: a short-side line segment estimation subunit, configured to estimate the short-side line segments corresponding to the original structure image according to the midpoint coordinates corresponding to the second line segment data; a standard short-side line segment acquisition subunit, configured to acquire the standard short-side line segments in the reference image; a scaling ratio calculation subunit, configured to determine the scaling ratio corresponding to the original structure image according to the short-side line segments and the standard short-side line segments.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1-8.

11. An electronic device, characterized in that, Including: A processor; And A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the method according to any one of claims 1-8 by executing the executable instructions.

12. A computer program product, characterized in that, It includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method according to any one of claims 1-8.

Images