A transparent screen defect positioning detection system and positioning detection method
By using image acquisition technology combined with linear slide rails and multi-angle cameras in the transparent screen detection system, the problem that the existing technology cannot distinguish the surface and interior of the transparent screen defect is solved, and automated detection is realized, improving detection accuracy and efficiency.
Patent Information
- Application Number
- CN202111490441.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-08
AI Technical Summary
The prior art cannot accurately determine whether the defect of the transparent screen is on the surface or inside, resulting in the need of manual inspection, the detection accuracy is not high, the efficiency is poor, and the vision damage is caused.
A linear slide rail and a transparent carrier platform are used to combine the first camera and the second camera for image acquisition, and a three-dimensional positioning detection is realized through the specular reflection layer and the diffuse reflection layer. The data processing unit calculates the depth and position of the defect and judges the defect on the screen surface. or internal.
Automatic detection of transparent screens is realized, replacing manual detection, improving detection accuracy and efficiency, and reducing costs.
Smart Images

Figure CN114166864B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of transparent screen detection, and in particular to a transparent screen defect positioning detection system and positioning detection method. Background Art
[0002] A screen is an electrical appliance used to display images and colors. Transparent display screens (such as mobile phone screens, Pad screens, etc.) are usually made of multiple layers of materials with different functions. During the production process, scratches, adhered particles, adhered fingerprints, dirt and other defects will inevitably occur on the surface and inside of the screen. If there are defects such as particles and dirt on the surface, they can be removed by cleaning and other means. If there are defects inside, the product is unqualified.
[0003] The prior art (CN201811036012.4) discloses a system and method for detecting fine appearance defects of a micro camera module lens, wherein the system comprises a metallographic microscopic imaging platform, a stage and an industrial computer, wherein the metallographic microscopic imaging platform is arranged above the stage, and the industrial computer is arranged below the stage; the metallographic microscopic imaging platform comprises an industrial camera, a coaxial light source, and a microscopic magnification device, wherein the industrial camera, the microscopic magnification device and the coaxial light source are arranged from top to bottom; firstly, the microscopic magnification device is used to optically magnify the camera module, then the coaxial light source is used for lighting, and finally, the industrial camera is used to collect a complete image of the camera module; the stage comprises an X-axis encoder and a grating ruler arranged on the X-axis encoder, a Y-axis encoder and a grating ruler arranged on the Y-axis encoder, and a motor box, wherein the motor box is connected to the X-axis encoder and the Y-axis encoder. Axis encoder; by controlling the speed of the X-axis and Y-axis motors in the motor box, the displacement of the X-axis and Y-axis of the stage is controlled respectively, and the grating ruler measures and feeds back the displacement to realize a full closed loop; the industrial computer is used to control the displacement of the stage and realize precise defect detection. The present invention can realize precise detection of micron-level defects on the appearance of the camera module, improve detection accuracy and automation, and provide guarantee for product quality.
[0004] However, existing inspection technology can only determine whether the inspected object has defects and the two-dimensional position of the defects (i.e., it cannot determine whether the defects are on the surface or inside). Therefore, manual inspection is required during the production process, that is, to inspect whether the product has defects and the spatial position of the defects by naked eye observation. The inspection accuracy is low, the efficiency is poor, and it also has a significant damage to the eyesight of the inspectors.
[0005] Therefore, providing a transparent screen defect positioning detection system and positioning detection method is an urgent problem to be solved in the field. Summary of the invention
[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a transparent screen defect positioning detection system and positioning detection method.
[0007] The objective of the present invention is achieved through the following technical solutions:
[0008] A first aspect of the present invention provides a transparent screen defect positioning detection system, comprising:
[0009] Linear slides;
[0010] A transparent loading platform is arranged on a linear slide rail and is driven to move by a motor; a mirror reflection layer is arranged on the lower surface, and an upper surface is used to place a detection object, and the detection object is a transparent screen;
[0011] A first camera, wherein a first optical axis of the first camera is perpendicular to a plane where the detection object is located;
[0012] a second camera, wherein a second optical axis of the second camera forms an angle α with the first optical axis, and α is less than 90°;
[0013] A light source is arranged above the transparent loading platform;
[0014] The data processing unit is used to process the images acquired by the first camera and the second camera to realize three-dimensional positioning detection.
[0015] Furthermore, the system further comprises:
[0016] The encoder receives the acquisition line frequency set by the camera and outputs a speed signal to the motor; the motor controls the transparent loading platform to move at a uniform speed on the linear slide rail according to the output speed signal.
[0017] Furthermore, the image of the first camera and the image of the second camera are images of the detection object after the detection object completely passes through the camera shooting area, and the camera can capture a complete image including the detection object.
[0018] Furthermore, a non-transparent diffuse reflection layer is provided on the upper surface of the transparent loading platform for rapid positioning.
[0019] A second aspect of the present invention provides a method for positioning and detecting defects of a transparent screen. Based on the positioning and detecting system, the method comprises the following steps:
[0020] The image A of the first camera and the image B of the second camera are registered, and the two images are adjusted to the same coordinate system; in the composite image, P" is the image of the defect P in image A, P' is the image of the defect P in image B, and Q' is the image of the reflection Q of the defect P in image B; wherein the position P" of the defect P in image A is known;
[0021] According to the position of P" and P"P' in the composite figure x Length, and calculated O'P' y / O'P" y The constraint relationship is to search for the position of point P' in a specific area of the image; where P"P' x represents the projection of P"P' on the x-axis, O'P' y represents the projection of O'P' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction, O represents the focus of the second camera, and O' represents the center point of the imaging plane; the positive direction of the x-axis is defined as a direction parallel to and opposite to the direction of motion, and the positive direction of the y-axis is defined as a direction perpendicular to the x-axis and parallel to the imaging plane;
[0022] According to the position of P" and P"Q' in the composite diagram x Length, and calculated O'Q' y / O'P" y The constraint relationship is to search for the Q' point position in a specific area of the image; where P"Q' x represents the projection of P”Q' on the x-axis, O'Q' y represents the projection of O'Q' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction;
[0023] When P' and Q' meet the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the image B and the projection of the reflection Q of the defect P. At this time, the depth AP of the defect P is calculated.
[0024] Furthermore, the method further comprises the following steps:
[0025] Depth AP determines whether the defect is on the screen surface or inside the screen. If the defect is inside the screen, determine between which two material layers the defect is.
[0026] Further, according to the position of P" and the P"P' in the composite graph x Length, and calculated O'P' y / O'P" y The constraint relationship is obtained by searching for the position of point P' in a specific area of the image, including the following sub-steps:
[0027] In the xz projection plane, we have:
[0028] P″A′ x =d sinα
[0029] P″B′ x =(dcosα+d1)tanα
[0030] P″A′ x ≤P″P′ x ≤P″B′ x
[0031] Where d represents the distance from the imaging plane to the upper surface of the detection object, d1 represents the thickness of the detection object, P”A’ x represents the projection of P”A’ on the x-axis, P”B’ x represents the projection of P"B' in the x-axis direction; the xz projection plane represents the plane composed of the x-axis and the z-axis and the projection of the point and line objects photographed by the second camera along the y-axis on the xz plane, the z-axis represents the direction parallel to the optical axis of the first camera, and the positive direction is vertically downward; A', B', Q' are the projections of A, B, Q on the imaging plane respectively;
[0032] On the yz projection plane, according to the imaging principle, we have:
[0033]
[0034] The yz projection plane represents a plane formed by the y axis and the z axis and is a projection of the point or line object captured by the second camera along the x axis on the yz plane;
[0035] On the y-z' projection plane, according to the imaging principle, we have:
[0036]
[0037]
[0038]
[0039] Among them, OP y =OQ y (b)
[0040] Where f represents the distance from the focus O to the imaging plane, AP z represents the projection of AP in the z-axis direction, OP y represents the projection of OP on the y-axis, d2 represents the thickness of the transparent loading platform, OQ y Represents the projection of OQ in the y-axis direction; y-z' projection plane is the plane formed by the y-axis and the z'-axis and the projection of the point and line objects captured by the second camera along the x-axis on the y-z' plane;
[0041] Therefore, by matching the calculation of the second formula in formula (a) and formula (b), we have:
[0042]
[0043] In the composite graph, according to the position of P” and P”P’ in the composite graphx Length, and calculated O'P' y / O'P" y The constraint relationship is determined by searching for the position of point P' in a specific area of the image.
[0044] Further, according to the position of P" and the P"Q' in the synthetic graph x Length, and calculated O'Q' y / O'P" y The constraint relationship is obtained by searching the Q' point position in a specific area of the image, including the following sub-steps:
[0045] On the xz projection plane, we have:
[0046] (dcosα+d1+2d2)tanα≤P″Q′ x ≤(dcosα+2d1+2d2)tanα
[0047] On the y-z' projection plane, according to the imaging principle, the calculation matching of the second formula in equation (a) and equation (b) yields:
[0048]
[0049] Therefore, according to the position of P" and the composite diagram P"Q' x Length, and calculated O'Q' y / O'P" y The constraint relationship is determined by searching the specific area of the image to obtain the position of point Q'.
[0050] Furthermore, when P' and Q' satisfy the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the image B and the projection of the reflection Q of the defect P. At this time, the depth AP of the defect P is calculated, including the following sub-steps:
[0051] When P' and Q' meet the following parameter conditions:
[0052] P″P′ x +P″Q′ x =(2dcosα+2d1+2d2)tana
[0053] It can be determined that P' and Q' are the projection of defect P in image B and the projection of defect P's reflection Q respectively. At this time, the depth AP of defect P is calculated:
[0054]
[0055] Furthermore, the method further comprises the following steps:
[0056] The acquired image A from the first camera and the acquired image B from the second camera are preprocessed including distortion correction.
[0057] The beneficial effects of the present invention are:
[0058] (1) In an exemplary embodiment of the present invention, a first camera whose optical axis is perpendicular to the plane where the inspection object is located and a second camera which is at an angle α to the first optical axis are used to collect images. Since a mirror reflection layer is provided on the lower surface of the transparent loading platform, the first camera can collect the image of the defect P in image A, and the second camera can collect the image of the defect P in image B and the image of the reflection Q of the defect P in image B. The data processing unit can then distinguish the three-dimensional spatial position of the defect based on corresponding calculations, thereby realizing automatic detection of the screen, replacing manual detection, effectively reducing costs, and improving detection accuracy and efficiency.
[0059] (2) In another exemplary embodiment of the present invention, the transparent loading platform is fixed on a linear slide rail controlled by a servo motor, and the motor is driven by software control to enable the detection object to move on the linear slide rail at any speed within a certain range. The speed is controlled by an encoder connected to a camera to match the camera acquisition frequency with the movement speed of the detection object.
[0060] (3) In another exemplary embodiment of the present invention, a non-transparent diffuse reflection layer is provided on the upper surface of the transparent loading platform for rapid positioning.
[0061] (4) In yet another exemplary embodiment of the present invention, a specific implementation of the positioning detection method is disclosed. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 It is a structural schematic diagram of a positioning detection system provided in an exemplary embodiment of the present invention;
[0063] Figure 2 A flowchart of a positioning detection method provided in an exemplary embodiment of the present invention;
[0064] Figure 3 A schematic diagram of a composite image provided in an exemplary embodiment of the present invention;
[0065] Figure 4 It is a schematic diagram of projection of objects such as points and lines photographed by a second camera along the y-axis on the xz plane in an exemplary embodiment of the present invention;
[0066] Figure 5 A schematic diagram of projection of a point, line or other object captured by a second camera along the x-axis on the yz plane in an exemplary embodiment of the present invention;
[0067] Figure 6 It is a schematic diagram of projection of a point, line or other object photographed by a second camera along the x-axis on the y-z' plane provided in an exemplary embodiment of the present invention;
[0068] Figure 7 A three-dimensional schematic diagram provided in an exemplary embodiment of the present invention;
[0069] In the figure, 1-first camera, 2-second camera, 3-light source, 4-detection object, 401-upper surface of detection object, 402-lower surface of detection object, 5-transparent loading platform, 501-lower surface of transparent loading platform, 6-marking pattern, 7-imaging plane, 8-point O trajectory. DETAILED DESCRIPTION
[0070] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0071] In the description of the present invention, it should be noted that the directions or positional relationships indicated by “center”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “inside” and “outside”, etc., are directions or positional relationships based on the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0072] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0073] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0074] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination". In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0075] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0076] A screen is an electrical appliance used to display images and colors. Transparent display screens are usually made of multiple layers of materials with different functions. During the production process, scratches, adhered particles, adhered fingerprints, dirt and other defects will inevitably occur on the surface and inside of the screen. If there are defects such as particles and dirt on the surface, they can be removed by cleaning and other means. If there are defects inside, the product is unqualified. However, existing detection technology can only determine whether the detection object has defects and the two-dimensional position of the defects (that is, it cannot determine whether the defects are on the surface or inside). Therefore, manual inspection is required during the production process, that is, to detect whether the product has defects and the spatial position of the defects by naked eye observation. The detection accuracy is not high, the efficiency is poor, and it also causes great damage to the eyesight of the inspectors.
[0077] Therefore, the following exemplary embodiments will solve the above problems:
[0078] See also Figure 1 , Figure 1 A transparent screen defect positioning detection system provided by an exemplary embodiment of the present invention is shown, comprising:
[0079] Linear slides;
[0080] A transparent loading platform 5 is arranged on a linear slide rail and is driven by a motor to move the transparent loading platform 5; a mirror reflection layer is arranged on the lower surface, and the upper surface is used to place a detection object 4, and the detection object 4 is a transparent screen;
[0081] A first camera 1, wherein a first optical axis of the first camera 1 is perpendicular to a plane where the detection object 4 is located;
[0082] A second camera 2, wherein a second optical axis of the second camera 2 forms an angle α with the first optical axis, and α is less than 90°;
[0083] A light source 3 is disposed above the transparent loading platform 5;
[0084] The data processing unit is used to process the images acquired by the first camera 1 and the second camera 2 to achieve three-dimensional positioning detection.
[0085] Specifically, in this exemplary embodiment, the first camera 1 and the second camera 2 are two high-resolution line scan cameras (including lenses) at different angles to collect images of the detection object 4. The detection object of thickness d1 is placed on a transparent loading platform 5, which is equipped with transparent glass of thickness d2, and its lower surface is coated with a material that can achieve mirror reflection; and in an exemplary embodiment, a local area of the upper surface is coated with a non-transparent material that can achieve diffuse reflection, that is, a logo pattern 6 for rapid positioning of the algorithm. The transparent loading platform 5 is fixed on a linear slide controlled by a servo motor, and the motor is driven by software to drive the slide, so that the detection object 4 can move at any speed within a certain range on the linear slide, and in an exemplary embodiment, the speed is controlled by an encoder connected to the camera to match the camera acquisition frequency with the movement speed of the detection object. In addition, a light source 3 (LED light source) for lighting is installed above the transparent loading platform 5, and the brightness of the light source 3 can be controlled by a controller.
[0086] When testing is required, the following steps may be involved:
[0087] (1) Turn on the light source 3 and adjust the brightness to an appropriate range.
[0088] (2) During detection, the transparent carrier platform 5 is controlled to drive the detection object 4 to pass through the camera shooting area along the movement direction; and in one of the preferred exemplary embodiments, the camera outputs a signal to the encoder according to the set acquisition line frequency, and the encoder outputs a speed signal to the servo motor to control the transparent carrier platform 5 to move at a uniform speed on the linear slide rail.
[0089] (3) Start the first camera 1 and the second camera 2 to start collecting data. In one preferred exemplary embodiment, the transparent loading platform 5 drives the detection object 4 to pass through the camera shooting area along the movement direction. When the detection object passes through the area completely, the camera can collect a complete image containing the detection object.
[0090] (4) The first camera 1 and the second camera 2 transmit the acquired images to the data processing unit respectively.
[0091] (5) The data processing unit processes the image to obtain three-dimensional positioning of the defect.
[0092] Compared with the prior art which can only determine whether the inspection object has defects and the two-dimensional position and three-dimensional position of the defects, and the detection requires naked eye observation with low accuracy, the positioning inspection system in this exemplary embodiment uses a first camera 1 whose optical axis is perpendicular to the plane where the inspection object 4 is located and a second camera 2 which is at an angle α to the first optical axis for image acquisition. Since the lower surface of the transparent loading platform 5 is provided with a mirror reflection layer, the first camera 1 can acquire the image of the defect P in image A, and the second camera 2 can acquire the image of the defect P in image B and the image of the reflection Q of the defect P in image B. The data processing unit can then distinguish the three-dimensional spatial position of the defect based on the corresponding calculation, thereby realizing automatic detection of the screen, replacing manual detection, effectively reducing costs, and improving detection accuracy and efficiency.
[0093] See also Figure 2 , Figure 2 A flowchart of a method for positioning and detecting a defect of a transparent screen provided in another exemplary embodiment of the present invention is shown. Based on the positioning and detecting system, the method comprises the following steps:
[0094] S02: Align the image A of the first camera and the image B of the second camera, and adjust the two images to the same coordinate system; in the composite image, P" is the image of the defect P in image A, P' is the image of the defect P in image B, and Q' is the image of the reflection Q of the defect P in image B; wherein, the position P" of the defect P in image A is known;
[0095] S04: According to the position of P” and P”P’ in the composite image x Length, and calculated O'P' y / O'P" y The constraint relationship is to search for the position of point P' in a specific area of the image; where P"P' x represents the projection of P"P' on the x-axis, O'P' y represents the projection of O'P' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction, O represents the focus of the second camera (specifically, O represents the optical center of the lens, the same below. Optical center: a special point in the lens, any light passing through this point, its propagation direction remains unchanged), O' represents the center point of the imaging plane 7; the positive direction of the x-axis is defined as the direction parallel to and opposite to the motion direction, and the positive direction of the y-axis is defined as the direction perpendicular to the x-axis and parallel to the imaging plane;
[0096] S06: Based on the position of P” and the P”Q’ in the composite image x Length, and calculated O'Q' y / O'P" yThe constraint relationship is to search for the Q' point position in a specific area of the image; where P"Q' x represents the projection of P”Q' on the x-axis, O'Q' y represents the projection of O'Q' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction;
[0097] S08: When P' and Q' meet the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P and the projection of the reflection Q of the defect P in the image B, and the depth AP of the defect P is calculated.
[0098] Specifically, in this exemplary embodiment, in step S02, the image A of the first camera and the image B of the second camera must first be registered, and the two images must be adjusted to the same coordinate system. In a preferred exemplary embodiment, before the synthesis of step S02, the image A of the first camera and the image B of the second camera obtained are first preprocessed including distortion correction, i.e., step S01; after the preprocessing, the position of the defect in image A is obtained by means of an image algorithm, as shown by point P" in FIG3; finally, the two images are adjusted to the same coordinate system, and the schematic diagram of the superimposed synthetic image is as shown in FIG3. Figure 3 As shown, in the composite image, P" is the image of the defect P in image A, P' is the image of the defect P in image B, and Q' is the image of the reflection Q of the defect P in image B. In addition, Figure 3 As shown, there are four identification patterns 6, which are located at four corners respectively.
[0099] The principle of image processing can be Figure 4 and Figure 5 As shown, Figure 4 shows a schematic diagram of the measurement principle (xz section), and Figure 5 A schematic diagram of the measurement principle (y-z' section) is shown, in which O represents the focus, the distance from the focus O to the imaging plane 7 is f, the distance from the imaging plane 7 to the upper surface of the detection object 4 is d, the thickness of the detection object 4 is d1, the thickness of the transparent carrier platform 5 is d2, O' represents the center point of the imaging plane 7, P represents the defect inside the detection object 4, A and B represent the highest and lowest positions where the defect may appear (i.e., the upper surface 401 of the detection object and the upper surface 402 of the detection object), Q represents the image of the defect on the lower surface 501 of the transparent carrier platform, and A', B', and Q' are the projections of A, B, and Q on the imaging plane 7, respectively.
[0100] Afterwards, in step S04 and step S06, the point positions P' and Q' can be searched in specific areas of the image through corresponding limitations; and in step S08, when P' and Q' meet the parameter conditions, it can be determined that P' and Q' are the projection of the defect P in image B and the projection of the reflection Q of the defect P, respectively, and the depth AP of the defect P is calculated at this time.
[0101] Therefore, this method can distinguish the three-dimensional spatial position of defects, thereby realizing automatic detection of the screen, replacing manual detection, effectively reducing costs, and improving detection accuracy and efficiency.
[0102] More preferably, in an exemplary embodiment, the method further comprises the following steps:
[0103] S10: Determine whether the defect is on the screen surface or inside the screen based on the depth AP, and if the defect is inside the screen, determine between which two material layers the defect is.
[0104] Specifically, in this exemplary embodiment, when it is judged according to the depth AP that the defect is on the screen surface (upper surface or lower surface), if it is a defect such as particles or dirt, it can be removed by cleaning or other means; when it is judged according to the depth AP that the defect is inside the screen, the product can be judged as unqualified, and because the transparent display screen is usually composed of multiple layers of materials with different functions, it is possible to determine between which two layers of materials the defect occurs according to the depth, thereby inspecting the corresponding process equipment or corresponding steps to avoid the recurrence of defects in the future.
[0105] The following preferred exemplary embodiments will elaborate on steps S04, S06 and S08 in detail. Figure 7 is a three-dimensional schematic diagram of the shooting line of the second camera (the point O trajectory 8 in the figure is the trajectory of the focus), and the following projection planes can refer to Figure 7 To project:
[0106] More preferably, in an exemplary embodiment, in step S04, the position of P" and P"P' in the composite image are x Length, and calculated O'P' y / O'P" y The constraint relationship is obtained by searching for the position of point P' in a specific area of the image, including the following sub-steps:
[0107] like Figure 4 As shown, in the xz projection plane of the picture taken by the second camera, we have:
[0108] P″A′ x =d sinα
[0109] P″B′ x=(dcosα+d1)tanα
[0110] P″A′ x ≤P″P′ x ≤P″B′ x
[0111] Where d represents the distance from the imaging plane 7 to the upper surface of the detection object, d1 represents the thickness of the detection object, P”A’ x represents the projection of P”A’ on the x-axis, P”B’ x represents the projection of P"B' in the x-axis direction; the xz projection plane represents the plane composed of the x-axis and the z-axis and the projection of the points, lines and other objects photographed by the second camera along the y-axis on the xz plane, and the z-axis represents the direction parallel to the optical axis of the first camera, and the positive direction is vertically downward; (the imaging plane in this figure is the imaging plane of the second camera, so d = AA', which can be referred to Figure 7 )
[0112] like Figure 5 As shown, on the yz projection plane, according to the imaging principle, we have:
[0113]
[0114] The yz projection plane represents a plane formed by the y-axis and the z-axis and is a projection of an object such as a point or line captured by the second camera along the x-axis on the yz plane.
[0115] like Figure 6 As shown, on the y-z' projection plane, according to the imaging principle, we have:
[0116]
[0117]
[0118]
[0119] Among them, OP y =OQ y (b)
[0120] Where, f represents the distance from the focus O to the imaging plane 7, AP z represents the projection of AP in the z-axis direction, OP y represents the projection of OP in the y-axis direction, d2 represents the thickness of the transparent loading platform, OQ y Represents the projection of OQ in the y-axis direction; y-z' projection plane is a plane composed of the y-axis and the z'-axis and a schematic diagram of the projection of points, lines and other objects photographed by the second camera along the x-axis in the y-z' plane, where the z'-axis represents a direction parallel to the optical axis of the second camera, and the positive direction is obliquely downward;
[0121] Therefore, by matching the calculation of the second formula in formula (a) and formula (b), we have:
[0122]
[0123] In the composite graph, according to the position of P” and P”P’ in the composite graph x Length, and calculated O'P' y / O'P" y The constraint relationship is determined by searching the specific area of the image to obtain the position of point P' (obtained through image algorithm, which is the same as the method for obtaining the position of P").
[0124] More preferably, in an exemplary embodiment, in step S06, the position of P" and P"Q' in the composite image are x Length, and calculated O'Q' y / O'P" y The constraint relationship is obtained by searching the Q' point position in a specific area of the image, including the following sub-steps:
[0125] like Figure 4 As shown, on the xz projection plane, we have:
[0126] (dcosα+d1+2d2)tanα≤P″Q′ x ≤(dcosα+2d1+2d2)tanα
[0127] like Figure 6 As shown, on the y-z' projection plane, according to the imaging principle, the calculation matching of the second formula in formula (a) and formula (b) yields:
[0128]
[0129] Therefore, according to the position of P" and the composite diagram P"Q' x Length, and calculated O'Q' y / O'P" y The constraint relationship is determined by searching in a specific area of the image to obtain the Q' point position (obtained through image algorithm, which is the same as the method for obtaining the P" position).
[0130] More preferably, in an exemplary embodiment, in step S06, when P' and Q' satisfy the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the image B and the projection of the reflection Q of the defect P. At this time, the depth AP of the defect P is calculated, including the following sub-steps:
[0131] When P' and Q' meet the following parameter conditions:
[0132] P″P′ x +P″Q′x =(2dcosα+2d1+2d2)tanα
[0133] It can be determined that P' and Q' are the projection of defect P in image B and the projection of defect P's reflection Q respectively. At this time, the depth AP of defect P is calculated:
[0134]
[0135] With the same inventive concept as the above exemplary embodiment, another exemplary embodiment of the present invention provides a storage medium having computer instructions stored thereon, wherein when the computer instructions are executed, steps of a method for locating and detecting defects of a transparent screen are executed, and the method for locating and detecting defects of a transparent screen is performed, and the method for locating and detecting defects comprises the following steps:
[0136] The image A of the first camera and the image B of the second camera are registered, and the two images are adjusted to the same coordinate system; in the composite image, P" is the image of the defect P in image A, P' is the image of the defect P in image B, and Q' is the image of the reflection Q of the defect P in image B; wherein the position P" of the defect P in image A is known;
[0137] According to the position of P" and P"P' in the composite figure x Length, and calculated O'P' y / O'P" y The constraint relationship is to search for the position of point P' in a specific area of the image; where P"P' x represents the projection of P"P' on the x-axis, O'P' y represents the projection of O'P' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction, O represents the focus of the second camera, and O' represents the center point of the imaging plane 7; the positive direction of the x-axis is defined as a direction parallel to and opposite to the moving direction, and the positive direction of the y-axis is defined as a direction perpendicular to the x-axis and parallel to the imaging plane;
[0138] According to the position of P" and P"Q' in the composite diagram x Length, and calculated O'Q' y / O'P" y The constraint relationship is to search for the Q' point position in a specific area of the image; where P"Q' x represents the projection of P”Q' on the x-axis, O'Q' y represents the projection of O'Q' on the y-axis, O'P" y represents the projection of O'P" in the y-axis direction;
[0139] When P' and Q' meet the parameter conditions, it can be determined that P' and Q' are the projection of the defect P in the image B and the projection of the reflection Q of the defect P, respectively. At this time, the depth AP of the defect P is calculated;
[0140] Among them, the first optical axis of the first camera 1 is perpendicular to the plane where the detection object 4 is located; the second optical axis of the second camera 2 is at an angle α with the first optical axis, and α is less than 90°; the lower surface of the transparent loading platform 5 is provided with a mirror reflection layer, and the upper surface is used to place the detection object 4, and the detection object 4 is a transparent screen.
[0141] Correspondingly, in an exemplary embodiment, the method further comprises the following steps:
[0142] Depth AP determines whether the defect is on the screen surface or inside the screen. If the defect is inside the screen, determine between which two material layers the defect is.
[0143] Correspondingly, in an exemplary embodiment, the position of P" and P"P' in the composite image x Length, and calculated O'P' y / O'P" y The constraint relationship is obtained by searching the position of point P' in a specific area of the image, including the following sub-steps:
[0144] In the xz projection plane, we have:
[0145] P″A′ x =dsinα
[0146] P″B′ x =(dcosα+d1)tanα
[0147] P″A′ x ≤P″P′ x ≤P″B′ x
[0148] Where d represents the distance from the imaging plane to the upper surface of the detection object, d1 represents the thickness of the detection object, P”A' x represents the projection of P”A’ on the x-axis, P”B’ x represents the projection of P"B' in the x-axis direction; the xz projection plane represents the plane composed of the x-axis and the z-axis and the projection of the point and line objects photographed by the second camera along the y-axis on the xz plane, the z-axis represents the direction parallel to the optical axis of the first camera, and the positive direction is vertically downward; A', B', Q' are the projections of A, B, Q on the imaging plane respectively;
[0149] On the yz projection plane, according to the imaging principle, we have:
[0150]
[0151] The yz projection plane represents a plane formed by the y axis and the z axis and is a projection of the point or line object captured by the second camera along the x axis on the yz plane;
[0152] On the y-z' projection plane, according to the imaging principle, we have:
[0153]
[0154]
[0155]
[0156] Among them, OP y =OQ y (b)
[0157] Where f represents the distance from the focus O to the imaging plane, AP z represents the projection of AP in the z-axis direction, OP y represents the projection of OP on the y-axis, d2 represents the thickness of the transparent loading platform, OQ y Represents the projection of OQ in the y-axis direction; y-z' projection plane is the plane formed by the y-axis and the z'-axis and the projection of the point and line objects captured by the second camera along the x-axis on the y-z' plane;
[0158] Therefore, by matching the calculation of the second formula in formula (a) and formula (b), we have:
[0159]
[0160] In the composite graph, according to the position of P” and P”P’ in the composite graph x Length, and calculated O'P' y / O'P" y The constraint relationship is determined by searching for the position of point P' in a specific area of the image.
[0161] Correspondingly, in an exemplary embodiment, the position of P" and P"Q' in the composite graph x Length, and calculated O'Q' y / O'P" y The constraint relationship is obtained by searching the Q' point position in a specific area of the image, including the following sub-steps:
[0162] On the xz projection plane, we have:
[0163] (dcosα+d1+2d2)tanα≤P″Q′ x ≤(dcosα+2d1+2d2)tanα
[0164] On the y-z' projection plane, according to the imaging principle, the calculation matching of the second formula in equation (a) and equation (b) yields:
[0165]
[0166] Therefore, according to the position of P" and the composite diagram P"Q' x Length, and calculated O'Q' y / O'P" y The constraint relationship is determined by searching the specific area of the image to obtain the position of point Q'.
[0167] Correspondingly, in an exemplary embodiment, when P' and Q' satisfy the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the image B and the projection of the reflection Q of the defect P. At this time, the depth AP of the defect P is calculated, including the following sub-steps:
[0168] When P' and Q' meet the following parameter conditions:
[0169] P″P′ x +P″Q′ x =(2dcosα+2d1+2d2)tanα
[0170] It can be determined that P' and Q' are the projection of defect P in image B and the projection of defect P's reflection Q respectively. At this time, the depth AP of defect P is calculated:
[0171]
[0172] Correspondingly, in an exemplary embodiment, the method further comprises the following steps:
[0173] The acquired image A from the first camera and the acquired image B from the second camera are preprocessed including distortion correction.
[0174] Among them, the specific content in this exemplary embodiment, the parts that are the same as the positioning detection system and the positioning detection system are not repeated here.
[0175] Having the same inventive concept as the above exemplary embodiment, another exemplary embodiment of the present invention provides a device, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the steps of a method for locating and detecting defects of a transparent screen when running the computer instructions.
[0176] Based on this understanding, the technical solution of this embodiment can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0177] The memory is used to store various types of data to support the operation of the device, which data may include, for example, instructions for any application or method operating on the device, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0178] Obviously, the above embodiments are merely examples for clear explanation, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.
Claims
1. A method for positioning and detecting defects of a transparent screen, based on a positioning and detecting system, characterized in that: The system comprises: Linear slides; A transparent loading platform is arranged on a linear slide rail and is driven to move by a motor; a mirror reflection layer is arranged on the lower surface, and an upper surface is used to place a detection object, and the detection object is a transparent screen; A first camera, wherein a first optical axis of the first camera is perpendicular to a plane where the detection object is located; a second camera, wherein a second optical axis of the second camera forms an angle α with the first optical axis, and α is less than 90°; A light source is arranged above the transparent loading platform; A data processing unit, used to process the first image acquired by the first camera and the second image acquired by the second camera to realize three-dimensional positioning detection; The method comprises the following steps: The first image of the first camera and the second image of the second camera are registered, and the two images are adjusted to the same coordinate system; in the composite image, P" is the image of the defect P in the first image, P' is the image of the defect P in the second image, and Q' is the image of the reflection Q of the defect P in the second image; wherein the position P" of the defect P in the first image is known; According to the position of P” and the length of P”P'x in the composite image, and the constraint relationship of O'P'y / O'P”y obtained by calculation, the position of point P' is searched in the specific area of the image; wherein, P”P'x represents the projection of P”P' in the x-axis direction, O'P'y represents the projection of O'P' in the y-axis direction, O'P”y represents the projection of O'P” in the y-axis direction, O represents the focus of the second camera, and O' represents the center point of the imaging plane; the positive direction of the x-axis is defined as the direction parallel to and opposite to the direction of movement, and the positive direction of the y-axis is defined as the direction perpendicular to the x-axis and parallel to the imaging plane; According to the position of P”, the length of P”Q'x in the synthetic image, and the constraint relationship of O'Q'y / O'P”y calculated, the position of point Q' is searched in a specific area of the image; where P”Q'x represents the projection of P”Q' in the x-axis direction, O'Q'y represents the projection of O'Q' in the y-axis direction, and O'P”y represents the projection of O'P” in the y-axis direction; When P' and Q' meet the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the second image and the projection of the reflection Q of the defect P, and the depth AP of the defect P is calculated at this time; The method of searching for the position of point P' in a specific area of the image according to the position of P', the length of P'P'x in the composite image, and the constraint relationship of O'P'y / O'P"y obtained by calculation, comprises the following sub-steps: In the xZ projection plane, we have: P″A′x=dsinα P″B′ x =(dcosα+d1)tanα P″A′ x ≤P″P′ x ≤P"B′ x Where d represents the distance from the imaging plane to the upper surface of the detection object, d1 represents the thickness of the detection object, P”A’ x represents the projection of P”A’ in the x-axis direction, and P”B’x represents the projection of P”B’ in the x-axis direction; the xZ projection plane represents the plane composed of the x-axis and the z-axis and the projection of the point and line objects photographed by the second camera along the y-axis on the xZ plane, and the z-axis represents the direction parallel to the optical axis of the first camera, and the positive direction is vertically downward; A’, B’, Q’ are the projections of A, B, and Q on the imaging plane, respectively, and A and B represent the highest and lowest positions where defects may appear, respectively; On the yz projection plane, according to the imaging principle, we have: The yz projection plane represents a plane formed by the y axis and the z axis and is a projection of the point or line object captured by the second camera along the x axis on the yz plane; On the y-z' projection plane, according to the imaging principle, we have: Among them, OP y =OQ y (b) Where f represents the distance from the focus O to the imaging plane, AP z represents the projection of AP in the z-axis direction, OP y represents the projection of OP on the y-axis, d2 represents the thickness of the transparent loading platform, OQ y represents the projection of OQ in the y-axis direction; the y-z' projection plane represents the plane formed by the y-axis and the z'-axis and the projection of the point and line objects photographed by the second camera along the x-axis on the y-z' plane, and the z'-axis represents the direction parallel to the optical axis of the second camera, and the positive direction is obliquely downward; Therefore, by matching the calculation of the second formula in formula (a) and formula (b), we have: In the composite image, according to the position of P” and the length of P”P’x in the composite image and the constraint relationship of O’P’y / O’P”y obtained by calculation, the position of point P’ is searched in a specific area of the image; The method of searching for the position of point Q' in a specific area of the image according to the position of P'' and the length of P'Q'x in the composite image and the constraint relationship of O'Q'y / O'P'y obtained by calculation includes the following sub-steps: On the xZ projection plane, we have: (dkosα+d1+2d2)tanα≤P″Q′ x ≤(dcosα+2d1+2d2)tanα On the y-z' projection plane, according to the imaging principle, the calculation matching of the second formula in equation (a) and equation (b) yields: Therefore, according to the position of P” and the length of P”Q'x in the composite image, and the constraint relationship of O'Q'y / O'P”y obtained by calculation, the position of point Q' is searched in the specific area of the image; When P' and Q' meet the parameter conditions, it can be determined that P' and Q' are respectively the projection of the defect P in the second image and the projection of the reflection Q of the defect P. At this time, the depth AP of the defect P is calculated, including the following sub-steps: When P' and Q' meet the following parameter conditions: P″P′ x +P″Q′ x =(2dcosα+2d1+2d2)tanα It can be determined that P' and Q' are the projection of the defect P in the second image and the projection of the reflection Q of the defect P, respectively. At this time, the depth AP of the defect P is calculated:
2. The method for locating and detecting transparent screen defects according to claim 1, characterized in that: Also includes: The encoder receives the acquisition line frequency set by the camera and outputs a speed signal to the motor; the motor controls the transparent loading platform to move at a uniform speed on the linear slide rail according to the output speed signal.
3. A method for detecting transparent screen defects according to claim 2, characterized in that: The first image of the first camera and the second image of the second camera are complete images containing the detection object that can be captured by the camera after the detection object completely passes through the camera shooting area.
4. The method for locating and detecting transparent screen defects according to claim 1, characterized in that: The upper surface of the transparent loading platform is provided with a non-transparent diffuse reflection layer for rapid positioning.
5. The method for locating and detecting transparent screen defects according to claim 1, characterized in that: The method further comprises the following steps: Depth AP determines whether the defect is on the screen surface or inside the screen. If the defect is inside the screen, determine between which two material layers the defect is.
6. The method for locating and detecting transparent screen defects according to claim 1, characterized in that: The method further comprises the following steps: The acquired first image of the first camera and the acquired second image of the second camera are preprocessed including distortion correction.
