Method, device, equipment and medium for detecting adhesive path on screen of electronic equipment

Abstract

The present invention discloses a method, device, equipment and medium for detecting glue paths on electronic device screens, and relates to the technical field of glue coating detection. The key points of the technical solution are: determining the reference plane of the frame from a depth image; segmenting the first point cloud containing the glue path and the frame in the depth image according to the reference plane; performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the frame and a third point cloud representing the glue path; fitting the frame line of the frame according to the second point cloud; calculating multiple parameter equations representing the glue path slices according to two adjacent frame points of the frame line, dividing the points in the third point cloud into corresponding glue path slices according to the multiple parameter equations, projecting the divided points onto the corresponding glue path slices, and obtaining a detection surface; removing non-glue path points in the detection surface to obtain a glue path detection surface, and detecting the screen glue path based on the glue path detection surface. The present invention solves the problem of being unable to evenly slice and accurately segment at the arched glue path, thereby improving the accuracy of glue path defect detection.

Description

Technical Field

[0001] The present invention relates to the field of glue coating detection technology, and more specifically, to a method, device, equipment and medium for detecting glue paths on screens of electronic devices. Background Art

[0002] During the assembly process of the screen of electronic devices (such as mobile phones, tablets, etc.), glue needs to be applied along the frame of the electronic device to make the screen bonded to the electronic device. If the glue is applied unevenly or the amount is inappropriate, the screen and the middle frame will not fit well, which will affect the user experience of the electronic device. In response to this problem, there is a need for testing whether the glue path meets the standards. At present, the main reasons for the failure of the glue path to meet the standards are: glue collapse, glue creeping, and glue shortage. Among them, glue shortage has the greatest impact and will cause serious quality problems in electronic equipment.

[0003] Nowadays, the glue path detection of electronic devices mainly involves slicing the glue path and then analyzing each slice. This type of technology mainly divides the glue path into many segments and detects whether each segment meets the set requirements in turn. The implementation method is to take a photo of the frame of the electronic device coated with glue to generate a depth image, then generate point cloud data from the depth image, and finally slice the point cloud data to obtain the distribution map of the glue path on each slice. However, the glue path of the electronic device is not a straight line. For example, the arched glue path at the chamfer, how to accurately divide the arched glue path and slice it evenly is the difficulty of this technology. Summary of the invention

[0004] The present invention provides a method, device, equipment and medium for detecting glue paths on electronic device screens, so as to solve the problem in the prior art that arched glue paths cannot be sliced ​​evenly and segmented accurately, resulting in an inability to accurately detect whether the glue paths have defects.

[0005] A first aspect of the present invention provides a method for detecting adhesive paths on a screen of an electronic device, the method comprising:

[0006] Determining a reference plane of the border from a depth image of the border of the electronic device;

[0007] Segment the first point cloud containing the glue path and the border in the depth image according to the reference plane;

[0008] Performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path;

[0009] Fitting a border line of the border according to the second point cloud;

[0010] Calculate multiple parameter equations representing the glue path slices according to two adjacent border points of the border line, divide the points in the third point cloud into corresponding glue path slices according to the multiple parameter equations, and project the divided points to the corresponding glue path slices to obtain the detection surface; wherein one parameter equation corresponds to one glue path slice, and one glue path slice passes through one border point of the border line;

[0011] The non-glue road points in the detection surface are removed to obtain the glue road detection surface, and the glue road of the electronic device screen is detected based on the glue road detection surface.

[0012] In one implementation, determining a reference plane of a border from a depth image includes: extracting three reference points of the border from the image, and determining a reference plane where the border is located according to the three reference points.

[0013] In one implementation scheme, a first point cloud including a glue path and a border is segmented out from a depth image according to a reference plane, specifically, a first distance from each point in the depth image to the reference plane is calculated, when the first distance is less than or equal to a preset threshold value, the point is a point representing the glue path or a point representing the border, the points representing the glue path or the points representing the border are aggregated to obtain a first point cloud representing the glue path and representing the border.

[0014] In one implementation, fitting a border line of the border according to the second point cloud includes:

[0015] Substitute the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the bounding box;

[0016] Solve the parametric equation of the fitted circle to obtain the coordinates of the center of the fitted circle;

[0017] According to the coordinates of the center of the fitting circle, the coordinates of each point in the second point cloud are converted into polar coordinates relative to the center of the fitting circle;

[0018] The polar coordinates are divided into n parts according to the polar angle, and the point with the smallest polar diameter in each part is taken as the border point on the border line, so as to fit the border line of the border; where n is a positive integer.

[0019] In one implementation, points in the third point cloud are divided into corresponding glue path slices according to a plurality of parameter equations, and the divided points are projected onto the corresponding glue path slices to obtain the detection surface, specifically:

[0020] Calculate the second distance from the point in the third point cloud to each glue path slice. If the second distance from the i-th point to the k-th glue path slice is less than the set threshold, assign the i-th point to the k-th glue path slice. After all points in the third point cloud are assigned to the corresponding glue path slices, calculate the projection coordinates of each point in the third point cloud on the corresponding glue path slice; wherein i and k are both positive integers;

[0021] Each point in the third point cloud is projected onto the corresponding glue path slice according to the projection coordinates to obtain the detection surface.

[0022] In one implementation, removing non-glue path points from a plurality of detection surfaces having glue path points to obtain a contour surface of the glue path includes:

[0023] The midpoint of the electronic device is selected; wherein the midpoint is the coordinate of the center of the fitting circle, and the steps of determining the coordinate of the center of the fitting circle are as follows: substituting the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the frame, solving the parametric equation of the fitting circle to obtain the coordinate of the center of the fitting circle;

[0024] Calculate the third distance between each point in each detection surface and the midpoint, and calculate the fourth distance between the midpoint and a border point of each detection surface passing through the border line;

[0025] Determine whether the third distance and the fourth distance meet a preset removal condition, and whether the difference between each detection surface point and the frame point corresponding to the detection surface in the vertical direction is less than a preset difference threshold; wherein the removal condition includes that the third distance is less than the fourth distance, and the fourth distance is greater than the preset threshold;

[0026] If there are points in each detection surface that meet both the removal condition and the difference threshold, they will be retained, otherwise they will be removed.

[0027] In one implementation scheme, after calculating multiple parametric equations characterizing the glue path slice based on two adjacent border points of the border line, dividing the points in the third point cloud into corresponding glue path slices according to the multiple parametric equations, and projecting the divided points to the corresponding glue path slices, before obtaining the detection surface, the method also includes: performing mean smoothing on the border line to obtain the smoothed border line.

[0028] A second aspect of the present invention provides a device for detecting adhesive paths on screens of electronic devices, the device comprising:

[0029] A reference plane determination module, used to determine a reference plane of a frame from a depth image of the frame of the electronic device;

[0030] A segmentation module, used for segmenting a first point cloud including a glue path and a border in the depth image according to the reference plane;

[0031] A clustering module, used for performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path;

[0032] A border line determination module, used for fitting a border line of the border according to the second point cloud;

[0033] A glue path slicing module is used to calculate multiple parameter equations characterizing the glue path slices according to two adjacent border points of the border line, divide the points in the third point cloud into corresponding glue path slices according to the multiple parameter equations, and project the divided points to the corresponding glue path slices to obtain a detection surface; wherein one parameter equation corresponds to one glue path slice, and one glue path slice passes through one border point of the border line;

[0034] The glue path detection module is used to remove non-glue path points in the detection surface to obtain the glue path detection surface, and detect the glue path of the electronic device screen based on the glue path detection surface.

[0035] According to a third aspect of the present invention, an electronic device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of a method for detecting glue paths on an electronic device screen according to the first aspect of the present invention are implemented.

[0036] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a computer program, and when the computer program is executed by one or more processors, the computer program implements a method for detecting glue paths on an electronic device screen according to the first aspect of the present invention.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In a method for detecting glue paths on the screen of an electronic device provided in an embodiment of the present invention, first, a reference plane of the frame is determined from a depth image of the frame of the electronic device, and then a first point cloud containing the glue path and the frame in the depth image is segmented from the reference plane; Euclidean clustering is performed on the first point cloud to remove noise in the point cloud to ensure accurate classification of the glue path and the frame, thereby obtaining a second point cloud representing the frame and a third point cloud representing the glue path, so that the second point cloud after clustering processing fits the frame line of the frame, and then, multiple parameter equations representing the glue path slices are calculated based on two adjacent frame points of the frame line, and the points in the third point cloud are divided into corresponding glue paths according to the multiple parameter equations. The glue road is sliced ​​and the divided points are projected to the corresponding glue road slices to obtain the detection surface; based on this process of using adjacent border points on the border line to slice the glue road, the distance of each glue road slice is uniform and the direction is smooth, thereby solving the problem that the prior art cannot evenly slice and accurately divide the arched glue road, resulting in an inability to detect whether the glue road has defects; finally, the non-glue road points in the detection surface are further removed to obtain the glue road detection surface. Based on the glue road detection surface, the width and height of the glue road can be accurately measured, and the detection surface can intuitively show whether there are defects in the glue road (glue collapse, glue creeping, glue deficiency), thereby completing the detection of the glue road of the electronic device screen.

[0039] In addition, an embodiment of the present invention also provides an electronic device screen glue line detection device, equipment and medium, which have the same beneficial effects as the above-mentioned electronic device screen glue line detection method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the present invention, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0041] Figure 1 A schematic diagram of a flow chart of a method for detecting adhesive paths on an electronic device screen provided by an embodiment of the present invention;

[0042] Figure 2 A principle block diagram of a device for detecting adhesive path on an electronic device screen provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0044] Reference numerals and description of the drawings:

[0045] 210, reference plane determination module; 220, segmentation module; 230, clustering module; 240, border line determination module; 250, glue path slicing module; 260, glue path detection module; 300, electronic device; 310, processor; 320, memory; 321, one or more programs; 330, communication interface. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0047] It should be noted that the terms "include" or "may include" used in various embodiments of the present invention indicate the presence of the function, operation or element applied for, and do not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present invention, the terms "include", "have" and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0048] In various embodiments of the present invention, the expression "or" or "at least one of B or / and C" includes any combination or all combinations of the words listed at the same time. For example, the expression "B or C" or "at least one of B or / and C" may include B, may include C, or may include both B and C.

[0049] It should be understood that terms such as "first", "second", "third", and "fourth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0050] As described in the background, the glue path detection of electronic devices nowadays mainly involves slicing the glue path and then analyzing each slice. This type of technology mainly divides the glue path into many segments and detects whether each segment meets the set requirements in turn. The implementation method is to take a photo of the frame of the electronic device coated with glue to generate a depth image, then generate point cloud data from the depth image, and finally slice the point cloud data to obtain the distribution map of the glue path on each slice. However, the glue path of the electronic device is not a straight line. For example, the arched glue path is formed during the gluing process at the chamfer of the electronic device. How to accurately divide the arched glue path and slice it evenly is the difficulty of this technology.

[0051] Therefore, the embodiments of the present invention provide a method, device, equipment and medium for detecting the glue path on the screen of an electronic device, so as to solve the problem that the prior art cannot evenly slice and accurately divide the arched glue path, resulting in the inability to accurately detect whether the glue path has defects.

[0052] The following will describe a method for detecting adhesive path on an electronic device screen provided by an embodiment of the present invention in combination with a specific implementation method. Figure 1 , Figure 1 A schematic diagram of a flow chart of a method for detecting adhesive paths on an electronic device screen provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0053] S101, determining a reference plane of the frame from a depth image of the frame of the electronic device.

[0054] In this embodiment, as is common knowledge among those skilled in the art, the depth image of the frame of the electronic device can be obtained by photographing the surface bonded to the electronic screen by an image collector such as a laser radar or a depth camera, wherein the depth image reflects the geometric shape of the surface of the frame of the electronic device.

[0055] Secondly, the reference plane of the border is determined from the depth image, specifically: three reference points of the border are extracted from the image, and the reference plane where the border is located is determined according to the three reference points.

[0056] Specifically, in three-dimensional space, determining a reference surface (also called a plane) based on three reference points is a basic geometric problem. Usually, the coordinates of these three reference points can be used to solve the parametric equation of this reference surface. The general form of the plane equation in three-dimensional space is: Ax+By+Cz+D=0 ,in A , B , C are the components of the normal vector, D is a constant.

[0057] It should be noted that when the image collector is used to take a photo of an electronic device, the electronic device may be tilted to a certain extent, so the three reference points of the border extracted in this embodiment should be as far away as possible. For example, for a general electronic device, such as a smart phone, the three selected reference points can be the midpoints of two short borders and the midpoint of a long border.

[0058] Step 1: Find the normal vector of the reference plane

[0059] Assume that the three given points are P 1( x 1, y 1, z 1), P 2( x 2, y 2, z 2) and P 3( x 3, y 3, z 3); among which, P 1. P 2. P 3 are three reference points, ( x 1, y 1, z 1) ( x 2, y 2, z 2) and ( x 3, y 3, z 3) are the coordinate values ​​of the three reference points in the x, y, and z directions respectively.

[0060] Calculate two vectors: First, calculate two vectors, such as from P 1 to P Vector of 2 vecv 1 and from P 1 toP Vector of 3 vecv 2. Among them, the vector vecv 1 and vecv 2 is expressed as follows:

[0061] vecv 1=( x 2− x 1, y 2− y 1, z 2− z 1);

[0062] vecv 2=( x 3− x 1, y 3− y 1, z 3− z 1);

[0063] Calculating vectors vecv 1 and vecv 2: The normal vector of the plane is vecv 1 and vecv The cross product of 2 is denoted by n =( A , B , C ).

[0064] The formula for calculating the cross product is vecn =( y 2− y 1)( z 3− z 1)−( z 2− z 1)( y 3− y 1),( z 2− z 1)( x 3− x 1)−( x 2− x 1)( z 3− z 1),( x 2− x 1)( y 3− y 1)−( y 2− y 1)( x 3− x 1);

[0065] Step 2: Determine the constant D

[0066] With the normal vector n =( A , B , C ), you can select any point on the plane (for example P 1) To find out D .Will P Substitute the coordinates of 1 into the plane equation Ax + By + C + D =0, we can solve D .

[0067] In summary, the parametric equation of the reference surface is obtained Ax + By + C + D =0, D is the constant obtained by the above method. This equation uniquely determines the P 1, P 2, P 3 Defined reference plane that describes the plane on which the electronic device is located.

[0068] S102, segmenting a first point cloud including a glue path and a border in the depth image according to the reference plane.

[0069] In this embodiment, the first distance from each point in the depth image to the reference plane is calculated by the distance formula from the point to the reference plane. When the first distance is less than or equal to a preset threshold, the point is a point representing the glue path or a point representing the border. The points that are points representing the glue path or points representing the border are summarized to obtain the first point cloud representing the glue path and representing the border.

[0070] It should be noted that the distance formula from a point to a reference plane is common knowledge and is not described in detail in this embodiment.

[0071] S103, performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path.

[0072] In this embodiment, the core idea of ​​Euclidean clustering is to first regard each point as a separate class, and then merge the points whose distance is less than a threshold into a new class. The distance between the merged classes is defined as the distance between the two nearest points in the two classes.

[0073] Euclidean clustering of the first point cloud can obtain the second point cloud representing the border and the third point cloud representing the glue path. The number of the second point cloud is greater than that of the third point cloud. The reason is that the border is an existing entity, and the number of its point clouds must be the number of point clouds of the redundant glue path. At the same time, Euclidean clustering can also remove noise points (set the minimum number of point clouds of the class, and remove all classes less than the threshold to achieve the purpose of denoising. Because there are fewer points near the noise points, the number of class point clouds formed is small).

[0074] S104: Fitting a border line of the border according to the second point cloud.

[0075] In this embodiment, the second point cloud obtained after Euclidean clustering represents the overall border of the electronic device, which is not convenient to use in the subsequent slicing process. Therefore, it is necessary to transform the overall border into a border line. Therefore, the coordinates of each point in the fourth point cloud are substituted into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the border; the parametric equation of the fitting circle is solved to obtain the coordinates of the center of the fitting circle; according to the coordinates of the center of the fitting circle, the coordinates of each point in the fourth point cloud are converted into polar coordinates relative to the center of the fitting circle; the polar coordinates are divided into n parts according to the polar angle, and the point with the smallest polar diameter in each part is taken as the border point on the border line, so as to fit the border line of the border; wherein n is a positive integer.

[0076] It should be noted that the parametric equation of a circle is common knowledge, and this embodiment does not describe its equation.

[0077] The expression of the parametric equation of the fitted circle is:

[0078] . x0, x1, …, x n Respectively represent the horizontal coordinates of each point in the second point cloud, y0, y1, ..., y n They respectively represent the ordinate of each point in the second point cloud, n represents the total number of points in the second point cloud, T represents the intermediate quantity, and x′ and y′ respectively represent the abscissa and ordinate of the fitted circle.

[0079] Secondly, polar coordinates are composed of polar diameter and polar angle, which is common knowledge and will not be described in detail in this embodiment.

[0080] S105, calculate multiple parametric equations characterizing the glue path slice according to two adjacent border points of the border line, divide the points in the third point cloud into corresponding glue path slices according to the multiple parametric equations, project the divided points to the corresponding glue path slices, and obtain the detection surface; wherein, one parametric equation corresponds to one glue path slice, and one glue path slice passes through one border point of the border line.

[0081] In this embodiment, since the slicing method provided by the current prior art cannot slice the arched glue path of the electronic device more evenly, this embodiment uses the border points of the border line to determine multiple parameter equations of the glue path slicing. The reason is that the border is a fixed shape, and the border points on the border line are used to establish slices of the glue path points. Then, based on these slices as the projection coordinates of the glue path points within the slicing range, at least one point in the third point cloud is projected onto the corresponding slice based on the projection coordinates, and multiple detection surfaces with glue path points are obtained, so that the glue path is sliced ​​evenly.

[0082] Among them, the parametric equation of the slice is the same as the parametric equation of the reference plane described in the above embodiment. The normal vector of the parametric equation of the glue path slice is a vector composed of two adjacent border points on the border line, and it is stipulated that each glue path slice passes through a point on the border line. For example, the number of border points is 350, and the combination of border point 1 and border point 2, border point 2 and border point 3, and so on is followed until the parametric equation of the last glue path slice of the glue path is calculated from border point 349 and border point 350, that is, the first glue path slice passes through border point 1, and the 349th glue path slice passes through border point 349.

[0083] After obtaining the parameter equations of each glue road slice, since there are many glue road slices, it is obvious that only the points located on the glue road slice (the points refer to the points in the third point cloud) can be projected onto the glue road slice. The process of determining the detection surface is as follows: calculate the second distance from the points in the third point cloud to each glue road slice. If the second distance from the i-th point to the k-th glue road slice is less than the set threshold, the i-th point is assigned to the k-th glue road slice. After all points in the third point cloud are assigned to the corresponding glue road slices, calculate the projection coordinates of each point in the third point cloud on the corresponding glue road slice; wherein i and k are both positive integers; project each point in the third point cloud onto the corresponding glue road slice according to the projection coordinates to obtain the detection surface.

[0084] In this embodiment, if the second distance is less than a preset threshold, it means that the point conforms to the slice, so the projection coordinates of the point in the third point cloud on the corresponding glue path slice are calculated according to the parameter equation of the corresponding glue path slice.

[0085] The calculation formula of the projection coordinates is:

[0086] , Among them, x, y, z represent the coordinates of the point to be projected, and x′, y′, z′ represent the coordinates after projection.

[0087] Exemplarily, the corrected expression is: x' = x + or - A* (A*x + B*y + C*z) / A*A + B*B +C*C, where + or - depends on where the point is on the plane. This formula can be understood as: the point moves in the same or opposite direction as the plane normal vector, depending on where the point is on the plane. The distance the point moves is exactly the distance from the point to the plane. For example, the parametric equation of the xoy plane is 0x + 0y + 1z + 0 = 0, obviously the normal vector is (0, 0, 1), and the plane passes through the point (0, 0, 0). Point (1, 1, 5) is above the plane (based on the direction of the normal vector, if the normal vector of the plane is (0, 0, -1) then point (1, 1, 5) is below the plane). To determine whether a point is above or below the plane, just substitute the coordinates of the point into the plane parametric equation. If the result is greater than 0, the point is above the plane (0*1 + 0*1 + 1*5 = 5>0), and if it is less than 0, the point is below the plane. Points above move in the opposite direction of the normal vector, and points below move in the direction of the normal vector. So the projection of (1, 1, 5) on the xoy plane is (1-0, 1-0, 5-5) = (1, 1, 0).

[0088] Then, based on the projection coordinates, all points in the third point cloud can be projected to the corresponding glue path slices to obtain the detection surface.

[0089] S106, removing non-glue points on the detection surface to obtain a glue detection surface, and detecting the glue path of the electronic device screen based on the glue detection surface.

[0090] In this embodiment, although most of the non-glue road points have been removed through the implementation method described above, there are still some non-glue road points on the projected detection surface. Therefore, this embodiment further removes the non-glue road points in the detection surface to obtain a glue road detection surface. The removal process specifically includes: first, selecting the midpoint of the electronic device; wherein the midpoint is the coordinate of the center of the fitting circle, and the steps for determining the coordinate of the center of the fitting circle are as follows: substitute the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the border, and solve the parametric equation of the fitting circle to obtain the fitting The coordinates of the center of the circle; secondly, calculate the third distance between each point in each detection surface and the midpoint, and calculate the fourth distance between the midpoint and the border point of each detection surface passing through the border line; then, determine whether the third distance and the fourth distance meet the preset removal conditions, and whether the difference in the vertical direction between the point of each detection surface and the border point corresponding to the detection surface is less than the preset difference threshold; wherein the removal conditions include that the third distance is less than the fourth distance, and that the fourth distance is greater than the preset threshold; finally, if there are points in each detection surface that meet both the removal conditions and the difference threshold, then retain them, otherwise remove them.

[0091] In summary, in a method for detecting glue paths on the screen of an electronic device provided in an embodiment of the present invention, first, a reference plane of the frame is determined from a depth image of the frame of the electronic device, and then a first point cloud containing the glue path and the frame in the depth image is segmented from the reference plane; Euclidean clustering is performed on the first point cloud to remove noise in the point cloud to ensure accurate classification of the glue path and the frame, thereby obtaining a second point cloud representing the frame and a third point cloud representing the glue path; therefore, the second point cloud after clustering is fitted with a frame line of the frame, and then, multiple parameter equations representing the glue path slices are calculated based on two adjacent frame points of the frame line, and the points in the third point cloud are divided into corresponding points according to the multiple parameter equations. The glue path is sliced ​​and the divided points are projected to the corresponding glue path slices to obtain the detection surface; based on this process of using adjacent border points on the border line to slice the glue path, the distance of each glue path slice is uniform and the direction is smooth, thereby solving the problem that the prior art cannot evenly slice and accurately divide the arched glue path, resulting in an inability to well detect whether the glue path has defects; finally, the non-glue path points in the detection surface are further removed to obtain the glue path detection surface, based on the glue path detection surface, the width and height of the glue path can be accurately measured, and the detection surface can intuitively show whether the glue path has defects (glue collapse, glue creeping, glue lack), thereby completing the detection of the glue path of the electronic device screen.

[0092] In some embodiments, before step S105, the method further includes: performing mean smoothing processing on the border line to obtain a smoothed border line.

[0093] Specifically, since the frame line obtained by fitting in step S104 is not smooth, it needs to be smoothed. The smoothing method is to replace the current point with the average of the m points before and after it. If there are less than m points before, after and after the current point, the current point is used to supplement it. The frame line after smoothing is obviously smoother, and the distribution of the frame points is more uniform, so that the calculated glue path slice is uniform and oversmooth.

[0094] For example, when m=2, it means that the calculation is done with the two points before and after. For example, when calculating the smooth coordinates of p5, the coordinate mean of p3, p4, p5, p6, and p7 must be calculated first, and then the smooth coordinates of p5 are replaced with their coordinate mean. When there are less than two points before and after, such as calculating the smooth coordinates of p1, the coordinate mean of p1, p1, (the first two p1s are because there are no points before p1, so p1 is directly used to replace them) p1, p2, and p3 are calculated, and then p1 is replaced with the coordinate mean; the same is true for the last point, and the point that is not behind it is replaced by itself.

[0095] The embodiment of the present invention also provides an electronic device screen glue path detection device, which can be used to perform each step of an electronic device screen glue path detection method described in the above embodiment of the present invention. Figure 2 , Figure 2 A schematic diagram of a device for detecting adhesive path on an electronic device screen provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the device comprises:

[0096] A reference plane determination module 210, configured to determine a reference plane of a frame from a depth image of the frame of the electronic device;

[0097] A segmentation module 220, configured to segment a first point cloud including a glue path and a border in the depth image according to the reference plane;

[0098] A clustering module 230 is used to perform Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path;

[0099] A frame line determination module 240, configured to fit a frame line of a frame according to the second point cloud;

[0100] The glue path slicing module 250 is used to calculate a plurality of parameter equations representing the glue path slices according to two adjacent border points of the border line, divide the points in the third point cloud into corresponding glue path slices according to the plurality of parameter equations, and project the divided points onto the corresponding glue path slices to obtain the detection surface; wherein one parameter equation corresponds to one glue path slice, and one glue path slice passes through one border point of the border line;

[0101] The glue path detection module 260 is used to remove non-glue path points in the detection surface to obtain a glue path detection surface, and detect the glue path of the electronic device screen based on the glue path detection surface.

[0102] An electronic device screen glue path detection device in an embodiment of the present invention is Figure 1 The electronic device screen glue path detection method shown is a technical solution based on the same inventive concept. Through the detailed description of the electronic device screen glue path detection method provided in the above embodiment, technical personnel in this field can clearly understand the implementation process of the electronic device screen glue path detection device in this embodiment, so for the sake of conciseness of the specification, it will not be repeated here.

[0103] Correspondingly, the electronic device screen glue path detection device provided by the present embodiment first determines the reference plane of the frame from the depth image of the electronic device frame, and then segments the first point cloud containing the glue path and the frame in the depth image from the reference plane; Euclidean clustering is performed on the first point cloud to remove noise in the point cloud to ensure accurate classification of the glue path and the frame, thereby obtaining a second point cloud representing the frame and a third point cloud representing the glue path, so that the second point cloud after clustering processing fits the frame line of the frame, and then, multiple parameter equations representing the glue path slices are calculated based on two adjacent frame points of the frame line, and the points in the third point cloud are divided into corresponding glue paths according to the multiple parameter equations. Slicing, projecting the divided points to the corresponding glue path slices to obtain the detection surface; based on this process of using adjacent border points on the border line to slice the glue path, the distance of each glue path slice is uniform and the direction is smooth, thereby solving the problem that the prior art cannot evenly slice and accurately divide the arched glue path, resulting in an inability to detect whether the glue path has defects; finally, further remove the non-glue path points in the detection surface to obtain the glue path detection surface, based on the glue path detection surface, the width and height of the glue path can be accurately measured, and the detection surface can intuitively show whether the glue path has defects (glue collapse, glue creeping, glue lack), thereby completing the detection of the glue path of the electronic device screen.

[0104] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 300 includes a processor 310, a memory 320, a communication interface 330, and at least one communication bus for connecting the processor 310, the memory 320, and the communication interface 330. The memory 320 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (PROM), or a portable CD-ROM, and the memory 320 is used for related instructions and data.

[0105] The communication interface 330 is used to receive and send data. The processor 310 can be one or more CPUs. When the processor 310 is a CPU, the CPU can be a single-core CPU or a multi-core CPU. The processor 310 in the electronic device 300 is used to read one or more programs 321 stored in the memory 320, and perform the following operations: determine the reference plane of the frame from the depth image of the frame of the electronic device; segment the first point cloud containing the glue path and the frame in the depth image according to the reference plane; perform Euclidean clustering on the first point cloud to obtain the second point cloud representing the frame and the third point cloud representing the glue path; fit the frame line of the frame according to the second point cloud; calculate multiple parameter equations representing the glue path slice according to two adjacent frame points of the frame line, divide the points in the third point cloud into corresponding glue path slices according to the multiple parameter equations, and project the divided points to the corresponding glue path slices to obtain the detection surface; wherein, one parameter equation corresponds to one glue path slice, and one glue path slice passes through one frame point of the frame line; remove the non-glue path points in the detection surface to obtain the glue path detection surface, and detect the glue path of the electronic device screen based on the glue path detection surface.

[0106] It should be noted that the specific implementation of each operation can be Figure 1 According to the corresponding description of the method embodiment shown, the electronic device 300 can be used to execute an electronic device screen glue path detection method of the above method embodiment of the present invention, which will not be described in detail here.

[0107] In the embodiments of the present disclosure, a computer-readable storage medium is also provided, and the computer-readable storage medium is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the above-mentioned embodiment of a method for detecting a screen glue path of an electronic device. It should be understood by those skilled in the art that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting adhesive lines on electronic device screens, characterized in that: Methods include: Determining a reference plane of the border from a depth image of the border of the electronic device; Segment the first point cloud representing the glue path and the border in the depth image according to the reference plane; Performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path; Fitting a border line of the border according to the second point cloud; wherein the fitting process of the border line includes: substituting the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the border; solving the parametric equation of the fitting circle to obtain the coordinates of the center of the fitting circle; according to the coordinates of the center of the fitting circle, converting the coordinates of each point in the second point cloud into polar coordinates relative to the center of the fitting circle; dividing the polar coordinates into n parts according to the polar angle, taking the point with the smallest polar diameter in each part as the border point on the border line, thereby fitting the border line of the border; wherein n is a positive integer; According to two adjacent border points of the border line, multiple parameter equations characterizing the glue road slices are calculated, and according to the multiple parameter equations, the points in the third point cloud are divided into corresponding glue road slices, and the divided points are projected onto the corresponding glue road slices to obtain the detection surface; wherein, one parameter equation corresponds to one glue road slice, and one glue road slice passes through one border point of the border line; wherein, the process of obtaining the detection surface is specifically as follows: the second distance from the point in the third point cloud to each glue road slice is calculated, and if the second distance from the i-th point to the k-th glue road slice is less than a set threshold, the i-th point is divided into the k-th glue road slice, until all points in the third point cloud are divided into corresponding glue road slices, and the projection coordinates of each point in the third point cloud on the corresponding glue road slice are calculated; wherein i and k are both positive integers; according to the projection coordinates, each point in the third point cloud is projected onto the corresponding glue road slice to obtain the detection surface; The non-glue road points in the detection surface are removed to obtain the glue road detection surface, and the glue road of the electronic device screen is detected based on the glue road detection surface.

2. The method for detecting adhesive path on an electronic device screen according to claim 1, characterized in that: Determining a reference plane of a border from a depth image includes: extracting three reference points of the border from the image, and determining a reference plane where the border is located according to the three reference points.

3. The method for detecting adhesive path on an electronic device screen according to claim 1, characterized in that: The first point cloud representing the glue path and the border in the depth image is segmented according to the reference plane. Specifically, the first distance from each point in the depth image to the reference plane is calculated. When the first distance is less than or equal to a preset threshold, the point is a point representing the glue path or a point representing the border. The points that are points representing the glue path or points representing the border are aggregated to obtain the first point cloud representing the glue path and the border.

4. The method for detecting adhesive path on screen of electronic equipment according to claim 1, characterized in that: The non-glue path points in the multiple detection surfaces with glue path points are removed to obtain the contour surface of the glue path, including: The midpoint of the electronic device is selected; wherein the midpoint is the coordinate of the center of the fitting circle, and the steps of determining the coordinate of the center of the fitting circle are as follows: substituting the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the frame, solving the parametric equation of the fitting circle to obtain the coordinate of the center of the fitting circle; Calculate the third distance between each point in each detection surface and the midpoint, and calculate the fourth distance between the midpoint and a border point of each detection surface passing through the border line; Determine whether the third distance and the fourth distance meet a preset removal condition, and whether the difference between each detection surface point and the frame point corresponding to the detection surface in the vertical direction is less than a preset difference threshold; wherein the removal condition includes that the third distance is less than the fourth distance, and the fourth distance is greater than the preset threshold; If there are points in each detection surface that meet both the removal condition and the difference threshold, they will be retained, otherwise they will be removed.

5. The method for detecting adhesive path on an electronic device screen according to claim 1, characterized in that: Before calculating multiple parametric equations characterizing the glue path slice based on two adjacent border points of the border line, dividing the points in the third point cloud into corresponding glue path slices according to the multiple parametric equations, and projecting the divided points to the corresponding glue path slices to obtain the detection surface, the method also includes: performing mean smoothing on the border line to obtain the smoothed border line.

6. An electronic device screen glue path detection device, characterized in that: The device includes: A reference plane determination module, used to determine a reference plane of a frame from a depth image of the frame of the electronic device; A segmentation module, used for segmenting a first point cloud representing a glue path and a first point cloud representing a border in the depth image according to a reference plane; A clustering module, used for performing Euclidean clustering on the first point cloud to obtain a second point cloud representing the border and a third point cloud representing the glue path; A border line determination module is used to fit the border line of the border according to the second point cloud; wherein the border line fitting process includes: substituting the coordinates of each point in the second point cloud into the parametric equation of the circle to obtain the parametric equation of the fitting circle of the border; solving the parametric equation of the fitting circle to obtain the coordinates of the center of the fitting circle; according to the coordinates of the center of the fitting circle, converting the coordinates of each point in the second point cloud into polar coordinates relative to the center of the fitting circle; dividing the polar coordinates into n parts according to the polar angle, taking the point with the smallest polar diameter in each part as the border point on the border line, thereby fitting the border line of the border; wherein n is a positive integer; A glue road slicing module is used to calculate multiple parameter equations characterizing the glue road slices according to two adjacent border points of the border line, divide the points in the third point cloud into corresponding glue road slices according to the multiple parameter equations, and project the divided points to the corresponding glue road slices to obtain a detection surface; wherein, one parameter equation corresponds to one glue road slice, and one glue road slice passes through one border point of the border line; wherein, the process of obtaining the detection surface is specifically as follows: calculate the second distance from the point in the third point cloud to each glue road slice, if the second distance from the i-th point to the k-th glue road slice is less than a set threshold, then divide the i-th point into the k-th glue road slice, until all points in the third point cloud are divided into corresponding glue road slices, calculate the projection coordinates of each point in the third point cloud on the corresponding glue road slice; wherein i and k are both positive integers; project each point in the third point cloud onto the corresponding glue road slice according to the projection coordinates to obtain a detection surface; The glue path detection module is used to remove non-glue path points in the detection surface to obtain the glue path detection surface, and detect the glue path of the electronic device screen based on the glue path detection surface.

7. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of a method for detecting glue paths on an electronic device screen as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, and when the computer program is executed by one or more processors, the computer program implements a method for detecting glue paths on an electronic device screen as described in any one of claims 1 to 5.

Images