Offset detection methods, devices, electronic equipment and storage media

Abstract

This application provides an offset detection method, apparatus, electronic device, and storage medium. The method includes: acquiring reference point cloud data and test point cloud data for a target component; constructing a reference depth image for the target component based on the reference point cloud data; constructing a test depth image for the target component based on the test point cloud data; determining matching feature point pairs in the reference depth image and the test depth image; determining whether the test depth image is an anomalous image based on the matching feature point pairs; and determining whether the target component's position in the electronic device has shifted based on the determination result of whether the test depth image is an anomalous image. This improves detection efficiency and provides technical support for achieving efficient detection of target component shifts.

Description

Technical Field

[0001] This application relates to the field of defect detection, and more particularly to an offset detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Electronic devices such as laptops typically contain surface-mount components. If these components shift, it can affect the overall quality of the electronic device. Current methods for detecting these shifts are inefficient. Therefore, developing a highly efficient method for detecting component shifts has become a pressing technical challenge. Summary of the Invention

[0003] This application provides an offset detection method, apparatus, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this application, an offset detection method is provided, the method comprising:

[0005] Acquire reference point cloud data and test point cloud data for the target component;

[0006] Based on the reference point cloud data, a reference depth image for the target component is constructed;

[0007] Based on the point cloud data to be measured, a depth image to be measured for the target component is constructed;

[0008] Identify matching feature point pairs in the reference depth image and the depth image to be measured;

[0009] Based on matching feature point pairs, determine whether the depth image to be measured is an anomalous image;

[0010] Based on the determination result of whether the depth image to be measured is an abnormal image, it is determined whether the position of the target component in the electronic device has shifted.

[0011] In the above scheme, determining the matching feature point pairs in the reference depth image and the depth image to be measured includes:

[0012] Feature corner points are extracted from the reference depth image to obtain the target feature points in the reference depth image;

[0013] Feature corner points are extracted from the depth image to be measured to obtain the target feature points in the depth image to be measured;

[0014] Feature matching is performed on the target feature points in the reference depth image and the target feature points in the depth image to be measured to obtain matching feature point pairs in the reference depth image and the depth image to be measured.

[0015] In the above scheme, determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes:

[0016] If the number of matching feature point pairs is less than or equal to a first preset threshold, the depth image to be measured is determined to be an abnormal image.

[0017] In the above scheme, determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes:

[0018] If the number of matching feature point pairs is greater than a first preset threshold, the depth image to be measured is determined to be an abnormal image based on the average Euclidean distance between all matching feature point pairs.

[0019] In the above scheme, determining whether the depth image to be measured is an anomalous image based on the average Euclidean distance between all matching feature point pairs includes:

[0020] If the average Euclidean distance is less than or equal to the second preset threshold, the depth image to be measured is determined to be an abnormal image.

[0021] If the average Euclidean distance is greater than a second preset threshold, the depth image to be measured is determined to be an abnormal image.

[0022] In the above scheme, obtaining reference point cloud data for the target component includes:

[0023] Acquire the initial test point cloud data and the initial reference point cloud data for the target component; register the initial test point cloud data and the initial reference point cloud data to obtain the transformation matrix;

[0024] Based on the transformation matrix, the initial reference point cloud data is transformed to obtain intermediate reference point cloud data;

[0025] From the intermediate reference point cloud data, the reference point cloud data for the target component is determined.

[0026] In the above scheme, acquiring the point cloud data to be tested for the target component includes:

[0027] Find the neighboring points of the reference point cloud data from the intermediate reference point cloud data;

[0028] Point cloud data whose neighboring points in the initial test point cloud data have the same characteristics as the neighboring points in the reference point cloud data are used as the test point cloud data.

[0029] According to a second aspect of this application, an offset detection device is provided, the device comprising:

[0030] The acquisition unit is used to acquire reference point cloud data and test point cloud data for the target component;

[0031] The first building unit is used to construct a reference depth image for the target component based on the reference point cloud data;

[0032] The second building unit is used to construct a depth image of the target component based on the point cloud data to be measured.

[0033] The first determining unit is used to determine matching feature point pairs in the reference depth image and the depth image to be measured;

[0034] The second determining unit is used to determine whether the depth image to be measured is an abnormal image based on the matching feature point pairs;

[0035] The third determining unit is used to determine whether the position of the target component in the electronic device has shifted based on the determination result of whether the depth image to be measured is an abnormal image.

[0036] According to a third aspect of this application, an electronic device is provided, comprising:

[0037] At least one processor; and

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0040] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods described in this application.

[0041] In this application, a reference depth image and a test depth image for the target component are obtained by constructing depth images of the acquired reference point cloud data and test point cloud data for the target component, respectively. Based on the matching feature point pairs in the reference depth image and the test depth image, it is determined whether the test depth image is an anomalous image. Based on the determination result of whether the test depth image is an anomalous image, it is determined whether the position of the target component in the electronic device has shifted, thus achieving efficient offset detection of the target component.

[0042] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0043] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein:

[0044] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0045] Figure 1 A schematic diagram illustrating the implementation flow of the offset detection method according to an embodiment of this application is shown;

[0046] Figure 2 A schematic diagram of the application implementation process in an embodiment of this application is shown;

[0047] Figure 3 A schematic diagram of the composition structure of the offset detection device according to an embodiment of this application is shown;

[0048] Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0049] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] It's understandable that electronic devices typically contain attached components (target components). If these components shift, the quality of the electronic device will be affected. Currently, deep learning-based offset detection methods are commonly used to detect offsets in these components. However, these methods require collecting a large number of samples and using a neural network through annotation and training to develop an offset defect detection model. This model is then used to identify whether the attached component has shifted. This method is resource-intensive and has low detection efficiency. Improving the detection efficiency for offsets in attached components would undoubtedly save on detection costs and further ensure the quality of the electronic device.

[0051] In this embodiment, a reference depth image and a test depth image for the target component are obtained by constructing depth images of reference point cloud data and test point cloud data for the target component, respectively. Based on the matching feature point pairs in the reference depth image and the test depth image, it is determined whether the test depth image is an anomalous image. Based on the determination result of whether the test depth image is an anomalous image, it is determined whether the position of the target component in the electronic device has shifted. Compared with current deep learning-based offset detection methods, this application does not require a large amount of manpower and resources for sample collection, annotation, and training. It only needs to use the reference point cloud data and test point cloud data for the target component to achieve offset detection, thus improving the efficiency of target component offset detection.

[0052] The offset detection method of this application embodiment will be described in detail below.

[0053] This application provides an offset detection method, such as... Figure 1 As shown, the method includes:

[0054] S101: Acquire reference point cloud data and test point cloud data for the target component.

[0055] In this step, the target component is an adhesive component inside the electronic device. The number of target components can be one, two, or more. Taking a laptop computer as an example, the target components include adhesive components such as Mylar and conductive cloth.

[0056] In this application, reference point cloud data and test point cloud data for a target component are identified to obtain the reference point cloud data and test point cloud data for that component. Two types of point cloud data exist for the same target component: reference point cloud data and test point cloud data. The reference point cloud data is the point cloud data of the target component when it is in a standard position inside the electronic device. The test point cloud data is the point cloud data of the target component when it is in its actual position inside the electronic device. Using the technical solution of this application is equivalent to detecting whether the position of the target component has shifted based on the point cloud data of the same target component in its actual position and the point cloud data when it is in its standard position.

[0057] S102: Based on the reference point cloud data, construct a reference depth image for the target component.

[0058] In this step, the reference point cloud data is a collection of points containing three-dimensional coordinates. If each point is represented by three-dimensional coordinates (x, y, z), then z represents the depth value of that point in the point cloud data. The depth values ​​of each point in the reference point cloud data are used as grayscale values ​​in the reference depth image to be constructed, the x-value of each point is used as the abscissa value of that point in the reference depth image to be constructed, and the y-value of each point is used as the ordinate value of that point in the reference depth image to be constructed. Based on the abscissa, ordinate, and grayscale values ​​of each point in the reference depth image to be constructed, the reference depth image to be constructed is constructed, and the constructed image can be used as the reference depth image for the target component.

[0059] S103: Based on the point cloud data to be tested, construct a depth image of the target component to be tested.

[0060] In this step, the point cloud data to be measured is a collection of points containing three-dimensional coordinates. If each point is represented by three-dimensional coordinates (x, y, z), then z represents the depth value of that point in the point cloud data. The depth values ​​of each point in the point cloud data are used as grayscale values ​​in the depth image to be constructed, the x-value of each point is used as the abscissa value of that point in the depth image to be constructed, and the y-value of each point is used as the ordinate value of that point in the depth image to be constructed. Based on the abscissa, ordinate, and grayscale values ​​of each point in the depth image to be constructed, the depth image to be constructed is performed, and the constructed image can be used as the depth image to be measured for the target component.

[0061] In steps S102 and S103, the 3D point cloud data is converted into a 2D depth image. On one hand, processing 2D data is faster than processing 3D point cloud data. On the other hand, because point cloud data has a relatively high dimensionality, the methods for processing it are usually limited; processing 2D data is easier and more convenient.

[0062] S104: Determine matching feature point pairs in the reference depth image and the depth image to be measured.

[0063] In this step, feature point pairs are matched, which are pairs of feature points formed by two feature points in the reference depth image and two feature points in the depth image to be measured. Typically, the two feature points forming a pair in both the reference and depth images share the same characteristics. For example, if the grayscale values ​​and distributions of the surrounding images of two feature points are the same, then these two feature points have the same characteristics. Alternatively, if the relationship, invariant moments, and angles between the two feature points and their surrounding feature points are the same, then these two feature points have the same characteristics.

[0064] S105: Based on matching feature point pairs, determine whether the depth image to be measured is an anomalous image.

[0065] In this step, the presence of anomalies in the depth image to be measured is determined by comparing the attributes of matching feature point pairs in the depth image and the depth image to be measured. The attributes of the matching feature point pairs include the number of matching feature point pairs and the average Euclidean distance between them. Taking the number of matching feature point pairs as an example, if the number of matching feature point pairs is less than or equal to a preset threshold, then the depth image to be measured is determined to be anomaly. Similarly, taking the average Euclidean distance between matching feature point pairs as an example, if the average Euclidean distance between matching feature point pairs is greater than a preset distance threshold, then the depth image to be measured is determined to be anomaly.

[0066] S106: Based on the determination result of whether the depth image to be measured is an abnormal image, determine whether the position of the target component in the electronic device has shifted.

[0067] In this step, based on whether the depth image to be measured is an abnormal image, it is determined whether the target component has shifted position. If the depth image to be measured is abnormal compared to the reference depth image, it is determined that the target component has shifted position within the electronic device. If the depth image to be measured is not abnormal compared to the reference depth image, it is determined that the target component has not shifted position within the electronic device.

[0068] In the schemes shown in S101 to S106, a reference depth image and a test depth image for the target component are obtained by constructing depth images of the acquired reference point cloud data and test point cloud data for the target component, respectively. Based on the matching feature point pairs in the reference depth image and the test depth image, it is determined whether the test depth image is an anomalous image. Based on the determination result of whether the test depth image is an anomalous image, it is determined whether the position of the target component in the electronic device has shifted. Compared with current deep learning-based offset detection methods, this application does not require a large amount of manpower and resources for sample collection, annotation, and training. It only needs to use the reference point cloud data and test point cloud data for the target component to achieve offset detection, thus improving the efficiency of target component offset detection.

[0069] In an optional scheme, determining the matching feature point pairs in the reference depth image and the depth image to be measured includes:

[0070] Feature corner points are extracted from the reference depth image to obtain the target feature points in the reference depth image;

[0071] Feature corner points are extracted from the depth image to be measured to obtain the target feature points in the depth image to be measured;

[0072] Feature matching is performed on the target feature points in the reference depth image and the target feature points in the depth image to be measured to obtain matching feature point pairs in the reference depth image and the depth image to be measured.

[0073] In this application, feature corner points are feature points with prominent feature attributes. Methods for feature corner point extraction include Scale-invariant feature transform (SIFT) algorithm, Oriented Fast and Rotated BRIEF algorithm, etc., and this application does not limit the methods used.

[0074] For example, ORB corner point extraction is performed on a reference depth image to obtain multiple target feature points. ORB corner point extraction is then performed on the depth image to be tested to obtain multiple target feature points. Feature matching is performed between the multiple target feature points of the reference depth image and the multiple target feature points of the depth image to be tested to obtain matching feature point pairs. For example, if point (x1, y1) in the reference depth image matches point (a1, b1) in the depth image to be tested, then points (x1, y1) and (a1, b1) constitute a matching point pair. Whether two feature points match is determined by their respective feature parameters. Feature parameters include the grayscale values ​​and distribution of the surrounding image, the relationship between the feature point and surrounding feature points, invariant moments, and angles. When the feature parameters of two feature points are the same or their similarity reaches a certain preset threshold, the two feature points are considered matching feature points, forming a matching feature point pair.

[0075] In this application, feature corner points are extracted from a reference depth image and a depth image to be measured, resulting in target feature points in both the reference depth image and the depth image to be measured. This method is simple and easy to implement in engineering, improving detection efficiency. Matching feature point pairs are determined by using the feature attributes of the target feature points in the reference depth image and the depth image to be measured, ensuring the accuracy of offset detection.

[0076] In an optional scheme, determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes:

[0077] If the number of matching feature point pairs is less than or equal to a first preset threshold, the depth image to be measured is determined to be an abnormal image.

[0078] In this application, a pre-set quantity threshold (first preset threshold) is used to determine the anomalies in the depth image under test based on the relationship between the number of matching feature point pairs and the quantity threshold. The fewer the number of matching feature points, the lower the degree of matching between the target feature points in the depth image under test and the target feature points in the reference depth image. When the number of matching feature points is less than or equal to the preset quantity threshold, the depth image under test is determined to be an abnormal image.

[0079] In an optional scheme, determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes:

[0080] If the number of matching feature point pairs is greater than a first preset threshold, the depth image to be measured is determined to be an abnormal image.

[0081] In this application, the method of determining whether the depth image to be measured is an abnormal image by comparing the number of matching feature point pairs with the size of a first preset threshold is simple and easy to implement, and it also improves the detection efficiency.

[0082] In a preferred embodiment, determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes:

[0083] If the number of matching feature point pairs is greater than a first preset threshold, the depth image to be measured is determined to be an abnormal image based on the average Euclidean distance between all matching feature point pairs.

[0084] In this application, the formula for calculating the average Euclidean distance is shown in equation (1) below:

[0085]

[0086] Where (model_x[i], model_y[i]) are the coordinates of the target feature points in the reference depth image that can form matching feature point pairs with feature points in the depth image to be measured. (dst_x[i], dst_y[i]) are the coordinates of the target feature points in the depth image to be measured that can form matching feature point pairs with feature points in the reference depth image. N is the total number of matching feature point pairs in the reference depth image and the depth image to be measured, and i is a natural number from 0 to N.

[0087] This application proposes a scheme to further determine whether the depth image under test is an abnormal image based on the average Euclidean distance between all matching feature point pairs when the number of matching feature point pairs exceeds a first preset threshold. This scheme provides a secondary determination of whether the depth image under test is abnormal, further ensuring the accuracy of offset detection.

[0088] In an optional scheme, determining whether the depth image to be measured is an anomalous image based on the average Euclidean distance between all matching feature point pairs includes:

[0089] If the average Euclidean distance is less than or equal to the second preset threshold, the depth image to be measured is determined to be an abnormal image.

[0090] If the average Euclidean distance is greater than a second preset threshold, the depth image to be measured is determined to be an abnormal image.

[0091] In this application, a distance threshold (second preset threshold) is preset, and anomalies in the depth image to be measured are determined based on the relationship between the average Euclidean distance and the distance threshold. Euclidean distance is the true distance between two points in a specific dimensional space. When the number of matching feature pairs meets a first preset range (greater than a number threshold), and the average Euclidean distance between all matching feature point pairs is less than or equal to the distance threshold, the depth image to be measured is determined to be a normal image. When the average Euclidean distance is greater than the distance threshold, the depth image to be measured is determined to be an abnormal image.

[0092] This application presents a method for determining whether a depth image to be measured is an abnormal image by means of the relationship between the average Euclidean distance and a second preset threshold. This method is simple and easy to implement, and provides a secondary method for determining whether a depth image to be measured is abnormal, thus achieving accurate offset detection.

[0093] In an optional embodiment, acquiring reference point cloud data for the target component includes:

[0094] Acquire the initial test point cloud data and the initial reference point cloud data for the target component; register the initial test point cloud data and the initial reference point cloud data to obtain the transformation matrix;

[0095] Based on the transformation matrix, the initial reference point cloud data is transformed to obtain intermediate reference point cloud data;

[0096] From the intermediate reference point cloud data, the reference point cloud data for the target component is determined.

[0097] In this application, the initial point cloud data to be tested includes point cloud data for the target component, and the initial reference point cloud data includes reference point cloud data for the target component. It can be understood that, taking a laptop computer as an example, the initial point cloud data to be tested is the point cloud data of the internal structure of the laptop computer under test, including point cloud data for the target component, as well as point cloud data of other internal structures of the laptop computer, such as the hard drive. The initial reference point cloud data is the point cloud data of the internal structure of the laptop computer with a location standard, including point cloud data for the target component, as well as point cloud data of other internal structures of the laptop computer, such as the hard drive.

[0098] Initial reference point cloud data and initial test point cloud data can be obtained by photographing the interior of electronic devices with a 3D camera, or by drawing them using computer-aided drafting (CAD) software and then exporting the results. Point cloud registration is performed on the initial reference point cloud data and the initial test point cloud data to obtain a transformation matrix. The initial reference point cloud data is then transformed based on this transformation matrix to obtain intermediate reference point cloud data. Specifically, assuming that the initial reference point cloud data and the initial test point cloud data obtain a transformation matrix T through point cloud registration, the initial reference point cloud data is cross-multiplied with the transformation matrix to obtain the corrected initial reference point cloud data, i.e., the intermediate reference point cloud data.

[0099] The initial reference point cloud data is transformed by rotation, translation and other transformations based on the transformation matrix to obtain intermediate reference point cloud data. The initial reference point cloud data is transformed to the same spatial dimension as the initial test point cloud data, so that the spatial position difference between the two point cloud data is minimized, which facilitates the detection of whether the target component has shifted position.

[0100] Extract the point cloud data region where the target component is located from the intermediate reference point cloud data, and identify the reference point cloud data for each target component.

[0101] In this application, a scheme is proposed that involves point cloud registration of the initial point cloud data to be tested and the initial reference point cloud data to obtain a transformation matrix, and then transforming the initial reference point cloud data based on the transformation matrix to obtain intermediate reference point cloud data. This improves the efficiency of offset detection. Simultaneously, transforming the initial reference point cloud data to a spatial dimension consistent with the initial point cloud data to be tested facilitates accurate detection of whether the target component has shifted.

[0102] In an optional embodiment, acquiring the point cloud data to be tested for the target component includes:

[0103] Find the neighboring points of the reference point cloud data from the intermediate reference point cloud data;

[0104] Point cloud data whose neighboring points in the initial test point cloud data have the same characteristics as the neighboring points in the reference point cloud data are used as the test point cloud data.

[0105] In this application, after determining the reference point cloud data for the target component from the intermediate reference point cloud data, the features of the surrounding points of the reference point cloud data for a target component are obtained by searching the neighboring points of the reference point cloud data. The neighboring points are points centered on the reference point cloud data and located within a preset range. The preset range is a pre-set range designed to restrict the position and distance of the neighboring points to be searched, facilitating accurate location of the neighboring points. Point cloud data with the same surrounding point features as the neighboring point features of the reference point cloud data are searched in the initial test point cloud data, and this point cloud data is used as the test point cloud data for the same target component. For example, for a target component A in the intermediate reference point cloud data, by searching the features of the points surrounding component A, the location of component A can be accurately located in the initial test point cloud data, thereby accurately detecting whether component A has experienced a positional shift in the initial test point cloud data.

[0106] In one specific embodiment, taking a portable laptop computer as the electronic device, a single target component, and a Mylar as the target component as an example, the offset detection method of this application will be described.

[0107] like Figure 2 As shown, a 3D camera is used to capture images of the interior of both a standard laptop and a laptop to be tested, obtaining initial reference point cloud data and initial test point cloud data. Point cloud registration is performed between the initial reference point cloud data and the initial test point cloud data to obtain a transformation matrix. Based on the transformation matrix, the initial reference point cloud data is transformed to obtain corrected initial reference point cloud data (used as intermediate reference point cloud data). From the corrected initial reference point cloud data, the point cloud data for the target component Myra is identified, i.e., the reference point cloud data is identified. By searching for neighboring points around Myra, the point cloud data at the location of Myra is determined in the initial test point cloud data, i.e., the test point cloud data is determined. A reference depth image is constructed from the point cloud data of Myra in the corrected initial reference point cloud data, and a test depth image is constructed from the point cloud data of Myra in the test point cloud data. Feature corner points are extracted from both the reference depth image and the test depth image, and feature matching is performed on the extracted feature corner points to obtain matching feature point pairs. Based on the relationship between the number of matching feature point pairs and a preset threshold (first preset threshold), the depth image to be tested is labeled according to the following formula (2):

[0108]

[0109] Where match_count is the number of matching feature point pairs, Thresh is the number threshold, and flag1 is the label of the depth image to be measured. When flag1 is 0, it indicates that the depth image to be measured is an abnormal image, and when flag1 is 1, it indicates that the depth image to be measured is not an abnormal image. When the number of matching feature point pairs is less than or equal to the preset number threshold, the depth image to be measured is labeled as 0, and is determined to be an abnormal image, that is, the position of Myra in the point cloud data to be measured has shifted. When the number of matching feature point pairs is greater than the preset number threshold, the depth image to be measured is labeled as 1, and is determined not to be an abnormal image. Based on the relationship between the average Euclidean distance between all matching feature point pairs and the preset distance threshold, the depth image to be measured labeled as 1 is re-labeled according to the following formula (3):

[0110]

[0111] Where *dis* is the average Euclidean distance between all matching feature point pairs, *T_dis* is a preset distance threshold, and *flag2* is the label of the depth image to be measured. When *flag2* is 0, it indicates that the depth image to be measured is an anomalous image; when *flag2* is 1, it indicates that the depth image to be measured is not an anomalous image. When the average Euclidean distance is less than or equal to the distance threshold, the depth image to be measured remains labeled 1, indicating it is not an anomalous image, meaning the position of Myra in the point cloud data has not shifted. When the average Euclidean distance is greater than the distance threshold, the label of the depth image to be measured changes from 1 to 0, indicating it is an anomalous image, meaning the position of Myra in the point cloud data has shifted.

[0112] In this application, depth images are constructed based on the acquired reference point cloud data and the point cloud data to be tested for the target component, resulting in a reference depth image and a depth image to be tested for the target component. Matching feature point pairs in the reference depth image and the depth image to be tested are used to determine whether the depth image to be tested is an anomalous image. Based on the determination of whether the depth image to be tested is an anomalous image, it is determined whether the position of the target component in the electronic device has shifted. Compared with current deep learning-based offset detection methods, this application does not require a large amount of manpower and resources for sample collection, annotation, and training. It only needs the reference point cloud data and the point cloud data to be tested for the target component to achieve offset detection, thus improving the efficiency of target component offset detection.

[0113] The foregoing content describes the offset detection method of this application using a portable laptop computer as the electronic device and Mylar as the target component as an example. For schemes where the electronic device is another device and the target component is another component, please refer to the explanation and it will not be repeated.

[0114] This application provides an offset detection device, such as... Figure 3As shown, the device includes:

[0115] The acquisition unit 301 is used to acquire reference point cloud data and test point cloud data for the target component;

[0116] The first construction unit 302 is used to construct a reference depth image for the target component based on the reference point cloud data;

[0117] The second building unit 303 is used to build a depth image of the target component based on the point cloud data to be measured.

[0118] The first determining unit 304 is used to determine matching feature point pairs in the reference depth image and the depth image to be measured;

[0119] The second determining unit 305 is used to determine whether the depth image to be measured is an abnormal image based on the matching feature point pairs;

[0120] The third determining unit 306 is used to determine whether the position of the target component in the electronic device has shifted based on the determination result of whether the depth image to be measured is an abnormal image.

[0121] In an optional scheme, the first determining unit 304 is used to extract feature corner points from the reference depth image to obtain target feature points in the reference depth image; extract feature corner points from the depth image to be tested to obtain target feature points in the depth image to be tested; and perform feature matching between the target feature points in the reference depth image and the target feature points in the depth image to be tested to obtain matching feature point pairs in the reference depth image and the depth image to be tested.

[0122] In an alternative embodiment, the second determining unit 305 is used to determine the depth image to be measured as an abnormal image when the number of matching feature point pairs is less than or equal to a first preset threshold.

[0123] In an optional embodiment, the second determining unit 305 is used to determine whether the depth image to be measured is an abnormal image based on the average Euclidean distance between all matching feature point pairs when the number of matching feature point pairs is greater than a first preset threshold.

[0124] In one alternative embodiment, the second determining unit 305 is configured to determine that the depth image to be measured is not an abnormal image when the average Euclidean distance is less than or equal to a second preset threshold; and to determine that the depth image to be measured is an abnormal image when the average Euclidean distance is greater than the second preset threshold.

[0125] In one optional embodiment, the acquisition unit 301 is used to acquire initial test point cloud data and initial reference point cloud data for the target component; register the initial test point cloud data and initial reference point cloud data to obtain a transformation matrix; transform the initial reference point cloud data based on the transformation matrix to obtain intermediate reference point cloud data; and determine the reference point cloud data for the target component from the intermediate reference point cloud data.

[0126] In an optional scheme, the acquisition unit 301 is used to find the neighboring points of the reference point cloud data from the intermediate reference point cloud data; and to take the point cloud data in the initial test point cloud data whose neighboring points have the same characteristics as the neighboring points of the reference point cloud data as the test point cloud data.

[0127] It should be noted that the offset detection device in this application embodiment solves the problem in a similar way to the aforementioned offset detection method. Therefore, the implementation process, implementation principle and beneficial effects of the device can be found in the description of the implementation process, implementation principle and beneficial effects of the aforementioned method, and the repeated parts will not be repeated.

[0128] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0129] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0130] like Figure 4 As shown, device 400 includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 402 or a computer program loaded from storage unit 408 into random access memory (RAM) 403. RAM 403 may also store various programs and data required for the operation of device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0131] Multiple components in device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the offset detection method. For example, in some embodiments, the offset detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the offset detection method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the offset detection method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0138] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0139] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0141] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An offset detection method, characterized in that, The method includes: Acquire reference point cloud data and test point cloud data for the target component; Based on the reference point cloud data, a reference depth image for the target component is constructed; Based on the point cloud data to be measured, a depth image to be measured for the target component is constructed; Identify matching feature point pairs in the reference depth image and the depth image to be measured; Based on matching feature point pairs, determine whether the depth image to be measured is an anomalous image; Based on the determination result of whether the depth image to be measured is an abnormal image, it is determined whether the position of the target component in the electronic device has shifted; The step of determining the matching feature point pairs in the reference depth image and the depth image to be measured includes: Feature corner points are extracted from the reference depth image to obtain the target feature points in the reference depth image; Feature corner points are extracted from the depth image to be measured to obtain the target feature points in the depth image to be measured; Feature matching is performed on the target feature points in the reference depth image and the target feature points in the depth image to be measured to obtain matching feature point pairs in the reference depth image and the depth image to be measured. The step of determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes: If the number of matching feature point pairs is greater than a first preset threshold, the depth image to be measured is determined to be an abnormal image based on the average Euclidean distance between all matching feature point pairs.

2. The method according to claim 1, characterized in that, The step of determining whether the depth image to be measured is an anomalous image based on matching feature point pairs includes: If the number of matching feature point pairs is less than or equal to a first preset threshold, the depth image to be measured is determined to be an abnormal image.

3. The method according to claim 1, characterized in that, The determination of whether the depth image to be measured is an anomalous image based on the average Euclidean distance between all matching feature point pairs includes: If the average Euclidean distance is less than or equal to the second preset threshold, the depth image to be measured is determined to be an abnormal image. If the average Euclidean distance is greater than a second preset threshold, the depth image to be measured is determined to be an abnormal image.

4. The method according to claim 1, characterized in that, The acquisition of reference point cloud data for the target component includes: Acquire the initial test point cloud data and the initial reference point cloud data for the target component; register the initial test point cloud data and the initial reference point cloud data to obtain the transformation matrix; Based on the transformation matrix, the initial reference point cloud data is transformed to obtain intermediate reference point cloud data; From the intermediate reference point cloud data, the reference point cloud data for the target component is determined.

5. The method according to claim 4, characterized in that, The acquisition of the test point cloud data for the target component includes: Find the neighboring points of the reference point cloud data from the intermediate reference point cloud data; Point cloud data whose neighboring points in the initial test point cloud data have the same characteristics as the neighboring points in the reference point cloud data are used as the test point cloud data.

6. An offset detection device, characterized in that, The device includes: The acquisition unit is used to acquire reference point cloud data and test point cloud data for the target component; The first building unit is used to construct a reference depth image for the target component based on the reference point cloud data; The second building unit is used to construct a depth image of the target component based on the point cloud data to be measured. The first determining unit is used to determine matching feature point pairs in the reference depth image and the depth image to be measured; The second determining unit is used to determine whether the depth image to be measured is an abnormal image based on the matching feature point pairs; The third determining unit is used to determine whether the position of the target component in the electronic device has shifted based on the determination result of whether the depth image to be measured is an abnormal image; The first determining unit is used to extract feature corner points from the reference depth image to obtain target feature points in the reference depth image; extract feature corner points from the depth image to be tested to obtain target feature points in the depth image to be tested; and perform feature matching between the target feature points in the reference depth image and the target feature points in the depth image to be tested to obtain matching feature point pairs in the reference depth image and the depth image to be tested. The second determining unit is used to determine whether the depth image to be measured is an abnormal image based on the average Euclidean distance between all matching feature point pairs when the number of matching feature point pairs is greater than a first preset threshold.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images