Object deformation detection method and device and computer readable storage medium

Abstract

The embodiment of the invention provides an object deformation detection method and device and a computer readable storage medium. The object deformation detection method according to the embodiment of the invention comprises the following steps: acquiring surface three-dimensional data of a to-be-detected object, and acquiring at least one reference polishing direction on the surface of the to-be-detected object; acquiring the surface direction of each to-be-detected point on the surface of the to-be-detected object according to the surface three-dimensional data; the polishing direction of each point to be detected is obtained according to the surface three-dimensional data and the at least one reference polishing direction; and according to the surface three-dimensional data, the surface direction of each to-be-detected point and the polishing direction, calculating a characteristic value of each to-be-detected point, and according to the characteristic value of each to-be-detected point, detecting deformation of the to-be-detected object.

Description

Technical Field

[0001] The present invention relates to the field of image analysis, and in particular to a method, a device and a computer-readable storage medium for detecting deformation of an object, which are mainly used for deformation detection of the surface of an object. Background Art

[0002] Surface quality inspection of products is an important quality control process in the production process. In the inspection of sheet metal parts or white bodies, the surface quality inspection methods often used in the early days include: visual inspection with strong light, touch inspection with gloves, grinding with oilstone, etc. However, these methods are time-consuming and subjective, and largely depend on the concentration and attention of the inspectors. In fact, it is difficult to meet the needs of automated production lines.

[0003] In order to improve the automation level of product production lines and improve detection efficiency, surface detection based on 3D sensors has become a new trend. However, surface detection of products using optical 3D sensors often leads to deviations in detection results due to insufficient detection information and huge time consumption.

[0004] Therefore, a method and device for detecting surface deformation of an object more quickly and accurately is needed to avoid the problems of low detection accuracy caused by insufficient information and huge time consumption of the above detection. Summary of the invention

[0005] To solve the above technical problems, according to one aspect of the present invention, a method for detecting deformation of an object is provided, comprising: obtaining surface three-dimensional data of an object to be detected, and obtaining at least one reference polishing direction on the surface of the object to be detected; obtaining the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; obtaining the polishing direction of each point to be detected according to the surface three-dimensional data and the at least one reference polishing direction; calculating the characteristic value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and detecting the deformation of the object to be detected according to the characteristic value of each point to be detected.

[0006] According to another aspect of the present invention, there is provided an object deformation detection device, comprising: an acquisition unit, configured to acquire surface three-dimensional data of an object to be detected, and acquire at least one reference polishing direction on the surface of the object to be detected; a surface direction acquisition unit, configured to acquire the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; a polishing direction acquisition unit, configured to acquire the polishing direction of each point to be detected according to the surface three-dimensional data and the at least one reference polishing direction; a detection unit, configured to calculate a feature value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detect the deformation of the object to be detected according to the feature value of each point to be detected.

[0007] According to another aspect of the present invention, there is provided an object deformation detection device, comprising: a processor; and a memory, in which computer program instructions are stored, wherein when the computer program instructions are executed by the processor, the processor performs the following steps: obtaining surface three-dimensional data of an object to be detected, and obtaining at least one reference polishing direction on the surface of the object to be detected; obtaining the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; obtaining the polishing direction of each point to be detected according to the surface three-dimensional data and the at least one reference polishing direction; calculating a characteristic value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detecting the deformation of the object to be detected according to the characteristic value of each point to be detected.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the following steps: obtaining surface three-dimensional data of an object to be detected, and obtaining at least one reference polishing direction on the surface of the object to be detected; obtaining the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; obtaining the polishing direction of each point to be detected according to the surface three-dimensional data and the at least one reference polishing direction; calculating a characteristic value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detecting the deformation of the object to be detected according to the characteristic value of each point to be detected.

[0009] According to the above-mentioned object deformation detection method, device and computer-readable storage medium of the present invention, the surface direction and grinding direction of each point to be detected can be obtained according to the surface three-dimensional data of the object to be detected and the reference grinding direction, and the characteristic value can be calculated according to the obtained surface direction and grinding direction of each point to be detected, so as to obtain the deformation detection result of the surface of the object to be detected. The object deformation detection method, device and computer-readable storage medium of the embodiment of the present invention can improve the accuracy of object surface deformation detection, improve the precision of object surface deformation detection, and avoid the problem of low detection precision caused by insufficient detection information and huge time consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the present invention will become more apparent by describing in detail the embodiments of the present invention in conjunction with the accompanying drawings.

[0011] Figure 1 A flow chart showing a method for detecting object deformation according to an embodiment of the present invention is shown;

[0012] Figure 2 A schematic diagram showing a reference grinding direction for a roof cover in an example of an embodiment of the present invention;

[0013] Figure 3 A schematic diagram showing a method of obtaining the surface direction of a point to be detected in an example of an embodiment of the present invention;

[0014] Figure 4 An example of obtaining a grinding direction and calculating a distance in an example of an embodiment of the present invention is shown;

[0015] Figure 5 An example of the grinding direction of each point to be detected on the surface of a roof cover in an example of an embodiment of the present invention is shown;

[0016] Figure 6 An example of a process for acquiring a section line to be detected of a point to be detected in an example of an embodiment of the present invention is shown;

[0017] Figure 7 An example of a process for obtaining a feature value of a point to be detected in an example of an embodiment of the present invention is shown;

[0018] Figure 8 A schematic diagram showing a characteristic diagram of an example of an embodiment of the present invention;

[0019] Fig. 9 A schematic diagram showing defect segmentation results of a feature graph of an embodiment of the present invention is shown;

[0020] Fig.10 A block diagram of an object deformation detection device according to an embodiment of the present invention is shown;

[0021] Fig.11 A block diagram of an object deformation detection device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] The following will describe the object deformation detection method, device and computer-readable storage medium according to the embodiments of the present invention with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same elements from beginning to end. It should be understood that the embodiments described herein are merely illustrative and should not be interpreted as limiting the scope of the present invention.

[0023] In the process of surface quality inspection for products, the surface inspection method based on optical 3D sensors is widely used due to its non-contact and full coverage characteristics. However, in the process of surface inspection based on optical 3D sensors, when using the inspection method based on data alignment, the data alignment process is often very time-consuming, and the accuracy is low due to insufficient information obtained during the inspection, which affects the effect of object deformation detection.

[0024] Based on the above problems, an embodiment of the present invention provides an object deformation detection method, device and computer-readable storage medium, which can simulate the oilstone grinding process in manual detection of object deformation, virtually grind the surface of the object to be detected and automatically perform defect detection.

[0025] Figure 1 A flow chart of an object deformation detection method 100 according to an embodiment of the present invention is shown.

[0026] like Figure 1 As shown, in step S101, three-dimensional surface data of an object to be detected is obtained, and at least one reference polishing direction on the surface of the object to be detected is obtained.

[0027] In an embodiment of the present invention, a three-dimensional sensor (e.g., an optical three-dimensional sensor) can be used to scan the surface of the object to be detected, thereby obtaining the three-dimensional surface data of the object to be detected. Considering the limitations of the scanning field of view of the three-dimensional sensor, if the surface range of the object to be detected is large, the three-dimensional sensor can be used to obtain the complete three-dimensional surface data of the object to be detected through multiple scanning and splicing. Of course, the above-mentioned method of obtaining the three-dimensional surface data of the object to be detected is only an example. In actual applications, any method of obtaining the three-dimensional surface data can be used, and there is no limitation here.

[0028] In an embodiment of the present invention, the acquired surface three-dimensional data of the object to be detected may include various information of multiple points. Optionally, the surface three-dimensional data of the object to be detected may include the position coordinates of the above-mentioned points (such as the position coordinates expressed by the XYZ coordinate system or other coordinate systems), and may further include the grayscale data, color data and some label information (such as the name, classification, attributes and other information of the object to be detected) of these points. In one example, all the points contained in the surface three-dimensional data may be the points to be detected on the surface of the object to be detected; in another example, a part of the points contained in the surface three-dimensional data may be the points to be detected on the surface of the object to be detected, which is not limited here.

[0029] In an embodiment of the present invention, the reference grinding direction obtained on the surface of the object to be detected can be indicated by at least two points on the surface of the object to be detected (such as at least two points to be detected). In one example, the reference grinding direction can be a reference grinding direction that is preset or pre-input; in another example, the reference grinding direction can also be a reference grinding direction obtained by judging based on information such as the name, classification, and attributes of the surface of the object to be detected. For example, the reference grinding direction can refer to the actual grinding direction of a certain type of object being polished with an oilstone in manual detection of object deformation.

[0030] In step S102, the surface direction of each point to be detected on the surface of the object to be detected is obtained according to the three-dimensional surface data.

[0031] In this step, the surface direction of each point to be detected on the surface of the object to be detected can be obtained one by one. Taking the process of obtaining the surface direction of the first point to be detected as an example, a plurality of adjacent points within a preset range around the first point to be detected can be first obtained. These adjacent points can be the remaining points to be detected on the surface of the object to be detected, or other points in the three-dimensional surface data; then, a fitting plane can be obtained based on the first point to be detected and the adjacent points; finally, the surface direction of the first point to be detected can be obtained based on the fitting plane. For example, the surface direction of the first point to be detected can be the direction of the normal vector of the fitting plane. Of course, the surface direction of the first point to be detected can also be other directions related to the fitting plane, which are not limited here.

[0032] In step S103, the polishing direction of each to-be-detected point is acquired according to the surface three-dimensional data and the at least one reference polishing direction.

[0033] In this step, the polishing direction of each point to be detected on the surface of the object to be detected can be obtained one by one. Taking the acquisition process of the polishing direction of the first point to be detected as an example, at least one sub-polishing direction can be generated according to at least one reference polishing direction; then, the distance from the first point to be detected to each sub-polishing direction is obtained, and the weight corresponding to the first point to be detected and each sub-polishing direction is calculated; finally, the polishing direction of the first point to be detected is calculated according to the distance from the first point to be detected to each sub-polishing direction and the weight corresponding to each sub-polishing direction.

[0034] In one example, one or more reference polishing directions may be selected from all the acquired reference polishing directions according to the position of the first point to be detected to determine the polishing direction of the first point to be detected. Optionally, one or more reference polishing directions may be selected according to the distance between the first point to be detected and each reference polishing direction. Optionally, the distance between the first point to be detected and the reference polishing direction may be expressed as the distance from the first point to be detected to the midpoint of the reference polishing direction (or the direction fitted thereto) or the length of a vertical line segment. Optionally, a reference polishing direction in which the distance between the first point to be detected and the reference polishing direction is less than a threshold distance may be selected to determine the polishing direction of the first point to be detected.

[0035] Optionally, corresponding sub-polishing directions may be obtained respectively according to the selected reference polishing directions. In the process of obtaining the sub-polishing direction according to the reference polishing direction, the sub-polishing direction on the reference polishing direction may be determined according to two or more adjacent or neighboring points on the reference polishing direction, that is, the sub-polishing direction may be at least a part of the reference polishing direction.

[0036] After obtaining at least one sub-polishing direction, the distance from the first point to be detected to the sub-polishing direction corresponding to each selected reference polishing direction can be obtained. Similarly, the distance between the first point to be detected and the sub-polishing direction can also be expressed as the distance from the first point to be detected to the midpoint of the sub-polishing direction (or the direction fitted thereto) or the length of a vertical line segment. And, then, the weight corresponding to the first point to be detected and each sub-polishing direction can also be calculated based on the obtained distance. For example, the weight can be inversely proportional to the distance between the first detection point and the sub-polishing direction. That is, the closer the distance between the first detection point and the sub-polishing direction is, the higher the corresponding weight can be.

[0037] Finally, the polishing direction of the first to-be-detected point may be counted according to each sub-polishing direction and the weight corresponding to each sub-polishing direction of the first to-be-detected point.

[0038] In step S104, a characteristic value of each point to be detected is calculated according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and the deformation of the object to be detected is detected according to the characteristic value of each point to be detected.

[0039] In this step, the characteristic value of each point to be detected on the surface of the object to be detected can be obtained one by one. Taking the acquisition process of the characteristic value of the first point to be detected as an example, the plane to be detected can be generated first according to the position of the first point to be detected in the three-dimensional data of the surface, the surface direction of the first point to be detected and the grinding direction; then the plane to be detected and the surface can be intersected to generate a section line to be detected, and a line segment to be detected including at least the first point to be detected can be obtained on the section line to be detected, and the line segment to be detected can include the first point to be detected (for example, the first point to be detected can be the midpoint of the line segment to be detected), and can also further include one or more other points to be detected, which also have corresponding surface directions and grinding directions, and the acquisition method is similar to the acquisition method of the first point to be detected; finally, the characteristic value of the first point to be detected can be calculated according to the surface direction and grinding direction of each point to be detected on the line segment to be detected including the first point to be detected. Optionally, the result of the weighted sum of the angles between the grinding direction and the surface direction of each point to be detected on the line segment to be detected can be used as the characteristic value of the first point to be detected. Of course, the above-mentioned calculation method of the characteristic value of the point to be detected is only an example. In practical applications, any representation method of the characteristic value corresponding to the point to be detected can be used, and there is no limitation here.

[0040] According to one embodiment of the present invention, a feature map of the surface of the object to be detected can be further obtained based on the feature value of each point to be detected. The feature map of the surface of the object to be detected can use, for example, color or grayscale to show the feature value of each point to be detected on the surface of the object to be detected, so as to intuitively show the distribution of defects on the surface of the object to be detected; then, the defects on the surface of the object to be detected can be segmented according to the feature map to obtain the detection results of the deformation of the object to be detected, such as the position, size, shape, depth, etc. of the defects on the surface of the object to be detected, and the deformation condition of the object to be detected can be evaluated based on the detection results.

[0041] According to the object deformation detection method of the present invention, the surface direction and grinding direction of each point to be detected can be obtained according to the surface three-dimensional data of the object to be detected and the reference grinding direction, and the characteristic value can be calculated according to the obtained surface direction and grinding direction of each point to be detected, so as to obtain the deformation detection result of the surface of the object to be detected. The object deformation detection method, device and computer-readable storage medium of the embodiment of the present invention can improve the accuracy of object surface deformation detection, improve the precision of object surface deformation detection, and avoid the problem of low detection precision caused by insufficient detection information and huge time consumption.

[0042] The following takes the object to be detected, which is a roof cover of a car, as an example to illustrate the specific implementation steps of the object deformation detection method according to an embodiment of the present invention.

[0043] In this example, firstly, the three-dimensional surface data of the vehicle roof cover is acquired, and at least one reference polishing direction on the surface of the vehicle roof cover is acquired.

[0044] In the embodiment of the present invention, an optical 3D sensor can be used to scan the surface of the roof cover to obtain the surface 3D data of the roof cover. Optionally, the optical 3D sensor can be used to obtain the complete surface 3D data of the roof cover through multiple scanning and splicing.

[0045] In the embodiment of the present invention, the obtained surface three-dimensional data of the roof cover may include various information of multiple points on the roof cover, for example, the position coordinates of each point in the XYZ coordinate system, grayscale data, color data, and name information or label information indicating that these points are located on the roof cover of the car, etc. In one example, all points included in the surface three-dimensional data may be used as points to be detected on the surface of the roof cover.

[0046] In one example, the reference grinding direction may be a preset or pre-input actual grinding direction of the roof cover during the manual detection of deformation of the reference object. Figure 2 FIG. 1 is a schematic diagram showing a reference grinding direction for a roof cover in an example of an embodiment of the present invention. Figure 2 As shown, the reference grinding direction for the roof cover may be grinding in the direction indicated by the arrow along the edge of the roof cover.

[0047] In the embodiment of the present invention, the reference grinding direction on the roof cover can be indicated by at least two points to be detected on the surface of the roof cover. For example, multiple reference grinding directions DG = {D1, ..., D i ,…,D d}, where each reference grinding direction D i It can be represented by two or more points to be detected on the roof surface: Each of the points to be tested The information may at least include the three-dimensional position information of the point to be detected in the XYZ coordinate system.

[0048] Subsequently, the surface direction of each to-be-detected point on the surface of the vehicle roof cover may be acquired according to the three-dimensional surface data.

[0049] In this step, the surface direction of each to-be-detected point on the surface of the vehicle roof cover may be acquired one by one. Figure 3 A schematic diagram of obtaining the surface direction of a point to be detected in an example of an embodiment of the present invention is shown. Figure 3 In the example, the first point to be detected is p i As an example, the process of obtaining the surface direction is shown. i A plurality of adjacent points within a preset range 1 around the first detected point can be other points to be detected on the surface of the object to be detected, or other points in the three-dimensional data of the surface; then, based on the first detected point p i and the adjacent points to obtain a fitting plane 2; finally, the first point to be detected p can be obtained based on the fitting plane 2 i The surface direction N i , for example, the surface direction N i It can be the direction of the normal vector of the fitting plane 2.

[0050] Thereafter, the polishing direction of each of the points to be detected may be acquired according to the three-dimensional surface data and the at least one reference polishing direction.

[0051] In this step, the grinding direction of each point to be detected on the surface of the object to be detected can be obtained one by one. i Taking the acquisition process of the grinding direction as an example, at least one sub-grinding direction can be generated according to at least one reference grinding direction; then, the first point to be detected p is obtained. i The distance to each sub-grinding direction and calculate the first point to be detected p i The weight corresponding to each sub-polishing direction; finally, according to the first point to be detected p i The distance to each sub-grinding direction and the weight corresponding to each sub-grinding direction are used to calculate the first point to be detected p i Grinding direction.

[0052] Figure 4 An example of obtaining the grinding direction and calculating the distance in an example of an embodiment of the present invention is shown. Figure 4 In the example, we can firstly calculate the first point to be detected based on iThe position of the first point to be detected is used to select two reference grinding directions DG={D1, D2} with a close distance from all reference grinding directions as references to determine the first point to be detected p i Grinding direction. Where D1={p1,p2,p3}, D2={p4,p5,p6,,p7}.

[0053] After determining the two reference grinding directions, the corresponding sub-grinding directions can be obtained according to the selected reference grinding directions. Figure 4 In the sub-grinding direction, G 12 , G 23 , G 45 , G 56 and G 67 , which can be calculated using the following formula:

[0054]

[0055]

[0056] in, and are the vector length and vector direction of the sub-polishing direction respectively.

[0057] After obtaining the sub-grinding direction G 12 , G 23 , G 45 , G 56 and G 67 Afterwards, the first point to be detected p can be obtained i In this example, the first point to be detected p i The distance from the sub-grinding direction can be expressed as the first point to be detected p i To the sub-grinding direction G 12 , G 23 , G 45 , G 56 and G 67 The distance between the midpoints can be expressed as L1,…,L5 respectively.

[0058] Then, the first to-be-detected point p can be calculated according to the acquired distance. i The weight corresponding to each sub-polishing direction can be related to the first detection point p i It is inversely proportional to the distance in the grinding direction, expressed as c1,…,c5. Optionally, any one of the following formulas (3-1) to (3-4) can be used to calculate the first point to be detected p i With a distance of L k The weight c corresponding to the sub-polishing direction k :

[0059]

[0060]

[0061]

[0062]

[0063] Finally, each sub-grinding direction G in the n sub-grinding directions can be k And the first point to be detected p i The weight c corresponding to each sub-grinding direction k , to count the first detection point p by the following formula i Grinding direction G i :

[0064]

[0065] Figure 5 An example of the grinding direction of each point to be detected on the roof surface in an example of an embodiment of the present invention is shown, wherein the grinding direction of each point to be detected is weightedly calculated by the reference grinding directions shown by the left and right lines.

[0066] Finally, the characteristic value of each point to be detected can be calculated according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and the deformation of the roof cover can be detected according to the characteristic value of each point to be detected.

[0067] In this step, the characteristic value of each point to be detected on the surface of the object to be detected can be obtained one by one. In the specific process, the section line to be detected of a certain point to be detected can be obtained first, and then the corresponding characteristic value can be calculated according to the characteristic points in the line segment to be detected on the section line to be detected.

[0068] Figure 6 An example of a process for acquiring a section line to be detected of a point to be detected in an example of an embodiment of the present invention is shown. Figure 6 In the example of FIG. 1 , the first point to be detected p in the three-dimensional surface data can be first detected. i The position of the first point to be detected p i The surface direction N i and the grinding direction G i Generate the plane F to be detected i Then the plane to be detected F i Intersecting with the surface of the roof cover to generate a section line CS to be detected i .

[0069] Figure 7 An example of a process for acquiring the feature value of a point to be detected in an example of an embodiment of the present invention is shown. First, the feature value of the point to be detected can be obtained by i The acquisition includes at least the first point to be detected p i The line segment l to be detected i , in the line segment l to be detected i In the example, the first point to be detected, p i is the midpoint, and its length can be L. The line segment to be detected is l i It may further include one or more other points to be detected, each of which has a corresponding surface direction and polishing direction, and the acquisition method is the same as the first point to be detected p i Finally, according to the line segment l to be detected, i The first point to be detected is included on i The surface direction and the polishing direction of each to-be-detected point are calculated by weighted summing the angles between the polishing direction and the surface direction of each to-be-detected point among a total of m to-be-detected points. i The eigenvalue of Among them, N i is the point p to be detected i The corresponding surface direction, G i is the point to be detected p i Corresponding grinding direction:

[0070]

[0071] After the feature value of each point to be detected is obtained, a feature map of the surface of the vehicle roof cover may also be obtained according to the feature value of each point to be detected. Figure 8 FIG. 1 is a schematic diagram showing a characteristic diagram of an example of an embodiment of the present invention. Figure 8 As shown, the feature map of the roof cover surface can use grayscale to distinguish the feature value of each point to be detected on the roof cover surface, so as to intuitively show the defect distribution of the roof cover surface. Subsequently, the defects on the roof cover surface can also be segmented according to the feature map. Fig. 9 A schematic diagram of a defect segmentation result of a feature map of an embodiment of the present invention is shown. Fig. 9 In the process, the defect location, size, shape and other information on the surface of the roof cover can be obtained through segmentation, and the deformation condition of the roof cover can be evaluated based on the detection results.

[0072] Below, refer to Fig.10 The object deformation detection device according to an embodiment of the present invention is described. Fig.10 FIG. 1 is a block diagram of an object deformation detection device 1000 according to an embodiment of the present invention. Fig.10As shown in FIG. 1 , the object deformation detection device 1000 includes an acquisition unit 1010, a surface direction acquisition unit 1020, a grinding direction acquisition unit 1030, and a detection unit 1040. In addition to these units, the object deformation detection device 1000 may also include other components. However, since these components are irrelevant to the content of the embodiment of the present invention, their illustration and description are omitted here. In addition, since the specific details of the following operations performed by the object deformation detection device 1000 according to the embodiment of the present invention are the same as those described above with reference to FIG. Figure 1 The details described are the same, so repeated description of the same details is omitted here to avoid repetition.

[0073] Fig.10 The acquisition unit 1010 of the object deformation detection device 1000 acquires the surface three-dimensional data of the object to be detected, and acquires at least one reference polishing direction on the surface of the object to be detected.

[0074] In an embodiment of the present invention, the acquisition unit 1010 can use a three-dimensional sensor (e.g., an optical three-dimensional sensor) to scan the surface of the object to be detected, thereby acquiring the surface three-dimensional data of the object to be detected. Considering the limitations of the scanning field of view of the three-dimensional sensor, if the surface range of the object to be detected is large, the three-dimensional sensor can be used to obtain the complete surface three-dimensional data of the object to be detected through multiple scanning and splicing. Of course, the above-mentioned method of acquiring the surface three-dimensional data of the object to be detected is only an example. In actual applications, any method of acquiring surface three-dimensional data can be used, and there is no limitation here.

[0075] In an embodiment of the present invention, the surface three-dimensional data of the object to be detected acquired by the acquisition unit 1010 may include various information of multiple points. Optionally, the surface three-dimensional data of the object to be detected may include the position coordinates of the above-mentioned points (such as the position coordinates expressed by the XYZ coordinate system or other coordinate systems), and may further include the grayscale data, color data and some label information of these points (such as the name, classification, attributes and other information of the object to be detected). In one example, all the points contained in the surface three-dimensional data may be the points to be detected on the surface of the object to be detected; in another example, a part of the points contained in the surface three-dimensional data may be the points to be detected on the surface of the object to be detected, which is not limited here.

[0076] In an embodiment of the present invention, the reference grinding direction on the surface of the object to be detected obtained by the acquisition unit 1010 can be indicated by at least two points on the surface of the object to be detected (such as at least two points to be detected). In one example, the reference grinding direction can be a reference grinding direction that is preset or pre-input; in another example, the reference grinding direction can also be a reference grinding direction obtained by judging based on information such as the name, classification, and attributes of the surface of the object to be detected. For example, the reference grinding direction can refer to the actual grinding direction of a certain type of object being polished with an oilstone in manual detection of object deformation.

[0077] The surface direction acquisition unit 1020 acquires the surface direction of each to-be-detected point on the surface of the to-be-detected object according to the surface three-dimensional data.

[0078] The surface direction acquisition unit 1020 can acquire the surface direction of each point to be detected on the surface of the object to be detected one by one. Taking the acquisition process of the surface direction of the first point to be detected as an example, a plurality of adjacent points within a preset range around the first point to be detected can be first acquired. These adjacent points can be the remaining points to be detected on the surface of the object to be detected, or other points in the three-dimensional surface data; then, a fitting plane can be acquired based on the first point to be detected and the adjacent points; finally, the surface direction of the first point to be detected can be acquired based on the fitting plane. For example, the surface direction of the first point to be detected can be the direction of the normal vector of the fitting plane. Of course, the surface direction of the first point to be detected can also be other directions related to the fitting plane, which are not limited here.

[0079] The polishing direction acquiring unit 1030 acquires the polishing direction of each to-be-detected point according to the surface three-dimensional data and the at least one reference polishing direction.

[0080] The polishing direction acquisition unit 1030 can acquire the polishing direction of each point to be detected on the surface of the object to be detected one by one. Taking the acquisition process of the polishing direction of the first point to be detected as an example, at least one sub-polishing direction can be first generated according to at least one reference polishing direction; then, the distance from the first point to be detected to each sub-polishing direction is acquired, and the weight corresponding to the first point to be detected and each sub-polishing direction is calculated; finally, the polishing direction of the first point to be detected is calculated according to the distance from the first point to be detected to each sub-polishing direction and the weight corresponding to each sub-polishing direction.

[0081] In one example, the polishing direction acquisition unit 1030 can select one or more reference polishing directions from all the acquired reference polishing directions according to the position of the first point to be detected to determine the polishing direction of the first point to be detected. Optionally, one or more reference polishing directions can be selected according to the distance between the first point to be detected and each reference polishing direction. Optionally, the distance between the first point to be detected and the reference polishing direction can be expressed as the distance from the first point to be detected to the midpoint of the reference polishing direction (or the direction fitted thereto) or the length of a vertical line segment. Optionally, a reference polishing direction whose distance between the first point to be detected and the reference polishing direction is less than a threshold distance can be selected to determine the polishing direction of the first point to be detected.

[0082] Optionally, the polishing direction acquiring unit 1030 may acquire corresponding sub-polishing directions according to the selected reference polishing directions. In the process of acquiring the sub-polishing direction according to the reference polishing direction, the sub-polishing direction on the reference polishing direction may be determined according to two or more adjacent or neighboring points on the reference polishing direction, that is, the sub-polishing direction may be at least a part of the reference polishing direction.

[0083] After acquiring at least one sub-polishing direction, the polishing direction acquisition unit 1030 can acquire the distance from the first point to be detected to the sub-polishing direction corresponding to each selected reference polishing direction. Similarly, the distance between the first point to be detected and the sub-polishing direction can also be expressed as the distance from the first point to be detected to the midpoint of the sub-polishing direction (or the direction fitted thereto) or the length of a vertical line segment. And, then, the weight corresponding to the first point to be detected and each sub-polishing direction can also be calculated based on the acquired distance. For example, the weight can be inversely proportional to the distance between the first detection point and the sub-polishing direction. That is, the closer the distance between the first detection point and the sub-polishing direction is, the higher the corresponding weight can be.

[0084] Finally, the polishing direction acquiring unit 1030 may count the polishing directions of the first to-be-detected points according to each sub-polishing direction and the weights corresponding to the first to-be-detected points and each sub-polishing direction.

[0085] The detection unit 1040 calculates a feature value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detects the deformation of the object to be detected according to the feature value of each point to be detected.

[0086] The detection unit 1040 can obtain the characteristic value of each point to be detected on the surface of the object to be detected one by one. Taking the acquisition process of the characteristic value of the first point to be detected as an example, the plane to be detected can be generated first according to the position of the first point to be detected in the three-dimensional data of the surface, the surface direction of the first point to be detected and the grinding direction; then the plane to be detected and the surface can be intersected to generate a section line to be detected, and a line segment to be detected including at least the first point to be detected can be obtained on the section line to be detected, and the line segment to be detected can include the first point to be detected (for example, the first point to be detected can be the midpoint of the line segment to be detected), and can also further include one or more other points to be detected, and these points to be detected also have corresponding surface directions and grinding directions, and the acquisition method is similar to the acquisition method of the first point to be detected; finally, the characteristic value of the first point to be detected can be calculated according to the surface direction and grinding direction of each point to be detected on the line segment to be detected including the first point to be detected. Optionally, the result of the weighted sum of the angles between the grinding direction and the surface direction of each point to be detected on the line segment to be detected can be used as the characteristic value of the first point to be detected. Of course, the above-mentioned calculation method of the characteristic value of the point to be detected is only an example. In practical applications, any representation method of the characteristic value corresponding to the point to be detected can be used, and there is no limitation here.

[0087] According to one embodiment of the present invention, the detection unit 1040 can further obtain a feature map of the surface of the object to be detected based on the feature value of each point to be detected. The feature map of the surface of the object to be detected can use, for example, color or grayscale to show the feature value of each point to be detected on the surface of the object to be detected, so as to intuitively show the distribution of defects on the surface of the object to be detected; then, the defects on the surface of the object to be detected can be segmented according to the feature map to obtain the detection results of the deformation of the object to be detected, such as the position, size, shape, depth, etc. of the defects on the surface of the object to be detected, and the deformation condition of the object to be detected can be evaluated based on the detection results.

[0088] According to the above-mentioned object deformation detection device of the present invention, the surface direction and grinding direction of each point to be detected can be obtained according to the surface three-dimensional data of the object to be detected and the reference grinding direction, and the characteristic value can be calculated according to the obtained surface direction and grinding direction of each point to be detected, so as to obtain the deformation detection result of the surface of the object to be detected. The object deformation detection method, device and computer-readable storage medium of the embodiment of the present invention can improve the accuracy of object surface deformation detection, improve the precision of object surface deformation detection, and avoid the problem of low detection precision caused by insufficient detection information and huge time consumption.

[0089] Below, refer to Fig.11 The object deformation detection device according to an embodiment of the present invention is described. Fig.11 FIG. 1 is a block diagram of an object deformation detection device 1100 according to an embodiment of the present invention. Fig.11 As shown, the device 1100 may be a computer or a server.

[0090] like Fig.11 As shown, the object deformation detection device 1100 includes one or more processors 1110 and a memory 1120. Of course, in addition to this, the object deformation detection device 1100 may also include an input device, an output device (not shown), etc. These components may be interconnected through a bus system and / or other forms of connection mechanisms. It should be noted that Fig.11 The components and structures of the object deformation detection device 1100 shown are merely exemplary and not restrictive. The object deformation detection device 1100 may also have other components and structures as required.

[0091] The processor 1110 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may utilize computer program instructions stored in the memory 1120 to execute desired functions, so that the steps executed by the processor 810 may include: obtaining surface three-dimensional data of the object to be detected, and obtaining at least one reference polishing direction on the surface of the object to be detected; obtaining the surface direction of each point to be detected on the surface of the object to be detected based on the surface three-dimensional data; obtaining the polishing direction of each point to be detected based on the surface three-dimensional data and the at least one reference polishing direction; calculating the characteristic value of each point to be detected based on the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detecting the deformation of the object to be detected based on the characteristic value of each point to be detected.

[0092] The memory 1120 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1110 may execute the program instructions to implement the functions of the object deformation detection device of the embodiment of the present invention described above and / or other desired functions, and / or may execute the object deformation detection method according to the embodiment of the present invention. Various applications and various data may also be stored in the computer-readable storage medium.

[0093] Below, a computer-readable storage medium according to an embodiment of the present invention is described, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the following steps: obtain surface three-dimensional data of an object to be detected, and obtain at least one reference polishing direction on the surface of the object to be detected; obtain the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; obtain the polishing direction of each point to be detected according to the surface three-dimensional data and the at least one reference polishing direction; calculate the characteristic value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected, and the polishing direction, and detect the deformation of the object to be detected according to the characteristic value of each point to be detected.

[0094] Of course, the above-mentioned specific embodiments are merely examples rather than limitations, and those skilled in the art can, based on the concept of the present invention, merge and combine some steps and devices from the various embodiments described separately above to achieve the effects of the present invention. Such merged and combined embodiments are also included in the present invention, and such merges and combinations are not described one by one herein.

[0095] Note that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above invention are only for the purpose of illustration and facilitating understanding, not limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0096] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0097] The step flow charts and the above method descriptions in the present invention are only illustrative examples and are not intended to require or imply that the steps of each embodiment must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter", "then", "next", etc. are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to a singular element, such as using the article "a", "an", or "the", is not to be construed as limiting the element to the singular.

[0098] In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, based on the concept of the present invention, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of the present invention.

[0099] Each operation of the method described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors.

[0100] The various illustrated logic blocks, modules and circuits described may be implemented or performed using a general purpose processor, digital signal processor (DSP), ASIC, field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but as an alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0101] The steps of the method or algorithm described in conjunction with the present invention can be directly embedded in hardware, in a software module executed by a processor, or in a combination of the two. A software module can exist in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. A storage medium can be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative manner, the storage medium can be integral with the processor. A software module can be a single instruction or many instructions, and can be distributed on several different code segments, between different programs, and across multiple storage media.

[0102] The method invented herein includes one or more actions for implementing the method described. Methods and / or actions can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions can be modified without departing from the scope of the claims.

[0103] The functions described can be implemented by hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as one or more instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device or any other tangible medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. As used herein, a disc includes a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disk and a blue disc.

[0104] Thus, a computer program product may perform the operations presented herein. For example, such a computer program product may be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which instructions may be executed by one or more processors to perform the operations described herein. The computer program product may include packaging materials.

[0105] Software or instructions may also be transmitted via a transmission medium. For example, the software may be transmitted from a website, server or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio or microwave.

[0106] In addition, the module and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station when appropriate. For example, such a device can be coupled to a server to facilitate the transmission of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage component (e.g., RAM, ROM, a physical storage medium such as a CD or a floppy disk, etc.) so that the user terminal and / or base station can obtain the various methods when being coupled to the device or providing a storage component to the device. In addition, any other appropriate technology for providing the methods and techniques described herein to a device can be utilized.

[0107] Other examples and implementations are within the scope and spirit of the present invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hard wiring, or any combination of these. Features that implement the functions can also be physically located in various locations, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, "or" used in the enumeration of items beginning with "at least one" indicates a separate enumeration, so that, for example, the enumeration of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the wording "exemplary" does not mean that the example described is preferred or better than other examples.

[0108] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present invention is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0109] The above description of the invented aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features of the present invention.

[0110] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the form invented herein. Although multiple example aspects and embodiments have been discussed, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for detecting deformation of an object, comprising: Acquire three-dimensional surface data of the object to be detected, and acquire at least one reference polishing direction on the surface of the object to be detected; Acquire the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; Acquire the polishing direction of each to-be-detected point according to the surface three-dimensional data and the at least one reference polishing direction; The characteristic value of each point to be detected is calculated according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and the deformation of the object to be detected is detected according to the characteristic value of each point to be detected.

2. The method of claim 1, wherein: Acquiring the three-dimensional surface data of the object to be detected includes: The surface of the object to be detected is scanned, and three-dimensional surface data of the object to be detected is obtained according to the scanning result.

3. The method of claim 1, wherein: The reference polishing direction is represented by at least two points to be detected on the surface of the object to be detected.

4. The method of claim 1, wherein: Acquiring the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data comprises: Acquire multiple adjacent points within a preset range around the first point to be detected; Acquire a fitting plane based on the first point to be detected and the adjacent points; The surface direction of the first to-be-detected point is acquired based on the fitting plane.

5. The method of claim 1, wherein: Acquiring the grinding direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data and the at least one reference grinding direction comprises: generating at least one sub-grinding direction according to at least one reference grinding direction; Obtaining the distance from the first to-be-detected point to each sub-polishing direction, and calculating the weight corresponding to the first to-be-detected point and each sub-polishing direction; The polishing direction of the first point to be detected is calculated according to each sub-polishing direction and the weight corresponding to the first point to be detected and each sub-polishing direction.

6. The method of claim 1, wherein: Calculating the characteristic value of each to-be-detected point according to the surface three-dimensional data, the surface direction of each to-be-detected point, and the polishing direction comprises: Generate a plane to be inspected according to the position of the first point to be inspected in the three-dimensional surface data, the surface direction of the first point to be inspected, and the grinding direction; Intersecting the plane to be detected and the surface to generate a section line to be detected, and obtaining a line segment to be detected including at least the first point to be detected on the section line to be detected; The characteristic value of the first point to be detected is calculated according to the surface direction and the polishing direction of each point to be detected on the line segment to be detected.

7. The method of claim 1, wherein: Detecting the deformation of the object to be detected according to the characteristic value of each point to be detected includes: Acquire a feature map of the surface of the object to be detected according to the feature value of each point to be detected; The defects on the surface of the object to be detected are segmented according to the feature map to obtain a detection result of the deformation of the object to be detected.

8. An object deformation detection device, comprising: an acquisition unit, configured to acquire three-dimensional surface data of the object to be detected, and acquire at least one reference polishing direction on the surface of the object to be detected; a surface direction acquisition unit, configured to acquire the surface direction of each to-be-detected point on the surface of the to-be-detected object according to the surface three-dimensional data; a polishing direction acquiring unit, configured to acquire the polishing direction of each to-be-detected point according to the surface three-dimensional data and the at least one reference polishing direction; The detection unit is configured to calculate the characteristic value of each point to be detected according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and detect the deformation of the object to be detected according to the characteristic value of each point to be detected.

9. An object deformation detection device, comprising: processor; and a memory having computer program instructions stored therein, Wherein, when the computer program instructions are executed by the processor, the processor is enabled to control the robotic arm, and the processor is capable of performing the following steps: Acquire three-dimensional surface data of the object to be detected, and acquire at least one reference polishing direction on the surface of the object to be detected; Acquire the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; Acquire the polishing direction of each to-be-detected point according to the surface three-dimensional data and the at least one reference polishing direction; The characteristic value of each point to be detected is calculated according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and the deformation of the object to be detected is detected according to the characteristic value of each point to be detected.

10. A computer-readable storage medium having computer program instructions stored thereon, wherein: When the computer program instructions are executed by a processor, the following steps are implemented: Acquire three-dimensional surface data of the object to be detected, and acquire at least one reference polishing direction on the surface of the object to be detected; Acquire the surface direction of each point to be detected on the surface of the object to be detected according to the surface three-dimensional data; Acquire the polishing direction of each to-be-detected point according to the surface three-dimensional data and the at least one reference polishing direction; The characteristic value of each point to be detected is calculated according to the surface three-dimensional data, the surface direction of each point to be detected and the polishing direction, and the deformation of the object to be detected is detected according to the characteristic value of each point to be detected.

Images