FINDING THE ARRANGEMENT OF AN OBJECT

Abstract

A system for determining the geometric orientation of an object within a coordinate system comprises a processor and a sensor array. The sensor array contains a variety of sensors oriented to scan a first and second edge of the object in order to identify the orientation of these edges within the coordinate system. The sensor array is electronically connected to the processor to signal the identified orientation of the object's first and second edges within the coordinate system. The processor is programmed to calculate the position of the intersection point of the first and second edges. From this calculated position of the intersection point, the processor determines the object's orientation within the coordinate system.

Description

PREVIOUS REGISTRATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 539,476, filed September 20, 2023, the contents of which are incorporated herein in their entirety. TECHNICAL FIELD

[0002] This application generally relates to finding the location of an object within a coordinate system. More specifically, this application relates to implementing sensors for accurately identifying the position of an object within the coordinate system for use in machine inspection of objects. BACKGROUND

[0003] Increasing the use of automation, and in particular sensor technology, to develop spatial recognition of an object within a coordinate system is intended to improve manufacturing efficiency, especially with regard to object inspection. Rigid objects are particularly suitable for inspection using various types of sensors. To inspect the assembly of components to a work surface of an object to verify assembly accuracy, the location of the object within a coordinate system of an inspection system should be estimated using a tool or set of tools. Previous inspection tools include digital cameras that use CCD and CMOS sensor arrays. Additional sensors capable of identifying geometric features can improve the estimation of the object's location within the defined coordinate system.These include laser sensors, lidar technology, radar technology, and the like, each of which can identify the distance of a surface from the sensor.

[0004] Inspection tools such as these are very useful in identifying the disposition of an object, and in particular, a stationary rigid body or an object that remains stationary during inspection and requires a single scan with the inspection tool. However, when the rigid body or object passes through the inspection system (i.e., a moving object), additional steps are required. First, detection of the object's presence within the inspection area is performed, involving obtaining the initial disposition or orientation of the rigid-body surface at the beginning of an inspection process. Second, the rigid body must be tracked to identify changes in its disposition or position over time to ensure accurate inspection.

[0005] Therefore, it is desirable for an inspection system to be able to accurately estimate the initial layout of the object to be inspected as part of an inspection process. An inspection surface requiring inspection defines corners and edges that establish a perimeter of the surface. Inspection of components attached to the work surface during an assembly process or other work performed, such as milling or machining contours or openings, is often necessary. It is particularly necessary to determine the initial layout of the work surface within a coordinate system so that the inspection is performed accurately and timely. For example, once the object to be inspected is located within a defined coordinate system, inspection of a work or assembly process can be initiated.This often requires the use of expensive and difficult-to-use fixtures. These fixtures often do not provide the accuracy needed to perform an inspection. Furthermore, movement of the object being inspected may require the inspection process to be repeated. Therefore, it would be desirable to provide an automated system that accurately locates an object to be inspected within a defined coordinate system. SUMMARY

[0006] A system for determining geometric location of an object within a coordinate system includes a processor and a sensor array. The sensor array includes a plurality of sensors oriented to scan a first edge and a second edge of an object to identify a location of the first edge and the second edge within a coordinate system. The sensor array is electronically connected to the processor to signal the identified location of the first edge and the second edge of the object within the coordinate system. The processor is programmed to calculate a position of an intersection of the first edge and the second edge. The processor calculates the location of the object within the coordinate system from the calculated position of the intersection.

[0007] Estimating the initial alignment and continuously estimating the alignment after defined or dynamic movement of an object within a defined coordinate system improves the speed and accuracy of an inspection process. Determining the intersection of a first edge and a second edge of the object, defining the intersection as an object reference point, and locating the reference point with the defined coordinate system allows the object to be tracked during dynamic or defined movement through an inspection cell at an accurate and rapid rate, enabling inspection of the object in a manner previously considered impractical. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Other advantages of the present invention will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which: Fig. shows a typical inspection surface with attached components; Fig. shows a graphical representation of corner detection; Fig. shows a schematic representation of the detection system of the present application; Fig. shows a schematic representation of an alternative embodiment of the invention of the present application, implementing a multi-sensor array; and Fig. shows a further schematic representation of the alternative embodiment of the invention of the present application, depicting an object in motion. DETAILED DESCRIPTION

[0009] Referring now to Fig. The scientific principles of the invention of the present application will now be explained. A disposition of an inspection surface within any given plane can be determined in two dimensions, i.e., within a two-dimensional coordinate system. Correlation or registration of a known point or datum on the inspection surface within the coordinate system of an inspection tool is determined by the orientation of the inspection surface within a common coordinate system defined between an inspection tool, i.e., sensors, cameras, and the like, and the inspection surface. It should be understood that reference to "inspection surface" within this application is made to the rigid-body surface of an object, but also includes a semi-rigid-body surface that is dimensionally stable.

[0010] If the shape of the rigid inspection surface takes the form of a polygon, any vertices of the polygon can be used as a known point to find the initial location of that surface within the common coordinate system. However, it should be understood that the overall shape of an inspection surface may include arcuate and nonlinear edges. It is desirable that at least two intersecting edges of the inspection surface be linear and form part of a polygon to implement the inspection method of the present invention. In this case, a vertex or intersection of the linear edges establishes a reference point by estimating the intersection or intersection of the adjacent linear edges.Identifying the intersection point as a datum as part of a computer-aided design (CAD) that adopts standardized general dimensioning and tolerancing (GD&T) principles, enables precise registration of the object with the desired coordinate system.

[0011] Referring now to Fig. An inspection tool 10 is provided with a view of a first edge 11 and a second edge 13. In this embodiment, the inspection tool 10 is a camera and defines a coordinate system defining at least two axes a and b. A location of each edge 11, 13 is estimated by the construction points 12 of the respective edge, from which an intersection point 14 is also estimated. A rigid construction of an inspection surface 16 enables use of the construction points 12 to determine the location of the inspection surface 16, as will be explained further herein. In this non-limiting example, the inspection surface 16 is shaped like a polygon. Any vertices defined by the polygon can be used as a known point to find the initial location of the surface within the coordinate system shared with the inspection tool 10.Thus, the position of a corner defined by the inspection surface 16 can be used as a reference point defined by the GD&T scheme as estimated as the intersection point of adjacent edges 11, 13 of the inspection surface 16 if the edges are substantially linear.

[0012] The position and orientation of each edge 11, 13 is obtained by identifying and locating at least two construction points 12 on each edge. Identifying and locating more than two construction points improves the estimation accuracy for each edge 11, 13. Identified construction points 12 on each edge 11, 13 are easily estimated using various tools including linear distance sensors 42 ( Fig. ), optical representation of the edge captured by a camera 10, point cloud data obtained using lidar, or even a depth camera. Each of these devices is within the scope of this invention and can be implemented either individually or in combination.

[0013] The process of determining the location of a rigid inspection surface is achieved by identifying a known corner point (reference point) 14, referred to within this application as "corner detection." A first embodiment is shown in Fig. , in which a schematic representation of the system for detecting the corner 14 of a rigid inspection surface 16 is shown generally at 20. The camera 10 is oriented to point downward in a perpendicular orientation to the inspection surface 16. The camera 10 is immovably mounted on a camera frame 17 above the inspection surface 16. The position and orientation of the camera 10 on the frame 17 are selected to ensure that the corner 14 defined by the inspection surface 16 is expected to be in a field of view 18 of the camera 10 that defines an area of ​​interest. In one embodiment, a black, non-reflective, and featureless background 22 is disposed at a position below the inspection surface 16, and more particularly below the field of view 18 in the area of ​​interest.In one embodiment, the field of view 18 of the camera 10 does not exceed the size of the background 22, even when the inspection surface 16 is not present. The visible contrast between the inspection surface 16 and the background 22 improves the accuracy of the measurements made by the camera 10 when comparing image pixels between different or consecutive images.

[0014] The camera 10 includes a camera lens 23 and a sensor 25. The sensor is intended to be a CCD or CMOS sensor and forms the sensor array as used in the context of this application. As explained further below, the sensor 25 generates image pixels used to determine the location of the inspection surface 16 within the defined coordinate system.

[0015] To achieve accurate estimation of the arrangement of the rigid-body surface 16 within the coordinate system by implementing an optical solution, the intrinsic and extrinsic parameters of the camera 10 should be known and predefined. The intrinsic parameters include, but are not limited to, horizontal and vertical focal lengths, the center of projection, and the distortion parameters of the lens 23. The extrinsic parameters include the position and orientation of the camera 10 on the frame 17 within the defined coordinate system.

[0016] The process by which the corner 14 of the inspection surface 16 is detected will now be explained. A difference image is generated from two consecutive images or frames taken by the camera 10 to capture both the stationary and moving object 27 situations that define the inspection surface 16 or the inspection services 16. A difference image is generated by subtracting pixels of a first image from pixels of a second image. The difference in pixel position between the first image and the second image indicates that movement of the inspection surface has occurred.

[0017] The difference image is known to contain noise, even in areas where the object 16 seen within the field of view 18 of the camera 10 has not changed or moved. To reduce noise, a number of Gaussian filters are applied to each image frame. The Gaussian filter is a linear filter that aids edge detection by replacing each image pixel with a weighted average of the values ​​of pixels within a rectangle or adjacent image area surrounding the pixel. Applying a Gaussian filter, even when a background 22 is present within the field of view 18, improves the accuracy of determining the location of the object 27 within the defined coordinate system.

[0018] The difference image, as set forth above, is obtained by subtracting corresponding pixels in two Gaussian-filtered images and retaining the absolute values ​​as the determined difference image. In one embodiment, the two Gaussian-filtered images are consecutive images.

[0019] An edge detection algorithm is then applied to the difference image by processor 48 to separately extract points on the first edge 11 and the second edge 13 of the inspection surface 16 by combing the image in the direction corresponding to each edge. By combing the image, processor 48 identifies construction points 12 that align linearly. Thus, construction points 12 are initially detected on each of the edges 11, 13.

[0020] From the initially detected construction points 12, a statistical best fit of a straight line through each set of construction points defined by each edge is estimated by the computer algorithm executed by a processor 48. From this estimate, a statistical best fit axis for each edge is established from the images.

[0021] An intersection point of the best fit lines defined by each of the first edge 11 and the second edge 13 is calculated to define the corner 14 or the reference point to the inspection surface in the image plane of the camera 10. It should be understood that the image plane in this embodiment is the two-dimensional plane perpendicular to the optical axis of the camera 10.

[0022] The now identified lines defined by the edges 11, 13, which intersect to form the corner 14 or reference point, are now transformed into the coordinate system defined by the camera 10 or another inspection system using intrinsic and extrinsic parameters. Thus, a common coordinate system is developed between the inspection surface 16 and the camera 10, enabling accurate initial location determination within the coordinate system of the inspection surface 16. In this non-limiting embodiment, the coordinate system defined by the camera 10 is a two-dimensional coordinate system along the axes a and b as in Fig. shown. Thus, the angular relationship of the lines defined by the construction points 12 on at least one of the edges 11, 13 to at least one of the axes a, b of the defined coordinate system establishes the orientation of the object 27 being inspected, while the calculated reference point (corner) 14 establishes the position of the object within the coordinate system.

[0023] An alternative embodiment is shown generally at 40 in Fig. . The alternative embodiment can be performed subsequently, concurrently with, or independently of the previous embodiment described herein. The alternative embodiment 40 is used for tracking a moving object 27 having a plurality of cooperative sensors 42. The system 40 includes a processor 48 that calculates an initial disposition of the object 44 from information received from the sensors 42. Additionally, the system 40 is also configured to track movement of the object 44 in a three-dimensional space by continuously calculating disposition as identified by the sensors 42. Thus, the system 40 is particularly useful for supporting automatic inspection of an object even when it is in motion, whether directional or dynamic motion.

[0024] The alternative system 40, in one embodiment, includes four sensors 42 for sensing in two dimensions. The sensors 42 are configured to sense the position of the object 44 to signal the processor 48 to calculate location, i.e., position and orientation of the object within the particular coordinate system. Location of the object 27 in this embodiment is defined by the position of a reference point 46 within a three-dimensional coordinate system (x, y, z coordinates) and its orientation or angle θ to the axes defined by the sensors 42. In one embodiment, three sensors 42a, 42b, 42c are used to measure distance to the object 27 in a specific direction, in this embodiment, a horizontal direction. Thus, the sensors 42a, 42b, 42c take the form of linear range sensors.A fourth sensor 42d measures distance in an alternative specific direction, for example, a vertical direction defined by a z-axis. Alternatively, the fourth sensor 42d is a range-scan sensor used to generate a horizontal profile of an edge 411, 13 of the object 44 being inspected. It should be understood that other sensors capable of detecting an edge 11, 13 to generate a horizontal profile are within the scope of this invention.

[0025] Sensors 42a and 42b are aligned along a horizontal X axis defined by the inspection coordinate system at predetermined, known positions. Sensors 42a and 42b are also positioned parallel to each other so that orientation sensing occurs in a common Z plane. Sensors 42a and 42b provide measurements of how far the object is in the Y direction from the sensors located on the X axis. Specifically, sensors 42a and 42b measure distance from the X axis to the second edge 13 of the object 44 to be inspected. The distance measurement to the construction points 12 on the second edge obtained by sensors 42a, 42b is used by processor 48 to calculate the orientation of object 44. The configuration of sensors 42a and 42b can be used to detect the existence of the object in the inspection area, as well as to determine the initial location.The initial location of the object 44 is identified when both sensors 42a and 42b register a valid measurement in the Y direction.

[0026] Sensor 42c is oriented at a predetermined angle and a known predetermined coordinate to the X and Y axes to ensure measurement from a predetermined side 13 of object 46. Sensor 42d is oriented to measure the location of object 44 in the Z direction. Therefore, sensor 42d is not arranged in a common plane with sensors 42a, 42b, and 42c and is positioned at a greater height at a known position in the Z direction of the coordinate system. Therefore, sensor 42d detects the spatial position of object 44 in a vertical direction when the other sensors 42a, 42b, and 42c detect the spatial position of object 44 in the horizontal direction.

[0027] The position of the object is estimated from the proposed sensor configuration, as in Fig. shown. Direction of movement of object 44 is shown in Fig. along arrow 50, so that sensor 42b registers measurement in the Y direction first. Sensor 42a also registers a measurement simultaneously identifying exact position of object 44 relative to sensor 42a at a precise time. A counter is included with processor 48 to establish a time at which measurements are taken. The measurement from sensor 42c at this time provides location of object 44 in the X direction of the coordinate system. The measurement from sensor 42a gives location of object 44 in the Y direction, while sensor 42d gives location of object 44 in the Z direction. At the same time, the processor calculates from 48 distance between sensor 42a and sensor 42b and the difference between measurement from sensor 42a and sensor 42b. Orientation θ of the object is calculated using the formula θ = arctan(dy / b).Thus, orientation of the object at any given time within the coordinate system is determined from the position of the construction points as measured by sensors 42a and 42b.

[0028] As mentioned above, the processor 48 is programmed to calculate the time it takes for the object 44 to move between sensor 42a and sensor 42b to determine the average velocity of the object 44. Average velocity is calculated rather than relying on a single velocity measurement and is used to correct for errors in the measurement or any sensor latency. Using this approach, the location of the object 44, represented by the reference point 46, is calculated using the algorithm set forth in the processor. If the object 44 is a rigid body, the estimated location of the reference point 46 within the common coordinate system is used to locate any component 52 disposed on the surface of the object 44 so that inspection of the accurate position of the component 52 can be verified.It should be understood that once the reference point 46 is located in the three-dimensional coordinate system, the computer-aided design (CAD) data is accessed to verify the accurate placement of the components 52 on the object 44. This is accomplished by processor alignment of the calculated coordinate system with the coordinate system of the CAD data.

[0029] After calculating the initial location of the object 44, continuous calculation can be implemented using the proposed configuration of sensors 42a, 42b, 42c, 42d. To achieve continuous calculation, sensor 42c is used to measure object location along the X direction according to a continuous direct view of the front part of the edge 46 of the object 44, which is the identified corner 46 or the reference point, as shown in Fig. shown. Thus, the distance x is calculated along the X axis from the sensor 42c to a line extending to the reference point 46, which is parallel to the Y axis. Sensors 42a and 42b provide measurements in the Y direction including the θ orientation. Measurement of the θ orientation of the object 44 is calculated in the same way as the initial location of the object 44. The angle θ is the angle of a line parallel to the Y axis and perpendicular to the X axis at the sensor 42a. Continuous measurement in the Y direction can be estimated from angle θ, distance Y1 and Y2, i.e. the distance measurements by sensors 42a and 42b shown in Fig.The calculation can be performed by tracing the extrapolation of a line defined by edge 46 of the object until it intersects the X axis. In a final measurement, sensor 42d provides a direct measurement of object 44 in the Z direction. The same configuration can also be used to terminate inspection when object 44 no longer registers measurements in sensors 42a and 42b, indicating that the object has passed through inspection system 40.

[0030] The invention has been described in an illustrative manner; many modifications and variations of the present invention are possible in light of the above teachings. It should therefore be understood that throughout the description, reference numerals are for convenience only and are not intended to be limiting in any way, and that the invention may be practiced otherwise than as specifically described. Therefore, the invention may be practiced otherwise than as specifically described within the scope of the appended claims following this first disclosed embodiment. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 539,476

[0001]

Claims

[1] A system for determining a geometric arrangement of an object within a coordinate system, comprising: a processor; a sensor array including a plurality of sensors oriented to scan a first edge and a second edge of an object, thereby identifying a location of the first edge and the second edge within a coordinate system; the sensor array is electronically connected to the processor to signal the identified arrangement of the first edge and the second edge of the object, and the processor is programmed to calculate a position of an intersection of the first edge and the second edge; and the processor calculates the arrangement of the object within the coordinate system from the calculated position of the intersection point. [2] The system of claim 1, wherein the sensor array includes a first arrangement of sensors located near the first edge of the object and a second arrangement of sensors located near the second edge of the object. [3] The system of claim 2, wherein the sensors comprise at least one of a distance sensor, a proximity sensor, and a camera sensor. [4] The system of claim 2, wherein the first arrangement of sensors includes a first proximal sensor and a first distal sensor each identifying a construction point on the first edge. [5] The system of claim 2, wherein the second arrangement of sensors includes a second proximal sensor and a second proximal sensor each identifying a construction point on the second edge. [6] The system of claim 1, wherein the sensor array includes a camera sensor disposed over the object for imaging a region of interest on the object. [7] The system of claim 6, wherein the camera sensor identifies the location of the first edge and the second edge of the object. [8] The system of claim 2, wherein the sensor array defines the coordinate system and the object is disposed within the coordinate system defined by the sensor array. [9] The system of claim 2, wherein the processor is programmed to correlate the detected position of the intersection of the first edge and the second edge with a reference point located within the coordinate system. [10] The system of claim 1, wherein the processor is programmed to correlate the reference point with objects defined on the object. [11] The system of claim 1, wherein the sensor array detects defined and dynamic movement of the object in the coordinate system from changes in the arrangement of the first edge and the second edge. [12] A method for identifying a location of an object within a coordinate system, comprising the steps of: Providing a sensor array and a processor for processing data received from the sensor array; Establish a coordinate system defined by the sensor array; Placing the object within the coordinate system defined by the sensor array; the sensor array detects a plurality of construction points on an edge of the object; the processor identifies a reference point of the object by extrapolating a line through the construction points to the reference point; and the processor determines an arrangement of the object within the coordinate system from the identified position of the construction points and the reference point. [13] The method of claim 12, wherein the step of the sensor array detecting a plurality of construction points on an edge of the object is further defined by the sensor array detecting construction points on a first edge and a second edge of the object. [14] The method of claim 13, wherein the step of extrapolating a line through the construction points is further defined by extrapolating a line through construction points detected on the first edge and the second edge and identifying the reference point by an intersection of the lines. [15] The method of claim 12, wherein the sensor array comprises a first distance sensor and a second distance sensor arranged in a parallel orientation along a first axis of the coordinate system and the first sensor detects a first distance to a first design point and the second sensor detects a second distance to a second design point. [16] The method of claim 15, further comprising a step of identifying movement of the object under investigation from the first sensor and the second sensor, detecting changes in detected distance to the first and second construction points. [17] The method of claim 12, wherein the sensor array comprises a camera and the camera detects the reference point from an image of an edge of a first edge and a second edge of the object. [18] The method of claim 12, further including a step of the sensor array identifying pixels from an image of the object. [19] The method of claim 18, wherein the processor identifies changes in pixels between successive images of the object, thereby calculating movement of the object. [20] The method of claim 12, wherein the step of identifying a coordinate system is further defined by identifying a three-dimensional coordinate system.

Images