Product inspection system and method

Abstract

A product inspection system and method are provided. The product inspection system includes: a camera and a plurality of reflectors arranged around an inspection position, each reflector positioned to reflect light from different sides of the product toward the camera, so that the camera generates an inspection image of the product, the image comprising multiple sub-images of different sides of the product; a calibration member having a plurality of correction patterns having a predetermined pattern position relationship formed on different sides along a circumferential direction thereof, each correction pattern corresponding to a reflector when the calibration member is placed at the inspection position, so that each reflector reflects light from the corresponding correction pattern toward the camera, so that the camera generates a calibration image of the calibration member; and a computer that determines a relative mirror position relationship between the plurality of reflectors based on the predetermined pattern position relationship and the calibration image, and splices the plurality of sub-images based on the relative mirror position relationship to form a spliced ​​image of the product.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to product inspection, and more particularly, to a product inspection system and method capable of stitching multiple sub-images of different sides of a product to obtain a complete inspection image of the product. Background Art

[0002] Product inspection, such as visual inspection, is widely used in industrial production. It involves image acquisition, which means that images of various surfaces of the product can be taken as needed. The captured images can be used to train artificial intelligence systems (AI) and machine learning systems (ML), and can also be used in situations where the surface of the product needs to be inspected.

[0003] In conventional technology, there are two main solutions for capturing images of multiple surfaces or the entire periphery of a product: the first solution is to use a single camera to capture images of multiple surfaces or the periphery of the product around the product, or to capture images of multiple surfaces or the periphery of the product while flipping the product, but this will bring problems such as time-consuming, inconvenient operation, and low shooting efficiency; the second solution is to use multiple cameras to capture images of multiple surfaces or the periphery of the product at the same time, but this will lead to problems such as high cost and the need for a large operating space.

[0004] In addition, the images of multiple surfaces of the product obtained by these two solutions are independent of each other, and the positional relationship between them cannot be determined. It is difficult to correspond or associate the obtained images with specific features on the product (such as patterns, defects, actual parts, etc.). Summary of the Invention

[0005] The present disclosure is proposed to overcome at least one of the above and other problems and drawbacks of the prior art.

[0006] According to one aspect of the present disclosure, there is provided a product inspection system, comprising:

[0007] an image acquisition system comprising a plurality of reflectors and a camera disposed above the reflectors, the plurality of reflectors being arranged around a product to be inspected to be placed at an inspection position, each reflector being positioned to reflect light from a different side of the product along a circumferential direction thereof toward the camera, the camera being configured to receive the light reflected by the reflectors to generate an inspection image of the product, the inspection image comprising a plurality of sub-images of different sides of the product;

[0008] a calibration member having a plurality of calibration patterns formed on different side surfaces along a circumferential direction thereof, the plurality of calibration patterns having a predetermined pattern position relationship therebetween, each calibration pattern corresponding to one of the plurality of reflectors when the calibration member is placed at the inspection position, such that each reflector reflects light from the corresponding calibration pattern toward the camera, the camera being further configured to receive the light from the calibration member reflected by the reflector to generate a calibration image of the calibration member; and

[0009] A computer communicates with the image acquisition system to receive the inspection image and the calibration image, the computer being configured to determine a relative mirror position relationship between the plurality of reflectors based on the predetermined pattern position relationship and the calibration image, and to stitch the plurality of sub-images based on the determined relative mirror position relationship to form a single stitched image of the product, the relative position relationship of the plurality of sub-images on the stitched image being determined based on the determined relative mirror position relationship.

[0010] In some embodiments, the plurality of reflectors are arranged circumferentially around the product or calibration member placed at the inspection position, such that a reflective surface of each reflector faces one side of the product or one calibration pattern of the calibration member.

[0011] In some embodiments, each reflector is arranged so that its reflective surface is oriented obliquely to a corresponding side of the product or a corresponding correction pattern of the calibration member to reflect light from the corresponding side or correction pattern towards the camera.

[0012] In some embodiments, the calibration member is a columnar member adapted to be placed vertically at the inspection position, the plurality of correction patterns are identical to each other, and the centers of the plurality of correction patterns are spaced apart along the circumferential direction of the columnar member and located at the same height.

[0013] In some embodiments, each correction pattern comprises an array of dot patterns arranged in a plurality of rows and columns, and dot patterns in each correction pattern having the same relative position with respect to the corresponding center are located at the same height.

[0014] In some embodiments, the calibration image includes a plurality of sub-calibration images distributed in a circular shape, and each sub-calibration image is an image of one of the correction patterns obtained by the camera via reflection from the corresponding reflector.

[0015] In some embodiments, the computer includes an image processor configured to determine a relative positional relationship between two adjacent reflective mirrors corresponding to two adjacent sub-calibration images based on a relative positional relationship between the two adjacent sub-calibration images.

[0016] In some embodiments, the image processor is configured to: determine the relative positions along the y direction on the stitched image between two sub-images among the multiple sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images based on the position difference between the two adjacent sub-calibration images in the radial direction relative to the center of the calibration image; and determine the relative positions along the x direction on the stitched image between two sub-images among the multiple sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images based on the relative positions between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image.

[0017] In some embodiments, the image processor is configured to: determine a height difference between the two adjacent reflectors based on the position difference; and determine the relative positions of two sub-images formed by the two adjacent reflectors in the plurality of sub-images along the y-direction on the stitched image based on the determined height difference. And / or, the image processor is configured to: determine the relative horizontal position between the two adjacent reflectors based on the relative position between the two adjacent calibration sub-images in a circumferential direction relative to the center of the calibration image; and determine the relative positions of two sub-images formed by the two adjacent reflectors in the plurality of sub-images along the x-direction on the stitched image based on the determined relative horizontal position between the two adjacent reflectors.

[0018] In some embodiments, the computer includes: a display device configured to display the inspection image, the calibration image and the stitched image; and / or a storage device for storing at least one of the predetermined pattern position relationship, the inspection image, the calibration image and the relative mirror position relationship.

[0019] According to another aspect of the present disclosure, there is also provided a method for inspecting a product using the product inspection system described in any one of the embodiments of the present disclosure, comprising:

[0020] placing the calibration member at the inspection position, with the plurality of reflectors arranged around the inspection position;

[0021] receiving, by a camera, the light from the calibration member reflected by the reflector to generate a calibration image of the calibration member;

[0022] determining a relative mirror position relationship between the plurality of reflective mirrors based on the predetermined pattern position relationship and the calibration image;

[0023] placing the product at the inspection location;

[0024] receiving, by a camera, light reflected by a reflector from different sides of the product to generate an inspection image including a plurality of sub-images of different sides of the product;

[0025] The multiple sub-images are spliced ​​based on the determined relative mirror position relationship to form a spliced ​​image of the product, and the relative position relationship of the multiple sub-images on the spliced ​​image is determined based on the determined relative mirror position relationship.

[0026] In some embodiments, the product or calibration member is positioned so that the reflective surface of each reflector faces one side of the product or one calibration pattern of the calibration member to reflect light from the corresponding side or calibration pattern toward the camera.

[0027] In some embodiments, the calibration member is a columnar member adapted to be placed vertically at the inspection position, the plurality of calibration patterns are identical to each other, and the calibration member is placed such that centers of the plurality of calibration patterns are located at the same height.

[0028] In some embodiments, the calibration image includes a plurality of sub-calibration images distributed in a circular shape, and each sub-calibration image is an image of one of the correction patterns obtained by the camera via reflection from the corresponding reflector.

[0029] In some embodiments, the computer includes an image processor, and the method further includes: determining, by the image processor, a relative positional relationship between two adjacent reflectors corresponding to two adjacent sub-calibration images based on the relative positional relationship between the two adjacent sub-calibration images of the calibration image.

[0030] In some embodiments, the method further comprises:

[0031] determining, by the image processor, a relative position along the y direction on the stitched image between two sub-images formed by two adjacent reflecting mirrors corresponding to the two adjacent sub-calibration images, among the plurality of sub-images, based on a position difference between the two adjacent sub-calibration images in a radial direction relative to a center of the calibration image; and

[0032] The image processor determines, based on the relative positions of the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image, the relative positions along the x-direction on the stitched image of two sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images.

[0033] In some embodiments, determining the relative position along the y direction by the image processor based on the position difference includes: determining the height difference between the two adjacent reflectors based on the position difference; and determining the relative position along the y direction on the stitched image between two sub-images formed by the two adjacent reflectors in the multiple sub-images based on the determined height difference; and / or determining the relative position along the x direction by the image processor includes: determining the relative horizontal position between the two adjacent reflectors based on the relative position between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image; and determining the relative position along the x direction on the stitched image between the two sub-images formed by the two adjacent reflectors in the multiple sub-images based on the determined relative horizontal position between the two adjacent reflectors.

[0034] In some embodiments, the computer includes a display device and a storage device, and the method further includes: displaying the inspection image, the calibration image and the stitched image by the display device; and / or storing at least one of the predetermined pattern position relationship, the inspection image, the calibration image and the relative mirror position relationship by the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other aspects, features and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0036] Figure 1 is a block diagram schematically illustrating a configuration of a product inspection system according to an exemplary embodiment of the present disclosure;

[0037] Figure 2 is a perspective view schematically illustrating a configuration of a product inspection system according to an exemplary embodiment of the present disclosure, in which an arrangement of reflecting mirrors is shown;

[0038] Figure 3 is a side perspective view schematically illustrating a configuration of a product inspection system according to an exemplary embodiment of the present disclosure, wherein relative positions of respective mirrors are determined using a calibration member;

[0039] Figure 4 is a perspective view schematically illustrating a structure of a calibration member for a product inspection system according to an exemplary embodiment of the present disclosure;

[0040] Figure 5A is a planar schematic diagram showing an inspection image of a calibration member simulated or calculated based on the design arrangement of a product inspection system according to an exemplary embodiment of the present disclosure;

[0041] Figure 5B is based on Figure 5AA plan view of a stitched image of a calibration component obtained by using the inspection image in FIG.

[0042] Figure 6A is a plan view schematically illustrating an inspection image of a calibration member obtained using a product inspection system according to an exemplary embodiment of the present disclosure;

[0043] Figure 6B is based on Figure 6A A plan view of a stitched image of a calibration component obtained by using the inspection image in FIG.

[0044] Figure 7 is a schematic diagram illustrating a scaling factor for acquiring an image of a pattern of a calibration member according to an embodiment of the present disclosure;

[0045] Figure 8A is a planar schematic diagram illustrating an inspection image of a product obtained using a product inspection system according to an exemplary embodiment of the present disclosure;

[0046] Figure 8B is based on Figure 8A A plan view of a stitched image of a calibration component obtained by inspecting the image in FIG; and

[0047] Figure 9 is a flowchart illustrating a method of inspecting a product using a product inspection system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] The following is a detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. In this specification, the same or similar components are indicated by the same or similar reference numerals. The following description of the various embodiments of the present disclosure with reference to the accompanying drawings is intended to illustrate the overall concept of the present disclosure and should not be construed as a limitation of the present disclosure.

[0049] In addition, in the following detailed description, for ease of illustration, numerous specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments can be practiced without these specific details. In other cases, well-known structures and devices are shown in diagrammatic form to simplify the accompanying drawings.

[0050] like Figure 1-3 As shown, according to an exemplary embodiment of the present disclosure, a product inspection system, such as a visual inspection system, is provided, which is suitable for multi-side inspection or circumferential inspection of products of various shapes, such products can be in the form of columns or cylinders, polyhedrons, etc.

[0051] As shown in the figure, a product inspection system according to an embodiment of the present disclosure includes an image acquisition system 100 for acquiring multiple images of different sides or circumferences of a product 10. As an example, the image acquisition system 100 includes multiple reflectors 110 and a single camera 110. The camera 110 is, for example, disposed above the reflectors 110. The multiple reflectors 110 are arranged around a product 10 to be inspected, which is to be placed at an inspection position.

[0052] For example, a plurality of reflectors 110 are arranged in a generally annular shape to define an inspection area 101, and each reflector 110 (specifically, its reflective surface 111) is positioned or oriented to reflect light from a product 10 placed at a predetermined inspection position within the inspection area 101, such as light from different sides of the product 10 along its circumferential direction, toward the camera 120. In the illustrated embodiment, six reflectors 110 are provided, but the present disclosure is not limited thereto, and two, three, four, five, or more reflectors may be provided.

[0053] In some examples, the plurality of reflectors 110 are arranged to reflect light from the portion to be inspected of the product 10 so that light from all positions along the complete circumference of the portion to be inspected is reflected at least to the camera 120. In other words, the plurality of reflectors 110 are arranged or configured to face the complete circumference of the portion to be inspected of the product 10, or all positions along the entire circumference of the portion to be inspected of the product 10 can be imaged in the reflectors 110, i.e., the virtual image of the portion to be inspected formed in the reflectors includes the virtual image of the complete circumference of the portion to be inspected along the complete circumference of the portion to be inspected, so that a single camera 120 can "see" the complete circumference of the portion to be inspected of the product 10 through the plurality of reflectors 110. The camera 120 receives the light reflected by the reflectors 110 to generate an inspection image of the outer circumference of the portion to be inspected of the product.

[0054] For example, light from all peripheral surfaces of a section of the inspected portion of the product 10 along its axial direction can be captured by the camera 120 through reflection from the reflector 110. For a polyhedral product, the peripheral surfaces described here may refer to multiple side surfaces of the product that are adjacent to each other in the circumferential direction; for example, when the top or bottom surface of a hexahedron faces the camera, light from all four side surfaces of the hexahedron can partially or completely enter the camera lens through reflection from the reflector.

[0055] Thus, in the product inspection system according to the embodiment of the present disclosure, by setting up such a reflector, a single camera can obtain or generate views or images of multiple sides or a complete circle of the product through a single shot in a manner similar to a kaleidoscope, without the need for the camera and the product to move circumferentially relative to each other, and without the need to set up multiple cameras along the circumference of the product.

[0056] For example, the reflectors may be evenly spaced around the circumference of the portion of the product to be inspected so as to face different parts or sides of the portion of the product to be inspected in different orientations. The virtual images of the different parts or sides of the product formed in each reflector may be different from each other or partially overlap. As an example, the reflectors may have reflective surfaces that are oriented to face both the portion or side of the product to be inspected and the camera.

[0057] The camera 120 receives the light reflected by the reflector 110 to generate an inspection image 10 ′ of the product 10 (see FIG. Figure 8A ), the inspection image 10' includes multiple sub-images of different sides of the product 10, that is, each sub-image is associated with or corresponds to a different side of the product 10. The distribution of the acquired sub-images on the inspection image is associated with the arrangement of the reflectors, such as in a ring shape. Thus, with a single camera shot or image capture, it is possible to obtain images of multiple different sides or the entire circumference of the product from multiple perspectives. The obtained inspection images can be used to inspect the product, such as visually inspecting or identifying features (such as patterns, defects, etc.) on different sides or circumferences of the product.

[0058] In an exemplary embodiment, a plurality of reflectors 110 or their reflective surfaces 111 are arranged approximately symmetrically around the inspection position or the product 10 placed at the inspection position, such as being centrally symmetrical or rotationally symmetrical relative to the inspection position. Preferably, the reflectors 110 are identical to one another, specifically, the reflective surfaces 111 of the reflectors have the same shape (e.g., a plane mirror surface) and have the same relative position and orientation relative to the inspection position or the product. For example, as shown in the figure, the plurality of reflectors 110 are arranged in a circular shape and are evenly spaced apart from one another, the reflective surfaces 111 of the reflectors 110 have a rectangular shape (a rectangular plane mirror surface) and face the sample or camera at the same tilt angle and orientation, the centers of the reflective surfaces 111 of the reflectors 110 are located on the same circle, and so on, thereby achieving a symmetrical arrangement of the reflectors around the inspection position or the product 10 placed at the inspection position.

[0059] By leveraging the positional relationships between the reflectors, the relative positional relationships between the sub-images of different sides of the product captured by the camera through the reflectors can be determined, thereby enabling the location of features on the product to be determined based on the obtained inspection images. The positional relationships between the reflectors can be determined using a position detection device before inspecting the product. It will be appreciated that in actual use, the positions of the reflectors may be difficult to accurately detect or may vary due to various reasons.

[0060] In an exemplary embodiment according to the present disclosure, a calibration component can be used to determine the relative position relationship between the various reflectors. The product inspection system according to an embodiment of the present disclosure is also equipped with a calibration component 20, such as Figure 3 and4 As shown, the calibration member 20 is formed with a plurality of calibration patterns 21 (indicated by numerals 1, 2, 3, 4, 5, and 6 in the figure) on different side surfaces along its circumferential direction. These calibration patterns 21 have a predetermined pattern position relationship. In the illustrated embodiment, a plurality of reflectors 110 are arranged circumferentially around the product or calibration member placed at the inspection position, such that the reflective surface 111 of each reflector faces a side surface of the product or a calibration pattern of the calibration member. Each reflector 110 can be arranged so that its reflective surface 111 is oriented obliquely with respect to the corresponding side surface of the product 10 or the corresponding calibration pattern 21 of the calibration member 20, so as to reflect light from the corresponding side surface or calibration pattern toward the camera 120.

[0061] When the calibration member 20 is placed at an inspection position within the inspection area 101, each calibration pattern 21 corresponds to (e.g., faces) one reflector 110, so that each reflector 110 reflects light from the corresponding calibration pattern 21 toward the camera 120. The camera 120 receives the light from the calibration member 20 or its calibration pattern 21 reflected by the reflector 110 to generate a calibration image 20' of the calibration member 20. The calibration image 20' includes sub-calibration images of the respective calibration patterns 21 (see FIG. Figure 5A and 6A As shown in the figure, the calibration image 20' includes a plurality of sub-calibration images distributed in a circular shape.

[0062] The calibration image is also an inspection image, which is an image of different sides (such as the circumferential surface) of the calibration component obtained by the camera through reflection from a reflector. The sub-calibration image is the part of the calibration image that includes the image of the corresponding correction pattern (for example, an image of a correction pattern obtained by the camera through reflection from the corresponding reflector).

[0063] As an example, Figure 3 and 4 As shown, the calibration member 20 may include a cylindrical member, such as a cylinder, which is suitable for vertical placement at an inspection position, for example. A plurality of calibration patterns 22 are formed on the outer circumferential surface of the cylindrical member at intervals along the circumferential direction, for example, evenly spaced along the circumferential direction of the calibration member. Preferably, each calibration pattern 22 is identical to another. When the calibration member is vertically placed at the inspection position or when the calibration member is placed at the inspection position in a posture suitable for inspection, the centers of each calibration pattern 22 are spaced apart along the circumferential direction of the cylindrical member and are located at the same height, such as on the same circle or polygon.

[0064] In the illustrated embodiment, each calibration pattern 22 includes an array of dot patterns 22 arranged in multiple rows and columns, such as a 3×3 rectangular dot matrix. As an example, the dot pattern 22 may include protrusions, convex points, recesses, openings, etc., but the present disclosure is not limited thereto. In a state where the calibration member is placed vertically at the inspection position or when the calibration member is placed at the inspection position in a posture suitable for inspection, the dot patterns in each calibration pattern that have the same relative position relative to the corresponding center are located at the same height, or on the same circle or polygon. For example, preferably, in the illustrated embodiment, the dot patterns of the corresponding rows (first to third rows) of each calibration pattern are respectively located at the same height or on the same circle or polygon, the columns of each calibration pattern extend in the vertical direction, the dot patterns of each calibration pattern have the same spacing, and the spacing of the dot patterns of each calibration pattern is the same.

[0065] like Figure 1 and 2 As shown, the product inspection system according to an embodiment of the present disclosure further includes a computer 200 (such as a workstation), which communicates with the image acquisition system 100 (for example, communicates with the camera 120) to receive the inspection image 10' and the calibration image 20'. According to an exemplary embodiment of the present disclosure, the computer 200 is configured to determine the relative mirror position relationship between the corresponding multiple reflectors 110 based on the predetermined pattern position relationship between the respective correction patterns 21 of the calibration member 20 and the obtained calibration image 20', and stitch the multiple sub-images in the obtained inspection image 10' of the product 10 based on the determined relative mirror position relationship to form a single stitched image of the product 10, wherein the relative position relationship of the respective sub-images on the stitched image corresponds to or is consistent with the actual position relationship of the side surfaces of the product associated with the respective sub-images. For example, the arrangement order of the respective sub-images on the stitched image corresponds to or is consistent with the arrangement order of the side surfaces of the product associated with the respective sub-images in the circumferential direction, so that the features on the product (such as patterns, defects, etc., and their positions) can be inspected or determined based on the stitched image. In the illustrated embodiment, an xy coordinate system is established on the stitched image, with the sub-images arranged along the x-direction on the stitched image. For example, the x-direction on the stitched image may correspond to or be associated with the circumferential direction of the unfolded product, while the position along the y-direction corresponds to or is associated with the z-direction position in the XYZ coordinate system of the product inspection system.

[0066] According to an exemplary embodiment of the present disclosure, the relative positional relationship of each sub-image on the stitched image is determined based on the relative positional relationship between each reflector determined by means of a calibration component, so that the part of the product actually photographed by the camera can be determined based on the stitched image, that is, the each sub-image on the stitched image can be associated with the part of the product actually photographed by the camera, thereby accurately inspecting or determining (such as locating) the features on the product.

[0067] like Figure 1 As shown, the computer 200 may include an image processor 210 configured to determine a relative positional relationship between two adjacent reflective mirrors 110 corresponding to two adjacent sub-calibration images of the calibration image 20 ′ based on the relative positional relationship between the two adjacent sub-calibration images.

[0068] In an exemplary embodiment, the image processor 210 may be configured to determine the relative positions along the y-direction on the stitched image between two sub-images formed by two adjacent reflective mirrors 110 corresponding to the two adjacent sub-calibration images based on a position difference between the two adjacent sub-calibration images in a radial direction relative to the center of the calibration image 20'. For example, the image processor 210 may be configured to determine a height difference between the two adjacent reflective mirrors based on the position difference; and determine the relative positions along the y-direction on the stitched image between the two sub-images formed by the two adjacent reflective mirrors 110 based on the determined height difference.

[0069] The image processor 210 may also be configured to determine, based on the relative position between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image 20, the relative positions along the x-direction on the stitched image between two sub-images formed by two adjacent reflectors 110 corresponding to the two adjacent sub-calibration images. For example, the image processor 210 may be configured to determine, based on the relative position between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image 20', the relative horizontal position between the two adjacent reflectors 110; and, based on the determined relative horizontal position between the two adjacent reflectors 110, determine, based on the determined relative horizontal position between the two adjacent reflectors 110, the relative positions along the x-direction on the stitched image between the two sub-images formed by the two adjacent reflectors 110.

[0070] Below, we will refer to Figures 5A to 6B Provide an illustrative description of the stitching of the inspection images. Figure 5A It is a calibration image (ie, inspection image) of the calibration member 20 obtained through simulation or calculation based on the design arrangement (eg, ideal arrangement) of the product inspection system, on which an X'-Y' coordinate system can be established. Figure 5AThe illustrated inspection image may also correspond to a situation where the multiple reflectors 110 of the product inspection system maintain a desired (e.g., designed) or predetermined relative positional relationship. For example, the multiple reflectors 110 are arranged in a centrally or rotationally symmetrical manner relative to the inspection position. In this case, the individual sub-calibration images 21' (as distinguished by 1', 2', 3', 4', 5', and 6' in the figure) are centrally or rotationally symmetrical about point O on the obtained calibration image 20'. The centers O1-O6 of each sub-calibration image coincide with the corresponding center point pattern image 22' of the calibration pattern and lie on the same circle with radius R1. The sub-calibration images have the same angular spacing θ. In this case, an x'-y' coordinate system can be established for each sub-calibration image 21' according to the right-hand rule. Figure 5A The calibration image in the image matches the desired or designed mirror arrangement and can be used as a reference image. Based on the design arrangement of the product inspection system (including the design arrangement of the mirrors, the arrangement of the calibration pattern of the calibration member, etc.), this reference image can be pre-calculated or simulated and stored in a storage device for future use.

[0071] for Figure 5A The calibration image shown can be based on a predetermined pattern position relationship between the calibration patterns 21 of the calibration member 20, that is, the centers of the calibration patterns are located on the same circle or polygon or at the same height and at the same spacing (angular spacing), and the obtained sub-calibration images 21' are stitched together in the arrangement order on the calibration image 20' (e.g., clockwise or counterclockwise) to form a stitched image, so that the centers O1 to O6 of the sub-calibration images are located on the same straight line along the x direction on the stitched image and are spaced at the same distance, and the sub-calibration images are in the same orientation on the stitched image (e.g., the coordinate axes of the x'-y' coordinate systems are oriented in the same direction), as shown. Figure 5B shown.

[0072] Figure 6A is the relative position between the plurality of reflectors 110 of the product inspection system relative to Figure 5A When the situation changes, the calibration image (ie, inspection image) of the calibration component 20 is obtained. Figure 6A As shown, the center O1' of the sub-calibration image 21' (marked as "1'") obtained by the camera 120 of the calibration pattern 21 marked as "1" on the calibration member 20 and the centers O2 to O6 of the sub-calibration images 21' of other calibration patterns obtained by the camera 120 are located on different circles, that is, on a circle with a radius of R2. Figure 5A There is a radial position difference R2-R1 in the reference image, which indicates that the reflector corresponding to the correction pattern 21 marked as "1" on the calibration member 20 has a change in height relative to the adjacent reflector (in Figure 6A In the case of the mirror being raised relative to the adjacent mirror, the mirror reflects light from the upper portion of the calibration pattern 21 identified as "1" towards the camera. Figure 6B In the stitched image shown, sub-calibration image 21' (labeled "1'") obtained by camera 120 of calibration pattern 21, labeled "1" on calibration member 20, is displaced by a distance h in the y-direction relative to the other sub-calibration images to correspond to the change in mirror height. Thus, the relative height relationship between adjacent mirrors corresponding to these sub-calibration images can be determined based on the relative radial positions of the adjacent sub-calibration images in the calibration image.

[0073] exist Figure 6A , the sub-calibration image 21′ (identified as “4′”) obtained by the camera 120 of the calibration pattern 21 identified as “4” on the calibration member 20 is closer to the sub-calibration image 21′ (identified as “3′”) obtained by the camera 120 of the calibration pattern 21 identified as “3” on the calibration member 20, that is, the angular distance between the two is Figure 5A θ in the reference image of θ is changed to θ', which indicates that the reflector corresponding to the calibration pattern 21 marked as "4" on the calibration member 20 is shifted or moved closer to the reflector corresponding to the calibration pattern 21 marked as "3" on the calibration member 20 in the circumferential direction. Figure 6B In the stitched image shown, the distance d in the x-direction between the sub-calibration image 21' (labeled "4'") of the calibration pattern 21 labeled "4" on the calibration member 20 and the sub-calibration image 21' (labeled "3'") of the calibration pattern 21 labeled "3" on the calibration member 20 and obtained by the camera 120 is reduced relative to the x-direction distances between other adjacent sub-calibration images in the stitched image to correspond to the change in the circumferential spacing or angular spacing between the reflectors 110. Thus, the relative angular positional relationship between adjacent reflectors corresponding to the adjacent sub-calibration images can be determined based on the relative angular positions between the adjacent sub-calibration images of the calibration image.

[0074] Therefore, a mapping relationship can be established between the determined displacement h and spacing d and the relative position between the reflectors. Subsequently, for the inspection image of the product, based on the above-mentioned relative position relationship between the determined reflectors, the sub-images of the inspection image of the product are spliced ​​in the same way, such as Figure 8A and Figure 8BFor example, in a stitched image of a product, a sub-image formed by a reflector associated with a displacement h is shifted in the y direction by a distance equal to h relative to a sub-image formed by another or adjacent reflector, and the spacing between sub-images formed by adjacent reflectors associated with a spacing d is adjusted to d, thereby obtaining a stitched image that more accurately reflects or represents the image of the portion of the product actually captured by the camera.

[0075] When performing the above-mentioned stitching, the spacing between the centers of the two adjacent sub-calibration images or sub-images along the x-direction on the stitched image can be associated with or correspond to (e.g., equal to or proportional to) the arc length corresponding to the angular spacing between the two adjacent sub-calibration images or sub-images on the calibration image or inspection image, or the spacing along the x-direction can be associated with or correspond to the circumferential spacing or angular spacing between the centers of the sides of adjacent calibration patterns or products based on the scaling parameters of the image capture system (it can be understood that the scaling parameters can be calculated using known or conventional methods through the imaging parameters of the camera, the position relative to the reflector, the arrangement of the reflector, the position of the reflector relative to the calibration component or product, etc.). Similarly, the spacing between the centers of the two adjacent sub-calibration images or sub-images along the y-direction on the stitched image can be associated with or correspond to (e.g., equal to or proportional to) the radial spacing between the two adjacent sub-calibration images or sub-images on the calibration image or inspection image, or the spacing along the y-direction can be associated with or correspond to (e.g., equal to or proportional to) the angular spacing between the centers of the sides of adjacent calibration patterns or products based on the scaling parameters of the image capture system. Figure 2 and 3 The spacing in the height direction (Z direction) is shown.

[0076] In addition to the above-mentioned changes in height or angular position between the reflectors, the reflectors may also rotate or turn (such as around Figure 3 The X, Y or Z axis rotation or steering in the calibration component is inconsistent with the desired orientation. Such rotation or steering may, for example, cause the pattern in the sub-image of the product formed by means of the reflector or the sub-calibration image of the calibration component to be deformed, such as scaled. The scaling of the image or pattern may also be caused by the displacement of the reflector closer to or away from the inspected product. The image processor can process such deformation so that the stitched image matches the image on the product. For example, when the relative position relationship of the reflectors is determined by using the calibration component, the reflector associated with the pattern deformation can be determined, the deformation parameters (such as the scaling factor) of the sub-calibration image of the calibration component relative to the above-mentioned reference image can be calculated, and before the sub-images of the product are stitched together, the pattern in the sub-image formed by means of the reflector that causes the pattern deformation is subjected to a reverse deformation (such as reverse scaling) so that the pattern in the subsequent stitched image matches the actual pattern on the product.

[0077] The following reference Figure 7 This illustrates the situation where the reflector 110 corresponding to or facing the calibration pattern 21 marked as "1" on the calibration member 20 rotates around the Z axis. Figure 7 Part (b) shows that some patterns in the sub-calibration image are deformed. Figure 7 Part (a) of FIG shows the undeformed patterns in the corresponding reference images; for example, compared with the reference images in part (b), the images in the left column are enlarged in the x' and y' directions, while the images in the right column are enlarged in the x' and y' directions. Figure 7 The deformed image in part (b) of is compared with the undeformed image (such as the image in the middle column), or with Figure 7 By comparing with the corresponding reference image shown in part (a), it can be determined Figure 7 Part (b) shows the scaling factors Sx and Sy along the x' direction and the y' direction for each pattern or part on the image.

[0078] Subsequently, after obtaining the inspection image of the product, a reverse deformation is applied to the pattern in the sub-image formed by means of a reflector that causes the above-mentioned pattern deformation (for example, enlarging or reducing it in reverse proportion to the absolute values ​​of the scaling factors Sx and Sy), so that the pattern in the sub-image matches the actual pattern on the product (for example, in a similar shape or pattern).

[0079] Figure 9 The flowchart of a method for inspecting a product using a product inspection system according to an exemplary embodiment of the present disclosure is shown. As shown in the figure, the method includes the following steps:

[0080] S101: placing the calibration member 20 at the inspection position, and arranging a plurality of reflectors 110 around the inspection position;

[0081] S102: receiving, by the camera 120 , the light from the calibration member 20 reflected by the reflector 110 to generate a calibration image 20 ′ of the calibration member;

[0082] S103: determining a relative mirror position relationship between the plurality of reflective mirrors 110 based on a predetermined pattern position relationship between the plurality of calibration patterns of the calibration member and the generated calibration image;

[0083] S104: placing the product 10 at the inspection position;

[0084] S105: The camera 120 receives the light reflected by the reflector 110 from different sides of the product 10 to generate an inspection image 10' including a plurality of sub-images of different sides of the product 10 (see Figure 8A );

[0085] S106: Based on the determined relative mirror position relationship, multiple sub-images on the inspection image of the product are spliced ​​together to form a spliced ​​image of the product (see Figure 8B ), wherein the relative positions of the multiple sub-images on the stitched image are determined based on the determined relative mirror position relationship, so that the relative positions of the multiple sub-images on the stitched image are consistent with or consistent with the arrangement of the patterns, features or parts on the product actually photographed by the camera, thereby being able to more accurately and quickly locate or inspect the features on the product.

[0086] In the above method, the product 10 or calibration member 20 may be placed so that the reflective surface 111 of each reflector 110 faces one side of the product or one calibration pattern 21 of the calibration member to reflect light from the corresponding side or calibration pattern toward the camera 120 .

[0087] A suitable calibration member can be selected or designed, such as a columnar member suitable for vertical placement at the inspection position, on which multiple calibration patterns can be identical to each other. In operation, the calibration member is placed so that the centers of its multiple calibration patterns are at the same height.

[0088] Exemplarily, the obtained calibration image includes a plurality of sub-calibration images 21 ′ distributed in a ring shape, and each sub-calibration image is an image of one calibration pattern 21 obtained by the camera via reflection from a corresponding reflector 110 .

[0089] As described above, the computer of the product inspection system may be provided with an image processor, and in the above method, the image processor may determine the relative positional relationship between two adjacent reflectors corresponding to two adjacent sub-calibration images based on the relative positional relationship between the two adjacent sub-calibration images of the calibration image.

[0090] Illustratively, the image processor may determine the relative positions along the y-direction on the stitched image between two sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images based on the position difference between the two adjacent sub-calibration images in the radial direction relative to the center of the calibration image. The image processor may also determine the relative positions along the x-direction on the stitched image between two sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images based on the relative positions between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image.

[0091] In some examples, determining, by the image processor, the relative position along the y direction based on the position difference may include: determining a height difference between the two adjacent reflectors based on the position difference; and determining, based on the determined height difference, a relative position along the y direction on the stitched image between two sub-images formed by the two adjacent reflectors in the multiple sub-images.

[0092] In some examples, determining, by the image processor, the relative position along the x-direction may include: determining, based on the relative position between the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image, the relative horizontal position between the two adjacent reflectors; and determining, based on the determined relative horizontal position between the two adjacent reflectors, the relative position along the x-direction on the stitched image between two sub-images formed by the two adjacent reflectors in the multiple sub-images.

[0093] The computer of the product inspection system may further include a display device and / or a storage device in communication with the image processor. The display device may display the stitched image and / or the inspection image, calibration image, or reference image for operator viewing or processing. The storage device may store at least one of the predetermined pattern position relationship, the inspection image, the calibration image, the reference image, and the relative mirror position relationship.

[0094] Although the embodiments of the present disclosure have been shown and described, it will be understood by those skilled in the art that these embodiments may be varied without departing from the principles and spirit of the present disclosure, and that the scope of protection of the present disclosure is defined by the appended claims and their equivalents. It should also be noted that, unless otherwise indicated, the terms "comprise," "include," and "have" used herein do not exclude other elements or steps. In addition, any element number in a claim should not be construed as limiting the scope of protection of the present disclosure.

Claims

1. A product inspection system comprising: An image acquisition system (100) comprising a plurality of reflectors (110) and a camera (120) disposed above the reflectors, the plurality of reflectors being arranged around a product (10) to be inspected that is placed at an inspection position, each reflector being positioned to reflect light from a different side of the product along its circumferential direction toward the camera, the camera being configured to receive the light reflected by the reflectors to generate an inspection image (10') of the product, the inspection image comprising a plurality of sub-images of different sides of the product; a calibration member (20) having a plurality of calibration patterns (21) formed on different side surfaces along a circumferential direction thereof, wherein the plurality of calibration patterns have a predetermined pattern position relationship, and each calibration pattern corresponds to one of the plurality of reflectors when the calibration member is placed at the inspection position, so that each reflector reflects light from the corresponding calibration pattern toward the camera, the camera being further configured to receive light from the calibration member reflected by the reflector to generate a calibration image (20') of the calibration member; and A computer (200) is provided, which communicates with the image acquisition system to receive the inspection image and the calibration image, wherein the computer is configured to determine a relative mirror position relationship between the plurality of reflectors based on the predetermined pattern position relationship and the calibration image, and to stitch the plurality of sub-images based on the determined relative mirror position relationship to form a single stitched image of the product, wherein the relative position relationship of the plurality of sub-images on the stitched image is determined based on the determined relative mirror position relationship.

2. The product inspection system according to claim 1, wherein: When the product is placed at the inspection position, the plurality of reflectors are arranged to circumferentially surround the product so that a reflective surface (111) of each reflector faces one side of the product; In a state where the calibration member is placed at the inspection position, the plurality of reflectors are arranged to circumferentially surround the calibration member so that a reflective surface (111) of each reflector faces one calibration pattern of the calibration member.

3. The product inspection system according to claim 2, wherein: Each reflector is arranged so that its reflective surface is oriented obliquely to a corresponding side surface of the product or a corresponding correction pattern of the calibration member to reflect light from the corresponding side surface or correction pattern toward a camera.

4. The product inspection system according to claim 1, wherein: The calibration member is a columnar member adapted to be placed vertically at the inspection position, the plurality of correction patterns are identical to one another, and the centers of the plurality of correction patterns are spaced apart in a circumferential direction of the columnar member and located at the same height.

5. The product inspection system according to claim 4, wherein: Each calibration pattern comprises an array of dot patterns (22) arranged in a plurality of rows and columns, with dot patterns in each calibration pattern having the same relative position relative to the corresponding center being located at the same height.

6. The product inspection system according to claim 4, wherein: The plurality of correction patterns are evenly spaced apart along a circumferential direction of the calibration member, and / or centers of the respective correction patterns are located on the same circle.

7. The product inspection system according to any one of claims 1 to 6, wherein: The calibration image includes a plurality of sub-calibration images distributed in a circular shape, and each sub-calibration image is an image of one of the correction patterns obtained by the camera via reflection from the corresponding reflector.

8. The product inspection system according to claim 7, wherein: The computer includes an image processor (210) configured to: The relative positional relationship between two adjacent reflecting mirrors corresponding to two adjacent sub-calibration images is determined based on the relative positional relationship between the two adjacent sub-calibration images of the calibration image.

9. The product inspection system according to claim 8, wherein: The image processor is configured to: determining, based on a position difference between the two adjacent sub-calibration images in a radial direction relative to a center of the calibration image, a relative position along the y-direction on the stitched image between two sub-images of the plurality of sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images; as well as Based on the relative position of the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image, the relative position of two sub-images in the multiple sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images along the x-direction on the stitched image is determined.

10. The product inspection system according to claim 9, wherein: The image processor is configured to: determining a height difference between the two adjacent reflectors based on the position difference; as well as Determining a relative position along the y direction on the stitched image between two sub-images formed by the two adjacent reflectors in the plurality of sub-images based on the determined height difference; And / or, the image processor is configured to: determining a relative horizontal position between the two adjacent reflective mirrors based on a relative position between the two adjacent sub-calibration images in a circumferential direction relative to a center of the calibration image; as well as The relative positions of two sub-images formed by the two adjacent reflectors in the plurality of sub-images along the x-direction on the stitched image are determined based on the determined relative horizontal position between the two adjacent reflectors.

11. The product inspection system according to any one of claims 1-6, 8-10, wherein: The computer comprises: A display device (220) configured to display the inspection image, the calibration image, and the stitched image; and / or A storage device (230) is used to store therein at least one of the predetermined pattern position relationship, the inspection image, the calibration image and the relative mirror position relationship.

12. A method for inspecting a product using the product inspection system of claim 1, comprising: placing the calibration member at the inspection position, with the plurality of reflectors arranged around the inspection position; receiving, by a camera, the light from the calibration member reflected by the reflector to generate a calibration image of the calibration member; determining a relative mirror position relationship between the plurality of reflective mirrors based on the predetermined pattern position relationship and the calibration image; placing the product at the inspection location; receiving, by a camera, light reflected by a reflector from different sides of the product to generate an inspection image including a plurality of sub-images of different sides of the product; The multiple sub-images are spliced ​​based on the determined relative mirror position relationship to form a spliced ​​image of the product, and the relative position relationship of the multiple sub-images on the spliced ​​image is determined based on the determined relative mirror position relationship.

13. The method according to claim 12, wherein: In a state where the product is placed at the inspection position, the product is placed so that the reflective surface (111) of each reflector faces one side of the product to reflect light from the corresponding side toward a camera; In a state where the calibration member is placed at the inspection position, the calibration member is placed so that the reflective surface (111) of each reflector faces one calibration pattern of the calibration member to reflect light from the corresponding calibration pattern toward a camera.

14. The method according to claim 12, wherein: The calibration member is a columnar member adapted to be vertically placed at the inspection position, the plurality of calibration patterns are identical to each other, and The calibration member is placed so that centers of the plurality of correction patterns are located at the same height.

15. The method according to any one of claims 12 to 14, wherein The calibration image includes a plurality of sub-calibration images distributed in a circular shape, and each sub-calibration image is an image of one of the correction patterns obtained by the camera via reflection from the corresponding reflector.

16. The method according to claim 15, wherein The computer includes an image processor, and the method further includes: The image processor determines a relative positional relationship between two adjacent reflecting mirrors corresponding to two adjacent sub-calibration images based on the relative positional relationship between the two adjacent sub-calibration images of the calibration image.

17. The method according to claim 16, wherein The method further comprises: determining, by the image processor, a relative position along the y direction on the stitched image between two sub-images formed by two adjacent reflecting mirrors corresponding to the two adjacent sub-calibration images, among the plurality of sub-images, based on a position difference between the two adjacent sub-calibration images in a radial direction relative to a center of the calibration image; and The image processor determines, based on the relative positions of the two adjacent sub-calibration images in the circumferential direction relative to the center of the calibration image, the relative positions along the x-direction on the stitched image of two sub-images formed by two adjacent reflectors corresponding to the two adjacent sub-calibration images.

18. The method according to claim 17, wherein: Determining, by an image processor, the relative position along the y direction based on the position difference comprises: determining a height difference between the two adjacent reflectors based on the position difference; and Determining a relative position along the y direction on the stitched image between two sub-images formed by the two adjacent reflectors in the plurality of sub-images based on the determined height difference; And / or, determining, by the image processor, the relative position along the x-direction includes: determining a relative horizontal position between the two adjacent reflecting mirrors based on a relative position between the two adjacent sub-calibration images in a circumferential direction relative to a center of the calibration image; and The relative positions of two sub-images formed by the two adjacent reflectors in the plurality of sub-images along the x-direction on the stitched image are determined based on the determined relative horizontal position between the two adjacent reflectors.

19. The method according to any one of claims 12 to 14, 16 to 18, wherein: The computer includes a display device and a storage device, and the method further includes: displaying the inspection image, the calibration image, and the stitched image by the display device; and / or At least one of the predetermined pattern position relationship, the inspection image, the calibration image, and the relative mirror position relationship is stored by the storage device.

Images