Tone correction system and tone correction method

Abstract

The present invention relates to a tone correction system and a tone correction method. A tone correction system includes: a receiving unit that receives a target workpiece image and design information of a target workpiece; a reference image data storage unit that stores a reference workpiece image; an image determination unit that determines whether the target workpiece is tilted from the reference workpiece; a calculation processing unit that estimates a tilt angle of the target workpiece based on the stored reference workpiece image and the target workpiece image; a tone data storage unit that stores a relationship between the tilt angle and a tone correction value; an image correction unit that corrects the tone of the target workpiece image based on the estimated tilt angle and a tone correction formula corresponding to the stored relationship; and an output unit that outputs the corrected target workpiece image.

Description

Technical Field

[0001] The present disclosure relates to a tone correction system and a tone correction method. Background Art

[0002] In the process of manufacturing a vehicle or the like, when assembling a component to a workpiece, an inspection operation is performed to photograph the assembled component in the workpiece and use the photographed image to determine the component assembly position. In this operation, it is usually necessary to use a satisfactory image so that an inspector can determine the component assembly position by viewing the image. Techniques related to image processing - for example, Japanese Unexamined Patent Application Publication Nos. 2017-072945 and 2005-159650 - are known. Summary of the Invention

[0003] However, in a system for inspection work, when determining the component assembly position in a workpiece based on an image, there is a problem that it is impossible to detect defective / missing components at the manufacturing site because the tone of the component in the image (including any feature of the visually recognizable color) changes according to the inclination of the workpiece.

[0004] This is caused by changes in the distance from the light source and the angle relative to the light source in any component on the workpiece due to the inclination of the workpiece. Since the illuminance and reflectance of the incident light change with the inclination of the workpiece, the tone of the component changes according to the inclination of the workpiece. Therefore, just by looking at the workpiece in the image, the tone of the workpiece becomes different from the reference color, and it is difficult to distinguish the components and detect defective / missing components.

[0005] On the other hand, Japanese Unexamined Patent Application Publication No. 2017-072945 discloses the following technique: obtaining a photographed image of an object, generating color correction information for performing color correction to eliminate the color feature based on the color feature of the image displayed on the display unit when photographing the image, and correcting the photographed image based on the color correction information. Japanese Unexamined Patent Application Publication No. 2005-159650 discloses the following technique: processing an image obtained by photographing an object using a photographing device to generate a grayscale conversion table representing the relationship between the value (brightness) and density, and correcting the image based on the grayscale conversion table. However, in these prior arts, color correction cannot be performed by the same light source, nor can color correction be performed according to the inclination of the object.

[0006] The present disclosure has been made in view of such circumstances, and provides a color correction system and a color correction method capable of preventing or minimizing the tone change of an image caused by the inclination of a workpiece.

[0007] An exemplary aspect of the present disclosure is a tone correction system for image determination when determining the component assembly position of a workpiece based on an image. The tone correction system includes: a receiving unit configured to receive a target workpiece image obtained by photographing a target workpiece and design information of the target workpiece; a reference workpiece image storage unit configured to store a reference workpiece image obtained by photographing a reference workpiece; an image determination unit configured to determine whether the target workpiece is tilted from the reference workpiece based on the received target workpiece image and the received design information; a calculation processing unit configured to, when it is determined that the target workpiece is tilted, estimate the tilt angle of the target workpiece based on the stored reference workpiece image and the target workpiece image; a tone data storage unit configured to store the relationship between the tilt angle and the tone correction value; an image correction unit configured to correct the tone of the target workpiece image based on the estimated tilt angle and the tone correction formula corresponding to the stored relationship; and an output unit configured to transmit and / or display the corrected target workpiece image.

[0008] In this exemplary aspect, the calculation processing unit may be configured to compare the reference workpiece image with the target workpiece image and estimate the tilt angle of the target workpiece based on the difference between the contour of the reference workpiece and the contour of the target workpiece and the spectral diffraction amount of each vertex.

[0009] In addition, the calculation processing unit may be configured to calculate the coordinates of a point of the target workpiece in the direction of the first axis based on the illuminance of a point of the reference workpiece and the illuminance of a point of the target workpiece, and estimate the tilt angle of the target workpiece around the first axis according to the calculated coordinates.

[0010] In addition, the calculation processing unit may further be configured to estimate the tilt angles of the target workpiece around the second axis and the third axis based on the tilt angle around the first axis, the three-dimensional coordinates of a point of the reference workpiece, and the three-dimensional coordinates of a point of the target workpiece.

[0011] An exemplary aspect of the present disclosure is a tone correction method for image determination when determining the component assembly position of a workpiece based on an image. The tone correction method includes: receiving a target workpiece image obtained by photographing a target workpiece and design information of the target workpiece; storing a reference workpiece image obtained by photographing a reference workpiece; determining whether the target workpiece is tilted from the reference workpiece based on the received target workpiece image and the received design information; when it is determined that the target workpiece is tilted, estimating the tilt angle of the target workpiece based on the stored reference workpiece image and the target workpiece image; storing the relationship between the tilt angle and the tone correction value; correcting the tone of the target workpiece image based on the estimated tilt angle and the tone correction formula corresponding to the stored relationship; and transmitting and / or displaying the corrected target workpiece image.

[0012] According to the present disclosure, a color correction system and a color correction method capable of preventing or minimizing changes in image tone due to tilting of a workpiece can be provided.

[0013] The above and other objects, features, and advantages of the present disclosure will be more fully understood from the following detailed description and the accompanying drawings given by way of illustration only, and should not be considered as limiting the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram showing a schematic configuration of a workpiece inspection system according to an embodiment of the present disclosure;

[0015] Figure 2 is a block diagram showing a schematic configuration of a tone correction system according to an embodiment of the present disclosure;

[0016] Figure 3 is a flowchart showing a processing flow of a workpiece inspection system according to an embodiment of the present disclosure;

[0017] Figure 4 is a flowchart showing a processing flow of an image acquisition process according to an embodiment of the present disclosure;

[0018] Figure 5 is a flowchart showing a processing flow of an image correction process according to an embodiment of the present disclosure;

[0019] Figure 6 shows an example of detection of the width of a workpiece;

[0020] Figure 7 shows an example of image determination when a workpiece is tilted;

[0021] Figure 8 shows an example of image determination when a workpiece is not tilted;

[0022] Figure 9 is a flowchart showing a processing flow of a process for calculating a workpiece tilt angle and a tone correction formula according to an embodiment of the present disclosure;

[0023] Figure 10 shows an example of calculation of illuminance of an image;

[0024] Figure 11 shows the relationship between illuminance and distance;

[0025] Figure 12 shows an example of coordinates of a workpiece with tilt and a workpiece without tilt;

[0026] Figure 13Shows an example of hue data corresponding to the tilt angle; and

[0027] Figure 14 is a flowchart showing a processing flow of image display processing according to an embodiment of the present disclosure. Detailed Description of the Invention

[0028] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, the present disclosure is not limited to the following embodiments. For the sake of clarity of description, the following description and drawings have been appropriately omitted and simplified. In each drawing, the same elements are denoted by the same reference numerals, and repeated descriptions thereof are omitted as necessary.

[0029] Figure 1 is a block diagram showing a schematic configuration of a workpiece inspection system according to an embodiment of the present disclosure. Figure 2 is a block diagram showing a schematic configuration of a hue correction system (value / hue correction system) included in the workpiece inspection system. The workpiece inspection system 1 according to the present embodiment inspects the component assembly state of a workpiece based on a workpiece image obtained by photographing the workpiece to which components (parts) are assembled. The hue correction system 2 (determination PC 400) corrects the hue of the workpiece image according to the tilt of the workpiece in the workpiece inspection system 1. The workpiece according to the present embodiment is, for example, a workpiece for manufacturing a vehicle, but is not limited to a vehicle and may be a workpiece of any product.

[0030] As Figure 1 shown, the workpiece inspection system 1 includes a photographing device 100, a production plan PC 200, a design unit server 300, a determination PC 400, and an inspector PC 500. The photographing device 100, the production plan PC 200, the design unit server 300, the determination PC 400, and the inspector PC 500 are connected via a communication network such as a LAN (local area network) so that communication can be performed between them.

[0031] The photographing device 100 is a device (system) for photographing a workpiece to which components are assembled in response to a production instruction. The photographing device 100 includes a production instruction receiving unit 110, a workpiece photographing unit 120, and a photographing data transmitting unit 130. The production instruction receiving unit 110, the workpiece photographing unit 120, and the photographing data transmitting unit 130 may be constituted by separate devices, or the functions required for the photographing device 100 may be implemented by one or more devices.

[0032] The production instruction receiving unit 110 receives the production instruction of the workpiece from the production plan PC 200. The production instruction is an instruction for producing the workpiece and is also a shooting instruction for shooting the workpiece to be produced. The production instruction receiving unit 110 may be an information processing device such as a PC (personal computer) or a tablet terminal, or may be a shooting device such as a camera identical to the workpiece shooting unit 120.

[0033] The workpiece shooting unit 120 is a shooting device (shooting unit) for shooting (imaging) the workpiece, such as a camera. The workpiece shooting unit 120 is fixed to a predetermined position near the workpiece assembling device, shoots the workpiece to which the components are assembled in response to the reception of the production instruction, and generates a two-dimensional workpiece image including the workpiece.

[0034] The shooting data transmission unit 130 is a transmission unit for transmitting (i.e., transferring) the shooting data to the determination PC 400. In this example, as the shooting data, the workpiece image shot by the workpiece shooting unit 120, the vehicle type information of the workpiece, and the measurement part information of the workpiece are transmitted. The vehicle type information is information indicating the vehicle type (e.g., vehicle type or vehicle model) to be manufactured by the workpiece, and the measurement part information is information indicating the part of the workpiece (e.g., the part of the vehicle or the position of the part in the vehicle). The vehicle type information and the measurement part information are also workpiece information for identifying the workpiece. The vehicle type information and the measurement part information may be preset, or may be obtained from the workpiece assembling device or the like when shooting the workpiece. When the workpiece shooting unit 120 shoots the workpiece, the shooting data transmission unit 130 transmits the shot workpiece image, the vehicle type information, and the measurement part information to the determination PC 400. The shooting data transmission unit 130 may be an information processing device such as a PC or a server, or may be other transmission devices.

[0035] The production plan PC 200 manages the production plan of vehicles and the like and guides the production of workpieces. The production plan PC 200 includes a production instruction receiving unit 210 and a production instruction transmission unit 220. The production plan PC 200 is constituted by an information processing device such as a PC or a server, for example, and instead, the functions required by the production plan PC 200 may be implemented by multiple devices.

[0036] The production instruction receiving unit 210 receives the production instruction of the workpiece. For example, the production instruction receiving unit 210 receives (receives or accepts) the production instruction by being input or received. That is, the production instruction receiving unit 210 may be an input device for the manager to input the production instruction operation, or a receiving device (receiving apparatus) for receiving the production instruction from another device.

[0037] The production instruction transmission unit 220 transmits production instructions to the imaging device 100. When the production instruction receiving unit 210 receives production instructions, the production instruction transmission unit 220 transmits the received production instructions to the imaging device 100 and other devices.

[0038] The design unit server 300 manages design information of workpieces and components such as vehicles, and transmits the design information in response to requests from other devices and the like. The design unit server 300 includes a design information accumulation unit 310 and a design information transmission unit 320. The design unit server 300 is constituted by an information processing device such as a server or a PC, and alternatively, the functions required by the design unit server 300 may be implemented by multiple devices.

[0039] The design information accumulation unit 310 accumulates design information of workpieces and components. The design information accumulation unit 310 may be a storage device or a database inside or outside the design unit server 300, or may be a storage area in a storage device. The design information transmission unit 320 transmits the design information to the determination PC 400. In response to a request from the determination PC 400, the design information transmission unit 320 transmits the design information of the relevant workpiece or component stored in the design information accumulation unit 310 to the determination PC 400.

[0040] The determination PC 400 is a device for determining the tilt of a workpiece image and correcting the tone. The tone correction system 2 according to the present embodiment is constituted by the determination PC 400, for example, but other devices and functions may be provided as needed. As Figure 2 shown, the determination PC 400 includes a receiving unit 410, an accumulation unit 420, an image processing unit 430, and an output unit 440. The determination PC 400 is constituted by an information processing device such as a PC or a server, and alternatively, the functions required by the determination PC 400 may be implemented by multiple devices.

[0041] The receiving unit 410 receives information from the imaging device 100 and the design unit server 300. The receiving unit 410 includes a captured data receiving unit 411 and a design information receiving unit 412. The captured data receiving unit 411 receives the workpiece image, vehicle type information, and measurement part information transmitted from the imaging device 100. The design information receiving unit 412 receives the design information of the workpiece transmitted from the design unit server 300.

[0042] The accumulation unit 420 is an accumulation unit (storage unit) that accumulates data required for processing the workpiece image. The accumulation unit 420 can be a storage device or database inside or outside the PC 400, or can be a storage area in a storage device. The accumulation unit 420 includes an unprocessed data storage unit 421, a reference image data storage unit 422, and a tone data storage unit 423.

[0043] The unprocessed data storage unit 421 stores the workpiece image, vehicle type information, and measurement part information received from the imaging device 100 by the imaging data receiving unit 411. The unprocessed data stored in the unprocessed data storage unit 421 is data before or during processing by the image processing unit 430. The workpiece image of the unprocessed data includes a reference workpiece image and an inspection target workpiece image. The reference workpiece image is an image obtained by imaging a workpiece (reference workpiece) in a reference state (non-tilted workpiece) before inspection, and is an image used as a reference for tilting and correction. The inspection target workpiece image is an image obtained by imaging an inspection target workpiece during inspection. One or both of the inspection target workpiece image and the reference workpiece image may be referred to as the workpiece image. The reference image data storage unit 422 stores the reference workpiece image. The reference image data storage unit 422 stores, for example, an image determined as the reference workpiece image by the image processing unit 430. Alternatively, the reference image data storage unit 422 may directly store the image received from the imaging device 100. The tone data storage unit 423 stores the relationship between the pre-learned workpiece tilt angle and tone data (tone correction value).

[0044] The image processing unit 430 determines and corrects the workpiece image. The image processing unit 430 includes an image determination unit 431, a calculation processing unit 432, and an image correction unit 433. The image determination unit 431 determines whether the workpiece of the workpiece image stored in the unprocessed data storage unit 421 is tilted based on the design information received from the design unit server 300. For example, the image determination unit 431 determines whether the inspection target workpiece is tilted from the reference workpiece. The image determination unit 431 can determine the presence or absence of tilt of the inspection target workpiece of the inspection target workpiece image based on the reference workpiece of the reference workpiece image.

[0045] The calculation processing unit 432 estimates the workpiece tilt angle of the workpiece to be inspected based on the reference workpiece image stored in the reference image data storage unit 422 and the workpiece image to be inspected stored in the unprocessed data storage unit 421. When the image processing unit 430 determines that the workpiece to be inspected is tilted, the calculation processing unit 432 estimates the workpiece tilt angle of the workpiece to be inspected. The workpiece tilt angle is the three-dimensional tilt (tilt angle) of the workpiece to be inspected relative to the reference workpiece. The calculation processing unit 432 obtains a tone correction formula based on the relationship between the workpiece tilt angle and the tone data stored in the tone data storage unit 423. The image correction unit 433 corrects the tone of the workpiece image to be inspected using the estimated workpiece tilt angle and the tone correction formula.

[0046] The output unit 440 transmits or outputs the tone-corrected image whose tone has been corrected as a display. The output unit 440 includes a corrected image transmission unit 441 and a display unit 442. The output unit 440 may include both the corrected image transmission unit 441 and the display unit 442, or may include one of them. The corrected image transmission unit 441 is a transmission unit that transmits the tone-corrected image to the inspector PC 500. The display unit 442 is a display device that displays the image after tone correction, such as a liquid crystal display device. Note that the image before correction and the image after correction can be output. The reference workpiece image can be output together with the workpiece image to be inspected. For example, the image can be output in a manner that shows the presence or absence of the tilt of the workpiece and the tilt angle.

[0047] The inspector PC 500 is a device for the inspector to perform inspections based on the displayed workpiece image. As Figure 1 shown, the inspector PC 500 includes a receiving unit 510 and a display unit 520. The inspector PC 500 is constituted by an information processing device such as a PC or a server, and alternatively, the functions required for the inspector PC can be implemented by multiple devices. The receiving unit 510 receives the tone-corrected image from the determination PC 400. The display unit 520 is a display device that displays the received tone-corrected image, such as a liquid crystal display device.

[0048] Figure 3 is a flowchart showing the processing flow of the workpiece inspection system 1 according to the present embodiment. As Figure 3 shown, the workpiece inspection system 1 executes processing in the order of an image acquisition process (S101) for acquiring a workpiece image, an image correction process (S102) for correcting the acquired workpiece image, and an image display process (S103) for displaying the corrected workpiece image.

[0049] First, the image acquisition process in S101 will be described. Figure 4It is a flowchart showing the processing flow of image acquisition processing. The image acquisition processing is executed by the imaging device 100. Through this processing, when the imaging device 100 pre-captures a reference workpiece, it is determined that the PC 400 acquires a reference workpiece image. After that, when the imaging device 100 captures an inspection target workpiece, it is determined that the PC 400 acquires an inspection target workpiece image.

[0050] As Figure 4 shown, the production instruction receiving unit 110 receives a production instruction from the production plan PC 200 (S201). When the production plan PC 200 transmits a production instruction for a workpiece, the production instruction receiving unit 110 receives the production instruction (imaging instruction) for the workpiece transmitted from the production plan PC 200.

[0051] Next, the workpiece imaging unit 120 images the workpiece (S202). When the production instruction receiving unit 110 receives a production instruction for a workpiece, the workpiece imaging unit 120 images the workpiece to which the component is assembled. For example, the workpiece assembly device assembles the component to the workpiece in response to the production instruction, and the workpiece imaging unit 120 images the workpiece after the component assembly is completed to generate a workpiece image.

[0052] Next, the imaging data transmission unit 130 transmits the workpiece image, vehicle type information, and measurement part information to the determination PC 400 (S203). When the workpiece imaging unit 120 images a workpiece image, the imaging data transmission unit 130 acquires the vehicle type information and measurement part information (workpiece information) of the imaged workpiece, and transmits the workpiece image, vehicle type information, and measurement part information to the unprocessed data storage unit 421 of the determination PC 400. Then, the unprocessed data storage unit 421 of the determination PC 400 receives and stores the workpiece image, vehicle type information, and measurement part information via the imaging data receiving unit 411.

[0053] Next, the Figure 3 image correction processing in S102 shown will be described. Figure 5 It is a flowchart showing the processing flow of image correction processing. This image correction processing is executed by the determination PC 400. Through this processing, the reference workpiece image is compared with the inspection target workpiece, the workpiece tilt angle is estimated based on the difference between the contour of the inspection target workpiece and the contour of the reference workpiece and the spectral diffraction amount of each vertex, and then the hue of the component caused by the tilt of the workpiece is corrected.

[0054] As Figure 5 shown, the image determination unit 431 acquires the workpiece image, vehicle type information, and measurement part information (S301). The image determination unit 431 acquires the workpiece image, vehicle type information, and measurement part information that are acquired from the imaging device 100 and stored in the unprocessed data storage unit 421.

[0055] Next, the image determination unit 431 acquires the design information of the relevant workpiece from the design unit server 300 (S302). The image determination unit 431 acquires the dimensional information of the workpiece of the captured workpiece image from the design information accumulation unit 310 in the design unit server 300 via the design information reception unit 412. Specifically, the image determination unit 431 requests the design information of the workpiece corresponding to the vehicle type information and the measurement part information obtained together with the workpiece image from the design unit server 300, and acquires the dimensional information of the relevant workpiece.

[0056] Next, the image determination unit 431 detects the contour of the workpiece in the workpiece image (S303). For example, the image determination unit 431 performs edge detection processing on the acquired workpiece image to extract the contour of the workpiece (workpiece area) in the image. The workpiece area in the image may be referred to as the workpiece image.

[0057] Next, the image determination unit 431 detects the widths (detection sizes) of the workpiece in the workpiece image in the X-axis direction and the Y-axis direction (S304). For example, as Figure 6 shown, the image determination unit 431 obtains the maximum value (maxx) and the minimum value (minx) of the X coordinates of each point along the contour of the detected workpiece (workpiece area), and calculates the detection width (x m ) in the X-axis direction by adding the maximum value and the minimum value of the X coordinates, as shown in the following formula (1). Similarly, the image determination unit 431 obtains the maximum value (maxy) and the minimum value (miny) of the Y coordinates of each point along the contour of the detected workpiece, and calculates the detection width (y m ) in the Y-axis direction by adding the maximum value and the minimum value of the Y coordinates, as shown in the following formula (1).

[0058] [Formula 1]

[0059]

[0060] Next, the image determination unit 431 calculates the design widths (design sizes) of the workpiece in the image in the X-axis direction and the Y-axis direction based on the design information (S305). The image determination unit 431 obtains the design width (x d ) of the workpiece (workpiece area) in the image in the X-axis direction and the design width (y d)。For example, the camera parameters of the workpiece photographing unit 120 are used to photograph the workpiece, and the dimensions of the workpiece in the X-axis direction and the dimensions of the workpiece in the Y-axis direction in the design drawing are converted into the width of the workpiece in the X-axis direction and the width of the workpiece in the Y-axis direction in the image. Note that the width of the workpiece using the reference workpiece image can be used instead of the design width.

[0061] Next, the image determination unit 431 determines whether the difference between the detected width of the workpiece and the designed width of the workpiece is within the design dimensional tolerance (S306). In this way, it is determined whether the photographed workpiece is tilted. The image determination unit 431 determines the detected width (x m ) of the workpiece in the X-axis direction in the workpiece image and the designed width (x d ) and determines whether the difference is within the design dimensional tolerance (h x ), and also determines the detected width (y m ) of the workpiece in the Y-axis direction in the workpiece image and the designed width (y d ) and determines whether the difference is within the design dimensional tolerance (h y ). Specifically, it is determined whether the following formula (2) is satisfied. In formula (2), and (h with bar symbols x and h y ) are the average values of the design dimensional tolerances in the X-axis direction and the Y-axis direction.

[0062] [Formula 2]

[0063]

[0064] For example, when one of the formulas (2) is not satisfied, it is determined that the workpiece is tilted (tilted relative to the reference), and when both of the formulas (2) are satisfied, it is determined that the workpiece is not tilted (not tilted relative to the reference). In S306, when the difference between the detected width of the workpiece and the designed width is within the design dimensional tolerance (not tilted), the determined workpiece image is stored in the reference image data storage unit 422 as a reference workpiece image (S307). For example, after storing the workpiece image as a reference workpiece image, an output image is output, and the correction from S311 is not performed. If necessary, the correction process of S308 to S310 can be performed.

[0065] The above S301 to S306 are image determination processes for determining the presence or absence of tilt of the workpiece photographed by the image determination unit 431 based on the workpiece image. Figure 7An example of image determination processing is shown when the workpiece is tilted in the X-axis direction, Y-axis direction, and Z-axis direction. In this example, the shape of the workpiece is square in the X-Y plane of the image (from the plane view of the camera), but it is not limited to a square, and instead, any shape can be used. There is also no limitation on the shape in the Z-axis direction.

[0066] As Figure 7 shown, the difference between the line segment of the detected value of the workpiece in the X-axis direction and the line segment of the design value in the captured workpiece image, and the difference between the line segment of the detected value of the workpiece in the Y-axis direction and the line segment of the design value are calculated based on the detected value and the design value of the captured workpiece image. That is, in the image determination processing, as described above, the difference between the design width (x d ) and the detected width (x m ) is obtained. The design width (x d ) is the length of the line segment of the workpiece in the X-axis direction in the image obtained from the design information, and the detected width (x m ) is the length of the line segment of the workpiece in the X-axis direction detected from the workpiece image. Similarly, the difference between the design width (y d ) and the detected width (y m ) is obtained. The design width (x d ) is the length of the line segment of the workpiece in the Y-axis direction in the image obtained from the design information, and the detected width (y m ) is the length of the line segment of the workpiece in the Y-axis direction detected from the workpiece image.

[0067] In Figure 7 the example, considering that the difference in the image scaling ratio (x d -x m ) in the X-axis direction is greater than the design dimensional tolerance (h x ), and considering that the difference in the image scaling ratio (y d -y m ) in the Y-axis direction is greater than the design dimensional tolerance (h y ). Therefore, it is determined that the captured workpiece is tilted. In this case, in the subsequent processing, the coordinates of the points (e.g., G0 and G1) offset due to the tilt are obtained, and the three-dimensional tilt is obtained using these coordinates to perform tone correction.

[0068] Figure 8 An example of image determination processing is shown when the workpiece is not tilted. As Figure 7 shown, the difference between the design width (x d ) of the workpiece in the X-axis direction and the detected width (x m ) of the workpiece is obtained, and the difference between the design width (y d ) of the workpiece in the Y-axis direction and the detected width (ym ) between. In Figure 8 example, considering the difference in the image scaling ratio (x d -x m ) falls within the design dimension tolerance (h x ) in the X-axis direction, and considering the difference in the image scaling ratio (y d -y m ) falls within the design dimension tolerance (h y ) in the Y-axis direction. Therefore, it is determined that the photographed workpiece is not tilted. In this case, tone correction is not performed in subsequent processing.

[0069] As Figure 5 shown, when it is determined in S306 that the difference between the detected width and the design width of the workpiece is greater than the design dimension tolerance (workpiece tilt), the calculation processing unit 432 performs the tone correction in the following S308 to S310. First, the calculation processing unit 432 acquires a reference workpiece image and unprocessed data (S308). In this case, the unprocessed data is the inspection target workpiece image. That is, the calculation processing unit 432 acquires the reference workpiece image stored in the reference image data storage unit 422 and the inspection target workpiece image stored in the unprocessed data storage unit 421.

[0070] Next, the calculation processing unit 432 performs processing for calculating the workpiece tilt angle and the tone correction formula (S309). Figure 9 is a flowchart showing the processing flow for calculating the workpiece tilt angle and the tone correction formula (S309).

[0071] As Figure 9 shown, the calculation processing unit 432 calculates the illuminance at a point (any point) of the inspection target workpiece (S401). The calculation processing unit 432 extracts the workpiece (workpiece area) of the inspection target workpiece image and designates points along the contour of the extracted workpiece. For example, as Figure 10 shown, the calculation processing unit 432 designates the positions of the vertices R1 to R4 of the inspection target workpiece as the points. The calculation processing unit 432 obtains the hues A1 to A4, saturations B1 to B4, and values C1 to C4 of the vertices R1 to R4 from the HSV values (H: Hue, S: Saturation, V: Value (brightness)) of the inspection target workpiece image, for example, and calculates the illuminance of the vertices R1 to R4 based on the obtained values. Note that not only HSV values can be used, but also RGB values and values in other color spaces. Alternatively, the spectral diffraction amount (spectral diffraction illuminance) of each vertex can be obtained by other methods.

[0072] Next, the tilt angle of the target workpiece is estimated based on the difference between the contour of the reference workpiece and the contour of the target workpiece, as well as the spectral diffraction amount of each vertex. In this example, in the following S402 to S404, the three-dimensional tilt angle of the workpiece is estimated based on the illuminance (spectral diffraction amount) of the points of the workpiece and the transformation of the coordinates of the points (difference between the contours) caused by the tilt of the workpiece based on the two-dimensional image. Specifically, the calculation processing unit 432 calculates the Z-axis coordinate of a point (any point) of the workpiece to be inspected (S402). The calculation processing unit 432 compares the reference workpiece image with the workpiece image to be inspected, and calculates the Z-axis coordinate (for example, the coordinate in the first axis direction) at this point based on the relationship between the distance from the light source and the illuminance, and the illuminance at the point based on the reference workpiece image and the workpiece image to be inspected. For example, calculate the Z-axis coordinate of the vertex of the workpiece whose illuminance is obtained as described above. Here, it is assumed that the direction of the plane perpendicular to the X-axis direction (for example, the second axis direction) and the Y-axis direction (for example, the third axis direction) of the reference workpiece is the Z-axis direction, and light is emitted from a light source at a position in the Z-axis direction to the workpiece.

[0073] As Figure 11 shown, the illuminance at a point depends on the distance from the light source. When the illuminance at point P1 located at a distance d1 from the light source is defined as E1 and the illuminance at point P2 located at a distance d2 from the light source is defined as E2, the following formula (3) is satisfied. For example, the Z-axis coordinate of this point (any point) is obtained from the relational expression between the distance from the light source and the illuminance. Formula (3) is an example of the relational expression, and other formulas can be used.

[0074] [Formula 3]

[0075]

[0076] The distance (Z-axis coordinate) of each vertex of the reference workpiece in the reference workpiece image is a predetermined distance. Therefore, formula (3) can be used to obtain the Z-axis coordinate of one of the vertices of the workpiece to be inspected based on the illuminance of the vertices of the reference workpiece in the reference workpiece image and the illuminance of the vertices of the workpiece to be inspected in the workpiece image to be inspected.

[0077] Next, the calculation processing unit 432 estimates the rotation angle of the workpiece to be inspected around the Z axis (S403). The calculation processing unit 432 calculates the rotation angle θz around the Z axis based on the Z-axis coordinate of the point calculated as described above. For example, the position of the center point O is specified, and the angle around the Z axis between the line extending from the center point O to one of the vertices of the reference workpiece in the reference workpiece image and the line extending from the center point O to the corresponding vertex of the workpiece to be inspected in the workpiece image to be inspected is calculated.

[0078] Next, the calculation processing unit 432 estimates the rotation angle about the X-axis and the rotation angle about the Y-axis of the workpiece to be inspected (S404). For example, as Figure 12 shown, the three-dimensional coordinates of point G0 (any point) of the workpiece without inclination (reference workpiece) and the three-dimensional coordinates of point G1 (any point) of the workpiece with inclination (workpiece to be inspected) are obtained. The X-axis coordinate and the Y-axis coordinate can be obtained from the image plane, and the Z-axis coordinate can be calculated by the method described above. The rotation angle about the X-axis and the rotation angle about the Y-axis are obtained from the transformation from point G0 to point G1 caused by the inclination of the workpiece and the rotation angle θz calculated as described above. For example, using the following formula (4), the calculation processing unit 432 calculates the rotation angle θx about the X-axis and the rotation angle θy about the Y-axis based on the three-dimensional coordinates of the points of the workpiece with inclination and the corresponding points of the workpiece without inclination and the rotation angle about the Z-axis. Formula (4) is an example, and other formulas can be used. In this way, the three-dimensional inclination angle (θx, θy, θz) of the workpiece to be inspected is estimated.

[0079] [Formula 4]

[0080]

[0081] Next, the calculation processing unit 432 calculates a tone correction formula based on the correction value and the workpiece inclination angle (S405). As Figure 13 shown, tone data corresponding to the inclination angle at the point of the workpiece is learned in advance, and the inclination angle and the tone data associated with each other at that point are stored in the tone data storage unit 423. The stored tone data is the difference from the reference value and is a tone correction value for correction according to the inclination angle. For example, the tone data is the RGB value and the HSV value corresponding to the inclination angle (θx, θy, θz). Both the RGB value and the HSV value can be used as the tone data, or only one of them can be used as the tone data. Alternatively, values in other color spaces can be used as the tone data. For example, the tone is a color specified in a predetermined color space, and the tone correction device corrects the position in the predetermined color space. That is, the tone in the present embodiment includes elements (any elements) in RGB, HSV, or any other color space.

[0082] The calculation processing unit 432 calculates a tone correction formula based on the relationship between the inclination angle and the tone difference (tone data). Specifically, the tone correction formula U is obtained from the correction value W and the inclination angle θ using the following formula (5). The correction value W is based on the relationship between the difference between tones and the inclination angle, and is, for example, the correction value of the RGB value. For example, the tone correction formula for each point can be obtained.

[0083] [Formula 5]

[0084]

[0085] Next, as Figure 5 shown, the image correction unit 433 corrects the image based on the workpiece tilt angle and the tone correction formula (S310). The image correction unit 433 uses the workpiece tilt angle estimated by the calculation processing unit 432 and the tone correction formula of formula (5) to correct the tone of the inspection target workpiece image. For example, the tone correction formula for each point is used to correct the tone of the entire workpiece.

[0086] Next, the corrected image transfer unit 441 transfers the tone-corrected image to the inspector PC 500 (S311). The corrected image transfer unit 441 transfers the inspection target workpiece image corrected by the image correction unit 433 to the receiving unit 510 of the inspector PC 500. The corrected inspection target workpiece image is displayed on the display unit 442 of the determination PC 400.

[0087] Next, the image display process in S103 shown Figure 3 will be described. Figure 14 is a flowchart showing the processing flow of the image display process. This image display process is executed by the inspector PC 500.

[0088] As Figure 14 shown, the receiving unit 510 receives the corrected image from the determination PC 400 (S501). When the determination PC 400 corrects the tone of the inspection target workpiece image and then transfers the corrected inspection target workpiece image, the receiving unit 510 receives the corrected inspection target workpiece image. Next, the display unit 520 displays the received corrected image (S502). When the receiving unit 510 receives the corrected inspection target workpiece image, the display unit 520 displays the received corrected inspection target workpiece image. The inspector determines the component assembly position of the workpiece based on the displayed image.

[0089] As described above, in this embodiment, in an inspection system for determining the component assembly position in a workpiece based on an image, a reference image database (including tilted images) is prepared, and based on the reference image and the image obtained by photographing the inspection target workpiece, the tilt angle of the workpiece is estimated based on the difference between the contour of the reference workpiece and the contour of the inspection target workpiece caused by the tilt of the workpiece and the illuminance at each vertex. Then, the tone of the components caused by the tilt of the workpiece in the inspection target workpiece image is corrected. Therefore, when determining the component assembly position in the workpiece based on the image, even if the tone of the components changes due to the tilt of the workpiece, the tone of the image is corrected according to the tilt of the workpiece, making the image clear. Therefore, the component assembly position in the workpiece can be appropriately determined, and defective / missing components can be detected at the manufacturing site.

[0090] Note that the present disclosure is not limited to the above-described embodiments and can be appropriately modified without departing from its scope. For example, in the above embodiments, a tone correction system for detecting the presence or absence of inclination of a workpiece, estimating the inclination angle, and correcting the tone of the workpiece image has been described. However, it may be a device that detects the presence or absence of inclination of a workpiece and outputs it, or it may be a device that estimates and outputs the inclination angle of the workpiece.

[0091] Each configuration in the above embodiments may include hardware and / or software, may include one of hardware or software, or may include multiple pieces of hardware or software. The functions (processing) of each device can be implemented by a computer including a CPU (Central Processing Unit), a memory, etc. For example, a program for executing a method according to an embodiment (e.g., a tone correction method) may be stored in a storage device, and each function can be implemented by the CPU executing the program stored in the storage device.

[0092] Any type of non-transitory computer-readable medium can be used to store a program and provide it to a computer. Non-transitory computer-readable media include any type of tangible storage medium. Examples of non-transitory computer-readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Compact Disc Read-Only Memory), CD-R (Read-Only Memory), CD-R / W (Rewritable Compact Disc), and semiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory), etc.). Any type of transitory computer-readable medium can be used to provide a program to a computer. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. A transitory computer-readable medium can provide a program to a computer via a wired communication line (e.g., wires and optical fibers) or a wireless communication line.

[0093] From the disclosure thus described, it is obvious that the embodiments of the present disclosure can be changed in various ways. Such variations should not be regarded as departing from the spirit and scope of the present disclosure, and it is obvious to those skilled in the art that all such modifications are intended to be included within the scope of the appended claims.

Claims

1. A tone correction system for image determination when determining the component assembly position of a workpiece based on an image, the tone correction system comprising: A receiving unit configured to receive a target workpiece image obtained by photographing a target workpiece and design information of the target workpiece; A reference workpiece image storage unit configured to store a reference workpiece image obtained by photographing a reference workpiece; An image determination unit configured to determine whether the target workpiece is tilted from the reference workpiece based on the received target workpiece image and the received design information; A calculation processing unit configured to, when it is determined that the target workpiece is tilted, estimate the tilt angle of the target workpiece based on the stored reference workpiece image and the target workpiece image; A tone data storage unit configured to store the relationship between the tilt angle and the tone correction value; An image correction unit configured to correct the tone of the target workpiece image based on the estimated tilt angle and a tone correction formula corresponding to the stored relationship; And An output unit configured to transmit and / or display the corrected target workpiece image, wherein the calculation processing unit is configured to compare the reference workpiece image with the target workpiece image and estimate the tilt angle of the target workpiece based on the difference between the contour of the reference workpiece and the contour of the target workpiece and the spectral diffraction amount of each vertex; wherein the calculation processing unit is configured to calculate the coordinates of a point of the target workpiece in the first axis direction based on the illuminance of a point of the reference workpiece and the illuminance of a point of the target workpiece, and estimate the tilt angle of the target workpiece about the first axis according to the calculated coordinates; wherein the calculation processing unit is configured to estimate the tilt angles of the target workpiece about the second axis and the third axis based on the tilt angle about the first axis, the three-dimensional coordinates of a point of the reference workpiece, and the three-dimensional coordinates of a point of the target workpiece.

2. A tone correction method for image determination when determining the component assembly position of a workpiece based on an image, the tone correction method comprising: Receiving a target workpiece image obtained by photographing a target workpiece and design information of the target workpiece; Storing a reference workpiece image obtained by photographing a reference workpiece; Determining whether the target workpiece is tilted from the reference workpiece based on the received target workpiece image and the received design information; When it is determined that the target workpiece is tilted, estimating the tilt angle of the target workpiece based on the stored reference workpiece image and the target workpiece image; Storing the relationship between the tilt angle and the tone correction value; Correcting the tone of the target workpiece image based on the estimated tilt angle and a tone correction formula corresponding to the stored relationship; And Transmitting and / or displaying the corrected target workpiece image Estimating the tilt angle of the target workpiece includes comparing the reference workpiece image with the target workpiece image, and estimating the tilt angle of the target workpiece based on the difference between the contour of the reference workpiece and the contour of the target workpiece and the spectral diffraction amount of each vertex. Estimating the tilt angle of the target workpiece includes calculating the coordinates of the points of the target workpiece in the first axis direction based on the illuminance of the points of the reference workpiece and the illuminance of the points of the target workpiece, and estimating the tilt angle of the target workpiece around the first axis according to the calculated coordinates. Estimating the tilt angle of the target workpiece includes estimating the tilt angles of the target workpiece around the second axis and the third axis based on the tilt angle around the first axis, the three-dimensional coordinates of the points of the reference workpiece, and the three-dimensional coordinates of the points of the target workpiece.

Images