Inspection device, computer program, and inspection method

Abstract

To easily inspect arranged positions of constituent parts constituting a portion of an object.SOLUTION: An inspection device identifies a constituent part in a drawing using drawing data that indicates the drawing of a portion that includes the constituent part, identifies the position of a reference region of an object in the drawing, and acquires dimension information indicated in the drawing using the drawing data, the identification result of the constituent part in the drawing, and the identification result of the position of the reference region in the drawing. The inspection device identifies a constituent part in a picked-up image using picked-up image data that indicates a picked-up image that includes at least the portion of the object that includes the constituent part, identifies the reference region of the object in the picked-up image, and assesses whether or not the arrangement position of the constituent part in the picked-up image is a specific arrangement position defined by the dimension information using the identification result of the constituent part in the picked-up image, the identification result of the reference region in the picked-up image, and the dimension information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] This specification relates to a technology for inspecting the placement position of a component that constitutes part of an object using captured image data, for example, a technology for inspecting the attachment position of a label affixed to a product using captured image data. [Background technology]

[0002] There are known techniques for inspecting an object using image data obtained by capturing an image of the object. For example, a coating state inspection device disclosed in Patent Document 1 inspects the coating state of a sealant applied to a workpiece using image data captured by an imaging device. The coating state inspection device identifies the coating area within the captured image and identifies the center line of the coating area as the coating trajectory. The inspection device determines a reference point and reference direction for the coating trajectory based on the center of gravity of the coating area, and uses the reference point and reference direction to correct any misalignment between the coating trajectory and a reference coating trajectory. After correcting the misalignment, the inspection device compares the coating trajectory with the reference coating trajectory to determine whether the coating state is normal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-83529 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above technology aims to determine whether the shape of the coating area itself is normal, and the positional deviation correction is performed to correct the positional deviation between the imaging device and the workpiece. Therefore, even if the above technology is applied to inspecting the position of a label affixed to a product, it cannot be said that the inspection of the label position can be easily performed. This problem is not limited to inspecting the position of a label affixed to a product, but is a common problem when inspecting the position of a component that constitutes a part of an object.

[0005] This specification discloses a technique for inspecting the placement position of a component that constitutes a part of an object, for example, a technique for easily inspecting the placement position of a label attached to a product. [Means for solving the problem]

[0006] The technology disclosed in this specification has been made to solve at least part of the above-mentioned problems, and can be realized as the following application examples.

[0007] [Application Example 1] An inspection device that inspects the attachment position of a label affixed to a product, comprising: a drawing acquisition unit that acquires drawing data showing a drawing of at least a portion of the product with the label affixed in a specific attachment position, the drawing data showing the portion including the label; a drawing label identification unit that uses the drawing data to identify the label in the drawing; a reference position identification unit that uses the drawing data to identify a position of a reference portion of the product in the drawing, the reference portion being a portion that serves as a reference for defining the specific attachment position of the label on the product; and a dimension acquisition unit that acquires dimensional information shown in the drawing using the drawing data, a result of identifying the label in the drawing, and a result of identifying the position of the reference portion in the drawing, the dimensional information being information that defines the positional relationship between the reference portion and the specific attachment position, and including a tolerance of the specific attachment position. an image capturing unit that captures image data obtained by capturing an image of the product with the label affixed thereto, the image data representing the captured image including at least a portion of the product that includes the label; an image capturing label identifying unit that identifies the label in the captured image using the captured image data; a reference portion identifying unit that identifies a reference portion of the product in the captured image using the captured image data; and a position determining unit that determines whether the label affixed position in the captured image is the specific affixing position defined by the dimensional information, using the result of identifying the label in the captured image, the result of identifying the reference portion in the captured image, and the dimensional information.

[0008] According to the above configuration, the position of the reference portion of the product in the drawing and the label in the drawing are identified using the drawing data, and dimensional information defining the positional relationship between the reference portion and the specific label attachment position is obtained using these identification results.Then, the label and the reference portion in the captured image are identified using the captured image data, and these identification results and the dimensional information are used to determine whether the label attachment position is the specific label attachment position.As a result, the label attachment position on a product can be easily inspected using the drawing data and the captured image data. [Application example 2] The inspection device according to Application Example 1, the dimensional information includes the tolerance and a design value of a dimension between the label and a reference portion, The position determination unit Identifying a dimension between the label in the captured image and the reference portion in the captured image; An inspection device that determines whether the specified dimension is within a range based on the design value and the tolerance, thereby determining whether the label attachment position in the captured image is the specified attachment position. [Application example 3] The inspection device according to Application Example 1, The position determination unit Identifying a difference between the position of the reference portion in the captured image and the position of the reference portion in the drawing in a state where the position and size of the label in the captured image and the position and size of the label in the drawing match, An inspection device that determines whether the label attachment position in the captured image is the specific attachment position by determining whether the difference between the position of the reference part in the captured image and the position of the reference part in the drawing is within the tolerance. [Application example 4] The inspection device according to any one of Application Examples 1 to 3, further comprising: a correction unit that executes a distortion correction process on the captured image data to correct distortion in the captured image, The position determination unit determines whether the label attachment position in the captured image is the specific attachment position using the captured image data that has been subjected to the distortion correction process. [Application example 5] The inspection device according to any one of Application Examples 1 to 4, further comprising: a size change unit that executes a size change process on at least one of the captured image data and the drawing data to change the size of the image so that the size of the label in the captured image matches the size of the label in the drawing; The position determination unit determines whether the label attachment position in the captured image is the specific attachment position using the at least one of the image data that has been resized. [Application Example 6] The inspection device according to any one of Application Examples 1 to 5, the label includes a first label and a second label; the specific attachment positions include a first attachment position to which the first label is to be attached and a second attachment position to which the second label is to be attached; the reference portion includes a first reference portion that serves as a reference for defining the first attachment position relative to the product, and a second reference portion that serves as a reference for defining the second attachment position relative to the product, the dimension information includes first dimension information defining a positional relationship between the first reference portion and the first attachment position, and second dimension information defining a positional relationship between the second reference portion and the second attachment position, the drawing label identification unit identifies the first label and the second label within the drawing; the reference position specifying unit specifies the position of the first reference portion and the position of the second reference portion within the drawing; the dimension acquisition unit acquires the first dimension information and the second dimension information shown in the drawing, the captured label identification unit identifies the first label and the second label in the captured image; the reference portion identification unit identifies the first reference portion and the second reference portion within the captured image; The position determination unit determines whether the attachment position of the first label in the captured image is the first attachment position, and determines whether the attachment position of the second label in the captured image is the second attachment position. [Application Example 7] The inspection device according to Application Example 6, the first label is affixed to a first surface of the product, and the second label is affixed to a second surface of the product facing in a different direction from the first surface; the drawing data includes first drawing data representing a two-dimensional drawing of a portion of the first surface that includes at least the first label, and second drawing data representing a two-dimensional drawing of a portion of the second surface that includes at least the second label; The captured image data includes first captured image data representing a captured image of a portion of the first surface that includes at least the first label, and second captured image data representing a captured image of a portion of the second surface that includes at least the second label. [Application Example 8] The inspection device according to any one of Application Examples 1 to 7, the drawing data is bitmap data, The reference position specifying unit Identifying a first extension line tangent to the label in the drawing by analyzing the drawing data; Identifying the location of the reference feature by identifying a second extension line parallel to the first extension line and spaced apart from the label in the drawing, the second extension line being attached to the reference feature; The dimension acquisition unit By analyzing the drawing data, numerical values ​​associated with the first extension line and the second extension line are identified; An inspection device that acquires the dimension information based on the numerical values. [Application Example 9] The inspection device according to Application Example 8, The dimension acquisition unit Identifying at least one of a dimension line associated with the first extension line and the second extension line and a terminal symbol of the dimension line; An inspection device that identifies the numerical value by searching within a range based on at least one of the dimension line and the terminal symbol.

[0009] [Application example 10an inspection device for inspecting the placement position of a component that constitutes a part of an object, the inspection device comprising: a drawing acquisition unit that acquires drawing data showing a drawing of at least a portion of the object in which the component is placed in a specific placement position; a drawing component identification unit that uses the drawing data to identify the component in the drawing; a reference position identification unit that uses the drawing data to identify a position of a reference part of the object in the drawing, the reference part being a part that serves as a reference for defining the specific placement position of the component with respect to the object; and a dimension acquisition unit that acquires dimensional information shown in the drawing using the drawing data, a result of identifying the component in the drawing, and a result of identifying the reference part in the drawing, the dimensional information being information that defines a positional relationship between the reference part and the specific placement position. an image acquisition unit that acquires captured image data obtained by imaging the object with the component formed thereon, wherein the captured image shown by the captured image data includes a portion of the object that includes at least the component; an image component identification unit that identifies the component in the captured image using the captured image data; a reference portion identification unit that identifies the reference portion of the object in the captured image using the captured image data; and a position determination unit that determines whether the placement position of the component in the captured image is the specific placement position defined by the dimension information, using the result of identifying the component in the captured image, the result of identifying the position of the reference portion in the captured image, and the dimension information.

[0010] According to the above configuration, the position of a reference portion of an object in the drawing and a component in the drawing are identified using drawing data, and dimensional information defining the positional relationship between the reference portion and a specific placement position is obtained using these identification results.The component and the reference portion in the captured image are then identified using captured image data, and these identification results and the dimensional information are used to determine whether the placement position of the component is the specific placement position.As a result, the placement position of a component that constitutes a part of an object can be easily inspected using drawing data and captured image data. [Application Example 11] The inspection device according to Application Example 10, the dimensional information includes the tolerance and a design value of a dimension between the component and a reference portion, The position determination unit Identifying a dimension between the component in the captured image and the reference portion in the captured image; An inspection device that determines whether the specified dimension is within a range based on the design value and the tolerance, thereby determining whether the placement position of the component in the captured image is the specified placement position. [Application Example 12] The inspection device according to Application Example 10, The position determination unit Identifying a difference between the position of the reference part in the captured image and the position of the reference part in the drawing in a state where the position and size of the component in the captured image match the position and size of the component in the drawing; An inspection device that determines whether the placement position of the component in the captured image is the specific placement position by determining whether the difference between the position of the reference part in the captured image and the position of the reference part in the drawing is within the tolerance. [Application Example 13] The inspection device according to any one of Application Examples 10 to 12, further comprising: a correction unit that executes a distortion correction process on the captured image data to correct distortion in the captured image, The position determination unit determines whether the arrangement position of the component in the captured image is the specific arrangement position using the captured image data that has been subjected to the distortion correction process. [Application Example 14] The inspection device according to any one of Application Examples 10 to 13, further comprising: a size change unit that executes a size change process on at least one of the captured image data and the drawing data to change the size of the image so that the size of the component in the captured image matches the size of the component in the drawing; The position determination unit determines whether the arrangement position of the component in the captured image is the specific arrangement position using the at least one of the image data that has been resized. [Application Example 15] The inspection device according to any one of Application Examples 10 to 14, the component includes a first component and a second component, the specific placement position includes a first placement position where the first component is to be placed and a second placement position where the second component is to be placed, the reference portion includes a first reference portion serving as a reference for defining the first placement position with respect to the object, and a second reference portion serving as a reference for defining the second placement position with respect to the object; the dimension information includes first dimension information defining a positional relationship between the first reference portion and the first arrangement position, and second dimension information defining a positional relationship between the second reference portion and the second arrangement position, the drawing component identification unit identifies the first component and the second component within the drawing; the reference position specifying unit specifies the position of the first reference portion and the position of the second reference portion within the drawing; the dimension acquisition unit acquires the first dimension information and the second dimension information shown in the drawing, the imaging component identification unit identifies the first component and the second component in the captured image; the reference portion identification unit identifies the first reference portion and the second reference portion within the captured image; The position determination unit determines whether the placement position of the first component in the captured image is the first placement position, and determines whether the placement position of the second component in the captured image is the second placement position. [Application Example 16] The inspection device according to Application Example 15, the first component is disposed on a first surface of the object, and the second component is disposed on a second surface of the object facing in a different direction from the first surface; the drawing data includes first drawing data representing a two-dimensional drawing of a portion of the first surface that includes at least the first component part, and second drawing data representing a two-dimensional drawing of a portion of the second surface that includes at least the second component part, The captured image data includes first captured image data that shows an captured image of a portion of the first surface that includes at least the first component, and second captured image data that shows an captured image of a portion of the second surface that includes at least the second component. [Application Example 17] The inspection device according to any one of Application Examples 10 to 16, the drawing data is bitmap data, The reference position specifying unit Identifying a first extension line tangent to the component in the drawing by analyzing the drawing data; Identifying the location of the reference feature by identifying a second extension line parallel to the first extension line and spaced from the feature in the drawing, the second extension line being attached to the reference feature; The dimension acquisition unit By analyzing the drawing data, numerical values ​​associated with the first extension line and the second extension line are identified; An inspection device that acquires the dimension information based on the numerical values. [Application Example 18] The inspection device according to Application Example 17, The dimension acquisition unit Identifying at least one of a dimension line associated with the first extension line and the second extension line and a terminal symbol of the dimension line; An inspection device that identifies the numerical value by searching within a range based on at least one of the dimension line and the terminal symbol.

[0011] The technology disclosed in this specification can be realized in various forms, such as an inspection device, an inspection method, a computer program for realizing these devices and methods, a recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing the configuration of an inspection system according to an embodiment of the present invention. [Figure 2] FIG. [Figure 3] FIG. 1 shows an example of a drawing. [Figure 4] 10 is a flowchart of an inspection preparation process. [Figure 5] 10 is a flowchart of an inspection process. [Figure 6] FIG. 2 is a diagram showing an example of a captured image represented by captured image data. [Figure 7] 10 is a flowchart of an inspection process according to a second embodiment. [Figure 8] FIG. 10 is an explanatory diagram of an inspection process according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] A. First Example A-1. Configuration of data generation device Next, an embodiment will be described based on an example. Fig. 1 is a block diagram showing the configuration of an inspection system 1000 of this example. The inspection system 1000 includes an inspection device 100 and an imaging device 400. The inspection device 100 and the imaging device 400 are connected so as to be able to communicate with each other.

[0014] The inspection device 100 is, for example, a computer such as a personal computer. The inspection device 100 includes a CPU 110 as a controller of the inspection device 100, a volatile storage device 120 such as RAM, a non-volatile storage device 130 such as a hard disk drive, an operation unit 150 such as a mouse or keyboard, a display unit 140 such as a liquid crystal display, and a communication unit 170. The communication unit 170 includes a wired or wireless interface for communicatively connecting to an external device, for example, an imaging device 400.

[0015] The volatile storage device 120 provides a buffer area for temporarily storing various intermediate data generated when the CPU 110 performs processing. The non-volatile storage device 130 stores a computer program PG for the inspection device.

[0016] The computer program PG is provided, for example, by the manufacturer of the inspection device 100. The computer program PG may be provided, for example, in a form downloaded from a server or stored on a DVD-ROM or the like. The CPU 110 executes the computer program PG to perform inspection preparation processing and inspection processing, which will be described later.

[0017] The imaging device 400 is a digital camera that optically captures an image of a subject to generate image data (also referred to as captured image data) representing the subject. The imaging device 400 generates the captured image data and transmits it to the inspection device 100 under the control of the inspection device 100. In this embodiment, the imaging device 400 is used to capture an image of the product 300 that is the inspection target of the inspection process, and generate captured image data that shows a captured image including the appearance of the product 300.

[0018] 2 is a perspective view of product 300. In this embodiment, product 300 is a printer having a substantially rectangular parallelepiped housing 30. During the manufacturing process, rectangular labels L1 and L2 are affixed to predetermined positions on the front surface 31 (the surface on the -Y side) and the right surface 32 (the surface on the +X side) of housing 30. These labels L1 and L2 bear various information such as the manufacturer's and product's brand logo, model number, lot number, etc.

[0019] The nonvolatile storage device 130 further stores drawing data DD1 and DD2. The drawing data DD1 and DD2 are image data representing drawings DI1 and DI2, respectively. In this embodiment, the drawing data is bitmap data representing an image including a plurality of pixels, and more specifically, is RGB image data that represents the color of each pixel using RGB values. The RGB values ​​are gradation values ​​of three color components (hereinafter also referred to as component values), i.e., color values ​​of the RGB color system including R, G, and B values. The R, G, and B values ​​are, for example, gradation values ​​of a predetermined number of gradations (e.g., 256).

[0020] FIG. 3 shows an example of a drawing. Drawings DI1 and DI2 are drawings used in instructions for the process of attaching labels L1 and L2. Drawing DI1 includes a line drawing BP1 of a portion of the front surface 31 of the housing 30 that includes label L1. Hereinafter, the label shown in line drawing BP1 will also be referred to as label BL1, with the symbol of the actual label L1 followed by a "B." The front surface of the housing shown in line drawing BP1 will also be referred to as front surface B31, with the symbol of the actual front surface 31 followed by a "B."

[0021] The diagram BP1 further includes extension lines A11a, A12a, A11b, and A12b for defining the position of the label L1 relative to the housing 30, dimension lines E1a and E1b, numerical values ​​N1a and N1b, and arrows R1a and R1b.

[0022] The extension line A11a is a line tangent to the right end of the label BL1 and indicates the horizontal position of the right end of the label BL1. The extension line A12a is a line drawn on the reference position BS1a on the front surface B31 and indicates the horizontal position of the reference position BS1a. The reference position BS1a in the diagram BP1 corresponds to the reference position S1a (FIG. 2) on the housing 30. The reference position is a part used as a reference for determining the position to attach the label. The reference position is a characteristic part that appears on the exterior of the housing 30, such as the boundary between multiple components that make up the housing 30, or a line that constitutes the design or outline of the housing 30. The two extension lines A11a and A12a extend vertically and are parallel to each other. The dimension line E1a is a line corresponding to the two extension lines A11a and A12a. Specifically, the dimension line E1a intersects perpendicularly with two extension lines A11a and A12a. A pair of arrows R1a are added to the intersections of the dimension line E1a and the two extension lines A11a and A12a. The arrows R1a are a type of terminal symbol. The numerical value N1a is a numerical value associated with the dimension line E1a and is located near the dimension line E1a.

[0023] The extension line A11b is a straight line tangent to the bottom end of the label BL1 and indicates the vertical position of the bottom end of the label BL1. The extension line A12b is a straight line drawn on the reference portion BS1b on the front surface B31 and indicates the vertical position of the reference portion BS1b. The reference portion BS1b on the front surface B31 corresponds to the reference portion S1b (FIG. 2) on the housing 30. Like the reference portion S1a, the reference portion S1b is a characteristic portion that appears on the exterior of the housing 30. The two extension lines A11b and A12b extend horizontally and are parallel to each other. The extension line E1b is a straight line corresponding to the two extension lines A11b and A12b. Specifically, the extension line E1b perpendicularly intersects with the two extension lines A11b and A12b. A pair of arrows R1b is added to the intersection of the dimension line E1b and the two extension lines A11b and A12b. A numerical value N1b is a numerical value associated with the dimension line E1b and is located near the dimension line E1b.

[0024] Drawing DI2 includes a line drawing BP2 of a portion of right side surface 32 of housing 30 that includes label L2. Hereinafter, the label shown in line drawing BP2 will also be referred to as label BL2, with the letter "B" added to the reference number of actual label L2. The right side surface of housing 30 shown in line drawing BP2 will also be referred to as right side surface B32, with the letter "B" added to the reference number of actual right side surface 32.

[0025] The diagram BP2 further includes extension lines A21a, A22a, A21b, and A22b for defining the position of the label L2 relative to the housing 30, dimension lines E2a and E2b, numerical values ​​N2a and N2b, and arrows R2a and R2b.

[0026] The extension line A21a is a straight line tangent to the right end of the label BL2 and indicates the horizontal position of the right end of the label BL2. The extension line A22a is a straight line drawn on the reference position BS2a on the right side B32 and indicates the horizontal position of the reference position BS2a. The reference position BS2a in the drawing DI2 corresponds to the reference position S2a (FIG. 2) of the housing 30. The reference position S2a is a characteristic portion that appears on the exterior of the housing 30, such as a line that constitutes the outline of the housing 30. The two extension lines A21a and A22a extend vertically and are parallel to each other. The dimension line E2a is a straight line corresponding to the two extension lines A21a and A22a. Specifically, the dimension line E1a perpendicularly intersects with the two extension lines A21a and A22a. A pair of arrows R2a are added to the intersection of the dimension line E2a and the two extension lines A21a and A22a. A numerical value N2a is a numerical value associated with the dimension line E2a and is located near the dimension line E2a.

[0027] The extension line A21b is a line tangent to the upper end of the label BL2 and indicates the vertical position of the upper end of the label BL2. The extension line A22b is a line drawn on the reference position BS2b on the right side surface B32 and indicates the vertical position of the reference position BS2b. The reference position BS2b in drawing DI2 corresponds to the reference position S2b (FIG. 2) of the housing 30. The two extension lines A21b and A22b extend horizontally and are parallel to each other. The dimension line E2b is a line corresponding to the two extension lines A21b and A22b. Specifically, the dimension line E2b intersects the two extension lines A21b and A22b perpendicularly. A pair of arrows R2b are added at the intersection of the dimension line E2b and the two extension lines A21b and A22b. The numerical value N2b is a numerical value associated with the dimension line E2b and is located near the dimension line E2b.

[0028] A-2. Inspection preparation process The inspection preparation process is a process for acquiring dimensional information to be used in the inspection process described later using the drawing data DD1 and DD2. The inspection preparation process is executed prior to the inspection process. Figure 4 is a flowchart of the inspection preparation process.

[0029] In S100, the CPU 110 selects one piece of target drawing data from the drawing data DD1 and DD2 stored in the nonvolatile storage device 130. In the following, a case where the drawing data DD1 indicating the drawing DI1 in FIG. 3(A) is the target drawing data will be mainly described as an example.

[0030] In S105, the CPU 110 identifies a label in the drawing indicated by the target drawing data (also referred to as the target drawing). For example, if the target drawing is the drawing DI1 in FIG. 3A, the label BL1 in the drawing DI1 is identified. For example, the CPU 110 displays the drawing DI1 on the display unit 140. The worker specifies the position of the label BL1 on the displayed drawing DI1 using a predetermined pointing device. The CPU 110 analyzes the vicinity of the specified position in the drawing DI1 and identifies a rectangle that includes the specified position. A known closed figure search process is used as the identification method. Alternatively, a method such as image recognition processing using pattern matching or a machine learning model may be used.

[0031] In S110, the CPU 110 identifies extension lines tangent to the identified label in the target drawing. For example, if the target drawing is drawing DI1 in FIG. 3A, extension lines A11a and A11b tangent to the label L1 are identified. For example, the CPU 110 identifies the extension line A11a extending vertically by searching for lines extending vertically near the left and right ends of the identified label L1. Furthermore, the CPU 110 identifies the extension line A11b extending horizontally by searching for lines extending horizontally near the top and bottom ends of the identified label L1.

[0032] In S115, the CPU 110 identifies extension lines that are paired with extension lines that are tangent to the labels of the target drawing. For example, if the target drawing is drawing DI1 in FIG. 3A, paired extension lines are identified for each of the two extension lines A11a and A11b that are tangent to the label L1. Specifically, the CPU 110 searches for a straight line parallel to the extension line A11a, starting from the position of the identified extension line A11a and moving away from the label BL1 (to the right in FIG. 3A). This identifies the extension line A12a that is paired with the extension line A11a. Identifying the extension line A12a is equivalent to identifying the horizontal position of the corresponding reference portion BS1a. Furthermore, the CPU 110 searches for a straight line parallel to the extension line A11b, starting from the position of the identified extension line A11b and moving away from the label BL1 (downward in FIG. 3A). This identifies the extension line A12b that pairs with the extension line A11b. Identifying the extension line A12b is equivalent to identifying the vertical position of the corresponding reference portion BS1b.

[0033] In S120, the CPU 110 identifies arrows associated with the identified extension lines in the target drawing. If the target drawing is drawing DI1 in FIG. 3(A), the CPU 110 identifies arrows associated with each of the extension lines A11a, A12a, A11b, and A12b. Specifically, the CPU 110 identifies a pair of arrows R1a by searching for arrows along each of the extension lines A11a and A12a. Furthermore, the CPU 110 identifies a pair of arrows R1b by searching for arrows along each of the extension lines A11b and A12b.

[0034] In S125, the CPU 110 identifies dimension lines associated with the identified arrows in the target drawing. If the target drawing is drawing DI1 in FIG. 3(A), dimension lines E1a and E1b associated with the arrows R1a and R1b are identified. Specifically, the CPU 110 identifies dimension line E1a by searching for a straight line connecting the pair of arrows R1a. Furthermore, the CPU 110 identifies dimension line E1b by searching for a straight line connecting the pair of arrows R1b.

[0035] In S130, the CPU 110 identifies the numerical values ​​associated with the identified dimension lines in the target drawing. If the target drawing is drawing DI1 in FIG. 3A, the CPU 110 identifies the numerical value N1a by searching for the numerical value along the dimension line E1a. Furthermore, the CPU 110 identifies the numerical value N1b by searching for the numerical value along the dimension line E1b.

[0036] In S135, the CPU 110 acquires horizontal and vertical dimension information based on the identified numerical values. The horizontal and vertical dimension information is acquired. The horizontal dimension information indicates the design value and tolerance of the horizontal dimension between either the left or right edge of the label and a reference portion located on either the left or right side of the label. The vertical dimension information indicates the design value and tolerance of the vertical dimension between either the vertical edge of the label and a reference portion located on either the top or bottom of the label.

[0037] Specifically, when the target drawing is drawing DI1 in FIG. 3A, CPU 110 recognizes the contents of the values ​​N1a and N1b by performing character recognition processing using a known optical character recognition (OCR) technique on the areas of the values ​​N1a and N1b. The values ​​N1a and N1b are written, for example, in a predetermined format: "design value ± tolerance (unit: mm)." Based on the value N1a, CPU 110 obtains dimensional information about the horizontal dimension d1a (FIG. 2) between the right edge of label L1 and the reference location S1a (X direction in FIG. 2). Based on the value N1b, CPU 110 obtains dimensional information about the vertical dimension d1b (FIG. 2) between the bottom edge of label L1 and the reference location S1b (Z direction in FIG. 2).

[0038] In S140, the CPU 110 registers the acquired horizontal and vertical dimension information. For example, if the target drawing is drawing DI1 in FIG. 3A, the horizontal and vertical dimension information is stored in the nonvolatile storage device 130 in association with drawing data DD1.

[0039] In S145, the CPU 110 determines whether all of the drawing data have been processed. If all of the drawing data have been processed (S145: YES), the CPU 110 ends the inspection preparation process. If there is unprocessed drawing data (S145: NO), the CPU 110 returns to S100.

[0040] As described above, the above inspection preparation process registers horizontal and vertical dimension information corresponding to label L1 for drawing data DD1. A similar process is also performed for drawing data DD2, registering horizontal and vertical dimension information corresponding to label L2. The horizontal dimension information corresponding to label L2 indicates the design value and tolerance of the horizontal dimension d2a (FIG. 2) between the right edge of label L2 and reference point S2a (Y direction). The vertical dimension information corresponding to label L2 indicates the design value and tolerance of the vertical dimension d2b (FIG. 2) between the top edge of label L2 and reference point S2b (Z direction).

[0041] A-3. Inspection processing The inspection process is a process for inspecting whether the labels L1 and L2 attached to the product 300 are attached in specific attachment positions. The specific label attachment positions are the positions indicated in the instructions for the label attachment process of the labels L1 and L2, i.e., the positions shown in the above-mentioned drawings DI1 and DI2.

[0042] 5 is a flowchart of the inspection process. The inspection process is performed for each product, and is started, for example, when the product is placed in a predetermined position where it can be imaged using the imaging device 400. In S200, the CPU 110 selects one target label from the labels L1 and L2 (FIG. 2) to be inspected. The following description will be given taking as an example a case where the label L1 is the target label.

[0043] In S205, the CPU 110 acquires captured image data representing a captured image including a target label. For example, in the example of FIG. 2, two labels L1 and L2 arranged on different sides of the housing 30 are the objects of inspection. For this purpose, the imaging device 400 includes a first imaging device (not shown) that captures an image of a portion including the label L1 on the front side 31 of the housing 30 in an imaging direction AR1 of FIG. 2, and a second imaging device (not shown) that captures an image of a portion including the label L2 on the right side 32 of the housing 30 in an imaging direction AR2. The imaging direction AR1 is the +Y direction, and the imaging direction AR2 is the −X direction. If the target label is label L1, the CPU 110 transmits an imaging instruction to the first imaging device and acquires captured image data from the first imaging device. The captured image data is bitmap data representing an image including a plurality of pixels, similar to drawing data, and is specifically RGB image data that represents the color of each pixel using RGB values.

[0044] In S208, distortion correction processing is performed on the acquired captured image data. The distortion correction processing includes, for example, trapezoidal distortion correction processing and lens distortion correction processing. The trapezoidal distortion correction processing is processing that corrects distortion of the subject in the captured image due to the shooting angle. The trapezoidal distortion correction processing is performed using a known algorithm, for example, a projective transformation function in OpenCV. The lens distortion correction processing is processing that corrects distortion that occurs in the subject in the captured image due to the lens of the imaging device. The lens distortion correction processing is performed using a known algorithm, for example, a distortion correction function in OpenCV.

[0045] FIG. 6 is a diagram showing an example of a captured image represented by captured image data. Captured image PI1 in FIG. 6(A) shows a portion including label L1 on front surface 31 of housing 30 of product 300. Captured image PI1 includes an image showing front surface 31 and an image showing label L1. Hereinafter, the label shown in captured image PI1 will be referred to as label PL1, with "P" added to the symbol of actual label L1. The front surface of the housing shown in captured image PI1 will also be referred to as front surface P31, with "P" added to the symbol of actual front surface 31. Similarly, the reference parts of the housing shown in captured image PI1 will also be referred to as reference parts PS1a and PS1b, with "P" added to the symbols of actual reference parts S1a and S1b (FIG. 2).

[0046] The captured image PI2 in FIG. 6(B) shows a portion of the right side surface 32 of the housing 30 of the product 300, including the label L2. The captured image PI2 includes an image showing the right side surface 32 and an image showing the label L2. Hereinafter, the label shown in the captured image PI2 will be referred to as the label PL2, with the letter "P" added to the reference number of the actual label L2. The right side surface of the housing shown in the captured image PI2 will also be referred to as the right side surface P32, with the letter "P" added to the reference number of the actual right side surface 32. Similarly, the reference parts of the housing shown in the captured image PI2 will also be referred to as the reference parts PS2a and PS2b, with the letter "P" added to the reference parts S2a and S2b.

[0047] In S210, the CPU 110 identifies a label PL1 in the captured image PI1. The label is identified using, for example, a method of object detection or semantic segmentation using a machine learning model. Examples of machine learning models that can be used include YOLO (You Only Look Once), SSD (Single Shot Detector), U-net, and SegNet. Alternatively, the label may be identified using another method such as pattern matching.

[0048] In S212, the CPU 110 acquires drawing data corresponding to the target label from the nonvolatile storage device 130. For example, if the target label is label L1 (FIG. 2), drawing data DD1 indicating drawing DI1 (FIG. 3A) used as instructions for the process of attaching label L1 is acquired. If the target label is label L2, drawing data DD2 indicating drawing DI2 (FIG. 3B) is acquired.

[0049] In S215, the CPU 110 performs pattern matching between the captured image and the drawing. For example, the CPU 110 performs pattern matching within the captured image PI1 of FIG. 6A using the drawing DI1 of FIG. 3A as a template. This identifies the positional relationship between the captured image PI1 and the drawing DI1. In the pattern matching of this step, for example, feature points P1, P2, and P3 (FIG. 3A) of the drawing DI1 and corresponding feature points P4, P5, and P6 (FIG. 6A) of the captured image PI1 are extracted using a known feature point extraction algorithm. Then, in the coordinate system of the captured image PI1, scaling, rotation, and translation processes are performed on the drawing DI1, and the positional relationship in which the feature points P1, P2, and P3 of the drawing DI1 match the corresponding feature points P4, P5, and P6 of the captured image PI1 is identified.

[0050] For example, in Fig. 6(A), drawing DI1 shown by a dashed rectangle is positioned relative to captured image PI1 in a positional relationship determined by pattern matching. In Fig. 6(B), drawing DI2 shown by a dashed rectangle is positioned relative to captured image PI2 in a positional relationship determined by pattern matching.

[0051] In S220, the CPU 110 identifies reference portions in the captured image corresponding to extension lines in the drawing. For example, in FIG. 6(A), extension lines A11a, A11b, A12a, and A12b are shown in a drawing DI1 indicated by a dashed line. In FIG. 6(B), extension lines A21a, A21b, A22a, and A22b are shown in a drawing DI2 indicated by a dashed line. As described above, of these extension lines, extension lines A12a, A12b, A22a, and A22b correspond to reference portions of the housing, and extension lines A11a, A11b, A21a, and A21b correspond to edges of the label. The positions in the captured image corresponding to these extension lines are identified by the pattern matching in S215. When the target label is label L1, the CPU 110 searches for linear edges parallel to the extension lines A12a and A12b in the vicinity of the extension lines A12a and A12b in the captured image PI1 of Fig. 6(A), and identifies the searched edges as reference portions. As a result, the reference portions PS1a and PS1b corresponding to the extension lines A12a and A12b are identified in the captured image PI1. When the target label is label L2, the CPU 110 identifies the reference portions PS2a and PS2b corresponding to the extension lines A22a and A22b in the captured image PI2 of Fig. 6(B).

[0052] In S225, the CPU 110 identifies edges of labels in the captured image that correspond to extension lines in the drawing. For example, the CPU 110 identifies edges of the labels identified in S210 that are located near the extension lines. As a result, if the target label is label L1, the right and bottom edges of label PL1 that correspond to extension lines A11a and A11b are identified in the captured image PI1. If the target label is label L2, the right and top edges of label PL2 that correspond to extension lines A21a and A21b are identified in the captured image PI2.

[0053] In S230, the CPU 110 calculates the dimension between the reference portion and the edge of the label in the captured image. For example, the CPU 110 counts the number of pixels between the reference portion and the edge of the label, and calculates the dimension (e.g., in mm) based on the number of pixels. In this embodiment, the internal parameters (e.g., focal length and optical center), external parameters (e.g., camera placement position), and field of view of the image capture device 400 are known, and the positional relationship between the product 300 and the image capture device 400 is fixed. For this reason, the relationship between the number of pixels and the dimension can be specified as an equation. The CPU 110 can convert the number of pixels into the dimension based on these equations.

[0054] When the target label is label L1, the horizontal dimension d1a between the reference portion PS1a and the right edge of label PL1 and the vertical dimension d1b between the reference portion PS1b and the bottom edge of label PL1 are calculated in the captured image PI1. When the target label is label L2, the horizontal dimension d2a between the reference portion PS2a and the right edge of label PL2 and the vertical dimension d2b between the reference portion PS2b and the top edge of label PL2 are calculated in the captured image PI2.

[0055] In S235, CPU 110 determines whether the horizontal and vertical dimensions between the label and the reference portion are within a dimensional range. The dimensional range is a range defined by the dimensional information registered for each label in the above-described inspection preparation process (FIG. 4).

[0056] For example, when the target label is label L1, the registered horizontal dimension information is the design value "10" and the tolerance "±3", and the vertical dimension information is the design value "8" and the tolerance "±2". Therefore, when the horizontal dimension d1a satisfies 7 < d1a < 13 and the vertical dimension d1b satisfies 6 < d1b < 10, the dimensions in the horizontal and vertical directions are determined to be within the dimension range. When the horizontal dimension d1a does not satisfy 7 < d1a < 13 or the vertical dimension d1b does not satisfy 6 < d1b < 10, the dimensions in the horizontal and vertical directions are determined not to be within the dimension range.

[0057] When the target label is label L2, the registered horizontal dimension information is the design value "16" and the tolerance "±3", and the vertical dimension information is the design value "8" and the tolerance "±2". Therefore, when the horizontal dimension d2a satisfies 13 < d2a < 19 and the vertical dimension d2b satisfies 6 < d2b < 10, the dimensions in the horizontal and vertical directions are determined to be within the dimension range. When the horizontal dimension d2a does not satisfy 13 < d2a < 19 or the vertical dimension d2b does not satisfy 6 < d2b < 10, the dimensions in the horizontal and vertical directions are determined not to be within the dimension range.

[0058] When the dimensions in the horizontal and vertical directions are within the dimension range (S235: YES), at S240, the CPU 110 determines that the attachment position of the target label is the designed position. When at least one of the dimensions in the horizontal and vertical directions is not within the dimension range (S235: NO), at S245, the CPU 110 determines that the attachment position of the target label is not the designed position.

[0059] In S250, CPU 110 determines whether all labels have been processed as target labels. If there are unprocessed labels (S250: NO), CPU 110 returns to S200. If all labels have been processed (S250: YES), in S255, CPU 110 outputs the determination result. For example, CPU 110 displays the determination result for each label on display unit 140. Once the determination result is output, the inspection process ends.

[0060] According to the present embodiment described above, the CPU 110 acquires drawing data DD1 indicating a drawing DI1 of a portion of the product 300 in which the label L1 is attached at a specific attachment position (in this embodiment, the design attachment position), the drawing data DD1 indicating at least the portion including the label L1 (S100 in FIG. 4, S212 in FIG. 5). The CPU 110 uses the drawing data DD1 to identify the label BL1 in the drawing DI1 (S105 in FIG. 4). The CPU 110 uses the drawing data DD1 to identify the extension lines A12a and A12b in the drawing DI1 (S115 in FIG. 4), thereby identifying the positions of the reference portions BS1a and BS1b in the drawing DI1. The CPU 110 acquires dimensional information indicated in the drawing DI1 using the drawing data DD1, the result of identifying the label BL1 in the drawing DI1, and the result of identifying the positions of the reference portions in the drawing DI1 (in this embodiment, the result of identifying the extension lines A12a and A12b) (S120 to S135 in FIG. 4). Furthermore, the CPU 110 acquires captured image data obtained by capturing an image of the product 300 with the label L1 attached thereto (S205 in FIG. 5). The CPU 110 uses the captured image data to identify the label PL1 in the captured image PI1 (S210 in FIG. 5). The CPU 110 uses the captured image data to identify the reference positions PS1a and PS1b in the captured image PI1 (S220 in FIG. 5). The CPU 110 uses the result of identifying the label PL1 in the captured image PI1, the result of identifying the reference positions PS1a and PS1b in the captured image PI1, and the dimensional information to determine whether the affixing position of the label PL1 in the captured image PI1 is a specific affixing position (in this embodiment, the affixing position according to the design) defined by the dimensional information (S225 to S245 in FIG. 5). As a result, the affixing position of the label L1 affixed to the product 300 can be easily inspected using the captured image data and the drawing data DD1. For example, there is no need to create inspection logic for each label, and various labels can be inspected simply by preparing drawing data for each label. Also, in this embodiment, drawing data DD1 indicating drawing DI1 (FIG. 3(A)) used as instructions for the process of attaching label L1 is used, so preparation of drawing data is also easy.

[0061] Furthermore, according to this embodiment, the dimensional information includes design values ​​and tolerances of the dimensions between the label L1 and the reference portions S1a and S1b (FIG. 3A). The CPU 110 identifies the dimensions between the label PL1 in the captured image PI1 and the reference portions PS1a and PS1b in the captured image PI1 (S225 and S230 in FIG. 5), and determines whether the identified dimensions are within the dimensional ranges based on the design values ​​and tolerances, thereby determining whether the affixing position of the label PL1 in the captured image PI1 is a specific affixing position (S235 to S245 in FIG. 5). As a result, the affixing position of the label L1 can be appropriately inspected based on the design values ​​and tolerances.

[0062] Furthermore, according to this embodiment, the CPU 110 performs distortion correction processing on the captured image data to correct distortion in the captured image PI1 (S208 in FIG. 5). The CPU 110 uses the distortion-corrected captured image data to determine whether the affixing position of the label PL1 in the captured image PI1 is a specific affixing position (S210 to S245 in FIG. 5). In this way, by using the distortion-corrected captured image data, it is possible to accurately determine whether the affixing position of the label L1 is a specific affixing position. For example, even if an image is captured with the front surface 31 of the housing 30 not perpendicular to the imaging direction AR1 of the imaging device 400 but tilted, it is possible to accurately determine whether the affixing position of the label L1 is a specific affixing position.

[0063] Furthermore, according to this embodiment, the labels to be inspected include label L1 and label L2, and the attachment positions of label L1 and label L2 are specified based on different reference portions (FIG. 3). CPU 110 acquires dimensional information for each of label L1 and label L2 (S135, S145, etc. in FIG. 4). CPU 110 executes S200 to S245 of the inspection process in FIG. 5 for each of label L1 and label L2. As a result, the attachment positions of the multiple labels L1 and L2 can be appropriately inspected.

[0064] Furthermore, according to this embodiment, label L1 is affixed to the front surface 31 of the housing 30, and label L2 is affixed to the right side surface 32 of the housing 30, which faces in a different direction from the front surface 31. Drawing data used in the inspection preparation process and the inspection process includes drawing data DD1 representing a two-dimensional drawing DI1 of a portion of the front surface 31 that includes at least label L1, and drawing data DD2 representing a two-dimensional drawing DI2 of a portion of the right side surface 32 that includes at least label L2 (see, e.g., FIGS. 1 and 3). Captured image data used in the inspection process includes captured image data representing a captured image PI1 of a portion of the front surface 31 that includes at least label L1, and captured image data representing a captured image PI2 of a portion of the right side surface 32 that includes at least label L2. As a result, the attachment positions of label L1 and label L2 can be appropriately inspected using pairs of different drawing data and captured image data, respectively. Furthermore, multiple labels can be easily inspected simply by preparing drawing data for each label.

[0065] Furthermore, according to the above embodiment, in the inspection preparation process, the CPU 110 analyzes the drawing data DD1, which is bitmap data, to identify extension lines A11a and A11b that are tangent to the label L1 and extension lines A12a and A12b that are parallel to the extension lines A11a and A11b and spaced apart from the label L1 (FIG. 3A, S110 and S115 in FIG. 4). The CPU 110 analyzes the drawing data DD1 to identify numerical values ​​N1a and N1b associated with these extension lines (S120 to S130 in FIG. 4), and acquires dimensional information based on the numerical values ​​N1a and N1b (S135 in FIG. 4). As a result, even if the drawing data DD1 is bitmap data, the CPU 110 can appropriately acquire dimensional information by analyzing the drawing data DD1 and identifying the numerical values ​​N1a and N1b associated with the extension lines.

[0066] More specifically, the CPU 110 identifies the dimension lines E1a and E1b associated with these extension lines and the arrows R1a and R1b that are terminal symbols of the dimension lines E1a and E1b (S120 and S125 in FIG. 4). The CPU 110 then searches within a range based on the dimension lines E1a and E1b to identify the numerical values ​​N1a and N1b (S130 in FIG. 4). As a result, by appropriately analyzing the drawing data DD1, it is possible to reliably acquire dimensional information. The drawing DI1 generally includes terminal symbols such as extension lines, dimension lines, and arrows according to rules established by standards such as JIS. For example, by analyzing the drawing data DD1 based on these rules, it is possible to reliably identify the terminal symbols such as extension lines, dimension lines, and arrows, and thus the numerical values ​​indicating the dimensions.

[0067] B. Second Example The second embodiment differs from the first embodiment in the configuration of the inspection process. Other configurations of the second embodiment are the same as those of the first embodiment. Figure 7 is a flowchart of the inspection process of the second embodiment.

[0068] In Figure 7, the same processes as in Figure 5 are assigned the same reference numerals as in Figure 5, and processes different from those in Figure 5 are assigned the suffix "B". In the inspection process in Figure 7, S215B is executed instead of S215 in Figure 5, and S225B and S235B are executed instead of S225, S230, and S235 in Figure 5. Other processes in the inspection process in Figure 7 are the same as those in Figure 5. Below, parts of the inspection process in Figure 7 that differ from Figure 5 will be described using an example where the target label is label L1.

[0069] In S215B, the CPU 110 performs pattern matching between the label PL1 in the captured image PI1 and the label BL1 in the drawing DI1. Specifically, the CPU 110 performs a resizing process on the drawing DI1 so that the size of the label BL1 in the drawing DI1 matches the size of the label PL1 in the captured image PI1. The resizing process is a process for changing the size of an image, and is either an enlargement process or a reduction process. The CPU 110 determines the position and angle of the drawing DI1 relative to the captured image PI1 so that the label BL1 in the resized drawing DI1 matches the label PL1 in the captured image PI1. Note that in a modified example, the resizing process may be performed on the captured image PI1 in the pattern matching.

[0070] 8A and 8B are explanatory diagrams of the inspection process of the second embodiment. Fig. 8A shows only the label BL1 and two extension lines A12a and A12b of the drawing DI1. Fig. 8B shows the state in which the label BL1 and two extension lines A12a and A12b of Fig. 8A are arranged on the captured image PI1 in the size and positional relationship determined by the pattern matching in S215B. In Fig. 8B, the label PL1 in the captured image PI1 and the label BL1 in the drawing DI1 completely overlap.

[0071] In S220, similar to S220 in FIG. 5, the CPU 110 identifies the reference portions PS1a and PS1b in the captured image corresponding to the extension lines A12a and A12b. In the state of FIG. 8B, when the distance between the label BL1 and the reference portion PS1a matches the design value, the reference portion PS1a and the extension line A12a overlap. When the distance between the label BL1 and the reference portion PS1a does not match the design value, the reference portion PS1a and the extension line A12a do not overlap. Similarly, the reference portion PS1b and the extension line A12b may or may not overlap. In the example of FIG. 8B, the reference portion PS1a and the extension line A12a and the reference portion PS1b and the extension line A12b do not overlap.

[0072] In S225B, the CPU 110 calculates the horizontal distance Δda between the reference portion PS1a in the captured image PI1 and the extension line A12a on the drawing DI1 in the state shown in FIG. 8B. For example, the CPU 110 calculates the distance dp (FIG. 8A) between the label BL1 and the extension line A12a in units of pixels. The CPU 110 determines a coefficient for converting the number of pixels into millimeters (mm) based on the ratio between the distance dp in units of pixels and the design value of the distance dp, i.e., the design value (unit: mm) shown as the numerical value N1a in FIG. 3A. Alternatively, the coefficient may be determined based on the ratio between the number of pixels in the width or height of the label BL1 and the pre-stored actual measurement value (unit: mm) of the width or height of the label L1. The design value shown as the numerical value N1a is included in the dimension information registered in the inspection preparation process. The CPU 110 calculates the distance Δda in units of the number of pixels, and then converts the distance Δda into millimeters using the coefficient described above. Using a similar method, the CPU 110 calculates the vertical distance Δdb between the reference site PS1b in the captured image PI1 and the extension line A12b on the drawing DI1 in the state of FIG. 8(B).

[0073] In S235B, the CPU 110 determines whether the horizontal and vertical distances Δda and Δdb are within the tolerances. The tolerances are included in the dimensional information registered in the inspection preparation process. If the horizontal and vertical distances Δda and Δdb are within the tolerances (S235B: YES), in S240 the CPU 110 determines that the affixing position of the target label (e.g., label L1) is the designed position. If at least one of the horizontal and vertical distances Δda and Δdb is not within the tolerances (S235B: NO), in S245 the CPU 110 determines that the affixing position of the target label (e.g., label L1) is not the designed position.

[0074] According to the second embodiment described above, in pattern matching, a resizing process is performed so that the size of the label LP1 in the captured image PI1 matches the size of the label BL1 in the drawing DI1, and the resized drawing data is used to determine whether the affixing position of the label PL1 in the captured image PI1 is a specific affixing position. It can be assumed that the size of the label is unlikely to fluctuate due to manufacturing errors or the like. Furthermore, since the label has a simple shape, such as a rectangle, the resizing process can be easily and accurately performed. As a result, the horizontal and vertical distances Δda and Δdb described above can be calculated with high accuracy, making it possible to accurately determine whether the affixing position of the label PL1 is a specific affixing position.

[0075] Furthermore, according to the second embodiment, in a state where the position and size of the label PL1 in the captured image PI1 match the position and size of the label BL1 in the drawing DI1 (FIG. 8B), the difference between the positions of the reference sites PS1a and PS1b in the captured image PI1 and the positions of the reference sites BS1a and BS1b in the drawing DI1 (i.e., the distances Δda and Δdb) is identified (S225B in FIG. 7). Then, by determining whether these distances Δda and Δdb are within the tolerance, it is determined whether the affixed position of the label PL1 in the captured image PI1 is a specific affixed position. As a result, it is possible to appropriately determine whether the affixed position of the label is a specific affixed position using a method different from the inspection process of the first embodiment.

[0076] C. Variations: (1) In the inspection preparation process of FIG. 4 in the above embodiment, dimension extension lines are identified in S110 and S115 of FIG. 4, and numerical values ​​associated with the dimension extension lines are identified in S120 to S130, thereby acquiring dimension information, but this is not limited to this.

[0077] For example, the CPU 110 may specify the positions of the reference portions BS1a and BS1b by specifying the reference portions BS1a and BS1b themselves in the drawing DI1 using a technique such as pattern matching, instead of specifying the extension lines A12a and A12b in S115 of Fig. 4. Then, the CPU 110 may obtain the dimension information by searching for numerical values ​​in the vicinity of the label LB1 and the reference portions BS1a and BS1b in the drawing DI1.

[0078] (2) In the above embodiment, the drawing data DD1 is bitmap data, but the drawing data may be CAD data including vector data indicating line drawings, extension lines, etc., and numerical data indicating design values ​​and tolerances. In this case, the CPU 110 may identify extension lines for defining the positional relationship between the label and the housing from the data on the extension lines included in the drawing data, and obtain numerical data indicating the design values ​​and tolerances associated with the extension lines.

[0079] (3) In the inspection process of Figure 5 in the above embodiment, distortion correction processing (S208) is performed, but for example, if the distortion of the captured image PI1 is small enough that it does not affect the inspection process, the distortion correction processing may be omitted.

[0080] (4) In the above embodiment, two labels are inspected, but the number of labels inspected may be one, three, or more. Furthermore, in the above embodiment, one piece of drawing data and one piece of photographed image data are used for one label. Alternatively, one piece of drawing data may be used for multiple labels, or one piece of photographed image data may be used for multiple labels. For example, if multiple labels are attached to the front surface 31, one piece of drawing data or one piece of photographed image data including the multiple labels may be used.

[0081] (5) In the inspection preparation process of the above embodiment, an arrow is identified as the terminal symbol of a dimension line (S120 in FIG. 4), and the dimension line associated with the arrow is identified (S125 in FIG. 4). Alternatively, a terminal symbol other than an arrow, such as a black circle or a diagonal line, may be identified. Furthermore, the dimension line may be identified by searching for a straight line near the extension line without identifying a terminal symbol.

[0082] (6) In the above embodiment, the shapes of the labels L1 and L2 are rectangular. Alternatively, the shapes of the labels L1 and L2 may be polygonal shapes other than rectangular shapes, such as triangles and pentagons, or may be circles or ellipses.

[0083] (7) In the above embodiment, the label attachment position on the housing is determined by specifying the horizontal distance between the label and the reference location and the vertical distance between the label and the reference location. Alternatively, all or part of the distance between the label and the reference location for determining the label attachment position on the housing may include a diagonal distance that is not parallel to either the horizontal or vertical direction. Even in this case, the system obtains dimensional information about the diagonal distance and determines whether the diagonal distance in the captured image is within a dimensional range defined by the dimensional information, thereby determining whether the label is in the attachment position.

[0084] (8) In the above embodiment, the inspection preparation process and the inspection process are performed by the inspection device 100 of FIG. 1. Alternatively, the inspection preparation process and the inspection process may each be performed by a different device. In this case, for example, the dimensional information registered by the inspection preparation process is stored in a storage device of the device that performs the inspection process. Also, all or part of the inspection preparation process and the inspection process may be performed by multiple computers (for example, so-called cloud servers) that can communicate with each other via a network.

[0085] (9) In the above-described embodiments, the positions of the labels L1 and L2 attached to the product 300 are inspected. However, the same techniques as those in the above-described embodiments can generally be employed when inspecting the position of a component that constitutes part of an object. That is, in each of the above-described embodiments, the product 300 to which the labels L1 and L2 are attached (the entire product 300 including the labels L1 and L2) is an example of an object, and the labels L1 and L2 are an example of a component.

[0086] For example, the object is not limited to the product 300, but may also be a component that constitutes the product 300, such as a print head used in a printer. The object may also be a combination of some of the components that constitute the product 300, such as a print head used in a printer and a carriage to which the print head is attached. The object may also be an unfinished intermediate product produced in the process of manufacturing the product 300.

[0087] For example, the constituent part is not limited to a label attached to the object, but may be a formed part formed on the object, specifically a screw hole, a pattern or name expressed by recesses and protrusions (brand logo of the manufacturer or product), or a painted part applied to the object.

[0088] Regardless of which of the above examples is used as the object or component, for example, at S100 in FIG. 4, CPU 110 acquires drawing data representing a drawing of at least a portion of the object including the component in a state where the component is arranged in a specific placement position. Then, at S105 in FIG. 4, CPU 110 uses the acquired drawing data to identify the component in the drawing. At S115 in FIG. 4, CPU 110 uses the drawing data to identify extension lines, etc. in the drawing to identify a reference portion in the drawing, i.e., a portion that serves as a reference for defining the specific placement position of the component relative to the object. CPU 110 acquires dimensional information shown in the drawing using the drawing data, the result of identifying the component in the drawing, and the result of identifying the position of the reference portion in the drawing. Furthermore, at S205 in FIG. 5, CPU 110 acquires captured image data obtained by capturing an image of the object in which the component is arranged. Then, in S210 of Fig. 5, CPU 110 identifies the component in the captured image using the captured image data, and in S220 of Fig. 5, identifies a reference portion in the captured image using the captured image data. In S225 to S245 of Fig. 5, CPU 110 determines whether the arrangement position of the component in captured image 1 is the specific arrangement position defined by the dimensional information, using the result of identifying the component in the captured image, the result of identifying the reference portion in the captured image, and the dimensional information. As a result, the arrangement position of the component that constitutes part of the object can be easily inspected using the captured image data and drawing data DD1.

[0089] (10) In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part or all of the configuration realized by software may be replaced by hardware. For example, all or part of the test preparation process and the test process may be executed by a hardware circuit such as an ASIC (Application Specific Integrated Circuit).

[0090] The present invention has been described above based on examples and modifications, but the above-described embodiments of the invention are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit and scope of the claims, and equivalents thereof are also included in the present invention. [Explanation of symbols]

[0091] 100... inspection device, 1000... inspection system, 110... CPU, 120... volatile storage device, 130... non-volatile storage device, 140... display unit, 150... operation unit, 170... communication unit, 30... housing, 300... product, 31... front, 32... right side, 400... imaging device, A11a, A11b, A12a, A12b, A21a, A21b, A22a, A22b... dimension extension lines, DD1, DD2... drawing data, DI1, DI2... drawings, E1a, E1b, E2a, E2b... dimension lines, L1, L2... labels, PG... computer program, PI1, PI2... captured image, R1a, R1b, R2a, R2b... arrows, S1a, S1b, S2a, S2b... reference portions

Claims

1. An inspection device that inspects the attachment positions of a first label attached to a first surface of a product and a second label attached to a second surface of the product facing in a different direction from the first surface, a drawing acquisition unit that acquires first drawing data representing a two-dimensional drawing of a portion of the first surface that includes at least the first label, and second drawing data representing a two-dimensional drawing of a portion of the second surface that includes at least the second label, of the product in a state in which the first label is affixed at a first affixing position and the second label is affixed at a second affixing position; a drawing label identifying unit that identifies the first label and the second label in the drawing using the first drawing data and the second drawing data; a reference position specifying unit that specifies a position of a reference portion of the product in the drawing using the first drawing data and the second drawing data, the reference portion including a first reference portion that serves as a reference for defining the first attachment position on the product, and a second reference portion that serves as a reference for defining the second attachment position on the product; a dimension acquisition unit that acquires dimensional information shown in the drawing using the first drawing data and the second drawing data, the identification result of the first label and the second label in the drawing, and the identification result of the first reference portion and the second reference portion in the drawing, wherein the dimensional information includes first dimensional information that defines a positional relationship between the first reference portion and the first attachment position, and second dimensional information that defines a positional relationship between the second reference portion and the second attachment position, and includes a tolerance between the first attachment position and the second attachment position; an image acquisition unit that acquires captured image data obtained by capturing an image of the product with the first label and the second label attached, the captured image data including first captured image data representing a captured image of a portion of the first surface that includes at least the first label, and second captured image data representing a captured image of a portion of the second surface that includes at least the second label; an image label identification unit that identifies the first label and the second label in the captured image using the first captured image data and the second captured image data; a reference portion specifying unit that specifies the first reference portion and the second reference portion of the product in the captured image using the first captured image data and the second captured image data; a position determination unit that determines whether or not the affixing position of the first label in the captured image is the first affixing position, and determines whether or not the affixing position of the second label in the captured image is the second affixing position, using the identification result of the first label and the second label in the captured image, the identification result of the first reference portion and the second reference portion in the captured image, and the first dimension information and the second dimension information; An inspection device comprising:

2. The inspection device according to claim 1, the first dimension information includes a tolerance of the first attachment position and a design value of a dimension between the first label and the first reference portion, The position determination unit Identifying a dimension between the first label in the captured image and the first reference portion in the captured image; An inspection device that determines whether the specified dimension is within a range based on the design value and the tolerance, thereby determining whether the attachment position of the first label in the captured image is the first attachment position.

3. The inspection device according to claim 1, The position determination unit identifying a difference between a position of the first reference portion in the captured image and a position of the first reference portion in the drawing in a state where the position and size of the first label in the captured image and the position and size of the first label in the drawing match; An inspection device that determines whether the affixing position of the first label in the captured image is the first affixing position by determining whether the difference between the position of the first reference portion in the captured image and the position of the first reference portion in the drawing is within the tolerance of the first affixing position.

4. The inspection device according to any one of claims 1 to 3, further comprising: a correction unit that executes a distortion correction process on the first captured image data to correct distortion in the captured image, The position determination unit determines whether the affixing position of the first label in the captured image is the first affixing position using the first captured image data that has been subjected to the distortion correction process.

5. The inspection device according to any one of claims 1 to 4, further comprising: a size change unit that executes a size change process on at least one of the first captured image data and the first drawing data to change the size of an image so that a size of the first label in the captured image matches a size of the first label in the drawing; The position determination unit determines whether the affixing position of the first label in the captured image is the first affixing position using the at least one of the image data that has been resized.

6. The inspection device according to any one of claims 1 to 5, the first drawing data is bitmap data, The reference position specifying unit Identifying a first extension line tangent to the first label in the drawing by analyzing the first drawing data; Identifying the location of the first reference feature by identifying a second extension line parallel to the first extension line and spaced apart from the first label in the drawing, the second extension line being attached to the first reference feature; The dimension acquisition unit By analyzing the first drawing data, numerical values ​​associated with the first extension line and the second extension line are identified; An inspection device that acquires the first dimension information based on the numerical value.

7. 7. The inspection device according to claim 6, The dimension acquisition unit Identifying at least one of a dimension line associated with the first extension line and the second extension line and a terminal symbol of the dimension line; An inspection device that identifies the numerical value by searching within a range based on at least one of the dimension line and the terminal symbol.

8. A computer program for inspecting the attachment positions of a first label attached to a first surface of a product and a second label attached to a second surface of the product facing in a different direction from the first surface, the computer program comprising: a drawing acquisition function for acquiring first drawing data representing a two-dimensional drawing of a portion of the first surface including at least the first label, of the product in a state in which the first label is attached at a first attachment position and the second label is attached at a second attachment position, and second drawing data representing a two-dimensional drawing of a portion of the second surface including at least the second label; a drawing label identification function that identifies the first label and the second label in the drawing using the first drawing data and the second drawing data; a reference position specifying function that specifies the position of a reference portion of the product in the drawing using the first drawing data and the second drawing data, the reference portion including a first reference portion that serves as a reference for defining the first attachment position on the product, and a second reference portion that serves as a reference for defining the second attachment position on the product; a dimension acquisition function that acquires dimensional information shown in the drawing using the first drawing data and the second drawing data, the result of identifying the first label and the second label in the drawing, and the result of identifying the first reference portion and the second reference portion in the drawing, wherein the dimensional information includes first dimensional information that defines a positional relationship between the first reference portion and the first attachment position, and second dimensional information that defines a positional relationship between the second reference portion and the second attachment position, and includes a tolerance between the first attachment position and the second attachment position; an image acquisition function that acquires captured image data obtained by capturing an image of the product with the first label and the second label attached, the captured image data including first captured image data representing a captured image of a portion of the first surface that includes at least the first label, and second captured image data representing a captured image of a portion of the second surface that includes at least the second label; an identification function that identifies the first label and the second label in the captured image using the first captured image data and the second captured image data; a reference portion specifying function that specifies the first reference portion and the second reference portion of the product in the captured image using the first captured image data and the second captured image data; a position determination function that determines whether or not the affixing position of the first label in the captured image is the first affixing position, and determines whether or not the affixing position of the second label in the captured image is the second affixing position, using the result of identifying the first label and the second label in the captured image, the result of identifying the first reference portion and the second reference portion in the captured image, and the first dimension information and the second dimension information; A computer program that enables a computer to realize the above.

9. An inspection method for inspecting the attachment positions of a first label attached to a first surface of a product and a second label attached to a second surface of the product facing in a different direction from the first surface, comprising: a drawing acquisition process for acquiring first drawing data representing a two-dimensional drawing of a portion of the first surface including at least the first label, and second drawing data representing a two-dimensional drawing of a portion of the second surface including at least the second label, of the product in a state in which the first label is attached at a first attachment position and the second label is attached at a second attachment position; a drawing label identifying step of identifying the first label and the second label in the drawing using the first drawing data and the second drawing data; a reference position specifying step of specifying a position of a reference portion of the product in the drawing using the first drawing data and the second drawing data, the reference portion including a first reference portion serving as a reference for defining the first attachment position on the product, and a second reference portion serving as a reference for defining the second attachment position on the product; a dimension acquisition process for acquiring dimensional information shown in the drawing using the first drawing data and the second drawing data, the result of identifying the first label and the second label in the drawing, and the result of identifying the first reference portion and the second reference portion in the drawing, wherein the dimensional information includes first dimensional information that defines a positional relationship between the first reference portion and the first attachment position, and second dimensional information that defines a positional relationship between the second reference portion and the second attachment position, and includes a tolerance between the first attachment position and the second attachment position; an image acquisition step of acquiring image data obtained by imaging the product with the first label and the second label attached, the image data including first image data representing an image of a portion of the first surface including at least the first label, and second image data representing an image of a portion of the second surface including at least the second label; an identifying step of identifying the first label and the second label in the captured image using the first captured image data and the second captured image data; a reference portion specifying step of specifying the first reference portion and the second reference portion of the product in the captured image using the first captured image data and the second captured image data; a position determination step of determining whether or not the affixing position of the first label in the captured image is the first affixing position, and determining whether or not the affixing position of the second label in the captured image is the second affixing position, using the result of identifying the first label and the second label in the captured image, the result of identifying the first reference portion and the second reference portion in the captured image, and the first dimension information and the second dimension information; An inspection method comprising:

10. An inspection device for inspecting the positioning of a first component arranged on a first surface of an object and a second component arranged on a second surface of the object facing in a different direction from the first surface, comprising: a drawing acquisition unit that acquires first drawing data representing a two-dimensional drawing of a portion of the first surface that includes at least the first component part, of the object in a state in which the first component part is arranged at a first placement position and the second component part is arranged at a second placement position, and second drawing data representing a two-dimensional drawing of a portion of the second surface that includes at least the second component part; a drawing component identification unit that identifies the first component and the second component in the drawing using the first drawing data and the second drawing data; a reference position specifying unit that specifies a position of a reference portion of the object in the drawing using the first drawing data and the second drawing data, the reference portion including a first reference portion that serves as a reference for defining the first placement position with respect to the object, and a second reference portion that serves as a reference for defining the second placement position with respect to the object; a dimension acquisition unit that acquires dimensional information shown in the drawing using the first drawing data and the second drawing data, an identification result of the first constituent part and the second constituent part in the drawing, and an identification result of the first reference part and the second reference part in the drawing, wherein the dimensional information includes first dimensional information that defines a positional relationship between the first reference part and the first placement position, and second dimensional information that defines a positional relationship between the second reference part and the second placement position, and includes a tolerance between the first placement position and the second placement position; a captured image acquisition unit that acquires captured image data obtained by capturing an image of the object in a state in which the first component part and the second component part are arranged, the captured image data including first captured image data representing a captured image of a portion of the first surface that includes at least the first component part, and second captured image data representing a captured image of a portion of the second surface that includes at least the second component part; an image component specifying unit that specifies the first component and the second component in the captured image using the first captured image data and the second captured image data; a reference portion specifying unit that specifies the first reference portion and the second reference portion of the object in the captured image using the first captured image data and the second captured image data; a position determination unit that determines whether or not the arrangement position of the first component part in the captured image is the first arrangement position, and determines whether or not the arrangement position of the second component part in the captured image is the second arrangement position, using the identification result of the first component part and the second component part in the captured image, the identification result of the first reference part and the second reference part in the captured image, and the first dimension information and the second dimension information; An inspection device comprising:

11. The inspection device according to claim 10, the first dimension information includes a tolerance of the first arrangement position and a design value of a dimension between the first component part and the first reference portion, The position determination unit Identifying a dimension between the first component part in the captured image and the first reference part in the captured image; An inspection device that determines whether the placement position of the first component part in the captured image is the first placement position by determining whether the identified dimension is within a range based on the design value and the tolerance.

12. The inspection device according to claim 10, The position determination unit identifying a difference between a position of the first reference portion in the captured image and a position of the first reference portion in the drawing in a state where the position and size of the first component portion in the captured image and the position and size of the first component portion in the drawing are consistent; An inspection device that determines whether the placement position of the first component in the captured image is the first placement position by determining whether the difference between the position of the first reference part in the captured image and the position of the first reference part in the drawing is within the tolerance of the first placement position.

13. The inspection device according to any one of claims 10 to 12, further comprising: a correction unit that executes a distortion correction process on the first captured image data to correct distortion in the captured image, An inspection device, wherein the position determination unit determines whether the arrangement position of the first component part in the captured image is the first arrangement position using the first captured image data that has been subjected to the distortion correction process.

14. The inspection device according to any one of claims 10 to 13, further comprising: a size change unit that executes a size change process on at least one of the first captured image data and the first drawing data to change the size of an image so that a size of the first component part in the captured image matches a size of the first component part in the drawing; The position determination unit determines whether the position of the first component in the captured image is the first position using the at least one image data that has been resized.

15. The inspection device according to any one of claims 10 to 14, the first drawing data is bitmap data, The reference position specifying unit By analyzing the first drawing data, a first extension line tangent to the first component part in the drawing is identified; Identifying the location of the first reference feature by identifying a second extension line parallel to the first extension line and spaced apart from the first component in the drawing, the second extension line being attached to the first reference feature; The dimension acquisition unit By analyzing the first drawing data, numerical values ​​associated with the first extension line and the second extension line are identified; An inspection device that acquires the first dimension information based on the numerical value.

16. 16. The inspection device according to claim 15, The dimension acquisition unit Identifying at least one of a dimension line associated with the first extension line and the second extension line and a terminal symbol of the dimension line; An inspection device that identifies the numerical value by searching within a range based on at least one of the dimension line and the terminal symbol.

Images