Sewing system and sewing method

Abstract

To reduce the number of operation man-hours regarding image recognition and the number of management man-hours of data in a sewing system which includes a plurality of sewing machines.SOLUTION: A sewing system includes: a plurality of imaging devices for acquiring an image of a sewing object; one or a plurality of sewing machines for performing sewing based on a recognition result acquired by image recognition processing of a sewing object; and a data processing part for acquiring template data of the sewing object created based on the image imaged in a first imaging device out of the plurality of imaging devices, and for transmitting it to another second imaging device. The sewing machine includes intrinsic data containing an imaging condition of the imaging device. The sewing machine having the second imaging device executes image recognition processing using the template data corrected based on the intrinsic data of the second imaging device, or corrects the recognition result using the template data, based on the intrinsic data of the second imaging device.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a sewing system and a sewing method. [Background technology]

[0002] In order to improve the design of a sewn object, stitches may be formed on the sewn object. Patent Document 1 discloses a technique for forming stitches on a cover material used for a vehicle seat. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-162957 A Summary of the Invention [Problem to be solved by the invention]

[0004] The upholstery material used for vehicle seats is thick and elastic. When stitches are formed on a sewing object having thickness and elasticity, the sewing object may shrink and the surface of the sewing object may be displaced. For example, when a second stitch is formed after a first stitch is formed, it is desirable to form the second stitch at a target position of the sewing object according to the displacement of the surface of the sewing object caused by the formation of the first stitch. As a countermeasure for forming a stitch at a target position of the sewing object, there is a method of photographing the surface of the sewing object before the sewing process and detecting the displacement of the surface of the sewing object. An image recognition process such as template matching is used for the displacement detection.

[0005] When the same product is manufactured using multiple sewing machines in a sewing factory or the like, the multiple sewing machines form stitches of the same shape on sewing objects of the same shape, so the template data used for image recognition processing is also common. However, in reality, the imaging conditions of the imaging device provided in each sewing machine differ due to individual differences in the imaging device, assembly errors, etc., so template data is prepared separately for each sewing machine. This increases the number of work steps and the number of data management steps required for the entire multiple sewing machines.

[0006] An aspect of the present invention aims to reduce the number of work steps and data management steps involved in image recognition in a sewing system equipped with multiple sewing machines. [Means for solving the problem]

[0007] According to a first aspect of the present invention, there is provided a sewing system comprising: a plurality of imaging devices which capture images of an object to be sewn; one or more sewing machines which perform sewing based on a recognition result obtained by an image recognition process of the object to be sewn; and a data processing unit which acquires template data of the object to be sewn, which is created based on an image captured by a first imaging device among the plurality of imaging devices, and transmits the template data to another second imaging device, wherein the sewing machines have unique data including imaging conditions of the imaging devices, and the sewing machine having the second imaging device performs image recognition processing using the template data corrected based on the unique data of the second imaging device, or corrects the recognition result using the template data based on the unique data of the second imaging device.

[0008] According to a second aspect of the present invention, there is provided a sewing method using a plurality of sewing machines, or a sewing machine having a plurality of the imaging devices, each having an imaging device that acquires an image of a sewing object and performs sewing based on a recognition result obtained by an image recognition process of the sewing object, the sewing method including: acquiring template data of the sewing object created based on an image captured by a first imaging device among the plurality of sewing machines or the plurality of the imaging devices, and transmitting the template data to another, second imaging device; and, in the sewing machine having the second imaging device, performing image recognition processing using the template data corrected based on unique data including the imaging conditions of the second imaging device, or correcting the recognition result using the template data based on the unique data. Effect of the Invention

[0009] According to an aspect of the present invention, it is possible to reduce the number of steps of work related to image recognition and data management in a sewing system equipped with multiple sewing machines. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a sewing system according to the present embodiment. [Diagram 2] FIG. 2 is a block diagram showing the sewing machine and the data processing unit according to the present embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of unique data possessed by a plurality of sewing machines. [Figure 4] FIG. 4 is a cross-sectional view showing a part of the sewing object. [Diagram 5] FIG. 5 is a plan view showing the sewing object. [Figure 6] FIG. 6 is a cross-sectional view showing a part of the sewing object. [Figure 7] FIG. 7 is a diagram for explaining template data to be recognized. [Figure 8] FIG. 8 is a diagram for explaining the correction of template data by the sewing system according to this embodiment. [Figure 9]FIG. 9 is a schematic diagram illustrating distribution of unique template data. [Figure 10] FIG. 10 is a diagram illustrating the sewing method according to this embodiment. [Figure 11] FIG. 11 is a perspective view showing the sewing machine according to the present embodiment. [Figure 12] FIG. 12 is a perspective view showing a part of the sewing machine according to the present embodiment. [Figure 13] FIG. 13 is a plan view illustrating an example of the arrangement of the imaging devices according to this embodiment. [Figure 14] FIG. 14 is a flowchart for explaining the sewing operation of the sewing machine according to this embodiment. [Figure 15] FIG. 15 is a schematic diagram for explaining the image joining process. [Figure 16] FIG. 16 is a schematic diagram for explaining the recognition target detection process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings, but the present invention is not limited thereto. The components of the embodiments described below can be appropriately combined. In addition, some components may not be used.

[0012] In this embodiment, a local coordinate system is defined for the sewing machine 1. In the following description, the local coordinate system defined for the sewing machine 1 is appropriately referred to as the sewing machine coordinate system. The sewing machine coordinate system is defined by an XYZ orthogonal coordinate system. In this embodiment, the positional relationship of each part will be described based on the sewing machine coordinate system. The direction parallel to the X axis in a predetermined plane is defined as the X axis direction. The direction parallel to the Y axis in a predetermined plane perpendicular to the X axis is defined as the Y axis direction. The direction parallel to the Z axis perpendicular to the predetermined plane is defined as the Z axis direction. In addition, the rotation direction or tilt direction around the X axis is defined as the θX direction. The rotation direction or tilt direction around the Y axis is defined as the θY direction. The rotation direction or tilt direction around the Z axis is defined as the θZ direction. In this embodiment, the plane including the X axis and the Y axis is appropriately referred to as the XY plane. The plane including the X axis and the Z axis is appropriately referred to as the XZ plane. The plane including the Y axis and the Z axis is appropriately referred to as the YZ plane. The XY plane is parallel to the predetermined plane. The XY plane, the XZ plane, and the YZ plane are orthogonal to each other. In this embodiment, the XY plane and the horizontal plane are parallel to each other. The Z axis direction is the up-down direction. The +Z direction is the upward direction, and the -Z direction is the downward direction. The XY plane may be inclined with respect to the horizontal plane.

[0013] [Outline of the sewing system] FIG. 1 is a diagram that illustrates a sewing system 100 according to the present embodiment. As illustrated in FIG. 1, the sewing system 100 includes a plurality of sewing machines 1, a plurality of gateways 101 corresponding to the respective sewing machines 1, a communication network 102, and a data processing unit 50. The sewing system 100 is installed, for example, in a sewing factory, which is an operating facility where a plurality of sewing machines 1 are operated. In the sewing factory, a sewn product is manufactured by sewing a sewing object S with sewing thread by the sewing machines 1. Note that, although FIG. 1 illustrates three sewing machines 1, the number of sewing machines 1 included in the sewing system 100 is not particularly limited and may be one, two, or four or more. Moreover, data may be exchanged between the devices via an external memory medium such as a USB memory without going through a communication network. Furthermore, even if there is only one sewing machine 1, and the imaging conditions change due to replacement of a camera or other device due to a malfunction, the template data can be corrected using the same principle as described below, thereby reducing the number of work steps.

[0014] The sewing machine 1 is an industrial sewing machine operated by an operator Ma. The sewing machine 1 of this embodiment is a cycle sewing machine that automatically performs pattern sewing based on a preset sewing pattern. The sewing machine 1 starts a process of sewing the preset sewing pattern when the operator Ma performs a preset operation with the sewing object S and sewing thread set in a preset position.

[0015] In this embodiment, the multiple sewing machines 1 are used to manufacture the same product. That is, the multiple sewing machines 1 sew the same pattern on sewing objects S having the same shape. For example, a sewing factory is provided with multiple production lines that manufacture the same product, and each sewing machine 1 is installed on each of the multiple production lines.

[0016] Sewing machine 1 connects to communication network 102 via a corresponding gateway 101. Gateway 101 is disposed on a communication line that connects sewing machine 1 to communication network 102. Gateway 101 is a device that relays the transmission and reception of various data in communication between sewing machine 1 and communication network 102, and converts the protocol of received data before transferring it.

[0017] The data processing unit 50 is an information terminal operated by an administrator Mb to manage data of the multiple sewing machines 1 of the sewing system 100. A typical example of the data processing unit 50 is a file server or NAS (Network Attached Storage) that manages files, but it may also be realized by utilizing a file management and sharing function of a desktop PC (Personal Computer), a notebook PC, a smartphone, a tablet terminal, or the like. One of the multiple sewing machines 1 can be designated as a master machine and also function as the data processing unit 50.

[0018] The data processing unit 50 is connected to a communication network 102 and can communicate with a plurality of sewing machines 1 via each gateway 101. The data processing unit 50 transmits and receives data necessary for sewing to and from each sewing machine 1 via the communication network 102.

[0019] [Sewing machines and control devices] Fig. 2 is a functional block diagram showing the sewing machine 1 and the data processing unit 50 according to this embodiment. In Fig. 2, only the schematic configuration of the sewing machine 1 is shown.

[0020] As shown in FIG. 2, the sewing machine 1 includes a sewing machine body 10, a control device 30, a storage device 35, an image processing unit 36, an imaging device 37, an illumination device 38, and a communication device 39.

[0021] The sewing machine body 10 is a mechanical portion that sews the sewing object S. The sewing machine body 10 has an actuator 10A and a drive amount sensor 10B.

[0022] The actuator 10A includes various drive mechanisms and drive sources that operate the various parts of the sewing machine body 10. The actuator 10A executes various operations for the sewing machine 1 to sew a predetermined sewing pattern that has been set.

[0023] The drive amount sensor 10B detects the drive amount of the actuator 10A. The drive amount sensor 10B includes, for example, an encoder that detects the rotation amount of the motor, a position sensor that detects the position of a holding member for the sewing needle, etc. The sewing machine main body 10 (actuator 10A) is controlled by the control device 30 based on the detection result of the drive amount sensor 10B.

[0024] The control device 30 includes a computer system and executes various controls on each part of the sewing machine 1. The control device 30 includes a processor such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The processor executes various arithmetic operations to realize the functions of the control device 30 based on a program stored in the storage device 35. The control device 30 controls the sewing machine main body 10 (actuator 10A) according to the sewing data set in the storage device 35.

[0025] The storage device 35 includes a primary storage device, which is a volatile memory such as a random access memory (RAM) that temporarily stores data, and a secondary storage device, which is a non-volatile memory such as a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), or a storage device that stores various programs and various data used by the processor for calculations. As will be described later, the storage device 35 stores unique data 60 relating to the imaging device 37 and the sewing machine main body 10 provided in the sewing machine 1 in addition to the program of the control device 30. The storage device 35 stores sewing data for executing the sewing process.

[0026] The sewing data is known data that can be derived from design data of the sewing object S, such as CAD (Computer Aided Design) data. The sewing data is referred to by the control device 30 in the sewing process. The sewing process refers to a process of forming stitches CH (see FIG. 6) in the sewing object S. The storage device 35 also stores template data used in image recognition processing for an image of the sewing object S captured by the imaging device 37.

[0027] The image processing unit 36 ​​includes a computer system. The image processing unit 36 ​​has a processor such as a CPU (Central Processing Unit), a storage device including a non-volatile memory such as a ROM (Read Only Memory) or storage and a volatile memory such as a RAM (Random Access Memory), and an input / output unit including an input / output interface. The image processing unit 36 ​​performs image recognition processing to capture an image of the sewing object S using an imaging device 37 and detect a recognition object UP (see FIG. 5) from the image in accordance with a computer program stored in the storage device.

[0028] The imaging device 37 captures an image of the sewing object S. The imaging device 37 has an optical system and an image sensor that receives light incident via the optical system. The image sensor includes a CCD (Couple Charged Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0029] The illumination device 38 irradiates light onto the imaging field of view of the imaging device 37. The illumination device 38 has a light source such as an LED light emitting element.

[0030] The communication device 39 communicates with the data processing unit 50 via the gateway 101 and the communication network 102. The communication device 39 includes a communication interface that complies with a wired or wireless communication standard.

[0031] The data processing unit 50 includes a computer system, and is provided with a processor 51, a storage device 52, and an input / output unit 53.

[0032] The data processing unit 50 may be connected to an input device and a display device (not shown). The input device is realized by, for example, a keyboard, a mouse, operation keys, a microphone for receiving voice input, etc. The display device displays various information. The display device includes, for example, a flat panel display such as a liquid crystal display or an organic EL display. Moreover, the data processing unit 50 may be a simple shared storage, in which case an input device or a display device is not required. Operation information to the input / output unit 53 is transmitted by communication.

[0033] The processor 51 is composed of a CPU, an MPU, and the like. The processor 51 operates based on a program stored in the storage device 52, and executes various processes. The processor 51 includes a master data management unit 51A and a data adjustment unit 51B. The master data management unit 51A and the data adjustment unit 51B are functional blocks that are realized by the processor 51 executing a program. The master data management unit 51A and the data adjustment unit 51B may be configured separately as dedicated processors.

[0034] The master data management unit 51A creates generalized template data 71 by generalizing the template data 70 of the first sewing machine SM1 in Fig. 8, for example. The generalized template data 71 is template data used for image recognition processing of an image of the sewing object S, and is data from which individual differences of each sewing machine 1 and image capturing conditions for each machine have been eliminated (generalized = corrected to an ideal form using design values). As in the configuration of Fig. 2, it is possible to perform data conversion processing in a centralized manner, but it is also possible to provide this function on the sewing machine side and build a system in a distributed manner. In general, a distributed type is easier to implement because it has a more efficient and simpler configuration.

[0035] Data adjustment unit 51B acquires unique data 60 of multiple imaging devices 37, and creates unique template data 72 (see second sewing machine SM2 in FIG. 8) for each imaging device based on generalized template data 71 and unique data 60. Unique template data 72 is template data that reflects individual differences and imaging conditions of each imaging device 37 in relation to generalized template data 71. This function is implemented by the same calculation as generalizing template data, and can be similarly provided on the sewing machine side. If the data conversion process is implemented on the sewing machine side in a distributed manner, data processing unit 50 only needs to manage generalized template data, and there is no need to centrally manage the data of all of the multiple sewing machines 1, resulting in an even simpler configuration.

[0036] The storage device 52 includes a primary storage device, which is a volatile memory such as a RAM that temporarily stores data, and a secondary storage device, which is a non-volatile memory such as a ROM, HDD, SSD, or storage that stores various programs and various data used by the processor 51 for calculations. The storage device 52 stores a program for implementing the functions of the data processing unit 50. The storage device 52 stores unique data 60 of each imaging device 37. The storage device 52 stores generalized template data 71 as master data. By centrally managing the unique data 60 of each imaging device 37, the safety of this data is guaranteed, but since the data is also managed on each sewing machine 1, this function can be eliminated in the data processing unit 50 to reduce the burden.

[0037] Input / output unit 53 communicates with each sewing machine 1 via communication network 102. Input / output unit 53 includes a communication interface that complies with a wired or wireless communication standard. For example, in FIG. 8, input / output unit 53 receives unique data 60A and template data 70 transmitted from first sewing machine SM1. Input / output unit 53 transmits to second sewing machine SM2 unique template data 72 that reflects unique data 60B of second sewing machine SM2 (imaging device 37 included in second sewing machine SM2) in generalized template data 71. When each sewing machine 1 is provided with a data conversion processing function as master data management unit 51A and data adjustment unit 51B, input / output unit 53 transmits and receives generalized template data 71 to and from each sewing machine 1.

[0038] [Unique data] 3 is an explanatory diagram of unique data 60 possessed by a plurality of sewing machines 1. The unique data 60 is data unique to each of the imaging devices 37 of the plurality of sewing machines 1, and differences in the unique data 60 represent individual differences between the imaging devices 37 of the individual sewing machines 1. In this embodiment, the unique data 60 includes information related to imaging of the sewing object S, i.e., imaging conditions of the imaging devices 37. Specifically, the unique data 60 includes information on imaging conditions unique to each sewing machine 1, which are caused by variation factors such as individual differences between the imaging devices 37 and lighting devices 38 of each sewing machine 1 and assembly errors. Note that the unique data 60 may include data other than that shown in FIG. 3. The unique data 60 is managed for each imaging device 37 of the individual sewing machine 1. Furthermore, by embedding the unique data 60 in the template data file each time an image of a template is captured, the data are always kept as a pair and are not separated even after being output from the sewing machine onto a communication line, allowing for reliable management. Of course, this is useful not only for sharing data between different sewing machines 1, but also for sharing data between different imaging devices 37 of the same sewing machine 1.

[0039] The inherent data 60 includes, as imaging conditions, a pixel rate 61 of the imaging device 37, an imaging offset 62, and illumination information 63.

[0040] The pixel rate 61 of the imaging device 37 is information indicating the actual distance per pixel of the captured image. The pixel rate 61 changes depending on the distance between the imaging device 37 and the subject (the sewing object S). The pixel rate 61 can be obtained by calibration, which involves capturing an image of a measuring jig with a known distance between two points and measuring the number of pixels between the two points in the image. The pixel rate 61 for any distance can be obtained by performing calibration at multiple distances and finding a regression line from each pixel rate at the multiple distances.

[0041] The imaging offset 62 is information indicating a deviation of the camera coordinate system of the imaging device 37 from a design value in the sewing machine coordinate system. The imaging offset 62 is information for correcting deviations caused by an assembly error of the imaging device 37, an individual difference in the optical system of the imaging device 37 (optical axis deviation, lens aberration), and the like. The imaging offset 62 includes information such as the XYZ position coordinates of the imaging device 37, the inclination of the camera optical axis (θX direction and θY direction), the rotation angle around the camera optical axis (θZ direction), and the lens distortion coefficient. The imaging offset 62 can be obtained by calibration in which a measuring tool of a known shape is positioned at a predetermined position, an image is captured, and an XY position deviation, image distortion, and angle deviation are calculated from the position and shape of the measuring tool captured in the image.

[0042] The lighting information 63 is information indicating the amount of illumination light from the lighting device 38. The lighting information can be acquired by performing calibration in which an image is captured of a predetermined subject (e.g., sewing object S) while changing the light amount setting value of the lighting device 38, and the light amount setting value that brings the contrast of the image into a desired range is obtained. When brightness adjustment is performed on the captured image by image processing, the lighting information 63 may further include information such as brightness adjustment parameters and a lookup table.

[0043] In the sewing system 100 according to the present embodiment, a calibration operation is performed in advance for each of the imaging devices 37 of the multiple sewing machines 1, so that unique data 60 including a pixel rate 61, an imaging offset 62, and illumination information 63 is acquired for each sewing machine 1. Each of the multiple sewing machines 1 holds the unique data 60 including the imaging conditions of the imaging device 37 in the storage device 35.

[0044] [Items to be sewn] Fig. 4 is a cross-sectional view showing a part of the sewing object S according to this embodiment. Fig. 5 is a plan view showing the sewing object S according to this embodiment. Figs. 4 and 5 show the sewing object S before sewing processing. In this embodiment, the sewing object S is a cover material used for a vehicle seat.

[0045] 4, the sewing object S has a surface material 4, a padding material 5, and a backing material 6. A hole 7 is provided in the surface material 4.

[0046] The surface of the cover material 4 is a seating surface that comes into contact with an occupant when the occupant sits on the vehicle seat. The cover material 4 includes at least one of a woven fabric, a nonwoven fabric, and leather. The pad material 5 has elasticity. The pad material 5 includes, for example, a urethane resin. The backing material 6 includes at least one of a woven fabric, a nonwoven fabric, and leather.

[0047] As shown in FIG. 5, a plurality of holes 7 are provided in the surface material 4. The holes 7 are formed so as to be scattered in the sewing object S. The holes 7 are arranged in a regular pattern. In other words, the sewing object S includes a plurality of reference patterns DPh. The reference pattern DPh is composed of a regular arrangement of a plurality of holes 7. In the example of FIG. 5, the reference pattern DPh is composed of 17 holes 7.

[0048] As shown in Fig. 5, the reference patterns DPh are arranged on the surface material 4 at intervals. The reference patterns DPh are arranged at equal intervals in both the X-axis direction and the Y-axis direction. A reference pattern DPh having a different position in the Y-axis direction is arranged between adjacent reference patterns DPh in the X-axis direction. No holes 7 are formed between adjacent reference patterns DPh.

[0049] Target patterns RP (RP1, RP2, RP3, RP4, RP5, RP6, RP7, RP8, RP9, RP10) of stitches CH to be formed on the sewing object S are defined in the areas between the reference patterns DPh. Fig. 5 shows a virtual target pattern RP of the stitches CH on the sewing object S before sewing. The sewing machine 1 forms the stitches CH so as to follow each target pattern RP in accordance with a preset sewing pattern.

[0050] When performing multiple sewing processes to form multiple stitches CH, a first sewing process is performed to form a first stitch CH on the sewing object S based on a first target pattern RP1, and after the first sewing process, a second sewing process is performed to form a second stitch CH on the sewing object S based on a second target pattern RP2. Thereafter, the third sewing process to the Nth sewing process are performed sequentially according to the number N of stitches CH to be formed. Fig. 5 shows an example where N=10, and the first target pattern RP1 to the tenth target pattern RP10 are defined.

[0051] Further, a plurality of recognition objects UP (UP1, UP2, UP3, UP4, UP5, UP6, UP7) are arranged on the sewing object S. In this embodiment, the recognition objects UP are an array of a plurality of holes 7 that exist in a predetermined pattern on the sewing object S. Specifically, the recognition objects UP are part of a reference pattern DPh. In the example of FIG. 5, the recognition objects UP (UP1, UP2, UP3, UP4, UP5, UP6, UP7) are a pattern (arrangement of holes 7) that includes one acute corner of the reference pattern DPh having a diamond-shaped outer shape. The recognition objects UP are patterns that can be specified by a template matching method, which is a type of image processing method.

[0052] With reference to FIG. 6, a description will be given of the displacement that occurs on the surface of the sewing object S when a stitch CH is formed on the sewing object S, which has thickness and elasticity. FIG. 6 is a cross-sectional view showing an example of the sewing object S according to the present embodiment. FIG. 6 shows the sewing object S after a sewing process has been performed. The sewing object S is thick and elastic. When a stitch CH is formed on the sewing object S, which has thickness and elasticity, as shown in FIG. 6, there is a high possibility that the sewing object S will shrink. When the sewing object S shrinks, there is a possibility that the surface of the sewing object S will be displaced. When the surface of the sewing object S is displaced, there is a high possibility that the target position of the stitch CH defined on the surface of the sewing object S will be displaced in the XY plane. When the target position of the stitch CH is displaced in the XY plane, it becomes difficult to form the stitch CH at the target position when the sewing object S is moved to the position coordinates of the target pattern RP in the design.

[0053] Therefore, in this embodiment, the image processing unit 36 ​​(see FIG. 2) acquires the amount of displacement (a correction amount for correcting the amount of displacement) of the surface of the sewing object S, and moves the sewing object S according to the amount of displacement so that the next stitch CH is formed at the target position, even if the sewing object S shrinks due to the formation of the stitch CH and the surface of the sewing object S is displaced. The image processing unit 36 ​​detects the recognition object UP from an image of the sewing object S captured by the imaging device 37, and acquires the amount of displacement based on the displacement of the recognition object UP.

[0054] [Template data] FIG. 7 is a diagram for explaining template data 70 of the recognition target UP. The template data 70 is created using a first imaging device 37, which is any one of the multiple imaging devices 37. Hereinafter, the sewing machine 1 having the first imaging device 37 used to create the template data 70 will be referred to as a "first sewing machine SM1." In addition, among the multiple sewing machines 1, a sewing machine 1 other than the first sewing machine SM1 will be referred to as a "second sewing machine SM2." The second sewing machine SM2 is a sewing machine 1 having a second imaging device 37 other than the first imaging device 37.

[0055] The template data 70 is data used in image recognition processing of the recognition target UP. Specifically, the template data 70 is an image of the recognition target UP, and is a reference image referred to in template matching. The template data 70 is acquired by capturing an image of the recognition target UP formed on the sewing object S by the first imaging device 37 included in the first sewing machine SM1.

[0056] Therefore, the template data 70 is an image that depends on the inherent data 60 of the first imaging device 37. In other words, parameters such as the size, distortion, tilt, average brightness, and contrast of the recognition target UP included in the image of the template data 70 reflect the imaging conditions (pixel rate 61, imaging offset 62, and lighting information 63) of the first imaging device 37 of the first sewing machine SM1.

[0057] In an actual sewing operation, an image captured by the first sewing machine SM1 (first imaging device 37) is captured under the same imaging conditions as the template data 70. Therefore, in the first sewing machine SM1, the image of the sewing object S involved in the sewing operation and the template data 70 have the same image parameters, and can be detected with high accuracy.

[0058] On the other hand, an image captured by the second imaging device 37 of the second sewing machine SM2 reflects the imaging conditions of the second imaging device 37 and has image parameters different from those of the image captured by the first imaging device 37. Therefore, if the template data 70 created by the first sewing machine SM1 (first imaging device 37) is used as is by the second sewing machine SM2 (second imaging device 37) during sewing work, the image parameters of the template data 70 and the image parameters of the image of the sewing object S captured by the second imaging device 37 may not match, and the detection accuracy of the recognition target UP may decrease.

[0059] Therefore, in the sewing system 100 according to this embodiment, the unique data 60 of the imaging device 37 possessed by each sewing machine 1 is used to correct any discrepancy between the imaging conditions of the template data 70 and the imaging conditions of the sewing object S at each imaging device 37.

[0060] [Template data correction] FIG. 8 is a diagram for explaining correction of the template data 70 by the sewing system 100 according to the present embodiment.

[0061] FIG. 8 shows a process of applying template data 70 created by one first sewing machine SM1 (first imaging device 37) to another second sewing machine SM2 (second imaging device 37). Here, the unique data 60 of the first sewing machine SM1 is referred to as unique data 60A, and the unique data 60 of the second sewing machine SM2 is referred to as unique data 60B. In this embodiment, the data processing unit 50 creates template data (unique template data 72) reflecting the unique data 60B of the second sewing machine SM2 from the template data 70 created by the first sewing machine SM1. The template data 70 created by the first sewing machine SM1 reflects the unique data 60A of the first sewing machine SM1, and is therefore also the unique template data 72 of the first sewing machine SM1.

[0062] Data processing unit 50 acquires template data 70 created by first sewing machine SM1 and unique data 60A of first sewing machine SM1 from first sewing machine SM1.

[0063] Data processing unit 50 uses master data management unit 51A to create generalized template data 71 by generalizing (normalizing) template data 70 based on template data 70 of first sewing machine SM1 and unique data 60A.

[0064] That is, the template data 70 is unique template data reflecting the unique data 60A of the first sewing machine SM1, but the unique data 60A is known through calibration. In order to simply explain the concept of normalization, attention will be focused on each of various parameters. The master data management unit 51A normalizes the image scale based on the pixel rate 61 of the unique data 60A. By being given information on the pixel rate 61, the generalized template data 71 becomes normalized data that can be converted into the scale of a captured image under desired imaging conditions. As a result, the generalized template data 71 can be made to match the pixel rate 61 of an image captured by the second sewing machine SM2 by applying the pixel rate 61 of the unique data 60B of the second sewing machine SM2 that uses the data.

[0065] Master data management unit 51A normalizes the luminance information of the image based on illumination information 63 of unique data 60A. By providing illumination information 63, generalized template data 71 becomes normalized data capable of reproducing the luminance distribution of an image captured under desired imaging conditions. As a result, generalized template data 71 can be made to match the luminance distribution (average luminance and contrast) of an image captured by second sewing machine SM2 by applying illumination information 63 of unique data 60B of second sewing machine SM2 that uses the data.

[0066] The master data management unit 51A normalizes the geometric information of the image based on the imaging offset 62 of the unique data 60A. That is, the image position shift, distortion, and rotation angle shift of the template data 70 are removed from the information of the imaging offset 62. That is, the data is converted into data for a case where the imaging device 37 is accurately attached at a position as designed. The generalized template data 71 becomes normalized data that can be converted into the geometric condition of the captured image under the desired imaging condition with a known relative positional relationship by transforming the generalized template data 71 to a certain reference position (here, the design value) based on the information of the imaging offset 62. As a result, the generalized template data 71 can be matched with the geometric information of the image captured by the second sewing machine SM2 by applying the imaging offset 62 of the unique data 60B of the second sewing machine SM2 that uses the data. In addition, if both the imaging offset of the unique data 60A and the imaging offset of the unique data 60B are available, it is also possible to directly calculate the conversion from the geometric information of the first sewing machine SM1 to the geometric information of the second sewing machine SM2 without using the generalized template.

[0067] Master data management unit 51A records the created generalized template data 71 in storage device 52 (see FIG. 2) as master data common to each sewing machine 1.

[0068] Data processing unit 50 acquires unique data 60B of second sewing machine SM2. Data processing unit 50 uses data adjustment unit 51B to create unique template data 72 for each sewing machine based on generalized template data 71 and unique data 60B.

[0069] In other words, data adjustment unit 51B acquires unique data 60B of second sewing machine SM2 to which generalized template data 71 is to be applied, and creates template data (unique template data 72) that reflects the unique data 60B of second sewing machine SM2 to which generalized template data 71 is to be applied, by applying the unique data 60B to generalized template data 71. One generalized template data 71 is created for every sewing machine 1 (sewing system 100), and one unique template data 72 is created separately for each sewing machine 1 (imaging device 37).

[0070] Fig. 9 is a schematic diagram for explaining the transmission of unique template data 72. Fig. 9 illustrates three second sewing machines SM2P, SM2Q, and SM2R. The second sewing machines SM2P, SM2Q, and SM2R each transmit unique data 60B to the data processing unit 50. Here, the unique data 60B of the second sewing machines SM2P, SM2Q, and SM2R are referred to as unique data 60P, 60Q, and 60R, respectively.

[0071] Data adjustment unit 51B creates unique template data 72P for second sewing machine SM2P by applying unique data 60P to generalized template data 71, and transmits this to second sewing machine SM2P. Data adjustment unit 51B creates unique template data 72Q for second sewing machine SM2Q by applying unique data 60Q to generalized template data 71, and transmits this to second sewing machine SM2Q. Data adjustment unit 51B creates unique template data 72R for second sewing machine SM2R by applying unique data 60R to generalized template data 71, and transmits this to second sewing machine SM2R.

[0072] The second sewing machines SM2P, SM2Q, and SM2R each perform image recognition processing using their own inherent template data 72P, 72Q, and 72R acquired from data processing unit 50. Note that the template data 70 created by the first sewing machine AM1 itself is the inherent template data 72 of the first sewing machine AM1, and therefore the first sewing machine AM1 performs image recognition processing using the template data 70 as is.

[0073] As described above, in this embodiment, data processing unit 50 acquires template data of the sewing object S created based on an image captured by first sewing machine SM1 (first imaging device 37) among the multiple sewing machines 1 (imaging devices 37), and transmits the template data to another, second sewing machine SM2 (second imaging device 37). Then, second sewing machine SM2 executes image recognition processing using unique template data 72 obtained by correcting generalized template data 71 based on its own machine's unique data 60.

[0074] Note that, although an example has been shown in which data processing unit 50 creates unique template data 72 for each second sewing machine SM2 from generalized template data 71, it is also possible for data processing unit 50 to distribute generalized template data 71 to second sewing machine SM2, and for each second sewing machine SM2 (image processing unit 36) to create unique template data 72 using its own machine's unique data 60.

[0075] Furthermore, the second sewing machine SM2 may execute image recognition processing using generalized template data 71 instead of using the unique template data 72, and may correct the recognition result using the generalized template data 71 based on the unique data 60 of the sewing machine itself. For example, when an error from the design value of the imaging conditions of the sewing machine itself becomes sufficiently small as a result of calibration, the recognition target UP can be appropriately detected using the generalized template data 71 (or the template data 70 of the first sewing machine SM1). Therefore, after the image recognition processing during sewing work, appropriate image recognition of the sewing object S can be achieved simply by correcting the XY position coordinates, rotation angle, etc. of the detected recognition target UP using the unique data 60 of the sewing machine itself.

[0076] [Sewing method] 10 is a diagram for explaining the sewing method according to the present embodiment. The flow of transmission of template data 70 in sewing system 100 will be described with reference to FIG.

[0077] First, in each of the multiple sewing machines 1 (multiple imaging devices 37), an operation of acquiring unique data 60 is performed (step S1). That is, through a calibration operation, unique data 60 including a pixel rate 61, an imaging offset 62, and illumination information 63 of the imaging device 37 is acquired for each imaging device 37. The acquisition of unique data 60 is performed in both the first sewing machine SM1 and the second sewing machine SM2 described above.

[0078] Next, the first sewing machine SM1 having the first imaging device 37, which is any one of the multiple imaging devices 37, creates template data 70 (step S2). The control device 30 of the first sewing machine SM1 actually images the sewing object S using the first imaging device 37 included in the first sewing machine SM1, and acquires an image of the recognition target UP. The image processing unit 36 ​​of the first sewing machine SM1 creates template data 70 (see FIG. 7) using the acquired image of the recognition target UP.

[0079] Next, data processing unit 50 creates generalized template data 71 (step S3). Data processing unit 50 acquires template data 70 created in step S2 and unique data 60A of first imaging device 37 from first sewing machine SM1 via input / output unit 53. Master data management unit 51A normalizes template data 70 using unique data 60A to create generalized template data 71, and records it in storage device 52.

[0080] Next, data processing unit 50 creates unique template data 72 (step S4). Data processing unit 50 acquires unique data 60B of second imaging device 37 from a second sewing machine SM2 other than first sewing machine SM1, via input / output unit 53. Data adjustment unit 51B creates unique template data 72 of second imaging device 37 of second sewing machine SM2 by applying unique data 60B to generalized template data 71 recorded in storage device 52. When there are multiple second sewing machines SM2, data adjustment unit 51B creates unique template data 72 for each individual machine using unique data 60B acquired from each second sewing machine SM2.

[0081] Next, data processing unit 50 transmits unique template data 72 to second sewing machines SM2 (step S5). Data processing unit 50 transmits corresponding unique template data 72 to each second sewing machine SM2 via input / output unit 53.

[0082] Thereafter, each of the multiple sewing machines 1 executes a sewing process. That is, each of the multiple sewing machines 1 acquires an image of the sewing object S with the imaging device 37, executes an image recognition process of the sewing object S with the image processing unit 36, and performs sewing based on the recognition result obtained by the image recognition process. At this time, each of the second sewing machines SM2 executes the image recognition process using the unique template data 72 corrected by its own unique data 60B. Also, the first sewing machine SM1 executes the image recognition process using the template data 70 created in step S2 as its own unique template data.

[0083] As described above, in the sewing method according to this embodiment, the second sewing machine SM2 may correct the recognition result using the template data (generalized template data 71) based on the unique data 60, instead of using the corrected template data (unique template data 72).

[0084] [sewing machine] Next, a specific example of the sewing machine 1 included in the sewing system 100 will be described below. Fig. 11 is a perspective view showing the sewing machine 1 according to this embodiment. Fig. 12 is a perspective view showing a part of the sewing machine 1 according to this embodiment. The sewing machine 1 shown in Fig. 11 includes a sewing machine main body 10, an operation device 20 operated by an operator Ma, a control device 30 that controls the sewing machine main body 10, an image processing unit 36, and a plurality of imaging devices 37.

[0085] Sewing machine body 10 is mounted on the upper surface of table 2. As shown in Fig. 12, sewing machine body 10 has sewing machine frame 11, needle bar 12 supported by sewing machine frame 11, needle plate 13 supported by sewing machine frame 11, holding member 15 supported by sewing machine frame 11 via support member 14, actuator 16 (see Fig. 11) that generates power to move needle bar 12, actuator 17 (see Fig. 11) that generates power to move holding member 15, and actuator 18 (see Fig. 11) that generates power to move at least a part of holding member 15. Actuator 16, actuator 17, and actuator 18 are specific examples of actuator 10A shown in Fig. 2.

[0086] The sewing machine frame 11 has a horizontal arm 11A extending in the Y-axis direction, a bed 11B (see Figure 11) arranged below the horizontal arm 11A, a vertical arm 11C (see Figure 11) connecting the +Y side end of the horizontal arm 11A to the bed 11B, and a head 11D arranged on the -Y side of the horizontal arm 11A.

[0087] Needle bar 12 holds sewing needle 3. Needle bar 12 holds sewing needle 3 so that sewing needle 3 is parallel to the Z-axis. Needle bar 12 is supported by head 11D so as to be movable in the Z-axis direction.

[0088] The needle plate 13 supports a sewing object S. The needle plate 13 supports a holding member 15. The needle plate 13 is supported by a bed 11B (see FIG. 11). The needle plate 13 is disposed below the holding member 15.

[0089] The holding member 15 holds the sewing object S. The holding member 15 is capable of moving while holding the sewing object S within an XY plane including a sewing position Ps directly below the sewing needle 3. The holding member 15 is capable of holding the sewing object S within an XY plane including a position directly below the imaging device 37. With the holding member 15 holding the sewing object S, it moves within the XY plane based on the sewing data. As a result, a stitch CH (see FIG. 6) is formed on the sewing object S that has passed the sewing position Ps. The holding member 15 is supported by the horizontal arm 11A via the support member 14.

[0090] The holding member 15 has a presser member 15A and a lower plate 15B facing the presser member 15A. The presser member 15A is a frame-shaped member. The presser member 15A is movable in the Z-axis direction. The lower plate 15B is disposed below the presser member 15A. The holding member 15 holds the sewing object S by sandwiching the sewing object S between the presser member 15A and the lower plate 15B.

[0091] When the presser member 15A moves in the +Z direction, the presser member 15A and the lower plate 15B are separated. This allows the operator Ma to place the sewing object S between the presser member 15A and the lower plate 15B. When the presser member 15A moves in the -Z direction with the sewing object S placed between the presser member 15A and the lower plate 15B, the sewing object S is sandwiched between the presser member 15A and the lower plate 15B. This allows the sewing object S to be held by the holding member 15. Furthermore, when the presser member 15A moves in the +Z direction, the holding of the sewing object S by the holding member 15 is released. This allows the operator Ma to remove the sewing object S from between the presser member 15A and the lower plate 15B.

[0092] 11, actuator 16 generates power to move needle bar 12 in the Z-axis direction. Actuator 16 includes a pulse motor. Actuator 16 is disposed on horizontal arm 11A.

[0093] A horizontal arm shaft extending in the Y-axis direction is disposed inside horizontal arm 11A. Actuator 16 is connected to the +Y side end of the horizontal arm shaft. The -Y side end of the horizontal arm shaft is connected to needle bar 12 via a power transmission mechanism disposed inside head 11D. Operation of actuator 16 rotates the horizontal arm shaft. Power generated by actuator 16 is transmitted to needle bar 12 via the horizontal arm shaft and the power transmission mechanism. As a result, sewing machine needle 3 held by needle bar 12 moves back and forth in the Z-axis direction.

[0094] A timing belt extending in the Z-axis direction is disposed inside the vertical arm 11C. A bed shaft extending in the Y-axis direction is disposed inside the bed 11B. Pulleys are disposed on each of the horizontal arm shaft and the bed shaft. The timing belt is hung on each of the pulleys disposed on the horizontal arm shaft and the pulleys disposed on the bed shaft. The horizontal arm shaft and the bed shaft are connected via a power transmission mechanism including the timing belt.

[0095] A shuttle is disposed inside bed 11B. A bobbin placed in a bobbin case is housed in the shuttle. The horizontal arm shaft and the bed shaft are each rotated by the operation of actuator 16. Power generated by actuator 16 is transmitted to the shuttle via the horizontal arm shaft, a timing belt, and the bed shaft. As a result, the shuttle rotates in synchronization with the reciprocating movement of needle bar 12 in the Z-axis direction.

[0096] Actuator 17 generates power to move holding member 15 in the XY plane. Actuator 17 includes a pulse motor. Actuator 17 includes an X-axis motor 17X that generates power to move holding member 15 in the X-axis direction, and a Y-axis motor 17Y that generates power to move holding member 15 in the Y-axis direction. Actuator 17 is disposed inside bed 11B.

[0097] The power generated by the actuator 17 is transmitted to the holding member 15 via the support member 14. This allows the holding member 15 to move in both the X-axis and Y-axis directions between the sewing needle 3 and the needle plate 13. By operating the actuator 17, the holding member 15 can move while holding the sewing object S within an XY plane including the sewing position Ps directly below the sewing needle 3.

[0098] The actuator 18 generates power to move the presser member 15A of the holding member 15 in the Z-axis direction. The actuator 18 includes a pulse motor. When the presser member 15A moves in the +Z direction, the presser member 15A and the lower plate 15B are separated from each other. When the presser member 15A moves in the -Z direction, the sewing object S is sandwiched between the presser member 15A and the lower plate 15B.

[0099] As shown in FIG. 12, the sewing machine body 10 has an intermediate presser member 19 arranged around the sewing needle 3. The intermediate presser member 19 holds down the sewing workpiece S around the sewing needle 3. The intermediate presser member 19 is supported by the head 11D so as to be movable in the Z-axis direction. An intermediate presser motor is arranged inside the head 11D, which generates power to move the intermediate presser member 19 in the Z-axis direction. Operation of the intermediate presser motor causes the intermediate presser member 19 to move in the Z-axis direction in synchronization with the needle bar 12. The intermediate presser member 19 prevents the sewing workpiece S from floating up due to movement of the sewing needle 3.

[0100] 11, the sewing machine 1 includes a drive amount sensor 31 that detects the drive amount of the actuator 16, and a drive amount sensor 32 that detects the drive amount of the actuator 17. The drive amount sensor 31 and the drive amount sensor 32 are specific examples of the drive amount sensor 10B in FIG.

[0101] The control device 30 controls the actuator 16 based on the detection data of the drive amount sensor 31. The control device 30 determines, for example, the operation timing of the actuator 16 based on the detection data of the drive amount sensor 31.

[0102] The drive amount sensor 32 includes an X-axis sensor 32X that detects the drive amount of the X-axis motor 17X of the actuator 17, and a Y-axis sensor 32Y that detects the drive amount of the Y-axis motor 17Y of the actuator 17. The X-axis sensor 32X includes an encoder that detects the rotation amount of the X-axis motor 17X. The Y-axis sensor 32Y includes an encoder that detects the rotation amount of the Y-axis motor 17Y. The detection data of the drive amount sensor 32 is output to the control device 30. The control device 30 controls the actuator 17 based on the detection data of the drive amount sensor 32. The control device 30 feedback-controls the actuator 17 based on the detection data of the drive amount sensor 32 so that the holding member 15 moves to a target position.

[0103] The drive amount sensor 32 functions as a position sensor that detects the position in the XY plane of the holding member 15. The drive amount of the actuator 17 and the movement amount of the holding member 15 correspond one-to-one.

[0104] The X-axis sensor 32X detects the amount of rotation of the X-axis motor 17X, thereby detecting the amount of movement of the holding member 15 in the X-axis direction from the origin in the sewing machine coordinate system. The Y-axis sensor 32Y detects the amount of rotation of the Y-axis motor 17Y, thereby detecting the amount of movement of the holding member 15 in the Y-axis direction from the origin in the sewing machine coordinate system.

[0105] The operation device 20 accepts operation input by the operator Ma. The sewing machine 1 is operated by operating the operation device 20. The operation device 20 includes an operation panel 21 and an operation pedal 22. The operation panel 21 is mounted on the upper surface of the table 2. The operation pedal 22 is disposed below the table 2. The operator Ma operates the operation pedal 22 with his / her foot. The sewing machine 1 is operated by the operator Ma operating at least one of the operation panel 21 and the operation pedal 22.

[0106] The multiple imaging devices 37 capture images of the sewing object S held by the holding member 15. The multiple imaging devices 37 are disposed above the needle plate 13 and the holding member 15. Each of the multiple imaging devices 37 captures an image of at least a portion of the sewing object S held by the holding member 15 from above. The number of imaging devices 37 is not particularly limited. FIG. 11 shows an example in which four imaging devices 37 are provided.

[0107] [Imaging device] 13 is a plan view illustrating an example of the arrangement of the imaging devices 37 according to this embodiment. The imaging devices 37 are arranged, for example, as shown in FIG.

[0108] The multiple imaging devices 37 are arranged at intervals in a horizontal plane so as to be aligned with the sewing machine body 10. The multiple imaging devices 37 are arranged within a sewing area SA. The sewing area SA is a range in the XY plane in which the sewing object S can be moved by the holding member 15 and sewn by the head 11D. The head 11D (i.e., the sewing position Ps) is arranged in the center of the sewing area SA. The multiple imaging devices 37 are arranged along the head 11D of the sewing machine body 10.

[0109] 13, the sewing machine 1 includes imaging units 40A and 40B, one adjacent to each side (+X side, -X side) of the head 11D in the X-axis direction. A plurality of imaging devices 37 can be attached to the imaging units 40A and 40B. The imaging units 40A and 40B are disposed above the sewing area SA. The imaging units 40A and 40B are fixed to the sewing machine 1.

[0110] 13, two imaging devices 37 are attached to each of mounting portions 41A and 41B of imaging unit 40A. The two imaging devices 37 of imaging unit 40A are arranged along the longitudinal direction (X-axis direction) of sewing area SA. The imaging devices 37 are attached to each of mounting portions 41C and 41D of imaging unit 40B. The two imaging devices 37 of imaging unit 40B are arranged along the lateral direction (Y-axis direction) of sewing area SA.

[0111] The imaging device 37 captures at least a part of an image of the sewing object S arranged within the imaging field of view. Each of the imaging devices 37 is arranged to have an overlapping area JA where the imaging field of view of the adjacent imaging device 37 partially overlaps with that of the other imaging device 37. Specifically, the imaging device 37 of the attachment part 41A has an imaging field of view 42A. The imaging device 37 of the attachment part 41B has an imaging field of view 42B. The imaging field of view 42A and the imaging field of view 42B have a rectangular shape and have an overlapping area JA extending in the Y-axis direction. The imaging field of view 42A and the imaging field of view 42B form an enlarged imaging area 43A. The enlarged imaging area is an area where the imaging fields of view of the multiple imaging devices 37 that share the overlapping area JA are combined. The imaging field of view 42A and the imaging field of view 42B arranged in the X-axis direction form a wide enlarged imaging area 43A extending in the X-axis direction.

[0112] The imaging device 37 of the mounting part 41C has an imaging field of view 42C. The imaging device 37 of the mounting part 41D has an imaging field of view 42D. The imaging field of view 42C and the imaging field of view 42D have a rectangular shape and have an overlapping area JA extending in the X-axis direction. The imaging field of view 42C and the imaging field of view 42D form an enlarged imaging area 43B. The imaging field of view 42C and the imaging field of view 42D arranged in the Y-axis direction form a wide enlarged imaging area 43B extending in the Y-axis direction.

[0113] In the configuration of FIG. 13, the imaging field size of the imaging device 37, the size of the overlapping area JA (the interval between the mounting parts), and the number of imaging devices 37 are set according to the dimensions of the target pattern RP (see FIG. 5) of the sewing object S. That is, the enlarged imaging areas 43A and 43B formed by the multiple imaging devices 37 are designed to have a shape that encompasses the entire target pattern RP of the sewing object S. The sewing object S on which a stitch CH extending in the X-axis direction is formed can be imaged by being placed in the enlarged imaging area 43A of the imaging unit 40A, and the sewing object S on which a stitch CH extending in the Y-axis direction is formed can be imaged by being placed in the enlarged imaging area 43B of the imaging unit 40B. Therefore, when a stitch CH extending in either direction is formed, the entire target pattern RP can be imaged without changing the mounting position of the imaging device 37.

[0114] The position of each imaging device 37 is fixed at one of six mounting positions. The relative position of the imaging device 37 and the sewing machine frame 11 is fixed. The relative position of the optical axis of the imaging device 37 and the sewing needle 3 (sewing position Ps) in the XY plane is fixed. Relative position data indicating the relative position of the center of the enlarged imaging areas 43A, 43B and the sewing needle 3 in the XY plane is known data that can be derived from the design data of the sewing machine 1.

[0115] The position of the image captured by the imaging device 37 is defined in the camera coordinate system. The position of the image defined in the camera coordinate system is transformed into the position of the image defined in the sewing machine coordinate system by a defined transformation formula or transformation matrix.

[0116] The imaging units 40A and 40B also hold a plurality of lighting devices 38A and 38B. The imaging units 40A and 40B hold the lighting devices 38A and 38B in a fixed state above the sewing area SA. The lighting devices 38A and 38B are a specific example of the lighting device 38 in FIG.

[0117] In the imaging unit 40A, one illumination device 38A is disposed on each of both outer sides of the arrangement of the mounting parts 41A and 41B along the X-axis direction and between the arrangements (three in total). Each illumination device 38A extends linearly along the Y-axis direction. Each illumination device 38A irradiates light in the form of epi-illumination to an imaging field set below the mounting parts 41A and 41B. In the imaging unit 40B, one illumination device 38B is disposed on each of both outer sides of the arrangement of the mounting parts 41C and 41D along the Y-axis direction and between the arrangements (three in total). Each illumination device 38B extends linearly along the X-axis direction. Each illumination device 38B irradiates light in the form of epi-illumination to an imaging field set below the mounting parts 41C and 41D.

[0118] [Image processing] Next, a description will be given of image processing using the multiple imaging devices 37. The image processing unit 36 ​​(see FIG. 11) acquires images of the sewing object S from the multiple imaging devices 37. When the sewing object S is positioned at a predetermined imaging position by the control device 30, the image processing unit 36 ​​controls the operation of the imaging device 37 and the lighting device 38 corresponding to the imaging position, and acquires the image of the sewing object S.

[0119] In FIG. 13, the imaging position Pf is set at the center of the enlarged imaging areas 43A and 43B. Imaging is performed with the center of the holding member 15 (sewing object S) being placed at the imaging position Pf of either of the enlarged imaging areas 43A and 43B. For example, in the imaging unit 40A, when the sewing object S is placed at the imaging position Pf of the enlarged imaging area 43A, the image processing unit 36 ​​turns on the illumination device 38A and causes the imaging device 37 of the mounting part 41A and the imaging device 37 of the mounting part 41B to perform imaging substantially simultaneously. As a result, an image corresponding to the imaging field of view 42A and an image corresponding to the imaging field of view 42B are acquired. In the imaging unit 40B, when the sewing object S is placed at the imaging position Pf of the enlarged imaging area 43B, the image processing unit 36 ​​turns on the illumination device 38B and causes the imaging device 37 of the mounting part 41C and the imaging device 37 of the mounting part 41D to perform imaging substantially simultaneously. The imaging may be performed in sequence.

[0120] When the recognition target UP is located on the boundary between the images of the multiple imaging devices 37, the image processing unit 36 ​​performs an image synthesis process to join the images of the multiple imaging devices 37. In this case, it is necessary to match the coordinate systems of the two imaging devices 37. For one of the imaging devices 37, the data conversion process is performed using the imaging conditions of the other imaging device 37, i.e., the unique data 60 in FIG. 3, not between the sewing machines 1, but between the imaging devices 37 of the same sewing machine 1. The image processing unit 36 ​​joins together the multiple images that make up the enlarged imaging area (43A or 43B) based on the image portions of the overlapping areas JA to generate one composite image 46 (see FIG. 15). Therefore, the composite image 46 is an image that captures a range equivalent to the enlarged imaging area 43A or 43B. The enlarged imaging areas 43A and 43B have a size that includes the multiple recognition targets UP (see FIG. 5) of the sewing object S. Therefore, the composite image 46 is an image that captures the multiple recognition targets UP. The composite image 46 preferably includes all the recognition targets UP (UP1 to UP7) detected when forming one stitch CH.

[0121] The image processing unit 36 ​​executes image recognition processing to detect a plurality of recognition targets UP from the composite image 46. The image processing unit 36 ​​acquires position coordinates of the recognition targets UP in the composite image 46 by matching with the unique template data 72. The image processing unit 36 ​​sequentially detects the plurality of recognition targets UP included in the composite image 46, and acquires position coordinates of each of the plurality of recognition targets UP.

[0122] The image processing unit 36 ​​calculates correction data for correcting the displacement of the surface of the sewing object S based on the detection result by the image processing unit 36.

[0123] Here, the sewing data stored in the storage device 35 (see FIG. 2) includes a target pattern RP of the stitch CH to be formed on the sewing object S, the position coordinates of the recognition target UP, and the movement conditions of the holding member 15.

[0124] The target pattern RP defines the target shape of the stitch CH to be formed on the sewing workpiece S and the target position of the stitch CH in the sewing machine coordinate system.

[0125] The movement conditions of the holding member 15 include a movement trajectory of the holding member 15 defined in the sewing machine coordinate system. The movement trajectory of the holding member 15 includes a movement trajectory of the holding member 15 in the XY plane. The movement conditions of the holding member 15 are determined based on the target pattern RP.

[0126] The target pattern RP of the sewing data is generated on the assumption that the sewing object S is not shrunk, and is stored in the control device 30. When the surface of the sewing object S is displaced, the image processing unit 36 ​​obtains the amount of displacement of the detected recognition object UP and generates correction data for correcting the sewing data.

[0127] The image processing unit 36 ​​transmits the correction data calculated by the image processing unit 36 ​​to the control device 30. The control device 30 executes the sewing process based on the correction data.

[0128] [Sewing operation] 14 is a flowchart for explaining the sewing operation of the sewing machine 1 according to this embodiment. The sewing operation of the sewing machine 1 includes an image capturing process S11, an image joining process S12, a recognition target detection process S13, a correction data calculation process S14, a sewing process S15, and an end determination process S16.

[0129] (Image capture processing) The imaging process S11 is a process of imaging the sewing object S by the multiple imaging devices 37. Whether to capture an image of the enlarged imaging area 43A extending in the X-axis direction by the imaging unit 40A or to capture an image of the enlarged imaging area 43B extending in the Y-axis direction by the imaging unit 40B is set in advance by the operator Ma or by the control device 30 based on the sewing data. The control device 30 moves the holding member 15 that holds the sewing object S to place the holding member 15 at a preset imaging position. When the holding member 15 is placed at the corresponding imaging position Pf, the image processing unit 36 ​​performs imaging by the set multiple imaging devices 37.

[0130] In the following, as an example, a case where an image of the enlarged imaging area 43A is acquired by each imaging device 37 of the imaging unit 40A in Fig. 13 will be described. In this case, the image processing unit 36 ​​acquires an image corresponding to the imaging field of view 42A and an image corresponding to the imaging field of view 42B by the imaging device 37 of the mounting part 41A and the imaging device 37 of the mounting part 41B.

[0131] (Image stitching process) The image joining process S12 is a process for joining together images of the sewing object S captured by the multiple imaging devices 37. Fig. 15 is a schematic diagram for explaining the image joining process S12.

[0132] 15, an image obtained by the imaging device 37 of the mounting part 41A is defined as a first image 44. An image obtained by the imaging device 37 of the mounting part 41B is defined as a second image 45. In this case, the second image 45 includes an area on the +X direction side with respect to the first image 44. The first image 44 corresponds to the imaging field of view 42A, and the second image 45 corresponds to the imaging field of view 42B. The first image 44 and the second image 45 share an overlapping area JA.

[0133] The overlapping area JA between the first image 44 and the second image 45 includes an image element GE that is common to at least a part of the recognition target UP. That is, as shown in FIG. 5, when the recognition target UP is an array of a plurality of holes 7 that exist in a predetermined pattern in the sewing object S, the image element GE is an image of each hole 7. The holes 7 are components that make up the recognition target UP and the reference pattern DPh. For convenience of explanation, FIG. 15 shows an example in which the arrangement of the holes 7 is simplified and the holes 7 are arranged vertically and horizontally in an array. Prior to the image stitching process S12, the image processing unit 36 ​​may binarize each image obtained in the imaging process S11. In this case, the template data 70 (generalized template data 71, specific template data 72) also becomes a binarized image.

[0134] As shown in FIG. 15, the image processing unit 36 ​​detects image elements GE included in the overlapping area JA of the first image 44 and image elements GE included in the overlapping area JA of the second image 45. Since the position coordinates of the imaging center of the imaging device 37 attached to the attachment parts 41A and 41B and the size of the imaging fields of view 42A and 42B are known, the range of the overlapping area JA in each image is known. The image processing unit 36 ​​selects three or more image elements GE included in the overlapping area JA, and transforms the second image 45 so that the selected image elements GE match the first image 44. The transformation of the second image 45 is performed, for example, by affine transformation. As a result of the transformation, the images of the overlapping area JA of the first image 44 and the transformed second image 45 (hereinafter referred to as the second image 45A) substantially match each other.

[0135] The image processing unit 36 ​​generates a composite image 46 by stitching together the first image 44 and the second image 45A. The pixels constituting the overlapping area JA may be selected from either the first image 44 or the second image 45A, or the pixel information of both may be averaged. In this manner, in the image stitching process S12, a composite image 46 having a size equivalent to the enlarged imaging area 43A is generated.

[0136] (Recognition target detection process) Next, the recognition target detection process S13 in Fig. 14 is a process for detecting the recognition target UP from the composite image 46 generated by the image stitching process S12 and acquiring the position coordinates of the detected recognition target UP. Fig. 16 is a schematic diagram for explaining the recognition target detection process S13.

[0137] As shown in FIG. 16, the image processing unit 36 ​​extracts an image of a recognition area AR in which detection processing of the recognition target UP is performed from the composite image 46. The recognition area AR is set in a range that takes into consideration the expected positional deviation amount of the recognition target UP accompanying stitch formation with respect to the design position coordinates of the recognition target UP set in the sewing data. FIG. 16 shows a schematic diagram of setting the recognition area AR for a target pattern RP of an arbitrary shape. The image processing unit 36 ​​sets a recognition area AR for each of a plurality of recognition targets UP to be detected in one stitch formation, and obtains an extracted image by cutting out the image of the set recognition area AR. For example, in the example of FIG. 5, a recognition area AR is set for each of the seven recognition targets UP from recognition target UP1 to recognition target UP7.

[0138] The image processing unit 36 ​​performs image recognition processing by the template matching method on each of the acquired extracted images in order. The image processing unit 36 ​​reads out the unique template data 72 from the storage device 35, and acquires the position coordinates of the recognition target UP in the extracted image (recognition area AR) by matching with the unique template data 72.

[0139] Through the recognition target detection process S13, the image processing section 36 acquires the position coordinates in the sewing machine coordinate system of each recognition target UP (UP1 to UP7) to be detected in one stitch formation.

[0140] (Correction data calculation process) The correction data calculation process S14 is a process for calculating correction data for correcting the displacement of the surface of the sewing object S based on the detection result by the recognition object detection process S13. The image processing unit 36 ​​calculates the displacement amount of the recognition object UP from the initial position to the current position based on the initial position and the current position of the recognition object UP related to the current sewing process. The initial position of the recognition object UP is the design position coordinate of the recognition object UP set in the sewing data. The current position of the recognition object UP is the position coordinate of the recognition object UP detected by the recognition object detection process S13. The image processing unit 36 ​​calculates the correction position coordinate of each correction point CP (see FIG. 14) on the target pattern RP based on the calculated displacement amount so that the stitch CH is formed at the target position in the sewing machine coordinate system, and calculates the correction data of the target pattern RP passing through the correction point CP. Note that in the initial sewing process (first sewing process), since no displacement of the surface of the sewing object S occurs due to stitch formation, no displacement of the recognition object UP occurs either. Therefore, in the initial sewing process (first sewing process), the target pattern RP in the initial state defined by the sewing data is used as is, and no correction data is calculated.

[0141] (Sewing process) The sewing process S15 is a process for forming a stitch CH based on a target pattern RP. The sewing process S15 is performed by controlling each part of the sewing machine 1 by the control device 30. The sewing process includes a first sewing process to an N-th sewing process. The first sewing process is performed based on the target pattern RP in the initial state defined by the sewing data. In the first sewing process, the control device 30 outputs a control command to the actuator 17 so that the stitch CH is formed according to the target pattern RP in the initial state defined by the sewing data. The second sewing process to the N-th sewing process are performed based on the correction data (the corrected target pattern RP) calculated by the correction data calculation process S14. After the second sewing process, the control device 30 acquires the correction data from the image processing unit 36 ​​and outputs a control command to the actuator 17 so that the stitch CH is formed according to the target pattern RP of the acquired correction data.

[0142] The end determination process S16 is a process for determining whether or not the sewing process of the sewing object S has been completed. The control device 30 determines whether or not the sewing process of the sewing object S has been completed based on the sewing data. When the first sewing process to the (N-1)th sewing process have been completed, the control device 30 determines in the end determination process S16 that the sewing process has not been completed. When the Nth sewing process has been completed, the control device 30 determines in the end determination process S16 that the sewing process has been completed.

[0143] Therefore, in the first sewing process to the Nth sewing process, each time one sewing process is completed, the image capturing process S11, the image stitching process S12, the recognition target detection process S13, and the correction data calculation process S14 are carried out to calculate the correction data for the next sewing process, and the next sewing process is carried out based on the calculated correction data. This series of processes is repeated.

[0144] 11 to 16, each of the sewing machines 1 included in the sewing system 100 includes a plurality of imaging devices 37 having overlapping areas JA where the imaging fields of view (42A and 42B, 42C and 42D) partially overlap. Then, an image recognition process is performed on a composite image 46 obtained by stitching together a plurality of images obtained from the plurality of imaging devices 37 based on template data (unique template data 72).

[0145] This allows a plurality of recognition targets UP detected in the sewing process of one stitch CH to be detected from a composite image 46 obtained by one imaging by the plurality of imaging devices 37. As a result, the processing time required for the imaging process S11, the recognition target detection process S13, and the correction data calculation process S14, which are performed for each sewing process, is reduced compared to the case where the recognition targets UP are imaged and detected one by one by one with one imaging device 37.

[0146] On the other hand, when images obtained from a plurality of imaging devices 37 are stitched together to create a composite image 46 covering a wide range, the imaging field of view of each imaging device 37 becomes large, and the influence of variations in imaging conditions for each imaging device 37 also becomes large. Therefore, compared with the case where the recognition target UP is simply imaged and detected one by one by one with one imaging device 37, the influence of the difference between the imaging conditions when the template data 70 is created by the first sewing machine SM1 and the imaging conditions of the second sewing machine SM2 on the image recognition process is also large. Therefore, the configuration of this embodiment, which can correct the difference between the imaging conditions of the template data 70 and the imaging conditions of the second sewing machine SM2 by the unique data 60, is effective for obtaining recognition accuracy in the image recognition process using the composite image 46.

[0147] [effect] As described above, according to this embodiment, the second sewing machine SM2 (sewing machine 1 having the second imaging device 37) executes the image recognition process using the template data (unique template data 72) corrected based on the unique data 60B of the second imaging device 37. As a result, the template data 70 created by the first sewing machine SM1 (sewing machine 1 having the first imaging device 37) and reflecting the imaging conditions unique to the first sewing machine SM1 can be corrected by the second sewing machine SM2 to match the imaging conditions unique to the second sewing machine SM2 and then the image recognition process can be executed. Therefore, it is not necessary to create the template data 70 separately for each of the multiple sewing machines 1 included in the sewing system 100. Furthermore, the administrator Mb only needs to manage one template data 70 (generalized template data 71) created by the first sewing machine SM1 in the data processing unit 50, and does not need to manage separate template data (unique template data 72) for each sewing machine 1. As a result, it is possible to reduce the number of work steps and the number of data management steps related to image recognition in the sewing system 100 including the multiple sewing machines 1. Note that the second sewing machine SM2 may perform image recognition processing using the template data 70 created by the first sewing machine SM1, and correct the recognition result based on the unique data 60B of the second sewing machine SM2. Even in this case, it is possible to reduce the number of work steps and the number of data management steps related to image recognition. Furthermore, as shown in Figures 12 and 13, even if one sewing machine 1 is configured to have multiple imaging devices 37 (i.e., a first imaging device 37 and a second imaging device 37), the same effect can be obtained by correcting the template data 70 created by the first imaging device 37 based on the unique data 60 of the second imaging device 37, or by correcting the image recognition results using the template data 70 based on the unique data 60 of the second imaging device 37.

[0148] Furthermore, according to the present embodiment, since data processing unit 50 has master data management unit 51A that creates generalized template data 71, elements resulting from unique data 60A of first sewing machine SM1 can be eliminated from template data 70 created by first sewing machine SM1 (sewing machine 1 having first imaging device 37). As a result, when creating unique template data 72 for second sewing machine SM2 (sewing machine 1 having second imaging device 37), it is not necessary to use unique data 60A of first sewing machine SM1, which is the source of creation of template data 70, every time, and therefore data management man-hours can be reduced. Furthermore, since data processing unit 50 has data adjustment unit 51B that creates unique template data 72 for each imaging device based on generalized template data 71 and unique data 60, unique template data 72 suitable for second sewing machine SM2 can be acquired without correcting the image recognition result of second sewing machine SM2 every time.

[0149] [Other embodiments] In the above embodiment, an example is shown in which the unique template data 72 for the second sewing machine SM2 is created from the template data 70 of the first sewing machine SM1 and transmitted to the second sewing machine SM2, but this is not limiting. The data processing unit 50 may transmit the template data 70 and unique data 60A of the first sewing machine SM1 to the second sewing machine SM2 as is, and the second sewing machine SM2 may correct the template data 70 (create the unique template data 72) based on the difference between the unique data 60A of the first sewing machine SM1 and the unique data 60B of the second sewing machine SM2. In addition, when the difference in the imaging conditions is small, the template data 70 of the first sewing machine SM1 may be used as is in the second sewing machine SM2 to perform image recognition processing, and the recognition result using the template data 70 may be corrected based on the difference between the unique data 60A of the first sewing machine SM1 and the unique data 60B of the second sewing machine SM2.

[0150] In the above embodiment, master data management unit 51A that creates generalized template data 71 is provided in data processing unit 50, but generalized template data 71 does not necessarily have to be created. As described above, template data 70 and unique data 60A of first sewing machine SM1 and unique data 60B of second sewing machine SM2 may be used to create unique template data 72 of second sewing machine SM2. In this case, template data 70 and unique data 60A of first sewing machine SM1 may be managed as master data instead of generalized template data 71.

[0151] In the above embodiment, data processing unit 50 is provided with data adjustment unit 51B that creates unique template data 72 for each sewing machine, but data processing unit 50 does not have to be provided with data adjustment unit 51B. Data processing unit 50 may transmit generalized template data 71 to each second sewing machine SM2, and each second sewing machine SM2 may create its own unique template data 72 using its own unique data 60B.

[0152] In the above embodiment, an example has been shown in which multiple imaging devices 37 are provided in the sewing machine 1 and image recognition processing is performed using a composite image 46 formed by stitching together multiple images, but the sewing machine 1 may be equipped with only one imaging device 37. The sewing machine 1 may also capture an image of the sewing object S multiple times and perform image recognition processing on each image. [Explanation of symbols]

[0153] 1... sewing machine, 2... table, 3... sewing machine needle, 4... surface material, 5... padding material, 6... backing material, 7... hole, 10... sewing machine body, 10A... actuator, 10B... drive amount sensor, 11... sewing machine frame, 11A... horizontal arm, 11B... bed, 11C... vertical arm, 11D... head, 12... needle bar, 13... needle plate, 14... support member, 15... holding member, 15A... presser member, 15B... lower plate, 16... actuator, 17... actuator, 17X... X-axis motor, 17Y... Y-axis Motor, 18... actuator, 19... intermediate presser member, 20... operation device, 21... operation panel, 22... operation pedal, 30... control device, 31... drive amount sensor, 32... drive amount sensor, 32X... X-axis sensor, 32Y... Y-axis sensor, 35... storage device, 36... image processing unit, 37... imaging device, 38, 38A, 38B... lighting device, 39... communication device, 40A, 40B... imaging unit, 41A, 41B, 41C, 41D... mounting portion, 42A, 42B, 42C, 42D... imaging field of view, 43A, 43B...Enlarged imaging area, 44...First image, 45, 45A...Second image, 46...Synthetic image, 50...Data processing unit, 51...Processor, 51A...Master data management unit, 51B...Data adjustment unit, 52...Storage device, 53...Input / output unit, 60, 60A, 60B, 60P, 60Q, 60R...Unique data, 61...Pixel rate, 62...Imaging offset, 63...Illumination information, 70...Template data, 71...Generalized template data, 72, 72P, 72Q, 72R...unique template data, 100...sewing system, 101...gateway, 102...communication network, AR...recognition area, CH...stitch, CP...correction point, DPh...reference pattern, GE...image element, JA...overlapping area, Ma...operator, Mb...manager, Pf...imaging position, Ps...sewing position, RP...target pattern, S...sewing object, SA...sewing area, SM1...first sewing machine, SM2, SM2P, SM2Q, SM2R...second sewing machine, UP...recognition object.

Claims

1. A plurality of imaging devices for acquiring images of a sewing object; one or more sewing machines that perform sewing based on a recognition result obtained by image recognition processing of the sewing object; a data processing unit that acquires template data of the sewing object created based on an image captured by a first imaging device among the plurality of imaging devices, and transmits the template data to another second imaging device, the sewing machine has unique data including imaging conditions of the imaging device; The sewing machine having the second imaging device executes an image recognition process using the template data corrected based on the unique data of the second imaging device, or corrects a recognition result using the template data based on the unique data of the second imaging device. Sewing system.

2. The specific data includes a pixel rate of the image capture device, an image capture offset, and illumination information. The sewing system of claim 1 .

3. the data processing unit has a master data management unit that creates generalized template data by generalizing the template data based on the template data and the unique data of the first imaging device; the sewing machine having the second imaging device executes an image recognition process using specific template data obtained by correcting the generalized template data based on the specific data of the sewing machine itself, or corrects a recognition result using the generalized template data based on the specific data of the sewing machine itself. The sewing system according to claim 1 or 2.

4. the data processing unit has a data adjustment unit that acquires the unique data of the plurality of imaging devices, and creates the unique template data for each of the imaging devices based on the generalized template data or the unique template data including the unique data of the imaging device used when creating the template, and the unique data; the sewing machine having the second imaging device executes an image recognition process using the unique template data of the sewing machine acquired from the data processing unit; The sewing system of claim 3 .

5. The sewing machine includes a plurality of the imaging devices having overlapping regions in which imaging fields of view partially overlap, and performs image recognition processing based on the template data on a composite image obtained by stitching together a plurality of images obtained from the plurality of the imaging devices. The sewing system according to claim 1 or 2.

6. A sewing method using a plurality of sewing machines having an imaging device for capturing an image of a sewing object and performing sewing based on a recognition result obtained by an image recognition process of the sewing object, or a sewing machine having a plurality of the imaging devices, acquiring template data of the sewing object created based on an image captured by a first imaging device among the plurality of sewing machines or the plurality of imaging devices, and transmitting the template data to another second imaging device; In the sewing machine having the second imaging device, executing an image recognition process using the template data corrected based on unique data including imaging conditions of the second imaging device, or correcting a recognition result using the template data based on the unique data. Sewing method.

Images