Sewing system and sewing method

Abstract

A sewing system (100) comprises: a plurality of imaging devices (37) that acquire an image of an object (S) to be sewn; one or a plurality of sewing machines (1) that perform sewing on the basis of a recognition result obtained by image recognition processing of the object (S) to be sewn; and a data processing unit (50) that acquires template data (70) of the object (S) to be sewn created on the basis of the image captured by a first imaging device (37) among the plurality of imaging devices (37), and transmits the template data (70) to a different, second imaging device (37). Each sewing machine (1) has unique data (60B) that includes imaging conditions of the imaging devices (37). The sewing machine (1) including the second imaging device (37) executes image recognition processing using template data (72) corrected on the basis of the unique data (60B) of the second imaging device (37), or corrects the result of recognition using the template data (70) on the basis of the unique data (60B) of the second imaging device (37).

Description

Sewing system and sewing method

[0001] The present disclosure relates to a sewing system and a sewing method.

[0002] BACKGROUND ART In order to enhance the design of an object to be sewn, stitches may be formed on the object to be sewn. Patent Document 1 discloses a technique for forming stitches on an upholstery material used for a vehicle seat.

[0003] Japanese Patent Application Publication No. 2013-162957

[0004] Upholstery materials used for vehicle seats are thick and elastic. When stitches are formed on a thick and elastic sewing object, the sewing object may shrink, causing a displacement of the surface of the sewing object. For example, when a second stitch is formed after a first stitch is formed, it is desirable to form the second stitch at a target position on the sewing object in accordance with the displacement of the surface of the sewing object caused by the formation of the first stitch. One way to form a stitch at a target position on the sewing object is to photograph the surface of the sewing object before sewing and detect the displacement of the surface of the sewing object. Image recognition processing such as template matching is used to detect the displacement.

[0005] When the same product is manufactured using multiple sewing machines in a sewing factory or the like, the multiple sewing machines form the same stitches on the same shaped sewing object, so the template data used for image recognition processing is also common. However, in reality, the imaging conditions of the imaging devices installed in each sewing machine vary due to individual differences in the imaging devices, assembly errors, etc., so separate template data is prepared for each sewing machine. This increases the number of work steps and the number of data management steps required for all multiple sewing machines.

[0006] An aspect of the present disclosure aims to reduce the number of work steps and data management steps involved in image recognition in a sewing system equipped with multiple sewing machines.

[0007] According to a first aspect of the present disclosure, there is provided a sewing system comprising: a plurality of imaging devices that acquire images of a sewing object; one or more sewing machines that perform sewing based on a recognition result obtained by image recognition processing of the sewing object; and a data processing unit that acquires template data of the sewing object created based on an image captured by a first imaging device among the plurality of imaging devices and transmits the template data to another second imaging device, wherein the sewing machines have unique data including imaging conditions of the imaging devices, and the sewing machine having the second imaging device performs image recognition processing using the template data corrected based on the unique data of the second imaging device, or corrects the recognition result using the template data based on the unique data of the second imaging device.

[0008] According to a second aspect of the present disclosure, there is provided a sewing method using a plurality of sewing machines, or a sewing machine having a plurality of the imaging devices, each having an imaging device that acquires an image of an object to be sewn and performing sewing based on a recognition result obtained by image recognition processing of the object to be sewn, the sewing method including: acquiring template data of the object to be sewn, created based on an image captured by a first imaging device among the plurality of sewing machines or the plurality of imaging devices, and transmitting the template data to another, second imaging device; and, in the sewing machine having the second imaging device, performing image recognition processing using the template data corrected based on unique data including imaging conditions of the second imaging device, or correcting the recognition result using the template data based on the unique data.

[0009] According to an aspect of the present disclosure, it is possible to reduce the number of work steps and data management steps involved in image recognition in a sewing system equipped with multiple sewing machines.

[0010] FIG. 1 is a diagram schematically illustrating a sewing system according to the present embodiment. FIG. 2 is a functional block diagram illustrating a sewing machine and a data processing unit according to the present embodiment. FIG. 3 is an explanatory diagram of unique data possessed by a plurality of sewing machines. FIG. 4 is a cross-sectional view illustrating a portion of a sewing object. FIG. 5 is a plan view illustrating a sewing object. FIG. 6 is a cross-sectional view illustrating an example of a sewing object. FIG. 7 is a diagram illustrating template data of a recognition object. FIG. 8 is a diagram illustrating correction of template data by the sewing system according to the present embodiment. FIG. 9 is a schematic diagram illustrating transmission of unique template data. FIG. 10 is a diagram illustrating a sewing method according to the present embodiment. FIG. 11 is a perspective view illustrating a sewing machine according to the present embodiment. FIG. 12 is a perspective view illustrating a portion of the sewing machine according to the present embodiment. FIG. 13 is a plan view illustrating an example arrangement of an imaging device according to the present embodiment. FIG. 14 is a flowchart for illustrating the sewing operation of a sewing machine according to the present embodiment. FIG. 15 is a schematic diagram illustrating image stitching processing. FIG. 16 is a schematic diagram illustrating recognition object detection processing.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited thereto. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0012] In this embodiment, a local coordinate system is defined for the sewing machine 1. In the following description, the local coordinate system defined for the sewing machine 1 will be referred to as the sewing machine coordinate system as appropriate. The sewing machine coordinate system is defined by an XYZ Cartesian coordinate system. In this embodiment, the positional relationship of each part will be described based on the sewing machine coordinate system. The direction parallel to the X axis in a predetermined plane is defined as the X-axis direction. The direction parallel to the Y axis in a predetermined plane that is perpendicular to the X axis is defined as the Y-axis direction. The direction parallel to the Z axis that is perpendicular to the predetermined plane is defined as the Z-axis direction. Furthermore, the direction of rotation or tilt around the X axis is defined as the θX direction. The direction of rotation or tilt around the Y axis is defined as the θY direction. The direction of rotation or tilt around the Z axis is defined as the θZ direction. Furthermore, in this embodiment, the plane including the X axis and the Y axis is defined as the XY plane as appropriate. The plane including the X axis and the Z axis is defined as the XZ plane as appropriate. The plane including the Y axis and the Z axis is defined as the YZ plane as appropriate. The XY plane is parallel to a predetermined plane. The XY plane, XZ plane, and YZ plane are each perpendicular to each other. In this embodiment, the XY plane and the horizontal plane are parallel. The Z-axis direction is the up-down direction on the predetermined plane. The +Z direction is the upward direction. The -Z direction is the downward direction. The XY plane may be inclined with respect to the horizontal plane.

[0013] [Overview of Sewing System] FIG. 1 is a diagram schematically illustrating a sewing system 100 according to the present embodiment. As shown in FIG. 1, the sewing system 100 includes a plurality of sewing machines 1, a plurality of gateways 101 corresponding to the respective sewing machines 1, a communication network 102, and a data processing unit 50. The sewing system 100 is installed, for example, in a sewing factory, which is an operating facility where a plurality of sewing machines 1 are operated. In the sewing factory, a sewn product is produced by the sewing machines 1 sewing an object S to be sewn with sewing thread. Although FIG. 1 illustrates three sewing machines 1, the number of sewing machines 1 included in the sewing system 100 is not particularly limited. The number of sewing machines 1 included in the sewing system 100 may be one, two, four or more. Furthermore, data may be exchanged between the sewing machines 1 and the data processing unit 50 without using the communication network 102. Data may be exchanged between the sewing machines 1 and the data processing unit 50 via an external memory medium such as a USB memory. Furthermore, even if there is only one sewing machine 1, and the imaging conditions change due to replacement of a camera or the like due to a malfunction, the template data can be corrected using the same principle as described below. This correction of the template data can reduce the number of work steps in the sewing system 100.

[0014] The sewing machine 1 is an industrial sewing machine operated by an operator Ma. The sewing machine 1 of this embodiment is a cycle sewing machine that automatically performs pattern sewing based on a predetermined sewing pattern that has been set. The sewing machine 1 starts the process of sewing the set predetermined sewing pattern when the operator Ma performs a predetermined operation while setting the sewing object S and sewing thread in predetermined positions.

[0015] In this embodiment, the multiple sewing machines 1 are used to manufacture the same product. That is, the multiple sewing machines 1 sew the same pattern on sewing objects S of the same shape. For example, a sewing factory is provided with multiple production lines that manufacture the same product. Each sewing machine 1 is installed in each of the multiple production lines that manufacture the same product.

[0016] The sewing machine 1 connects to the communication network 102 via a corresponding gateway 101. The gateway 101 is disposed on a communication line connecting the sewing machine 1 and the communication network 102. The gateway 101 is a device that relays the transmission and reception of various data in communication between the sewing machine 1 and the communication network 102, and converts the protocol of the received data before transferring it.

[0017] The data processing unit 50 is an information terminal operated by an administrator Mb to manage data of the multiple sewing machines 1 of the sewing system 100. Typical examples of the data processing unit 50 include a file server or a network-attached storage (NAS) that manages files. The data processing unit 50 may be realized by utilizing the file management and sharing functions of a desktop personal computer (PC), a notebook PC, a smartphone, a tablet terminal, or the like. One of the multiple sewing machines 1 can be designated as a master machine and also function as the data processing unit 50.

[0018] The data processing unit 50 is connected to the communication network 102 and can communicate with a plurality of sewing machines 1 via each gateway 101. The data processing unit 50 transmits and receives data necessary for sewing to and from each sewing machine 1 via the communication network 102.

[0019] [Sewing Machine and Management Device] Fig. 2 is a functional block diagram showing the sewing machine 1 and the data processing unit 50 according to this embodiment. Fig. 2 shows only the schematic configuration of the sewing machine 1.

[0020] As shown in FIG. 2 , the sewing machine 1 includes a sewing machine main body 10 , a control device 30 , a storage device 35 , an image processing unit 36 ​​, an imaging device 37 , an illumination device 38 , and a communication device 39 .

[0021] The sewing machine body 10 is a mechanical part that sews on the sewing object S. The sewing machine body 10 has an actuator 10A and a drive amount sensor 10B.

[0022] The actuator 10A includes various drive mechanisms and drive sources that operate the various parts of the sewing machine body 10. The actuator 10A executes various operations for the sewing machine 1 to sew a predetermined sewing pattern that has been set.

[0023] The drive amount sensor 10B detects the drive amount of the actuator 10A. The drive amount sensor 10B includes, for example, an encoder that detects the rotation amount of the motor, a position sensor that detects the position of a needle holder, etc. The actuator 10A of the sewing machine main body 10 is controlled by the control device 30 based on the detection result of the drive amount sensor 10B.

[0024] The control device 30 includes a computer system. The control device 30 executes various controls on each part of the sewing machine 1. The control device 30 includes a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor executes various arithmetic processes that realize the functions of the control device 30 based on programs stored in the storage device 35. The control device 30 controls the actuator 10A of the sewing machine main body 10 in accordance with the sewing data set in the storage device 35.

[0025] The storage device 35 includes a primary storage device, which is a volatile memory that temporarily stores data, and a secondary storage device, which is a nonvolatile memory that stores various programs and various data used by the processor for calculations. The primary storage device is, for example, a random access memory (RAM). The secondary storage device is, for example, a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), or other storage. As will be described later, the storage device 35 stores, in addition to the program for the control device 30, unique data 60 related to the imaging device 37 and the sewing machine main body 10 provided in the sewing machine 1. The storage device 35 stores sewing data for executing the sewing process.

[0026] The sewing data is known data that can be derived from design data of the sewing object S, such as CAD (Computer Aided Design) data. The sewing data is referenced by the control device 30 during sewing processing. The sewing processing refers to processing for forming stitches CH (see FIG. 6 ) in the sewing object S. The storage device 35 also stores template data used in image recognition processing for an image of the sewing object S captured by the imaging device 37.

[0027] The image processing unit 36 ​​includes a computer system. The image processing unit 36 ​​has a processor, a storage device including a non-volatile memory and a volatile memory, and an input / output unit including an input / output interface. The processor is, for example, a central processing unit (CPU). The non-volatile memory is, for example, a read-only memory (ROM) or storage. The volatile memory is, for example, a random access memory (RAM). The image processing unit 36 ​​performs image recognition processing in which an image of the sewing object S is captured by an imaging device 37 in accordance with a computer program stored in the storage device, and a recognition target UP (see FIG. 5) is detected from the image.

[0028] The imaging device 37 captures an image of the sewing object S. The imaging device 37 has an optical system and an image sensor that receives light incident thereon via the optical system. The image sensor includes a CCD (Couple Charged Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0029] The illumination device 38 irradiates light onto the imaging field of view of the imaging device 37. The illumination device 38 has a light source such as an LED light-emitting element.

[0030] The communication device 39 communicates with the data processing unit 50 via the gateway 101 and the communication network 102. The communication device 39 includes a communication interface that complies with a wired or wireless communication standard.

[0031] The data processing unit 50 includes a computer system, and includes a processor 51, a storage device 52, and an input / output unit 53.

[0032] The data processing unit 50 may be connected to an input device or display device (not shown). The input device is realized by, for example, a keyboard, a mouse, operation keys, a microphone for accepting voice input, etc. The display device displays various types of information. The display device includes, for example, a flat panel display. The flat panel display is, for example, a liquid crystal display or an organic EL display. The data processing unit 50 may also be a simple shared storage. If the data processing unit 50 is a simple shared storage, an input device or a display device is not required. If the data processing unit 50 is a simple shared storage, operation information is sent to the input / output unit 53 via communication.

[0033] The processor 51 is composed of a CPU, an MPU, etc. The processor 51 operates based on programs stored in the storage device 52 and executes various processes. The processor 51 includes a master data management unit 51A and a data adjustment unit 51B. The master data management unit 51A and the data adjustment unit 51B realize various functions when the processor 51 executes the programs. The master data management unit 51A and the data adjustment unit 51B may be configured as separate dedicated processors.

[0034] The master data management unit 51A creates, for example, generalized template data 71 shown in FIG. 8 . The generalized template data 71 in FIG. 8 is generalized template data obtained by generalizing the template data 70 for the first sewing machine SM1. The generalized template data 71 is template data used in image recognition processing of an image of the sewing object S, and is data that excludes individual differences between individual sewing machines 1 and image capture conditions for each machine. Excluding image capture conditions means generalizing the image and correcting it to an ideal form using design values. As in the configuration of FIG. 2 , data conversion processing can be performed in a centralized manner, but this function can also be provided on the sewing machine side to build a distributed system. In general, a distributed system is easier to implement because it is more efficient and simpler in configuration.

[0035] The data adjustment unit 51B acquires the unique data 60 of the multiple imaging devices 37 and creates unique template data 72 for each imaging device (see the second sewing machine SM2 in FIG. 8 ) based on the generalized template data 71 and the unique data 60. The unique template data 72 is template data that reflects the individual differences and imaging conditions of each imaging device 37 in comparison with the generalized template data 71. This function is implemented using the same calculations as generalizing the template data, and can therefore also be provided on the sewing machine side. If the data conversion process is distributed and implemented on the sewing machine side, the data processing unit 50 only needs to manage the generalized template data. This eliminates the need for centralized management of all data for the multiple sewing machines 1, resulting in an even simpler configuration.

[0036] The storage device 52 includes a primary storage device, which is a volatile memory that temporarily stores data, and a secondary storage device, which is a nonvolatile memory that stores various programs and data used by the processor 51 for calculations. The volatile memory is, for example, RAM. The nonvolatile memory is, for example, ROM, HDD, SSD, or storage. The storage device 52 stores programs for implementing the functions of the data processing unit 50. The storage device 52 stores unique data 60 for each imaging device 37. The storage device 52 stores generalized template data 71 as master data. By centrally managing the unique data 60 for each imaging device 37, the security of this data is ensured. However, because the data is also managed on each sewing machine 1, this function can be eliminated from the data processing unit 50, reducing the burden.

[0037] Input / output unit 53 communicates with each sewing machine 1 via communication network 102. Input / output unit 53 includes a communication interface that complies with wired or wireless communication standards. For example, in FIG. 8 , input / output unit 53 receives unique data 60A or template data 70 transmitted from first sewing machine SM1. Input / output unit 53 transmits to second sewing machine SM2 unique template data 72 that reflects unique data 60B of second sewing machine SM2 (including imaging device 37) in generalized template data 71. If each sewing machine 1 is provided with data conversion processing functions as master data management unit 51A and data adjustment unit 51B, input / output unit 53 transmits generalized template data 71 to and receives generalized template data 71 from each sewing machine 1.

[0038] [Unique Data] FIG. 3 is an explanatory diagram of unique data 60 possessed by a plurality of sewing machines 1. The unique data 60 is data unique to each of the imaging devices 37 of the plurality of sewing machines 1. Differences in the unique data 60 represent individual differences between the imaging devices 37 of the individual sewing machines 1. In the present embodiment, the unique data 60 includes information related to imaging of the sewing object S. The information related to imaging of the sewing object S is, in other words, the imaging conditions of the imaging devices 37. Specifically, the unique data 60 includes information on imaging conditions unique to each sewing machine 1. The information on the imaging conditions unique to each sewing machine 1 is information resulting from variability factors such as individual differences between the imaging devices 37 and lighting devices 38 of each sewing machine 1 and assembly errors. Note that the unique data 60 may include data other than that shown in FIG. 3. The unique data 60 is managed for each imaging device 37 of each sewing machine 1. Furthermore, the unique data 60 is embedded in the template data file each time an image of a template is captured. By embedding the unique data 60 in the template data file each time an image is captured, the data is always kept as a pair and is not separated even after being output from the sewing machine onto the communication line, allowing for reliable management. Of course, this is useful not only for sharing data between different sewing machines 1, but also for sharing data between different imaging devices 37 of the same sewing machine 1.

[0039] The specific data 60 includes, as imaging conditions, a pixel rate 61 of the imaging device 37, an imaging offset 62, and illumination information 63.

[0040] The pixel rate 61 of the imaging device 37 is information indicating the actual distance per pixel in the captured image. The pixel rate 61 changes depending on the distance between the imaging device 37 and the subject (the sewing object S). The pixel rate 61 can be obtained by calibration, in which an image of a measuring jig with a known distance between two points is captured and the number of pixels between the two points in the image is measured. The pixel rate 61 for any distance can be obtained by performing calibration at multiple distances and finding a regression line from the pixel rates for the multiple distances.

[0041] The imaging offset 62 is information indicating a deviation of the camera coordinate system of the imaging device 37 from a design value in the sewing machine coordinate system. The imaging offset 62 is information for correcting deviations due to assembly errors of the imaging device 37 and individual differences in the optical system of the imaging device 37. Examples of individual differences in the optical system of the imaging device 37 include optical axis deviation and lens aberration. The imaging offset 62 includes information such as the XYZ position coordinates of the imaging device 37, the tilt of the camera optical axis (in the θX direction and the θY direction), the rotation angle around the camera optical axis (in the θZ direction), and the lens distortion coefficient. The imaging offset 62 can be obtained by calibration, in which a measuring jig of a known shape is positioned at a predetermined position, an image is captured, and the XY position deviation, image distortion, and angle deviation are calculated from the position and shape of the measuring jig captured in the image.

[0042] The lighting information 63 is information indicating the amount of illumination light from the lighting device 38. The lighting information can be acquired by performing calibration, in which an image of a predetermined subject is captured while changing the light intensity setting value of the lighting device 38, and the light intensity setting value is determined so that the contrast of the image falls within a desired range. The predetermined subject is, for example, the sewing object S. When brightness adjustment is performed on the captured image by image processing, the lighting information 63 may further include information such as brightness adjustment parameters and a look-up table.

[0043] In the sewing system 100 according to the present embodiment, a calibration operation is performed in advance for each of the imaging devices 37 of the plurality of sewing machines 1, so that unique data 60 including a pixel rate 61, an imaging offset 62, and illumination information 63 is acquired for each sewing machine 1. Each of the plurality of sewing machines 1 stores the unique data 60 including the imaging conditions of the imaging device 37 in the storage device 35.

[0044] [Sewing Object] Fig. 4 is a cross-sectional view showing a part of the sewing object S according to this embodiment. Fig. 5 is a plan view showing the sewing object S according to this embodiment. Figs. 4 and 5 show the sewing object S before sewing processing. In this embodiment, the sewing object S is a cover material used for a vehicle seat.

[0045] 4, the sewing object S has a surface material 4, a padding material 5, and a backing material 6. A hole 7 is formed in the surface material 4.

[0046] The surface of the surface material 4 is the seating surface that comes into contact with the occupant when the occupant sits in the vehicle seat. The surface material 4 includes at least one of woven fabric, nonwoven fabric, and leather. The padding material 5 has elasticity. The padding material 5 includes, for example, urethane resin. The backing material 6 includes at least one of woven fabric, nonwoven fabric, and leather.

[0047] As shown in FIG. 5 , a plurality of holes 7 are provided in the surface material 4. The holes 7 are formed so as to be scattered throughout the sewing object S. The holes 7 are arranged in a regular pattern. In other words, the sewing object S includes a plurality of reference patterns DPh. The reference patterns DPh are configured by a regular arrangement of a plurality of holes 7. In the example of FIG. 5 , the reference pattern DPh is configured by 17 holes 7.

[0048] As shown in Figure 5, the reference patterns DPh are arranged on the surface material 4 at intervals. The reference patterns DPh are arranged at equal intervals in both the X-axis direction and the Y-axis direction. Between adjacent reference patterns DPh in the X-axis direction, a reference pattern DPh at a different position in the Y-axis direction is arranged. No holes 7 are formed between adjacent reference patterns DPh.

[0049] Target patterns RP (RP1, RP2, RP3, RP4, RP5, RP6, RP7, RP8, RP9, RP10) of stitches CH to be formed on the sewing object S are defined in the areas between the reference patterns DPh. Fig. 5 shows a virtual target pattern RP of the stitches CH on the sewing object S before sewing. The sewing machine 1 forms the stitches CH so as to follow each target pattern RP in accordance with a preset sewing pattern.

[0050] When performing multiple sewing processes to form multiple stitches CH, a first sewing process is performed to form a first stitch CH on the sewing object S based on a first target pattern RP1, and after the first sewing process, a second sewing process is performed to form a second stitch CH on the sewing object S based on a second target pattern RP2. Thereafter, the third to Nth sewing processes are performed sequentially depending on the number N of stitches CH to be formed. Figure 5 shows an example where N = 10, and the first to tenth target patterns RP1 to RP10 are defined.

[0051] Furthermore, a plurality of recognition targets UP (UP1, UP2, UP3, UP4, UP5, UP6, and UP7) are arranged on the sewing object S. In this embodiment, the recognition targets UP are an array of a plurality of holes 7 that exist in a predetermined pattern on the sewing object S. Specifically, the recognition targets UP are part of the reference pattern DPh. In the example of FIG. 5 , the recognition targets UP (UP1, UP2, UP3, UP4, UP5, UP6, and UP7) are a pattern (an array of holes 7) that includes one acute corner of the reference pattern DPh that has a diamond-shaped outer shape. The recognition targets UP are patterns that can be identified by a template matching method, which is a type of image processing method.

[0052] Using FIG. 6 , displacement that occurs on the surface of the sewing object S when a stitch CH is formed on the sewing object S, which has thickness and elasticity, will be described. FIG. 6 is a cross-sectional view showing an example of the sewing object S according to this embodiment. FIG. 6 shows the sewing object S after a sewing process has been performed. The sewing object S is thick and elastic. When a stitch CH is formed on the sewing object S, which has thickness and elasticity, there is a high possibility that the sewing object S will shrink, as shown in FIG. 6 . When the sewing object S shrinks, there is a possibility that the surface of the sewing object S will displace. When the surface of the sewing object S displaces, there is a high possibility that the target position of the stitch CH defined on the surface of the sewing object S will displace within the XY plane. If the target position of the stitch CH displaces within the XY plane, it will be difficult to form the stitch CH at the target position when the sewing object S is moved to the position coordinates of the target pattern RP in the design.

[0053] For this reason, in this embodiment, the image processing unit 36 ​​(see FIG. 2) acquires the amount of displacement (a correction amount for correcting the amount of displacement) of the surface of the sewing object S. Even if the sewing object S shrinks due to the formation of a stitch CH and the surface of the sewing object S is displaced, the acquired amount of displacement of the surface of the sewing object S is used to move the sewing object S in accordance with the amount of displacement so that the next stitch CH is formed at the target position. The image processing unit 36 ​​detects the recognition object UP from an image of the sewing object S captured by the imaging device 37, and acquires the amount of displacement based on the displacement of the recognition object UP.

[0054] [Template Data] FIG. 7 is a diagram illustrating template data 70 of the recognition target UP. The template data 70 is created using a first imaging device 37, which is any one of the multiple imaging devices 37. Hereinafter, a sewing machine 1 having the first imaging device 37 used to create the template data 70 will be referred to as a "first sewing machine SM1." Furthermore, among the multiple sewing machines 1, a sewing machine 1 other than the first sewing machine SM1 will be referred to as a "second sewing machine SM2." The second sewing machine SM2 is a sewing machine 1 having a second imaging device 37 other than the first imaging device 37.

[0055] The template data 70 is data used in image recognition processing of the recognition target UP. Specifically, the template data 70 is an image of the recognition target UP and is a reference image referred to in template matching. The template data 70 is acquired by capturing an image of the recognition target UP formed on the sewing object S using the first imaging device 37 provided in the first sewing machine SM1.

[0056] Therefore, the template data 70 is an image that depends on the inherent data 60 of the first imaging device 37. In other words, parameters such as the size, distortion, tilt, average brightness, and contrast of the recognition target UP included in the image of the template data 70 reflect the imaging conditions (pixel rate 61, imaging offset 62, and lighting information 63) of the first imaging device 37 of the first sewing machine SM1.

[0057] In an actual sewing operation, the image captured by the first sewing machine SM1 (first imaging device 37) is captured under the same imaging conditions as the template data 70. Therefore, in the first sewing machine SM1, the image of the sewing object S involved in the sewing operation and the template data 70 have the same image parameters, enabling highly accurate detection.

[0058] On the other hand, the image captured by the second imaging device 37 of the second sewing machine SM2 reflects the imaging conditions of the second imaging device 37 and has different image parameters from the image captured by the first imaging device 37. Therefore, if the template data 70 created by the first sewing machine SM1 (first imaging device 37) is used as is by the second sewing machine SM2 (second imaging device 37) during sewing work, the image parameters of the template data 70 will not match the image parameters of the image of the sewing object S captured by the second imaging device 37, which may result in a decrease in the detection accuracy of the recognition target UP.

[0059] Therefore, in the sewing system 100 according to this embodiment, the unique data 60 of the imaging device 37 of each sewing machine 1 is used to correct any discrepancy between the imaging conditions of the template data 70 and the imaging conditions of the sewing object S at each imaging device 37.

[0060] [Correction of Template Data] FIG. 8 is a diagram for explaining correction of the template data 70 by the sewing system 100 according to this embodiment.

[0061] 8 illustrates a process for applying template data 70 created by one first sewing machine SM1 (first imaging device 37) to another second sewing machine SM2 (second imaging device 37). Here, the unique data 60 of the first sewing machine SM1 is referred to as unique data 60A, and the unique data 60 of the second sewing machine SM2 is referred to as unique data 60B. In this embodiment, the data processing unit 50 creates template data (unique template data 72) that reflects the unique data 60B of the second sewing machine SM2 from the template data 70 created by the first sewing machine SM1. Because the template data 70 created by the first sewing machine SM1 reflects the unique data 60A of the first sewing machine SM1, it is also the unique template data 72 of the first sewing machine SM1.

[0062] Data processing unit 50 acquires template data 70 created by first sewing machine SM1 and unique data 60A of first sewing machine SM1 from first sewing machine SM1.

[0063] Data processing unit 50 uses master data management unit 51A to generalize (normalize) template data 70 based on template data 70 and unique data 60A of first sewing machine SM1 to create generalized template data 71.

[0064] That is, the template data 70 is unique template data that reflects the unique data 60A of the first sewing machine SM1, but the unique data 60A is known through calibration. To briefly explain the concept of normalization, let's focus on each of the various parameters. The master data management unit 51A normalizes the image scale based on the pixel rate 61 of the unique data 60A. By providing information about the pixel rate 61, the generalized template data 71 becomes normalized data that can be converted to the scale of a captured image under desired imaging conditions. As a result, by applying the pixel rate 61 of the unique data 60B of the second sewing machine SM2 that uses the generalized template data 71, the pixel rate 61 of the image captured by the second sewing machine SM2 can be matched to the pixel rate 61 of the image captured by the second sewing machine SM2.

[0065] Master data management unit 51A normalizes the luminance information of the image based on illumination information 63 in unique data 60A. By providing illumination information 63, generalized template data 71 becomes normalized data that can reproduce the luminance distribution of an image captured under desired imaging conditions. As a result, by applying illumination information 63 in unique data 60B of second sewing machine SM2 that uses the data, generalized template data 71 can match the luminance distribution (average luminance and contrast) of an image captured by second sewing machine SM2.

[0066] The master data management unit 51A normalizes the geometric information of the image based on the imaging offset 62 of the unique data 60A. In other words, the image position shift, distortion, and rotation angle shift of the template data 70 are removed from the information on the imaging offset 62. In other words, the data is converted into data that would be obtained if the imaging device 37 were installed accurately in the designed position. By transforming the generalized template data 71 to a certain reference position (here, the design value) based on the information on the imaging offset 62, the generalized template data 71 becomes normalized data that can be converted into geometric conditions of a captured image under desired imaging conditions with known relative positional relationships. By applying the imaging offset 62 of the unique data 60B of the second sewing machine SM2 that uses the data, the generalized template data 71 can be made to match the geometric information of an image captured by the second sewing machine SM2. Furthermore, if both the imaging offset of the unique data 60A and the imaging offset of the unique data 60B are available, it is also possible to directly calculate the conversion from the geometric information of the first sewing machine SM1 to the geometric information of the second sewing machine SM2 without using a generalized template.

[0067] The master data management unit 51A records the created generalized template data 71 in the storage device 52 (see FIG. 2) as master data common to each sewing machine 1.

[0068] Data processing unit 50 acquires specific data 60B of second sewing machine SM2. Data processing unit 50 uses data adjustment unit 51B to create specific template data 72 for each sewing machine based on generalized template data 71 and specific data 60B.

[0069] In other words, data adjustment unit 51B obtains specific data 60B of second sewing machine SM2 to which generalized template data 71 is to be applied, and applies specific data 60B to generalized template data 71 to create template data (specific template data 72) that reflects specific data 60B of second sewing machine SM2 to which generalized template data 71 is to be applied. One piece of generalized template data 71 is created for each sewing machine 1 (sewing system 100). One piece of specific template data 72 is created separately for each sewing machine 1 (imaging device 37).

[0070] 9 is a schematic diagram illustrating the transmission of unique template data 72. Three second sewing machines SM2P, SM2Q, and SM2R are illustrated in FIG. The second sewing machines SM2P, SM2Q, and SM2R each transmit unique data 60B to data processing unit 50. Here, the unique data 60B of the second sewing machines SM2P, SM2Q, and SM2R are referred to as unique data 60P, 60Q, and 60R, respectively.

[0071] Data adjustment unit 51B applies unique data 60P to generalized template data 71 to create unique template data 72P for second sewing machine SM2P, and sends this data to second sewing machine SM2P. Data adjustment unit 51B applies unique data 60Q to generalized template data 71 to create unique template data 72Q for second sewing machine SM2Q, and sends this data to second sewing machine SM2Q. Data adjustment unit 51B applies unique data 60R to generalized template data 71 to create unique template data 72R for second sewing machine SM2R, and sends this data to second sewing machine SM2R.

[0072] Second sewing machines SM2P, SM2Q, and SM2R each perform image recognition processing using their own unique template data 72P, 72Q, and 72R acquired from data processing unit 50. Note that, since template data 70 created by first sewing machine SM1 is unique template data 72 of first sewing machine SM1, first sewing machine SM1 performs image recognition processing using template data 70 as is.

[0073] As described above, in this embodiment, data processing unit 50 acquires template data of the sewing object S created based on an image captured by first sewing machine SM1 (first imaging device 37) among the multiple sewing machines 1 (imaging devices 37), and transmits the template data to another sewing machine, second sewing machine SM2 (second imaging device 37). Second sewing machine SM2 then performs image recognition processing using specific template data 72 obtained by correcting generalized template data 71 based on its own specific data 60.

[0074] Note that the example shown here is one in which data processing unit 50 creates unique template data 72 for each second sewing machine SM2 from generalized template data 71. However, the creation of unique template data 72 is not limited to the above example. Data processing unit 50 may distribute generalized template data 71 to second sewing machines SM2, and each second sewing machine SM2 (image processing unit 36) may then create unique template data 72 using its own unique data 60.

[0075] Furthermore, the second sewing machine SM2 may perform image recognition processing using generalized template data 71 instead of using the unique template data 72, and may correct the recognition results using the generalized template data 71 based on the unique data 60 of the sewing machine itself. For example, if the calibration results in a sufficiently small error from the design values ​​of the imaging conditions of the second sewing machine, the recognition target UP can be properly detected using the generalized template data 71 (or the template data 70 of the first sewing machine SM1). Therefore, after the image recognition processing during sewing work, proper image recognition of the sewing object S can be achieved simply by correcting the XY position coordinates, rotation angle, etc. of the detected recognition target UP using the unique data 60 of the sewing machine itself.

[0076] [Sewing Method] Fig. 10 is a diagram for explaining a sewing method according to this embodiment. The flow of transmission of template data 70 in sewing system 100 will be described with reference to Fig. 10 .

[0077] First, in each of the plurality of sewing machines 1 (plurality of image capture devices 37), an operation to acquire unique data 60 is performed (S (step) 1). That is, through a calibration operation, unique data 60 including a pixel rate 61, an image capture offset 62, and illumination information 63 of the image capture device 37 is acquired for each image capture device 37. The acquisition of unique data 60 is performed in both the first sewing machine SM1 and the second sewing machine SM2 described above.

[0078] Next, the first sewing machine SM1, which has the first imaging device 37, which is any one of the plurality of imaging devices 37, creates template data 70 (S2). The control device 30 of the first sewing machine SM1 uses the first imaging device 37 provided in the first sewing machine SM1 to actually capture an image of the sewing object S and acquire an image of the recognition target UP. The image processing unit 36 ​​of the first sewing machine SM1 creates template data 70 (see FIG. 7) using the acquired image of the recognition target UP.

[0079] Next, data processing unit 50 creates generalized template data 71 (S3). Data processing unit 50 acquires template data 70 created in S2 and unique data 60A of first imaging device 37 from first sewing machine SM1 via input / output unit 53. Master data management unit 51A normalizes template data 70 using unique data 60A to create generalized template data 71, and records the generalized template data 71 in storage device 52.

[0080] Next, data processing unit 50 creates unique template data 72 (S4). Data processing unit 50 acquires, via input / output unit 53, unique data 60B of second imaging device 37 from second sewing machine SM2 other than first sewing machine SM1. Data adjustment unit 51B creates unique template data 72 for second imaging device 37 of second sewing machine SM2 by applying unique data 60B to generalized template data 71 recorded in storage device 52. If there are multiple second sewing machines SM2, data adjustment unit 51B creates unique template data 72 for each individual second sewing machine SM2 using unique data 60B acquired from each second sewing machine SM2.

[0081] Next, data processing unit 50 transmits unique template data 72 to second sewing machines SM2 (S5). Data processing unit 50 transmits corresponding unique template data 72 to each second sewing machine SM2 via input / output unit 53.

[0082] Thereafter, the plurality of sewing machines 1 each perform a sewing process. That is, the plurality of sewing machines 1 acquire an image of the sewing object S using the imaging device 37, perform an image recognition process on the sewing object S using the image processing unit 36, and perform sewing based on the recognition results obtained by the image recognition process. At this time, each second sewing machine SM2 performs the image recognition process using unique template data 72 corrected using its own unique data 60B. Furthermore, the first sewing machine SM1 performs the image recognition process using the template data 70 created in S2 as its own unique template data.

[0083] As described above, in the sewing method according to this embodiment, the second sewing machine SM2 may correct the recognition result obtained by using the template data (generalized template data 71) based on the unique data 60, instead of using the corrected template data (unique template data 72).

[0084] [Sewing Machine] Next, a specific example of the sewing machine 1 included in the sewing system 100 will be described below. Fig. 11 is a perspective view showing the sewing machine 1 according to this embodiment. Fig. 12 is a perspective view showing a part of the sewing machine 1 according to this embodiment. The sewing machine 1 shown in Fig. 11 includes a sewing machine main body 10, an operation device 20 operated by an operator Ma, a control device 30 that controls the sewing machine main body 10, an image processing unit 36, and a plurality of imaging devices 37.

[0085] The sewing machine body 10 is mounted on the upper surface of the table 2. As shown in FIG. 12 , the sewing machine body 10 has a sewing machine frame 11, a needle bar 12, a needle plate 13, a holding member 15, an actuator 16, an actuator 17, and an actuator 18. The needle bar 12 and the needle plate 13 are each supported by the sewing machine frame 11. The holding member 15 is supported by the sewing machine frame 11 via a support member 14. The actuator 16 (see FIG. 11 ) generates power to move the needle bar 12. The actuator 17 (see FIG. 11 ) generates power to move the holding member 15. The actuator 18 (see FIG. 11 ) generates power to move at least a portion of the holding member 15. The actuators 16, 17, and 18 are examples of the actuator 10A shown in FIG. 2 .

[0086] The sewing machine frame 11 has a horizontal arm 11A, a bed 11B, a vertical arm 11C, and a head 11D. The horizontal arm 11A extends in the Y-axis direction. The bed 11B (see FIG. 11) is located below the horizontal arm 11A. The vertical arm 11C (see FIG. 11) is located so as to connect the +Y side end of the horizontal arm 11A to the bed 11B. The head 11D is located on the -Y side of the horizontal arm 11A.

[0087] Needle bar 12 holds sewing needle 3. Needle bar 12 holds sewing needle 3 so that sewing needle 3 is parallel to the Z axis. Needle bar 12 is supported by head 11D so as to be movable in the Z axis direction.

[0088] The needle plate 13 supports the sewing object S. The needle plate 13 supports a holding member 15. The needle plate 13 is supported by a bed 11B (see FIG. 11 ). The needle plate 13 is disposed below the holding member 15.

[0089] The holding member 15 holds the sewing object S. The holding member 15 is capable of moving the sewing object S within an XY plane that includes a sewing position Ps directly below the sewing needle 3. The holding member 15 is capable of holding the sewing object S within an XY plane that includes a position directly below the imaging device 37. The holding member 15 moves within the XY plane while holding the sewing object S, based on the sewing data. As a result, a stitch CH (see FIG. 6) is formed on the sewing object S that has passed the sewing position Ps. The holding member 15 is supported by the horizontal arm 11A via the support member 14.

[0090] The holding member 15 has a presser member 15A and a lower plate 15B facing the presser member 15A. The presser member 15A is a frame-shaped member. The presser member 15A is movable in the Z-axis direction. The lower plate 15B is disposed below the presser member 15A. The holding member 15 holds the sewing object S by sandwiching the sewing object S between the presser member 15A and the lower plate 15B.

[0091] When the presser member 15A moves in the +Z direction, the presser member 15A and the lower plate 15B separate. This allows the operator Ma to place the sewing object S between the presser member 15A and the lower plate 15B. When the presser member 15A moves in the -Z direction with the sewing object S placed between the presser member 15A and the lower plate 15B, the sewing object S is sandwiched between the presser member 15A and the lower plate 15B. This causes the sewing object S to be held by the holding member 15. Furthermore, when the presser member 15A moves in the +Z direction, the holding of the sewing object S by the holding member 15 is released. This allows the operator Ma to remove the sewing object S from between the presser member 15A and the lower plate 15B.

[0092] 11, actuator 16 generates power to move needle bar 12 in the Z-axis direction. Actuator 16 includes a pulse motor. Actuator 16 is disposed on horizontal arm 11A.

[0093] A horizontal arm shaft extending in the Y-axis direction is disposed inside horizontal arm 11A. Actuator 16 is connected to the +Y side end of the horizontal arm shaft. The -Y side end of the horizontal arm shaft is connected to needle bar 12 via a power transmission mechanism disposed inside head 11D. Operation of actuator 16 causes the horizontal arm shaft to rotate. Power generated by actuator 16 is transmitted to needle bar 12 via the horizontal arm shaft and the power transmission mechanism. As a result, sewing machine needle 3 held by needle bar 12 moves back and forth in the Z-axis direction.

[0094] A timing belt extending in the Z-axis direction is disposed inside the vertical arm 11C. A bed shaft extending in the Y-axis direction is disposed inside the bed 11B. Pulleys are disposed on both the horizontal arm shaft and the bed shaft. The timing belt is wound around the pulley disposed on the horizontal arm shaft and the pulley disposed on the bed shaft. The horizontal arm shaft and the bed shaft are connected via a power transmission mechanism including the timing belt.

[0095] A shuttle is disposed inside the bed 11B. A bobbin placed in a bobbin case is housed in the shuttle. The horizontal arm shaft and the bed shaft are rotated by the operation of the actuator 16. The power generated by the actuator 16 is transmitted to the shuttle via the horizontal arm shaft, timing belt, and bed shaft. As a result, the shuttle rotates in synchronization with the reciprocating movement of the needle bar 12 in the Z-axis direction.

[0096] Actuator 17 generates power to move holding member 15 within the XY plane. Actuator 17 includes a pulse motor. Actuator 17 includes an X-axis motor 17X that generates power to move holding member 15 in the X-axis direction, and a Y-axis motor 17Y that generates power to move holding member 15 in the Y-axis direction. Actuator 17 is disposed inside bed 11B.

[0097] The power generated by the actuator 17 is transmitted to the holding member 15 via the support member 14. This allows the holding member 15 to move in both the X-axis and Y-axis directions between the sewing needle 3 and the needle plate 13. By operating the actuator 17, the holding member 15 can move while holding the sewing object S within an XY plane that includes the sewing position Ps directly below the sewing needle 3.

[0098] The actuator 18 generates power to move the presser member 15A of the holding member 15 in the Z-axis direction. The actuator 18 includes a pulse motor. When the presser member 15A moves in the +Z direction, the presser member 15A and the lower plate 15B are separated from each other. When the presser member 15A moves in the -Z direction, the sewing object S is sandwiched between the presser member 15A and the lower plate 15B.

[0099] As shown in Figure 12, the sewing machine body 10 has an intermediate presser member 19 that is arranged around the sewing needle 3. The intermediate presser member 19 presses down on the sewing object S around the sewing needle 3. The intermediate presser member 19 is supported by the head 11D so as to be movable in the Z-axis direction. An intermediate presser motor that generates power to move the intermediate presser member 19 in the Z-axis direction is arranged inside the head 11D. Operation of the intermediate presser motor causes the intermediate presser member 19 to move in the Z-axis direction in synchronization with the needle bar 12. The intermediate presser member 19 prevents the sewing object S from lifting up due to movement of the sewing needle 3.

[0100] 11, the sewing machine 1 includes a drive amount sensor 31 that detects the drive amount of the actuator 16, and a drive amount sensor 32 that detects the drive amount of the actuator 17. The drive amount sensor 31 and the drive amount sensor 32 are examples of the drive amount sensor 10B in FIG.

[0101] The control device 30 controls the actuator 16 based on the detection data of the drive amount sensor 31. The control device 30 determines, for example, the operation timing of the actuator 16 based on the detection data of the drive amount sensor 31.

[0102] The drive amount sensor 32 includes an X-axis sensor 32X that detects the drive amount of the X-axis motor 17X of the actuator 17, and a Y-axis sensor 32Y that detects the drive amount of the Y-axis motor 17Y of the actuator 17. The X-axis sensor 32X includes an encoder that detects the rotation amount of the X-axis motor 17X. The Y-axis sensor 32Y includes an encoder that detects the rotation amount of the Y-axis motor 17Y. The detection data of the drive amount sensor 32 is output to the control device 30. The control device 30 controls the actuator 17 based on the detection data of the drive amount sensor 32. The control device 30 feedback-controls the actuator 17 based on the detection data of the drive amount sensor 32 so that the holding member 15 moves to a target position.

[0103] The drive amount sensor 32 functions as a position sensor that detects the position of the holding member 15 in the XY plane. The drive amount of the actuator 17 and the movement amount of the holding member 15 correspond one-to-one.

[0104] The X-axis sensor 32X detects the amount of rotation of the X-axis motor 17X, thereby detecting the amount of movement of the holding member 15 in the X-axis direction from the origin in the sewing machine coordinate system. The Y-axis sensor 32Y detects the amount of rotation of the Y-axis motor 17Y, thereby detecting the amount of movement of the holding member 15 in the Y-axis direction from the origin in the sewing machine coordinate system.

[0105] The operation device 20 accepts operation inputs from the operator Ma. The sewing machine 1 is operated by operating the operation device 20. The operation device 20 includes an operation panel 21 and an operation pedal 22. The operation panel 21 is mounted on the top surface of the table 2. The operation pedal 22 is disposed below the table 2. The operator Ma operates the operation pedal 22 with his / her foot. The sewing machine 1 is operated by the operator Ma operating at least one of the operation panel 21 and the operation pedal 22.

[0106] The multiple imaging devices 37 capture images of the sewing object S held by the holding member 15. The multiple imaging devices 37 are disposed above the needle plate 13 and the holding member 15. Each of the multiple imaging devices 37 captures an image of at least a portion of the sewing object S held by the holding member 15 from above. The number of imaging devices 37 is not particularly limited. Fig. 11 shows an example in which four imaging devices 37 are provided.

[0107] 13 is a plan view illustrating an example of the arrangement of the imaging devices 37 according to this embodiment. The imaging devices 37 are arranged, for example, as shown in FIG.

[0108] The multiple imaging devices 37 are arranged at intervals in a horizontal plane so as to be aligned with the sewing machine body 10. The multiple imaging devices 37 are arranged within the sewing area SA. The sewing area SA is the range in the XY plane within which the sewing object S can be moved by the holding member 15 and sewn by the head 11D. The head 11D (i.e., sewing position Ps) is located in the center of the sewing area SA. The multiple imaging devices 37 are arranged along the head 11D of the sewing machine body 10.

[0109] 13, the sewing machine 1 is equipped with imaging units 40A and 40B, one adjacent to each side of the head 11D in the X-axis direction (+X side, −X side). A plurality of imaging devices 37 can be attached to the imaging units 40A and 40B. The imaging units 40A and 40B are disposed above the sewing area SA. The imaging units 40A and 40B are fixed to the sewing machine 1.

[0110] 13, two imaging devices 37 are attached to mounting portion 41A of imaging unit 40A, and two imaging devices 37 are also attached to mounting portion 41B. The two imaging devices 37 of imaging unit 40A are arranged along the longitudinal direction (X-axis direction) of the sewing area SA. Two imaging devices 37 are attached to mounting portion 41C of imaging unit 40B, and two imaging devices 37 are also attached to mounting portion 41D. The two imaging devices 37 of imaging unit 40B are arranged along the lateral direction (Y-axis direction) of the sewing area SA.

[0111] The imaging device 37 captures an image of at least a portion of the sewing object S arranged within its imaging field of view. Each of the multiple imaging devices 37 is arranged so as to have an overlapping area JA where the imaging field of view of the adjacent imaging device 37 partially overlaps with that of the other imaging device 37. Specifically, the imaging device 37 of the mounting portion 41A has an imaging field of view 42A. The imaging device 37 of the mounting portion 41B has an imaging field of view 42B. The imaging field of view 42A and the imaging field of view 42B are rectangular, with an overlapping area JA extending in the Y-axis direction. The imaging field of view 42A and the imaging field of view 42B form an enlarged imaging area 43A. The enlarged imaging area 43A is the combined area of ​​the imaging fields of view of the multiple imaging devices 37 that share the overlapping area JA. The imaging field of view 42A and the imaging field of view 42B, which are aligned in the X-axis direction, form a wide enlarged imaging area 43A extending in the X-axis direction.

[0112] The imaging device 37 of the mounting portion 41C has an imaging field of view 42C. The imaging device 37 of the mounting portion 41D has an imaging field of view 42D. The imaging field of view 42C and the imaging field of view 42D have a rectangular shape and have an overlapping area JA extending in the X-axis direction. The imaging field of view 42C and the imaging field of view 42D form an enlarged imaging area 43B. The imaging field of view 42C and the imaging field of view 42D are aligned in the Y-axis direction, forming a wide enlarged imaging area 43B extending in the Y-axis direction.

[0113] 13 , the size of the imaging field of view of the imaging device 37, the size of the overlapping area JA (the spacing between the mounting portions), and the number of imaging devices 37 are set according to the dimensions of the target pattern RP (see FIG. 5 ) of the sewing object S. That is, the enlarged imaging areas 43A, 43B formed by the multiple imaging devices 37 are designed to have a shape that encompasses the entire target pattern RP of one sewing object S. The sewing object S on which a stitch CH extending in the X-axis direction is to be formed can be imaged by being positioned in the enlarged imaging area 43A of the imaging unit 40A, and the sewing object S on which a stitch CH extending in the Y-axis direction is to be formed can be imaged by being positioned in the enlarged imaging area 43B of the imaging unit 40B. Therefore, regardless of the direction in which the stitch CH extending in either direction is to be formed, the entire target pattern RP can be imaged without changing the mounting position of the imaging device 37.

[0114] The position of each imaging device 37 is fixed at one of six mounting positions. The relative position of the imaging device 37 and the sewing machine frame 11 is fixed. The relative position of the optical axis of the imaging device 37 and the sewing needle 3 (sewing position Ps) in the XY plane is fixed. Relative position data indicating the relative position of the center of the enlarged imaging areas 43A, 43B and the sewing needle 3 in the XY plane is known data that can be derived from the design data of the sewing machine 1.

[0115] The position of the image captured by the imaging device 37 is defined in the camera coordinate system. A defined transformation formula or transformation matrix is ​​used to convert the position of the image defined in the camera coordinate system into the position of the image defined in the sewing machine coordinate system.

[0116] The imaging units 40A and 40B also hold a plurality of lighting devices 38A and 38B. The imaging units 40A and 40B hold the lighting devices 38A and 38B in a fixed state above the sewing area SA. The lighting devices 38A and 38B are an example of the lighting device 38 in FIG. 2.

[0117] In the imaging unit 40A, one illumination device 38A is disposed on each side of the arrangement of the mounting portions 41A and 41B along the X-axis direction, and one illumination device 38A is disposed between the arrangement of the mounting portions 41A and 41B (a total of three illumination devices 38A). Each illumination device 38A extends linearly along the Y-axis direction. Each illumination device 38A irradiates an imaging field of view set below the mounting portions 41A and 41B with light in the form of epi-illumination. In the imaging unit 40B, one illumination device 38B is disposed on each side of the arrangement of the mounting portions 41C and 41D along the Y-axis direction, and one illumination device 38B is disposed between the arrangement of the mounting portions 41C and 41D (a total of three illumination devices 38B). Each illumination device 38B extends linearly along the X-axis direction. Each illumination device 38B irradiates an imaging field of view set below the mounting portions 41C and 41D with light in the form of epi-illumination.

[0118] [Image Processing] Next, a description will be given of image processing using the multiple imaging devices 37. The image processing unit 36 ​​(see FIG. 11) acquires images of the sewing object S from the multiple imaging devices 37. When the sewing object S is positioned at a predetermined imaging position by the control device 30, the image processing unit 36 ​​controls the operation of the imaging device 37 and the lighting device 38 corresponding to the imaging position, and acquires the image of the sewing object S.

[0119] 13 , an imaging position Pf is set at the center of the enlarged imaging areas 43A and 43B. Image capture is performed with the center of the holding member 15 (sewing object S) positioned at the imaging position Pf of either the enlarged imaging areas 43A or 43B. For example, in the imaging unit 40A, when the sewing object S is positioned at the imaging position Pf of the enlarged imaging area 43A, the image processing unit 36 ​​turns on the illumination device 38A and causes the imaging device 37 of the attachment portion 41A and the imaging device 37 of the attachment portion 41B to capture images substantially simultaneously. As a result, an image corresponding to the imaging field of view 42A and an image corresponding to the imaging field of view 42B are acquired. In the imaging unit 40B, when the sewing object S is positioned at the imaging position Pf of the enlarged imaging area 43B, the image processing unit 36 ​​turns on the illumination device 38B and causes the imaging device 37 of the attachment portion 41C and the imaging device 37 of the attachment portion 41D to capture images substantially simultaneously. The imaging may be performed in sequence.

[0120] When the recognition target UP is located at the boundary between images captured by multiple image capture devices 37, the image processing unit 36 ​​performs image synthesis processing to stitch together the images captured by the multiple image capture devices 37. In this case, the coordinate systems of the two image capture devices 37 must be aligned. The imaging conditions of one of the image capture devices 37 are aligned with those of the other image capture device 37. In other words, the data conversion processing using the unique data 60 in FIG. 3 is performed between the image capture devices 37 of the same sewing machine 1, rather than between the sewing machines 1. The image processing unit 36 ​​stitches together the multiple images constituting the enlarged image capture area (43A or 43B) based on the image portions of the overlapping area JA to generate a single composite image 46 (see FIG. 15). Therefore, the composite image 46 is an image that captures the area corresponding to the enlarged image capture area 43A or 43B. The enlarged image capture areas 43A and 43B are large enough to encompass the multiple recognition targets UP (see FIG. 5) on the sewing object S. Therefore, the composite image 46 is an image that captures multiple recognition targets UP. The composite image 46 preferably includes all the recognition targets UP (UP1 to UP7) detected when forming one stitch CH.

[0121] The image processing unit 36 ​​executes image recognition processing to detect a plurality of recognition targets UP from the composite image 46. The image processing unit 36 ​​acquires the position coordinates of the recognition targets UP in the composite image 46 by matching with the unique template data 72. The image processing unit 36 ​​sequentially detects the plurality of recognition targets UP included in the composite image 46 and acquires the position coordinates of each of the plurality of recognition targets UP.

[0122] The image processing unit 36 ​​calculates correction data for correcting the displacement of the surface of the sewing object S based on the detection result by the image processing unit 36 ​​.

[0123] The sewing data stored in the storage device 35 (see FIG. 2) includes the target pattern RP of the stitch CH to be formed on the sewing object S, the position coordinates of the recognition object UP, and the movement conditions of the holding member 15.

[0124] The target pattern RP defines the target shape of the stitch CH to be formed on the sewing object S and the target position of the stitch CH in the sewing machine coordinate system.

[0125] The movement conditions of the holding member 15 include a movement trajectory of the holding member 15 defined in the sewing machine coordinate system. The movement trajectory of the holding member 15 includes a movement trajectory of the holding member 15 in the XY plane. The movement conditions of the holding member 15 are determined based on the target pattern RP.

[0126] The target pattern RP of the sewing data is generated on the assumption that the sewing object S is not shrunk, and is stored in the control device 30. If a displacement occurs on the surface of the sewing object S, the image processing unit 36 ​​determines the amount of displacement of the detected recognition object UP and generates correction data for correcting the sewing data.

[0127] The image processing unit 36 ​​transmits the correction data calculated by the image processing unit 36 ​​to the control device 30. The control device 30 executes the sewing process based on the correction data.

[0128] 14 is a flowchart for explaining the sewing operation of the sewing machine 1 according to this embodiment. The sewing operation of the sewing machine 1 includes an image capturing process S11, an image joining process S12, a recognition target detection process S13, a correction data calculation process S14, a sewing process S15, and an end determination process S16.

[0129] (Image Capture Process) The image capture process S11 is a process of capturing images of the sewing object S using the multiple image capture devices 37. Whether the image capture unit 40A captures an image of the enlarged image capture area 43A extending in the X-axis direction or the image capture unit 40B captures an image of the enlarged image capture area 43B extending in the Y-axis direction is set in advance by the operator Ma or by the control device 30 based on the sewing data. The control device 30 moves the holding member 15 that holds the sewing object S and positions the holding member 15 at a preset image capture position. When the holding member 15 is positioned at the corresponding image capture position Pf, the image processing unit 36 ​​performs image capture using the multiple image capture devices 37 that have been set.

[0130] 13. In this case, the image processing unit 36 ​​acquires an image corresponding to the imaging field of view 42A and an image corresponding to the imaging field of view 42B using the imaging device 37 of the mounting portion 41A and the imaging device 37 of the mounting portion 41B.

[0131] (Image Joining Process) The image joining process S12 is a process of joining together images of the sewing object S captured by the multiple image capturing devices 37. Fig. 15 is a schematic diagram illustrating the image joining process S12.

[0132] 15 , the image obtained by the imaging device 37 of the mounting portion 41A is referred to as a first image 44. The image obtained by the imaging device 37 of the mounting portion 41B is referred to as a second image 45. In this case, the second image 45 includes an area on the +X direction side of the first image 44. The first image 44 corresponds to the imaging field of view 42A, and the second image 45 corresponds to the imaging field of view 42B. The first image 44 and the second image 45 share an overlapping area JA.

[0133] The overlapping area JA between the first image 44 and the second image 45 includes image elements GE that are common to at least a portion of the recognition target UP. That is, as shown in FIG. 5 , when the recognition target UP is an array of multiple holes 7 that exist in a predetermined pattern on the sewing object S, the image elements GE are images of the individual holes 7. The holes 7 are components that make up the recognition target UP and the reference pattern DPh. For ease of explanation, FIG. 15 shows an example in which the arrangement of the holes 7 is simplified and the holes 7 are arranged vertically and horizontally in an array. Prior to the image stitching process S12, the image processing unit 36 ​​may binarize each image obtained in the imaging process S11. In this case, the template data 70 (generalized template data 71, specific template data 72) also becomes a binarized image.

[0134] As shown in FIG. 15 , the image processing unit 36 ​​detects image elements GE included in the overlapping area JA of the first image 44 and image elements GE included in the overlapping area JA of the second image 45. Because the position coordinates of the imaging center of the imaging device 37 attached to the mounting portions 41A and 41B and the size of the imaging fields of view 42A and 42B are known, the extent of the overlapping area JA in each image is known. The image processing unit 36 ​​selects three or more image elements GE included in the overlapping area JA and deforms the second image 45 so that the selected image elements GE match the first image 44. The deformation of the second image 45 is performed, for example, by affine transformation. As a result of the deformation, the images in the overlapping area JA of the first image 44 and the deformed second image 45 (hereinafter referred to as second image 45A) substantially match.

[0135] The image processing unit 36 ​​generates a composite image 46 by stitching together the first image 44 and the second image 45A. The pixels constituting the overlapping area JA may be selected from either the first image 44 or the second image 45A, or the pixel information of both may be averaged. In this way, the image stitching process S12 generates a composite image 46 of a size corresponding to the enlarged imaging area 43A.

[0136] (Recognition Target Detection Process) Next, the recognition target detection process S13 in Fig. 14 is a process for detecting a recognition target UP from the composite image 46 generated by the image stitching process S12 and acquiring the position coordinates of the detected recognition target UP. Fig. 16 is a schematic diagram illustrating the recognition target detection process S13.

[0137] As shown in FIG. 16 , the image processing unit 36 ​​extracts an image of a recognition area AR from the composite image 46, in which detection processing of the recognition target UP is performed. The recognition area AR is set within a range that takes into account the expected positional deviation of the recognition target UP that occurs during stitch formation, relative to the design position coordinates of the recognition target UP set in the sewing data. FIG. 16 schematically shows the setting of a recognition area AR for a target pattern RP of an arbitrary shape. The image processing unit 36 ​​sets a recognition area AR for each of multiple recognition targets UP that are to be detected in one stitch formation, and obtains an extracted image by cutting out an image of the set recognition area AR. For example, in the example of FIG. 5 , a recognition area AR is set for each of the seven recognition targets UP, from recognition target UP1 to recognition target UP7.

[0138] The image processing unit 36 ​​performs image recognition processing by template matching on each of the acquired extracted images in turn. The image processing unit 36 ​​reads out the unique template data 72 from the storage device 35 and acquires the position coordinates of the recognition target UP in the extracted image (recognition area AR) by matching with the unique template data 72.

[0139] In the recognition target detection process S13, the image processing unit 36 ​​acquires the position coordinates in the sewing machine coordinate system of each recognition target UP (UP1 to UP7) to be detected in one stitch formation.

[0140] (Correction Data Calculation Process) The correction data calculation process S14 is a process for calculating correction data for correcting displacement of the surface of the sewing object S based on the detection results of the recognition object detection process S13. The image processing unit 36 ​​calculates the amount of displacement of the recognition object UP from its initial position to its current position based on the initial position and current position of the recognition object UP related to the current sewing process. The initial position of the recognition object UP is the design position coordinate of the recognition object UP set in the sewing data. The current position of the recognition object UP is the position coordinate of the recognition object UP detected by the recognition object detection process S13. Based on the calculated amount of displacement, the image processing unit 36 ​​calculates the corrected position coordinates of each correction point CP (see FIG. 14) on the target pattern RP so that stitches CH are formed at target positions in the sewing machine coordinate system, and calculates correction data for the target pattern RP that passes through the correction points CP. Note that in the initial sewing process (first sewing process), no displacement of the surface of the sewing object S occurs due to stitch formation, and therefore no displacement of the recognition object UP occurs either. Therefore, in the first sewing process (first sewing process), the target pattern RP in the initial state defined by the sewing data is used as is, and no correction data is calculated.

[0141] (Sewing Process) The sewing process S15 is a process for forming stitches CH based on the target pattern RP. The sewing process S15 is performed by the control device 30 controlling each part of the sewing machine 1. The sewing process includes a first sewing process to an N-th sewing process. The first sewing process is performed based on the target pattern RP in an initial state defined by the sewing data. In the first sewing process, the control device 30 outputs a control command to the actuator 17 so that the stitches CH are formed in accordance with the target pattern RP in the initial state defined by the sewing data. The second sewing process to the N-th sewing process are performed based on the correction data calculated in the correction data calculation process S14 (the corrected target pattern RP). From the second sewing process onwards, the control device 30 acquires correction data from the image processing unit 36 ​​and outputs a control command to the actuator 17 so that the stitches CH are formed in accordance with the target pattern RP of the acquired correction data.

[0142] The end determination process S16 is a process for determining whether or not the sewing process of the sewing object S has been completed. The control device 30 determines whether or not the sewing process of the sewing object S has been completed based on the sewing data. When the first sewing process to the (N-1)th sewing process have been completed, the control device 30 determines in the end determination process S16 that the sewing process has not been completed. When the Nth sewing process has been completed, the control device 30 determines in the end determination process S16 that the sewing process has been completed.

[0143] Therefore, in the first to Nth sewing processes, each time one sewing process is completed, the image capturing process S11, the image stitching process S12, the recognition target detection process S13, and the correction data calculation process S14 are carried out to calculate the correction data for the next sewing process, and the next sewing process is carried out based on the calculated correction data, and this series of processes is repeated.

[0144] 11 to 16, each of the sewing machines 1 included in the sewing system 100 includes a plurality of image capture devices 37 having overlapping areas JA where the image capture fields (42A and 42B, 42C and 42D) partially overlap. Then, an image recognition process based on template data (unique template data 72) is performed on a composite image 46 formed by stitching together a plurality of images obtained from the plurality of image capture devices 37.

[0145] As a result, the multiple recognition targets UP detected in the sewing process of one stitch CH can be detected from the composite image 46 obtained by one image capture by the multiple image capture devices 37. As a result, the processing time required for the image capture process S11, the recognition target detection process S13, and the correction data calculation process S14, which are performed for each sewing process, is reduced compared to when the image capture device 37 captures and detects the recognition targets UP one by one.

[0146] On the other hand, when creating a wide-area composite image 46 by stitching together images obtained from multiple imaging devices 37, the imaging field of view of each imaging device 37 becomes larger, and the influence of variations in imaging conditions among the imaging devices 37 also becomes greater. Therefore, compared to simply capturing and detecting the recognition target UP one by one with a single imaging device 37, the influence of differences between the imaging conditions used to create template data 70 on the first sewing machine SM1 and the imaging conditions used on the second sewing machine SM2 on the image recognition process becomes greater. Therefore, the configuration of this embodiment, which can correct the differences between the imaging conditions used on the template data 70 and the imaging conditions used on the second sewing machine SM2 using unique data 60, is effective for achieving high recognition accuracy in the image recognition process using the composite image 46.

[0147] [Effects] As described above, according to this embodiment, the second sewing machine SM2 (sewing machine 1 having the second imaging device 37) performs image recognition processing using template data (specific template data 72) corrected based on the specific data 60B of the second imaging device 37. As a result, the second sewing machine SM2 can correct template data 70, which was created by the first sewing machine SM1 (sewing machine 1 having the first imaging device 37) and reflects the imaging conditions specific to the first sewing machine SM1, to match the imaging conditions specific to the second sewing machine SM2 and perform image recognition processing. This eliminates the need to create template data 70 separately for each of the multiple sewing machines 1 included in sewing system 100. Furthermore, in data processing unit 50, administrator Mb only needs to manage one template data 70 (generalized template data 71) created by the first sewing machine SM1, and does not need to manage separate template data (specific template data 72) for each sewing machine 1. As a result, the number of steps required for image recognition and the number of steps required for data management can be reduced in a sewing system 100 that includes multiple sewing machines 1. Note that the second sewing machine SM2 may perform image recognition processing using the template data 70 created by the first sewing machine SM1 and correct the recognition results based on the unique data 60B of the second sewing machine SM2. In this case, the number of steps required for image recognition and the number of steps required for data management can also be reduced. Furthermore, as shown in FIGS. 12 and 13 , even in a configuration in which one sewing machine 1 includes multiple image capture devices 37 (i.e., the first image capture device 37 and the second image capture device 37), the same effect can be achieved by correcting the template data 70 created by the first image capture device 37 based on the unique data 60 of the second image capture device 37, or by correcting the image recognition results using the template data 70 based on the unique data 60 of the second image capture device 37.

[0148] Furthermore, according to the present embodiment, data processing unit 50 includes master data management unit 51A that creates generalized template data 71, and therefore elements attributable to specific data 60A of first sewing machine SM1 can be eliminated from template data 70 created by first sewing machine SM1 (sewing machine 1 having first imaging device 37). As a result, when creating specific template data 72 for second sewing machine SM2 (sewing machine 1 having second imaging device 37), it is not necessary to use specific data 60A of first sewing machine SM1, which was the source of template data 70, each time, thereby reducing the amount of data management work. Furthermore, data processing unit 50 includes data adjustment unit 51B that creates specific template data 72 for each imaging device based on generalized template data 71 and specific data 60. Therefore, specific template data 72 suitable for second sewing machine SM2 can be obtained without having to correct the image recognition results of second sewing machine SM2 each time.

[0149] Other Embodiments In the above embodiment, an example was shown in which unique template data 72 for second sewing machine SM2 was created from template data 70 of first sewing machine SM1 and transmitted to second sewing machine SM2, but the correction of template data in the present disclosure is not limited to the above example. Data processing unit 50 may transmit template data 70 and unique data 60A of first sewing machine SM1 directly to second sewing machine SM2, and second sewing machine SM2 may correct template data 70 (create unique template data 72) based on the difference between unique data 60A of first sewing machine SM1 and unique data 60B of second sewing machine SM2. In addition, if the difference in imaging conditions is small, the template data 70 of the first sewing machine SM1 can be used directly on the second sewing machine SM2 to perform image recognition processing, and the recognition results using the template data 70 can be corrected based on the difference between the unique data 60A of the first sewing machine SM1 and the unique data 60B of the second sewing machine SM2.

[0150] In the above embodiment, data processing unit 50 is provided with master data management unit 51A that creates generalized template data 71, but generalized template data 71 does not necessarily have to be created. As described above, template data 70 and specific data 60A of first sewing machine SM1 and specific data 60B of second sewing machine SM2 may be used to create specific template data 72 of second sewing machine SM2. In this case, template data 70 and specific data 60A of first sewing machine SM1 may be managed as master data instead of generalized template data 71.

[0151] In the above embodiment, data processing unit 50 is provided with data adjustment unit 51B that creates unique template data 72 for each sewing machine, but data processing unit 50 does not have to be provided with data adjustment unit 51B. Data processing unit 50 may send generalized template data 71 to each second sewing machine SM2, and each second sewing machine SM2 may create its own unique template data 72 using its own unique data 60B.

[0152] In the above embodiment, an example has been shown in which multiple image capturing devices 37 are provided in the sewing machine 1 and image recognition processing is performed using a composite image 46 formed by stitching together multiple images, but the sewing machine 1 may be equipped with only one image capturing device 37. The sewing machine 1 may also capture images of the sewing object S multiple times and perform image recognition processing on each image.

[0153] The present disclosure includes the following aspects: (1) A sewing system comprising: a plurality of imaging devices that capture images of a sewing object; one or more sewing machines that perform sewing based on a recognition result obtained by image recognition processing of the sewing object; and a data processing unit that acquires template data of the sewing object created based on an image captured by a first imaging device among the plurality of imaging devices and transmits the template data to another second imaging device, wherein the sewing machine has unique data including imaging conditions of the imaging devices, and the sewing machine having the second imaging device performs image recognition processing using the template data corrected based on the unique data of the second imaging device, or corrects the recognition result using the template data based on the unique data of the second imaging device. (2) The sewing system described in (1), wherein the unique data includes a pixel rate, an imaging offset, and lighting information of the imaging devices. (3) The sewing system according to (1) or (2), wherein the data processing unit includes a master data management unit that creates generalized template data by generalizing template data based on the template data and unique data of the first imaging device, and the sewing machine having the second imaging device performs image recognition processing using unique template data obtained by correcting the generalized template data based on the unique data of its own machine, or corrects a recognition result using the generalized template data based on the unique data of its own machine. (4) The sewing system according to (3), wherein the data processing unit includes a data adjustment unit that acquires the unique data of the multiple imaging devices and creates the unique template data for each of the imaging devices based on the generalized template data or the unique template data including the unique data of the imaging device used when creating the template and the unique data, and the sewing machine having the second imaging device performs image recognition processing using the unique template data of its own machine acquired from the data processing unit.(5) The sewing system according to any one of (1) to (4), wherein the sewing machine is provided with a plurality of the imaging devices having overlapping regions where imaging fields of view partially overlap, and performs image recognition processing based on the template data on a composite image obtained by joining together a plurality of images obtained from the plurality of the imaging devices. (6) A sewing method using a plurality of sewing machines, each having an imaging device that acquires an image of a sewing object and performing sewing based on a recognition result obtained by image recognition processing of the sewing object, or a sewing method using a sewing machine having a plurality of the imaging devices, comprising: acquiring template data of the sewing object created based on an image captured by a first imaging device of the plurality of sewing machines or the plurality of the imaging devices, and transmitting the template data to another, second imaging device; and performing image recognition processing in the sewing machine having the second imaging device using the template data corrected based on unique data including imaging conditions of the second imaging device, or correcting a recognition result using the template data based on the unique data.

[0154] This application is based on Japanese Patent Application No. 2023-108670, filed on June 30, 2023, the contents of which are incorporated herein by reference.

Claims

1. A sewing system comprising: a plurality of imaging devices which capture images of an object to be sewn; one or more sewing machines which perform sewing based on a recognition result obtained by an image recognition process of the object to be sewn; and a data processing unit which acquires template data of the object to be sewn which is created based on an image captured by a first imaging device among the plurality of imaging devices, and transmits the template data to another second imaging device, wherein the sewing machines have unique data including imaging conditions of the imaging devices, and the sewing machine having the second imaging device performs image recognition processing using the template data corrected based on the unique data of the second imaging device, or corrects the recognition result using the template data based on the unique data of the second imaging device.

2. The sewing system according to claim 1, wherein the specific data includes a pixel rate of the imaging device, an imaging offset, and lighting information.

3. The sewing system according to claim 1 or 2, wherein the data processing unit has a master data management unit which creates generalized template data by generalizing template data based on the template data and unique data of the first imaging device, and the sewing machine having the second imaging device executes image recognition processing using unique template data obtained by correcting the generalized template data based on the unique data of the sewing machine itself, or corrects a recognition result using the generalized template data based on the unique data of the sewing machine itself.

4. The sewing system according to claim 3, wherein the data processing unit acquires the unique data of the multiple imaging devices, and has a data adjustment unit that creates the unique template data for each of the imaging devices based on the generalized template data or the unique template data including the unique data of the imaging device used when creating the template and the unique data, and the sewing machine having the second imaging device executes image recognition processing using the unique template data of its own machine acquired from the data processing unit.

5. The sewing system according to claim 1 or 2, wherein the sewing machine is provided with a plurality of the imaging devices having overlapping areas in which the imaging fields of view partially overlap, and performs image recognition processing based on the template data on a composite image formed by stitching together a plurality of images obtained from the plurality of the imaging devices.

6. A sewing method using a plurality of sewing machines having an imaging device that acquires an image of an object to be sewn and that performs sewing based on a recognition result obtained by an image recognition process of the object to be sewn, or a sewing method using a sewing machine having a plurality of the imaging devices, comprising: acquiring template data of the object to be sewn that is created based on an image captured by a first imaging device among the plurality of sewing machines or the plurality of the imaging devices, and transmitting the template data to another second imaging device; and, in the sewing machine having the second imaging device, executing an image recognition process using the template data corrected based on unique data including the imaging conditions of the second imaging device, or correcting the recognition result using the template data based on the unique data.

Images