Decoration application system and holder

Abstract

[Solution] A decoration application system (1) is provided with a decoration position calculation unit (36), and a frame coordinate setting unit (35) and a superimposed image creation unit (37) serving as correction processing units. The decoration position calculation unit (36) calculates, on the basis of a holder image, which is a captured image of a holder capable of holding an object to be decorated, a decoration position, which is a position at which decoration is applied to the object to be decorated. The superimposed image creation unit (37) subjects the holder image to correction processing. The decoration position calculation unit (36) calculates the decoration position on the basis of the holder image subjected to the correction processing. The frame coordinate setting unit (35) calculates a gradient of an imaging object on the holder image with respect to a plane orthogonal to the optical axis of an imaging unit of an imaging device that has captured the holder image. The superimposed image creation unit (37) subjects the holder image to correction processing on the basis of the calculated gradient.

Description

Decoration application system and holder

[0001] The present disclosure relates to a decoration application system and a holder used in the decoration application system.

[0002] Japanese Patent Application Laid-Open No. 2021-010620 discloses an embroidery frame provided with a plurality of frame markers for calculating a frame coordinate system. The embroidery frame includes an outer frame and an inner frame. Four frame markers are provided on the rectangular inner frame. Each frame marker is provided at the intersection of two sides of the inner frame. With the inner frame mounted inside the outer frame to hold the embroidery target, a frame image including the inner frame and the outer frame is captured with a camera of a terminal such as a smartphone. The outer frame is mounted on the frame holder of the sewing machine. The CPU of the sewing machine calculates the deviation of the inner frame with respect to the outer frame based on the frame coordinate system set based on the positions of the four frame markers on the frame image.

[0003] In Japanese Patent Application Laid-Open No. 2021-010620, when a user or the like captures a frame image, for example, when the optical axis of the camera is inclined by a predetermined angle or more with respect to the plane perpendicular to the sewing surface of the embroidery frame due to camera shake or the like. In such a case, the embroidery frame on the frame image is inclined compared to the actual embroidery frame, and as a result, the frame coordinate system may also be inclined. It is difficult to perform the following processing in such a state, and it may be necessary to obtain a non-inclined frame image again. The following processing is, for example, a process of calculating the position where embroidery is to be performed. Obtaining a non-inclined frame image again may, for example, require the user to re-capture the image, which may lead to a decrease in the efficiency of operations such as system use.

[0004] One aspect of the present disclosure is to provide a decoration application system that can avoid a decrease in efficiency during system use, and a holder used in the decoration application system.

[0005] According to one feature of this disclosure, the decoration application system comprises a decoration position calculation unit and a correction processing unit. The decoration position calculation unit calculates the decoration position, which is the position where decoration is applied to the object to be decorated, based on a holder image, which is a captured image of a holder capable of holding the object to be decorated. The correction processing unit performs a correction process on the holder image. The decoration position calculation unit calculates the decoration position based on the corrected holder image. The correction processing unit calculates the gradient of the object to be photographed on the holder image with respect to the plane orthogonal to the optical axis of the photographing unit of the photographing device that photographed the holder image, and performs a correction process on the holder image based on the calculated gradient.

[0006] Therefore, because the correction processing is performed by the correction processing unit, the process related to adding decorations can continue even if the object on the held image is tilted, without having to acquire the image again. This avoids a decrease in efficiency when using the system.

[0007] According to other features of this disclosure, the holder is provided with at least one coordinate determination marker. The correction processing unit sets a holder coordinate system, which is a coordinate system based on the holder, based on at least one coordinate determination marker on the corrected holder image. The decoration position calculation unit calculates the position of the decoration in the holder coordinate system as the decoration position.

[0008] Therefore, the correction processing unit can set the holder coordinate system without having to acquire the image again if the object on the holder image is tilted. This avoids a decrease in efficiency when using the system.

[0009] According to other features of this disclosure, the holder is configured to be attachable to a decoration application device capable of applying decoration to an object to be decorated held by the holder. At least one coordinate determination marker includes a pair of vertical markers facing each other in the vertical direction, which is the front-to-back direction, and a pair of horizontal markers facing each other in the horizontal direction, which is the left-to-right direction, when the decoration application device to which the holder is attached is viewed from directly above. The correction processing unit sets a first line connecting the pair of vertical markers on the corrected holder image as the vertical axis of the holder coordinate system, sets a second line connecting the pair of horizontal markers on the holder image as the horizontal axis of the holder coordinate system, and sets the intersection of the first line and the second line as the origin of the holder coordinate system.

[0010] Therefore, the modification processing unit can set the holder coordinate system as a coordinate system that coincides with the forward, backward, left, and right directions in which the holder attached to the decoration device moves.

[0011] According to other features of this disclosure, the decoration position calculation unit calculates the decoration position based on the position of the position marker on the holder image.

[0012] According to other features of this disclosure, the decoration is applied to the object to be decorated by embroidery.

[0013] According to other features of this disclosure, the decoration application system comprises at least one decoration application device and a processing device. The processing device is connectable via a network to at least one terminal and at least one decoration application device. The processing device includes a decoration position calculation unit and a correction processing unit.

[0014] Therefore, while the decoration application device is applying decorations, the processing unit can perform calculation and correction processing of the decoration positions. This improves the efficiency of decoration application.

[0015] According to other features of this disclosure, the correction process includes a process of correcting the holder image to an orientation perpendicular to the optical axis based on a gradient.

[0016] Therefore, since the holder image is corrected by the correction processing unit, the process of applying decorations can continue even if the object on the holder image is tilted, without having to acquire the image again. This avoids a decrease in efficiency when using the system.

[0017] According to other features of this disclosure, the modification process includes deforming the image of the decoration to be applied to the retainer image based on a gradient, and superimposing the deformed image of the decoration onto the retainer image.

[0018] Therefore, the image of the deformed decoration is superimposed on the holder image by the correction processing unit, and the decoration position is calculated based on the image of the deformed decoration by the decoration position calculation unit. As a result, the decoration position can be calculated without acquiring the image again if a tilt occurs, thus avoiding a decrease in efficiency when using the system.

[0019] According to other features of this disclosure, the image of the deformed ornament is superimposed with respect to the position of the position marker on the retainer image.

[0020] According to one feature of this disclosure, the holder comprises a holder body and a plurality of markers. The holder body holds the object to be decorated. The plurality of markers are provided on the holder body. The plurality of markers include at least one coordinate determination marker used to set a holder coordinate system, which is a coordinate system based on the holder, and at least one identification marker having holder-specific information set for each holder.

[0021] Therefore, since the holder has a coordinate determination marker, it is easy to set up a holder coordinate system based on the holder. In addition, since the holder is provided with an identification marker that contains holder-specific information, it is easy to identify the holder.

[0022] According to other features of the present disclosure, at least one coordinate determination marker includes a pair of vertical markers facing each other in the front-to-back direction and a pair of horizontal markers facing each other in the left-to-right direction when the ornament-applying device to which the holder is attached is viewed from directly above.

[0023] Therefore, the holder coordinate system can be set as a coordinate system that coincides with the forward, backward, left, and right directions in which the holder attached to the decoration device moves.

[0024] According to other features of this disclosure, the decoration device is a sewing machine, and the holder is an embroidery hoop that can be attached to the sewing machine.

[0025] This is a schematic diagram of the system related to this disclosure. This is a schematic diagram of the sewing machine and embroidery hoop. This is a block diagram of the system. This is a diagram showing the target image displayed on the terminal. This is a diagram showing the superimposed image displayed on the terminal. This is a diagram showing the hoop image. This is a diagram illustrating the calculation of the gradient of the embroidery hoop on the hoop image. This is a diagram showing the hoop superimposed image. This is a diagram showing the user information, decoration type information, and hoop type information stored on the server. This is a diagram showing the identification information and linking information stored on the server. This is a flowchart of the order processing. This is a flowchart of the embroidery preparation processing. This is a flowchart of the embroidery processing.

[0026] A decoration application system for applying decoration to an object to be decorated includes a decoration position calculation unit that calculates a decoration position, which is the position where decoration is applied to the object to be decorated, based on a holder image, which is a captured image of a holder capable of holding the object to be decorated, and a correction processing unit that applies a correction process to the holder image. The decoration position calculation unit calculates the decoration position based on the holder image after the correction process has been applied. The correction processing unit calculates the gradient of the object to be photographed on the holder image with respect to the plane orthogonal to the optical axis of the photographing unit of the photographing device that photographed the holder image, and applies the correction process to the holder image based on the calculated gradient.

[0027] In the decoration application system, the holder may be provided with at least one coordinate determination marker. The correction processing unit may set a holder coordinate system, which is a coordinate system based on the holder, based on at least one of the coordinate determination markers on the holder image that has been corrected. The decoration position calculation unit may calculate the position of the decoration in the holder coordinate system as the decoration position.

[0028] In any of the above decoration application systems, the holder may be configured to be attachable to a decoration application device capable of applying decoration to the object to be decorated held by the holder. At least one of the coordinate determination markers may include a pair of vertical markers facing each other in the vertical direction, which is the front-to-back direction, and a pair of horizontal markers facing each other in the horizontal direction, which is the left-to-right direction, when the decoration application device to which the holder is attached is viewed from directly above. Furthermore, in any of the above decoration application systems, the correction processing unit sets a first line connecting the pair of vertical markers on the holder image that has undergone the correction processing as the vertical axis of the holder coordinate system, sets a second line connecting the pair of horizontal markers on the holder image as the horizontal axis of the holder coordinate system, and sets the intersection of the first line and the second line as the origin of the holder coordinate system.

[0029] In any of the above-described decoration application systems, the decoration position calculation unit may calculate the decoration position based on the position of the position marker on the holder image.

[0030] In any of the above-described decoration systems, the decoration may be applied by embroidery to the object to be decorated.

[0031] Any of the above decoration application systems may include at least one decoration application device and a processing device that can be connected to at least one of the decoration application devices via a network. In any of the above decoration application systems, the processing device may also include the decoration position calculation unit and the correction processing unit.

[0032] In any of the above-described decoration systems, the modification process may include a process of modifying the holder image to a orientation perpendicular to the optical axis based on the gradient.

[0033] In any of the above-described decoration application systems, the modification process may include a process of deforming the image of the decoration to be applied to the holder image based on the gradient, and superimposing the deformed image of the decoration onto the holder image.

[0034] In any of the above decoration systems, the deformed image of the decoration may be superimposed based on the position of the position marker on the holder image.

[0035] A holder that can be attached to a decoration application device comprises a holder body for holding an object to be decorated, and a plurality of markers provided on the holder body. The plurality of markers include at least one coordinate determination marker used to set a holder coordinate system which is a coordinate system based on the holder, and at least one identification marker having holder-specific information set for each holder.

[0036] In any of the above-described holders, at least one of the coordinate determination markers may include a pair of vertical markers facing each other in the front-to-back direction and a pair of horizontal markers facing each other in the left-to-right direction when the decoration device to which the holder is attached is viewed from directly above.

[0037] In any of the above-described holders, the decoration-applying device is a sewing machine, and the holder may be an embroidery hoop that can be attached to the sewing machine.

[0038] A decoration application system is a system that applies decoration to an object to be decorated. Decorations refer to things like patterns, letters, and symbols, and are applied by methods such as embroidery or printing. The object to be decorated is, for example, embroiderable fabric or printable paper.

[0039] The decoration system 1 in this embodiment is a system that applies decoration 3 to the object to be decorated 2 by embroidery. The object to be decorated 2 in this embodiment is a fabric material. However, the object to be decorated 2 is not limited to a fabric material, but can be any object that can be embroidered. The decoration application device in this embodiment is a sewing machine. The decoration system 1 in this embodiment will be described below with reference to Figures 1 to 13.

[0040] As shown in Figure 1, the overall configuration of the decoration application system 1 includes a plurality of sewing machines 10, a plurality of terminals 20, and a server 30. The plurality of sewing machines 10 and the plurality of terminals 20 are connected to the server 30 via a network 5. The plurality of sewing machines 10 are provided, for example, in each of the plurality of stores T1 and T2. The number of sewing machines 10 provided in each store T1 and T2 may be one or more.

[0041] The multiple terminals 20 are well-known smartphones. However, the multiple terminals 20 are not limited to smartphones; for example, they may be information terminals such as well-known tablet personal computers. Furthermore, the multiple terminals 20 may be installed in each of the multiple stores T1 and T2, or they may be owned by customers. Also, the models of the multiple terminals 20 may be different.

[0042] As shown in Figures 2 and 3, the configuration of the sewing machine 10 includes a decoration application unit 11, a frame holding unit 12, a CPU 13, a frame information acquisition unit 14, an input unit 15 which also serves as a display unit 16, a sewing machine transmission unit 17, and a sewing machine reception unit 18. As shown in Figure 2, the frame holding unit 12 can detachably hold an embroidery hoop 40 capable of holding the object to be decorated 2. There are multiple types of embroidery hoops 40 used in this embodiment, but the frame holding unit 12 can detachably hold these types of embroidery hoops 40. Note that only one type of embroidery hoop 40 may be used in this embodiment. The decoration application unit 11 can apply decoration to the object to be decorated 2 within the frame of the embroidery hoop 40 held in the frame holding unit 12 by embroidery. In the following description, various directions are defined based on the state in which the operator has attached the embroidery hoop 40 to the frame holding unit 12. Specifically, the front-to-back direction is defined as the side closest to the viewer when viewed from directly above, and the side furthest away is defined as the rear. The up-down, left-to-right, and right directions are defined as the direction viewed from the worker's perspective.

[0043] The decoration applying unit 11 includes a head driving unit 11a, a vertical axis driving unit 11b, and a horizontal axis driving unit 11c. The head driving unit 11a sequentially drives any one of the plurality of embroidery needles provided on the head of the decoration applying unit 11 for use in embroidery. The vertical axis driving unit 11b moves the frame holding unit 12 in the front-rear direction. The horizontal axis driving unit 11c moves the frame holding unit 12 in the left-right direction.

[0044] The frame holding unit 12 is movable in the front-rear direction and the left-right direction with respect to the decoration applying unit 11 by a driving source such as a motor provided in the decoration applying unit 11. Further, the frame holding unit 12 is movable integrally with the held embroidery frame 40. Based on the decoration information 54 (see FIG. 10) regarding the decoration 3, the decoration applying unit 11 embroiders at a predetermined position of the decoration target 2 that has moved together with the embroidery frame 40 by the frame holding unit 12 with an embroidery needle to which an embroidery thread of a predetermined color is attached. By the interlocking of the decoration applying unit 11 and the frame holding unit 12, the sewing machine 10 can apply a desired decoration 3 (see FIG. 8) to the decoration target 2 in a desired state (including position, orientation, size, etc.).

[0045] The decoration information 54 includes decoration type information 55 which is information regarding the type of decoration to be applied, and decoration position information 57 which is information regarding the position where the decoration is to be applied. The decoration type information 55 in the present embodiment includes information such as the type of patterns such as figures, patterns, characters, symbols, etc., embroidery pattern data (shape, color), and the size in the initial state.

[0046] The CPU 13 controls the driving of each structure of the sewing machine 10. The CPU 13 is electrically connected to each of the decoration applying unit 11, the frame information acquisition unit 14, the input unit 15, the sewing machine transmission unit 17, and the sewing machine reception unit 18.

[0047] The frame information acquisition unit 14 is a mechanism capable of acquiring frame information 60 (see FIG. 10) and the like regarding the embroidery frame 40. Specifically, the frame information acquisition unit 14 is a mechanism capable of reading tag information, and in this embodiment, it is an RFID reader. As will be described later, the identification tag 44 of the embroidery frame 40 is an RFID tag, and the frame information acquisition unit 14 can acquire the frame information 60 from the identification tag 44 by wireless communication by bringing the identification tag 44 closer. The frame information acquisition unit 14 is located at a position close to the identification tag 44 in a state where the embroidery frame 40 is held by the frame holding unit 12, and in this embodiment, it is located on the left side of the decoration application unit 11 in the sewing machine 10.

[0048] The input unit 15 is a mechanism through which a user can input information regarding decoration, and for example, it is a touch panel that also serves as the display unit 16.

[0049] As shown in FIG. 3, the sewing machine transmission unit 17 and the sewing machine reception unit 18 transmit and receive information to and from the server 30 via the network. The sewing machine transmission unit 17 and the sewing machine reception unit 18 may be an integrated transmission and reception unit, or may each have an individual structure. The sewing machine transmission unit 17 can transmit the frame information 60 and the like acquired by the frame information acquisition unit 14 to the server reception unit 34 of the server 30 via the network 5. Further, the sewing machine reception unit 18 can receive, for example, the linking information 53 (see FIG. 10) stored in the database 31 of the server 30 from the server transmission unit 33 of the server 30 via the network 5.

[0050] Regarding the configuration of the terminal 20, as shown in FIG. 3, the terminal 20 includes a CPU 21, a photographing unit 22, an input unit 23 that also serves as the display unit 24, a terminal transmission unit 25, and a terminal reception unit 26. The CPU 21 controls the operations of each part of the terminal 20. The CPU 21 is electrically connected to each of the photographing unit 22, the input unit 23 that also serves as the display unit 24, the terminal transmission unit 25, and the terminal reception unit 26. The photographing unit 22 is a photographing mechanism such as a camera. The input unit 23 is a mechanism through which a user can input information, and for example, it is a touch panel that also serves as the display unit 24.

[0051] The terminal transmission unit 25 and the terminal reception unit 26 transmit and receive information to and from the server 30 via the network. The terminal transmission unit 25 and the terminal reception unit 26 may be a single transmission / reception unit, or they may have separate structures. The terminal transmission unit 25 can transmit identification information 50, target images 71 and frame images 73 captured by the imaging unit 22, and signals requesting operations input by the input unit 23 to the server reception unit 34 of the server 30 via the network 5. The terminal reception unit 26 can also receive, for example, images of decorations 3 stored in the database 31 of the server 30, superimposed images 72 and frame superimposed images 74 edited by the server 30, etc., from the server transmission unit 33 of the server 30 via the network 5.

[0052] As shown in Figure 3, the server 30 comprises a database 31, a CPU 32, a server transmission unit 33, and a server reception unit 34. The database 31 is a database capable of storing various types of information and functions as a storage unit. The CPU 32 controls the operation of each part of the server 30 and is electrically connected to the database 31, the server transmission unit 33, and the server reception unit 34. The CPU 32 comprises a frame coordinate setting unit 35, a decoration position calculation unit 36, a superimposed image creation unit 37, a linking processing unit 38, and an extraction unit 39. The frame coordinate setting unit 35 sets the frame coordinate system, which will be described later. The decoration position calculation unit 36 ​​calculates decoration information 54, which is information regarding the position to which decoration 3 is applied, as will be described later. The superimposed image creation unit 37 superimposes the image of decoration 3 onto the target image 71, which will be described later. The linking processing unit 38 performs a linking process, which is the process of linking various types of information to the identification information 50. The extraction unit 39 extracts various types of information stored in the database 31.

[0053] The server transmission unit 33 and the server reception unit 34 transmit and receive information with multiple sewing machines 10 and multiple terminals 20 via the network. The server transmission unit 33 and the server reception unit 34 may be a single transmission / reception unit, or they may have separate structures. The various types of information that the server transmission unit 33 and the server reception unit 34 can transmit and receive are as described above.

[0054] Next, using Figure 10, examples of various types of information stored in the database 31 will be explained. The database 31 stores identification information 50, frame information 60, decoration information 54, and image information 70. In this embodiment, each of the frame information 60, decoration information 54, and image information 70 is stored in a state linked to the identification information 50.

[0055] The identification information 50 includes user information 51, which is information about the user, and order information 52, which is information about the order. As shown in Figure 9, user information 51 includes user information ID 51A, such as a member ID, password 51B, username 51C, contact information 51D, owned sewing machine ID 51E, etc., which are assigned to each user. Note that user information 51 is not limited to these, and any information that can identify the user is acceptable. The database 31 stores password 51B, username 51C, contact information 51D, owned sewing machine ID 51E, etc., linked to user information ID 51A, for example. For example, store T1 is assigned "AAA" as user information ID 51A, and the owned sewing machine 10 is assigned "NA1" as owned sewing machine ID 51E. Returning to Figure 10, order information 52 includes order information ID, which is an ID assigned to each order, order history, etc. Note that order information 52 is not limited to these, and any information related to orders for decoration is acceptable.

[0056] The frame information 60 is information about the embroidery frame 40 and includes frame type information 61 and frame-specific information 62. The frame type information 61 is information about the type of embroidery frame 40, also known as the frame type, and as shown in Figure 9, includes the frame type ID 61A, frame shape 61B, frame size 61C, embroidery area 61D, frame image 61E, etc. The database 31 stores the frame shape 61B, frame size 61C, embroidery area 61D, frame image 61E, etc., linked to, for example, the frame type ID 61A. The frame type ID 61A is an ID corresponding to the type of embroidery frame 40. The frame shape 61B is a classification of the outer shape of the embroidery frame 40. The frame size 61C is data such as dimensions related to the outer shape of the embroidery frame 40. The embroidery area 61D is the coordinates of the area that can be embroidered when using the embroidery frame 40. The frame image 61E is an overall image of the embroidery frame 40. Returning to Figure 10, the frame-specific information 62 is unique information set for each embroidery frame 40, and includes a frame-specific ID. Embroidery frames of the same type 40 are assigned the same frame type ID 61A, but even embroidery frames of the same type 40 are assigned different frame-specific IDs.

[0057] The decoration information 54 is information related to decorations and includes decoration type information 55, position marker information 56, and decoration position information 57. The decoration type information 55 is information related to the type of decoration 3 to be applied. In this embodiment, the decoration type information 55 includes information such as an image of the decoration 3, pattern data (shape, color), and initial size. As shown in Figure 9, the decoration type information 55 includes a decoration ID 55A, a decoration image 55B, decoration reference data 55C, etc. The database 31 stores the decoration image 55B, decoration reference data 55C, etc., linked to, for example, the decoration ID 55A. The decoration ID 55A is an ID assigned to each decoration 3. The decoration image 55B is, for example, an image of the pattern data of the decoration 3. The decoration reference data 55C is, for example, pattern data that includes information on the coordinates of each needle position from the start to the end of embroidery, embroidery thread color information, etc., based on the reference position (e.g., the center) of the decoration image 55B when embroidering the decoration image 55B with a reference size and reference orientation.

[0058] Returning to Figure 10, the position marker information 56 is information possessed by the position marker 4, and a different ID is set for each position marker 4. The position marker 4 is placed on the object to be decorated 2 and functions as a reference for the decoration position. The position marker 4 is a marker called an AR marker, for example, and an ArUco marker is one example. The position marker 4 is, for example, a sticker with a pattern on one side and adhesive material on the other side, and is attached to the surface of the object to be decorated 2. In other words, the position marker 4 is a marker that can be easily placed by the user at the desired position. The pattern of the position marker 4 is, for example, a shape in which the entire square is divided into 5x5 or 7x7 square cells, and each cell is drawn in a single color, such as white or black. However, the form of the position marker 4 is not limited to the above. The photographed image of the object to be decorated 2 with the position marker 4 placed on its surface is called the target image 71. In this embodiment, the position marker information 56 is calculated based on the position markers 4 on the target image 71. The target image 71 only needs to include at least a portion of the position markers 4 placed on the surface of the object to be decorated 2.

[0059] The decoration position information 57 is information about the position on which the decoration 3 is applied. The decoration position information 57 includes first decoration position information 57A and second decoration position information 57B. The first decoration position information 57A is information about the position on the object to be decorated 2 on which the decoration 3 is applied, with reference to the position of the position marker 4. In other words, the first decoration position information 57A is information that represents the positional relationship between the position marker 4 and the position on which the decoration 3 is applied. In this embodiment, the first decoration position information 57A includes information about the position, scale, and rotation angle of the decoration 3 with reference to the center 4a of the position marker 4 on the superimposed image 72. The second decoration position information 57B is information about the position on the object to be decorated 2 on which the decoration 3 is applied, in the frame coordinate system described later. In other words, the second decoration position information 57B is information that represents the positional relationship between the origin of the frame coordinate system and the position on which the decoration 3 is applied. In this embodiment, the first decoration position information 57A and the second decoration position information 57B are calculated by the decoration position calculation unit 36. The second decoration position information 57B includes information on the position, scale, and rotation angle of the decoration 3 relative to the frame coordinate system, and is calculated by performing a coordinate transformation on the first decoration position information 57A based on the frame coordinate system.

[0060] Image information 70 is information about various images, and includes information about the target image 71, the superimposed image 72, the frame image 73, and the frame superimposed image 74. As described above, the target image 71 is a photographed image of the object to be decorated 2 with the position marker 4 placed on its surface, as shown in Figure 4. In this embodiment, the target image 71 is photographed by the photographing unit 22 of the terminal 20. The superimposed image 72 is an image in which the image of the decoration 3 is superimposed on the target image 71, as shown in Figure 5. In this embodiment, the superimposed image is superimposed by the superimposed image creation unit 37 of the server 30 based on the position of the position marker 4 on the target image 71. The frame image 73 is a photographed image of the embroidery hoop 40 holding the object to be decorated 2 with the position marker 4 placed on its surface, as shown in Figure 6. In this embodiment, the frame image 73 is photographed by the photographing unit 22 of the terminal 20. The frame superimposed image 74 is an image in which the image of the decoration 3 is superimposed on the frame image 73, as shown in Figure 8. In this embodiment, the superimposed image is created by the superimposed image creation unit 37 of the server 30 based on the position of the position marker 4 on the frame image 73.

[0061] The linking processing unit 38 sequentially links the information contained in the frame information 60, decoration information 54, and image information 70 to the identification information 50. The frame information 60, decoration information 54, and image information 70 linked to the identification information 50 by the linking processing unit 38 in this way are collectively referred to as linked information 53. The information contained in linked information 53 is linked to each other. The identification information 50 and linked information 53 are stored in the database 31. The information stored in the database 31 can be extracted by the extraction unit 39.

[0062] Regarding the acquisition of the target image 71 and superimposed image 72 in terminal 20 and server 30, the shooting unit 22 photographs the object to be decorated 2 on which the position marker 4 is placed, and the CPU 21 of terminal 20 can acquire the target image 71 as shown in Figure 4. The acquired target image 71 is transmitted from terminal transmission unit 25 to server reception unit 34.

[0063] The CPU 32 of the server 30 acquires position marker information 56 and position information of the position marker 4 based on the position marker 4 of the target image 71 received by the server receiving unit 34. In this embodiment, the position information includes not only front, back, left, and right position information, but also size (scale) information and orientation (rotation angle) information. The target image 71 is stored in the database 31.

[0064] Furthermore, the superimposed image creation unit 37 creates a superimposed image 72. The superimposed image creation unit 37 obtains images of decorations 3 included in the decoration type information 55 which is linked to the target image 71 from the database 31. Then, the superimposed image creation unit 37 superimposes the obtained images of decorations 3 onto the target image 71. The superimposed target image 71 is referred to as the superimposed image 72. Decorations 3 are superimposed, for example, at predetermined positions on the target image 71 based on the position marker 4, such that the center of the image of decoration 3 coincides with the center 4a of the position marker 4. Alternatively, the front end, rear end, left end, right end, etc., of the image of decoration 3 is placed at the center 4a of the position marker 4 and superimposed. The superimposed image 72 is stored in the database 31 and transmitted by the server transmission unit 33 to the terminal 20, which is the source of the target image 71.

[0065] As shown in Figure 5, the display unit 24 of the terminal 20 displays the superimposed image 72 received by the terminal receiving unit 26. This display allows the user to confirm that the image of decoration 3 on the superimposed image 72 is positioned as desired. At this time, the display unit 24 also displays a modify button 23a and an exit button 23b. When the user presses the modify button 23a, the display on the display unit 24 ends. When the user presses the modify button 23a, the modification mode is activated. In modification mode, the position, scale, or rotation angle of the image of decoration 3 on the displayed superimposed image 72 can be changed in the front-back, left-right, or right-handed directions. The image of decoration 3 can be moved or rotated, for example, by a user flicking on the input unit 23. However, it is not limited to this. The position, scale, and rotation angle of the image of decoration 3 in the front-back, left-right, or right-handed directions are changed, for example, with respect to the center 4a of the position marker 4 on the superimposed image 72.

[0066] In modification mode, if the user makes changes to the position of the decoration 3 image, modification request information requesting modification of the superimposed image 72 is sent from the terminal transmission unit 25 to the server reception unit 34. Based on the modification request information, the superimposed image creation unit 37 modifies the superimposed image 72 by moving the decoration 3 image on the superimposed image 72 with respect to the center 4a of the position marker 4. The modified superimposed image 72 is linked to the identification information 50 by the linking processing unit 38 and is stored again in the database 31 in place of the original superimposed image 72. The modified superimposed image 72 is then sent again by the server transmission unit 33 to the terminal 20, which is the source of the target image 71, and is displayed again on the display unit 24 of the terminal 20. When the user presses the end button 23b, the modification mode ends, and the display on the display unit 24 also ends. In this case, the database 31 stores the superimposed image 72 in the state it was last edited in.

[0067] Regarding the acquisition of the frame image 73 and the frame superimposed image 74 in terminal 20 and server 30, the shooting unit 22 photographs the embroidery hoop 40 holding the object to be decorated 2 with the position marker 4 placed on its surface, and the CPU 21 of terminal 20 can acquire the frame image 73 as shown in Figure 6. The acquired frame image 73 is transmitted from terminal transmission unit 25 to server reception unit 34. Note that in Figures 6 and 7, only a portion of the actual objects on the frame image 73 (the embroidery hoop 40, the object to be decorated 2, and the position marker 4) are shown.

[0068] First, let's describe the structure of the embroidery hoop 40. The embroidery hoop 40 has a main body 41a and a holding portion 41b that protrudes outward from both the left and right sides of the main body 41a. The object to be decorated 2 is held with its surface stretched flat against the inner circumference of the inner edge 41c of the main body 41a. The holding portion 41b is configured to be attachable to the hoop holder 12 of the sewing machine 10, and when the holding portion 41b is attached to the hoop holder 12, the embroidery hoop 40 is held by the sewing machine 10 (see Figure 2).

[0069] As shown in Figure 6, the frame body 41a of the embroidery hoop 40 is provided with an identification marker 42, four coordinate determination markers 43, and an identification tag 44. The identification marker 42 is a square-shaped two-dimensional image marker similar to the position marker 4. The identification marker 42 has different frame-specific information 62 for each embroidery hoop 40. In the decoration application system 1, multiple types of embroidery hoops 40 may be provided, but each embroidery hoop 40 can be identified by attaching the identification marker 42 in this way. The identification marker 42 is provided on the upper side of the frame body 41a in the position in which the embroidery hoop 40 is held by the sewing machine 10, and at a position that avoids the front-to-back center and the left-to-right center of the frame body 41a (see Figure 2, etc.). Therefore, the user can easily check the top, bottom, front, back, left, and right orientation of the embroidery hoop 40 by looking at the identification marker 42. This prevents the user from attaching the embroidery hoop 40 to the frame holder 12 in the wrong orientation (for example, upside down, front-to-back, or left-to-right inversion).

[0070] The four coordinate determination markers 43 are two-dimensional, square-shaped image markers similar to the position marker 4. The four coordinate determination markers 43 are used as position references for setting the origin of the frame coordinate system when setting the frame coordinate system, as described later, and include two vertical markers 43a and two horizontal markers 43b. The four coordinate determination markers are provided on the upper surface of the frame body 41a in the posture in which the embroidery hoop 40 is held by the frame holding part 12. Of the four coordinate determination markers, the two vertical markers 43a are provided at positions opposite each other in the front-to-back direction (vertical direction) in that posture, and the two horizontal markers 43b are provided at positions opposite each other in the left-to-right direction (horizontal direction) in that posture (see Figure 2). Each coordinate determination marker 43 is provided such that the line connecting the centers of the two vertical markers 43a and the line connecting the centers of the two horizontal markers 43b are perpendicular to each other. Furthermore, at least one of the four coordinate determination markers 43 is a marker that has frame type information 61 indicating the frame type of the embroidery hoop 40.

[0071] Since the identification marker 42 is square-shaped, when the imaging unit 22 photographs the surface of the object to be decorated 2 perpendicular to its surface, the identification marker 42 on the frame image 73 will be rectangular in shape, with the length 4b in the first direction and the length 4c in the second direction perpendicular to the first direction being the same length. The same applies to the coordinate determination marker 43; the coordinate determination marker 43 on the frame image will be rectangular in shape, with the vertical length 43c in the front-to-back direction and the horizontal length 43d in the left-to-right direction being the same length. The case where the imaging unit 22 photographs from a direction inclined with respect to the direction perpendicular to the surface of the object to be decorated 2 will be described later.

[0072] The identification tag 44 is a tag provided on the surface or inside the frame body 41a. The identification tag 44 has frame-specific information 62 which is the same as the information held by the identification marker 42. In addition to the frame-specific information 62, the identification marker 42 and the identification tag 44 may also have frame type information 61. The identification tag 44 is a tag whose information can be read by wireless communication such as radio waves, which is called an RFID tag. However, the identification tag 44 is not limited to this, and any tag capable of transmitting information to the frame information acquisition unit 14 by wireless communication is acceptable. Furthermore, the identification tag 44 is provided in a location such that it is located near the frame information acquisition unit 14 of the sewing machine 10 when the embroidery hoop 40 is held by the sewing machine 10. This allows for smooth communication between the frame information acquisition unit 14 and the identification tag 44, and the frame information acquisition unit 14 can acquire the frame-specific information 62 from the identification tag 44.

[0073] Next, the image processing of the frame image 73 and the setting of the frame coordinate system by the server 30 will be described. The frame coordinate system is a two-dimensional coordinate system based on the embroidery hoop 40. First, as shown in Figure 6, the frame coordinate setting unit 35 sets the first line L1, which connects the centers of the two vertical markers 43a on the frame image 73, as the vertical axis of the frame coordinate system, and the second line L2, which connects the centers of the two horizontal markers 43b on the frame image 73, as the horizontal axis of the frame coordinate system. The frame coordinate setting unit 35 also sets the intersection point P where the first line L1 and the second line L2 intersect as the origin of the frame coordinate system. In this way, the vertical axis, horizontal axis, and origin of the frame coordinate system are set.

[0074] Incidentally, camera shake may occur when the imaging unit 22 photographs the embroidery hoop 40. In this case, as shown in Figure 7, the optical axis (shooting direction) of the imaging unit 22 is inclined with respect to the direction perpendicular to the surface of the object to be decorated 2 held in the embroidery hoop 40. As a result, the position marker 4, identification marker 42, coordinate determination marker 43, etc. on the hoop image 73 may become distorted from their original rectangular shape, for example, into a trapezoid or parallelogram shape. The hoop coordinate setting unit 35 can set the hoop coordinate system even from a hoop image 73 in such a state. The method for setting the hoop coordinate system will be explained below using the case where the rear coordinate determination marker 43 on the hoop image 73 is distorted as an example, but the same method can be used to set the hoop coordinate system even if other markers are distorted. Note that Figure 7 only shows the hoop body 41a, identification marker 42, and the four coordinate determination markers 43 of the embroidery hoop 40.

[0075] As shown in Figure 7, the vertical lengths 43e and 43f on the left and right sides, and the horizontal lengths 43g and 43h on the front and back sides of the coordinate determination marker 43, which is distorted on the frame image 73, are different from the vertical length 43c and horizontal length 43d shown in Figure 6. Also, the sides of the coordinate determination marker 43 are not mutually orthogonal. The frame coordinate setting unit 35 calculates the gradient of the object to be photographed on the frame image 73 with respect to the orthogonal plane, which is the plane perpendicular to the optical axis of the shooting unit 22, based on the shape, size, and orientation of the coordinate determination marker 43. In this embodiment, the object to be photographed is an embroidery hoop 40, but it is not limited to this. If there is no camera shake or anything like that and the shooting unit 22 is photographing the object to be decorated 2 held in the embroidery hoop 40 from a direction perpendicular to the plane, then the plane formed by the embroidery hoop 40 on the frame image 73 and the orthogonal plane will be parallel, and the gradient of the embroidery hoop 40 will be 0.

[0076] The frame coordinate setting unit 35 readjusts the frame coordinate system based on the calculated gradient, using the embroidery frame 40 as the reference, and modifies the frame image 73 by transforming its coordinates based on the readjusted frame coordinate system. Specifically, the frame coordinate setting unit 35 modifies the frame image 73 to a reference orientation so that the embroidery frame 40 on the frame image 73 is perpendicular to the optical axis of the imaging unit 22. Furthermore, the frame coordinate setting unit 35 rotates the frame image 73 so that the first straight line L1 extends parallel to the front-to-back direction and the second straight line L2 extends parallel to the left-to-right direction. The modified frame image 73 is in the state shown in Figure 6.

[0077] The superimposed image creation unit 37 superimposes the image of decoration 3 onto the frame image 73 to create a frame superimposed image 74. The frame image 73 used in creating the frame superimposed image 74 is the frame image 73 modified as described above. The position where the image of decoration 3 is superimposed on the frame image 73 is determined based on the position of the position marker 4 and the first decoration position information 57A calculated by the decoration position calculation unit 36. The same applies to the scale and rotation angle of the image of decoration 3. The created frame superimposed image 74 is stored in the database 31 and transmitted by the server transmission unit 33 to the terminal 20, which is the source of the frame image 73. The display unit 24 of the terminal 20 displays the frame superimposed image 74 received by the terminal receiving unit 26. By displaying it in this way, the user can confirm that the image of decoration 3 on the frame superimposed image 74 is placed in the desired position. The frame superimposed image 74 may also be displayed on the display unit 16 of the sewing machine 10.

[0078] Furthermore, the decoration position calculation unit 36 ​​calculates second decoration position information 57B, which represents the positional relationship between the origin of the frame coordinate system and the position where the decoration 3 is applied, based on the frame superimposed image 74. The calculated second decoration position information 57B is linked to the identification information 50 by the linking processing unit 38 and then stored in the database 31. In other words, the second decoration position information 57B is calculated by performing a coordinate transformation on the first decoration position information 57A based on the frame coordinate system.

[0079] The following describes a modified example in which the superimposed image creation unit 37 creates the frame superimposed image 74. The frame coordinate setting unit 35 deforms the image of the decoration 3 to a shape that follows the gradient of the embroidery hoop 40, based on the gradient of the embroidery hoop 40 on the frame image 73. Specifically, it deforms the shape of the image of the decoration 3 according to the coordinate transformation required when correcting the frame image 73 to a reference orientation. Then, the superimposed image creation unit 37 creates the frame superimposed image 74 by superimposing the deformed image of the decoration 3 at a position based on the position of the position marker 4 on the frame image 73. This reference position is determined based on the position of the position marker 4 and the first decoration position information 57A calculated by the decoration position calculation unit 36.

[0080] Some or all of the operations of the above-described parts may be performed by a decoration application, which is an application related to the decoration application system 1. The decoration application may be in a form that can be installed and used on the sewing machine 10 or terminal 20, or it may be in a form that can be accessed from the sewing machine 10 or terminal 20 using a browser or the like.

[0081] Next, the decoration application process for applying decoration 3 to the object to be decorated 2 will be described. Below, an example in which a user applies the desired decoration 3 to the object to be decorated 2 in the desired state (position, scale, rotation angle) using terminal 20 at store T1 will be described using Figures 11 to 13. The decoration application process includes order processing, embroidery preparation processing, and embroidery execution processing. Order processing is the process up to the determination of the first decoration position information 57A based on the position marker 4, as shown in Figure 11. Embroidery preparation processing is post-processing after order processing, and is the process up to the start of embroidery on the object to be decorated 2, as shown in Figure 12. Embroidery execution processing is the process of operating the sewing machine 10 to apply decoration 3 to the object to be decorated 2. Part or all of the decoration application process may be executed by a decoration application application. First, the order processing will be described using Figure 11.

[0082] The user can access the server 30 via the network 5 by operating the terminal 20 and use the decoration application service provided by the decoration application system 1. Order processing begins, for example, when the user logs in to the decoration application application. The terminal 20 displays the decoration application menu screen provided by the decoration application service on the display unit 24. The decoration application menu screen includes a screen displaying various buttons such as "Send User Information," "Send Decoration Type Information," "Decoration List," "Photograph the Object to be Decorated," "Send Target Image," "Modify Overlay Image," and "Send Frame Image."

[0083] In step S01 of the order processing, terminal 20 sends user information 51 and decoration type information 55 to server 30 when it receives an instruction to send user information 51 and decoration type information 55. Specifically, in step S01, terminal 20 sends user information 51 and decoration type information 55 to server 30 when the "Send User Information" and "Send Decoration Type Information" buttons are pressed on the decoration application menu screen. The timing of the transmission of user information 51 and decoration type information 55 may be simultaneous or different. In this embodiment, the transmission of user information 51 and decoration type information 55 constitutes an order for decoration application. However, the order for decoration application is not limited to this.

[0084] User information 51 is entered by the user, for example, when logging into the decoration application. However, the timing of user information 51 input is not limited to this. Also, the decoration image 55B of the decoration type information 55 is selected in advance by the user before step S01. Specifically, when "Decoration List" is selected by the user on the decoration application service screen, terminal 20 sends embroidery list request information to server 30. When server 30 receives the embroidery list request information, it sends embroidery list screen information, which displays a list of decoration images 55B stored in database 31, to terminal 20. Terminal 20 displays the embroidery list screen information on display unit 24 and allows the user to select the desired decoration image 55B. However, the method of selecting the decoration image 55B of the decoration type information 55 is not limited to this. Note that the decoration ID 55A corresponding to the decoration image 55B may be assigned in advance, or it may be assigned by server 30 after the user's selection result is sent to server 30.

[0085] Next, in step S02, the server 30 stores the decoration type information 55 linked to the identification information 50. Specifically, when the server receiving unit 34 receives user information 51 and decoration type information 55, the linking processing unit 38 links the decoration type information 55 to the user information 51. At this time, the linking processing unit 38 assigns an order information ID and retrieves the user's past order history, and links this information as order information 52 to the decoration type information 55 together with the user information 51. Then, the database 31 stores the user information 51, order information 52, and decoration type information 55, respectively, in the data table shown in Figure 10.

[0086] Returning to Figure 11, in step S03, the terminal 20 photographs the object to be decorated 2 on which the position marker 4 is placed on the surface. The user places the position marker 4 near the location on the object to be decorated 2 where they want to apply the decoration 3 at the desired timing. Then, when the user selects "Photograph the object to be decorated" from the decoration application menu screen, the photography unit 22 is activated. The photography unit 22 photographs the object to be decorated 2 on which the position marker 4 is placed on the surface, based on the user's operation.

[0087] Next, in step S04, when "Send target image" is selected from the decoration menu screen, the terminal 20 sends the target image 71 captured by the shooting unit 22 to the server 30. The user can then check the captured target image 71 on the display unit 24 of the terminal 20, and if it is the desired image, select "Send target image," or if it is not the desired image, take another picture.

[0088] Next, in step S05, the server 30 stores the target image 71 and the location marker information 56 linked to the identification information 50. Specifically, the linking processing unit 38 recognizes the location marker 4 on the received target image 71 and calculates the location marker information 56. The linking processing unit 38 then links the target image 71 and the location marker information 56 to the identification information 50. The database 31 then adds and assigns the target image 71 and the location marker information 56 to the data table and stores them.

[0089] Next, in step S06, the server 30 superimposes the decorative image 55B on the target image 71 based on the position marker 4 to create a superimposed image 72. The superimposed image 72 is created by the superimposed image creation unit 37 using the method described above. The decorative position calculation unit 36 ​​calculates the first decorative position information 57A based on the method described above. The server 30 then transmits the created superimposed image 72 to the terminal 20.

[0090] Next, in step S07, the terminal 20 displays the received superimposed image 72 on the display unit 24. By checking the superimposed image 72 displayed on the terminal 20, the user can confirm the overall image (image after decoration) when the decoration 3 is applied to the object to be decorated 2.

[0091] Next, in step S08, the terminal 20 determines whether "Modify superimposed image" is selected on the decoration menu screen. If "Modify superimposed image" is not selected, the terminal proceeds to step S11. If "Modify superimposed image" is selected, the terminal 20 displays a screen (see Figure 5) on the display unit 24 that includes the superimposed image 72 as well as the modification button 23a and the exit button 23b.

[0092] When the correction button 23a is pressed, terminal 20 starts correction mode. In correction mode, when the image of decoration 3 on the superimposed image 72 is corrected using the method described above, correction request information including correction amount information is sent to server 30. The correction amount information includes information such as the amount of translation, scaling, and rotation of the image of decoration 3 on the superimposed image 72, with respect to the center 4a of the position marker 4. If the exit button 23b is pressed before correction mode is started, the process proceeds to step S11.

[0093] Next, in step S09, the server 30 modifies the superimposed image 72 based on the modification amount information included in the modification request information. Specifically, upon receiving the modification request information, the superimposed image creation unit 37 modifies and updates the state (translation amount, scaling amount, rotation amount) of the decoration image 55B related to the image of decoration 3 superimposed on the superimposed image 72 based on the modification amount information included in the modification request information. Then, the server 30 transmits the updated superimposed image 72 to the terminal 20. In addition, the decoration position calculation unit 36 ​​recalculates and updates the first decoration position information 57A calculated in step S06 based on the modification amount information.

[0094] Next, in step S10, the terminal 20 receives the updated superimposed image 72 and displays it on the display unit 24.

[0095] Next, in step S11, terminal 20 determines whether the exit button 23b has been pressed. If the exit button 23b has been pressed (YES), the process proceeds to step S12. If the exit button 23b has not been pressed, but the correct button 23a has been pressed (NO), the process returns to step S08, and the correction request information, including the correction amount information, is sent again.

[0096] Next, in step S12, terminal 20 sends correction completion information to server 30 indicating that the correction of the superimposed image 72 has been completed.

[0097] Next, in step S13, when the server 30 receives the correction completion information, the linking processing unit 38 links the superimposed image 72 and the first decoration position information 57A in the last updated state to the identification information 50. Then, the database 31 adds and assigns both the superimposed image 72 and the first decoration position information 57A to the data table and stores them.

[0098] In the order processing described above, an order is received from the user, and the type and state of the decoration 3 (position, scale, rotation angle, etc.) for the object to be decorated 2 is determined.

[0099] Next, the embroidery preparation process will be explained using Figure 12. The embroidery preparation process is a process in which the needle position coordinates and other parameters in the embroidery operation are calculated based on the frame coordinate system before the decoration 3, which was determined in the order processing, is actually applied to the object to be decorated 2.

[0100] The user selects the desired embroidery hoop 40 and attaches the object to be decorated 2, which has a position marker 4 placed (attached) to its surface, as used in the order processing shown in Figure 11, to the embroidery hoop 40. The user attaches the object to be decorated 2 so that the position marker 4 is positioned on the upper side when the embroidery hoop 40 is attached to the sewing machine 10, and so that the position to be decorated 3 and the position marker 4 are located within the embroidery hoop 40 (see Figure 8).

[0101] In step S20 of the embroidery preparation process, terminal 20 photographs the embroidery hoop 40 that holds the object to be decorated 2, on which the position marker 4 is placed on the surface. When "Photograph the object to be decorated" is selected by the user from the decoration application menu screen, the photography unit 22 is activated. The photography unit 22 photographs the embroidery hoop 40 according to the user's operation.

[0102] Next, in step S21, when the user selects "Send frame image" from the decoration application menu screen, the terminal 20 sends the frame image 73 captured by the shooting unit 22 to the server 30.

[0103] Next, in step S22, the server 30 stores the frame image 73 and frame information 60 linked to the identification information 50. Specifically, the linking processing unit 38 recognizes the coordinate determination marker 43 on the received frame image 73 and obtains the frame type ID (frame type information 61). The linking processing unit 38 also recognizes the identification marker 42 on the received frame image 73 and obtains the frame unique ID (frame unique information 62). Furthermore, the linking processing unit 38 links the frame image 73, frame type information 61, and frame unique information 62 to the identification information 50. Then, the database 31 adds and assigns the frame image 73, frame type information 61, and frame unique information 62 to the data table and stores them.

[0104] Next, in step S23, the server 30 calculates the gradient of the embroidery hoop 40 on the frame image 73. The processing in steps S23 and S24 is one aspect of the correction process. Specifically, the frame coordinate setting unit 35 calculates the gradient of the object to be photographed on the frame image 73 with respect to the optical axis (shooting direction) of the shooting unit 22 using the method described above. The object to be photographed in this embodiment is the embroidery hoop 40. However, it is not limited to this.

[0105] Next, in step S24, the server 30 corrects the frame image 73 to the reference orientation based on the gradient of the embroidery hoop 40. Specifically, first, the frame coordinate setting unit 35 sets a frame coordinate system on the frame image 73 with the embroidery hoop 40 as the reference, based on the gradient of the embroidery hoop 40 calculated in step S23, using the method described above. Then, the frame coordinate setting unit 35 uses the method described above to transform the coordinates of the frame image 73 based on the set frame coordinate system and creates an orientation-corrected frame image corrected to the reference orientation orthogonal to the optical axis of the imaging unit 22. Subsequently, the frame coordinate setting unit 35 uses the method described above to correct the orientation (rotation state) of the embroidery hoop 40 in the front-back, left-right, and right directions on the orientation-corrected frame image to the reference orientation, based on the image of the coordinate determination marker 43 on the orientation-corrected frame image. The corrected frame image 73 is referred to as the corrected frame image (see Figures 6 and 7).

[0106] Next, in step S25, the server 30 sets the frame coordinate system based on the coordinate determination marker 43 on the corrected frame image. The frame coordinate setting unit 35 sets the first straight line L1 connecting a pair of vertical markers 43a on the corrected frame image as the vertical axis, sets the second straight line L2 connecting a pair of horizontal markers 43b on the corrected frame image as the horizontal axis, and sets the intersection of the first straight line L1 and the second straight line L2 on the corrected frame image as the origin P (see Figure 6).

[0107] Next, in step S26, the server 30 creates a frame superimposed image 74 by superimposing the image of decoration 3 onto the modified frame image. The image of decoration 3 is superimposed in a state that has been transformed (position, scale, rotation) to a predetermined state.

[0108] More specifically, the frame coordinate setting unit 35 recognizes the size and orientation of the position marker 4 on the corrected frame image and the size and orientation of the position marker 4 captured on the target image 71. The frame coordinate setting unit 35 then updates the first decoration position information 57A (movement amount, scaling amount, rotation amount) stored in the database 31 by converting it based on the size and orientation of the position marker 4 on the corrected frame image and the size and orientation of the position marker 4 on the target image 71. The server 30 then uses the superimposed image creation unit 37 to move the image of the decoration 3 based on the position, scaling, and rotation amount included in the updated first decoration position information 57A and superimposes it onto the corrected frame image. Thus, the superimposed image creation unit 37 creates a frame superimposed image 74 (see Figure 8). Through the above process, the server 30 reproduces the image of the decoration 3 on the superimposed image 72 onto the frame superimposed image 74, using the position marker 4 as a reference. Server 30 sends the created frame-overlay image 74 to terminal 20.

[0109] Next, in step S27, the terminal 20 displays the received frame superimposed image 74 on the display unit 24. By checking the frame superimposed image 74 displayed on the display unit 24 of the terminal 20, the user can confirm the overall image (image after decoration is applied) when the decoration 3 is applied to the object to be decorated 2.

[0110] Next, in step S28, the server 30 associates the frame-overlay image 74 with the identification information 50 and stores it in the database 31. Specifically, the association processing unit 38 associates the frame-overlay image 74 with the identification information 50. Then, the database 31 adds the frame-overlay image 74 to the data table, assigns it, and stores it.

[0111] Next, in step S29, the server 30 calculates the second decoration position information 57B. Specifically, the decoration position calculation unit 36 ​​calculates the second decoration position information 57B in the manner described above, based on the decoration reference data 55C, the first decoration position information 57A which was updated in step S26 and stored in the linking information 53, and the image of the position marker 4 on the frame superimposed image 74. The second decoration position information 57B is calculated by converting the coordinates of the position marker 4 of the first decoration position information 57A to coordinates in a frame coordinate system with the intersection P of the embroidery hoop 40 (i.e., the origin of the frame coordinate system) as the reference position.

[0112] Next, in step S30, the server 30 associates the second decorative position information 57B with the identification information 50 and stores it in the database 31. Specifically, the association processing unit 38 associates the second decorative position information 57B with the identification information 50. Then, the database 31 adds the second decorative position information 57B to the data table, assigns it, and stores it.

[0113] In the embroidery preparation process described above, the needle position coordinates during the embroidery operation, i.e., the second decoration position information 57B, are calculated.

[0114] Next, the embroidery execution process will be explained. The embroidery execution process is a process in which the sewing machine 10, to which the embroidery hoop 40 that held the object to be decorated 2 in the embroidery preparation process was attached, executes embroidery using the second decoration position information 57B calculated in the embroidery preparation process.

[0115] The user attaches the embroidery hoop 40, which holds the object to be decorated 2 during the embroidery preparation process, to the sewing machine 10. It is preferable that the position marker 4 placed (attached) to the object to be decorated 2 be removed, but if the position marker 4 is sufficiently far from the position to be decorated, it is not necessary to remove the position marker 4.

[0116] In step S31 of the embroidery execution process, the sewing machine 10 acquires frame-specific information 62. Specifically, the frame information acquisition unit 14 acquires frame-specific information 62 from the identification tag 44 provided on the embroidery frame 40 using the method described above.

[0117] Next, in step S32, the sewing machine 10 sends decoration data request information, including frame-specific information 62, to the server 30.

[0118] Next, in step S33, the server 30 transmits the decoration type information 55 and decoration position information 57, which are linked to the frame-specific information 62 included in the received decoration data request information, to the sewing machine 10. Specifically, when the server 30 receives the decoration data request information, the extraction unit 39 acquires the frame-specific information 62 included in the decoration data request information and extracts the decoration type information 55 and second decoration position information 57B, which are linked to the frame-specific information 62, from the information stored in the database 31. Then, the server 30 transmits the extracted decoration type information 55 and second decoration position information 57B to the sewing machine 10, which is the source of the decoration data request information.

[0119] Next, in step S34, the sewing machine 10 applies decoration to the object to be decorated 2 by embroidering based on the received decoration type information 55 and decoration position information 57. Specifically, when the sewing machine 10 receives an instruction from the user to apply decoration, the decoration application unit 11 applies decoration to the object to be decorated 2 held in the embroidery hoop 40 by embroidering based on the received decoration type information 55 and second decoration position information 57B. An instruction to apply decoration is, for example, pressing the "Execute Embroidery" button on the decoration application menu screen displayed on the display unit 16, but is not limited to this.

[0120] Once the application of decorations based on the decoration information 54 is complete, the embroidery execution process ends. In the embroidery execution process described above, decoration 3 is applied to the object to be decorated 2 by embroidery.

[0121] Some or all of the above processes may be executed by a decoration application, which is an application related to the decoration application system 1. The decoration application may be in a form that can be installed and used on the sewing machine 10 or terminal 20, or it may be in a form that can be accessed from the sewing machine 10 or terminal 20 using a browser or the like.

[0122] The decoration application system 1 in this embodiment includes a database 31 corresponding to a storage unit and a decoration application unit 11. The database 31 stores decoration type information 55, which is information about the type of decoration 3 to be applied, and decoration location information 57, which is information about the location where the decoration 3 is applied, linked to identification information 50. The decoration application unit 11 applies decoration 3 to the object to be decorated 2. The identification information 50 includes user information 51, which is information about the user, and order information 52, which is information about the order. The decoration application unit 11 applies decoration 3 to the object to be decorated 2 based on the decoration type information 55 and decoration location information 57 linked to the identification information 50.

[0123] Therefore, the decoration type information 55 and decoration position information 57 are centrally managed in a state linked to the identification information 50, which includes user information 51 and order information 52. As a result, even if there are multiple orders, confusion of decoration information 54 can be suppressed.

[0124] In this embodiment, the decoration application unit 11 acquires decoration type information 55 and decoration position information 57 stored in the database 31, and applies decoration to the object to be decorated based on the acquired decoration type information 55 and decoration position information 57.

[0125] Therefore, the decoration type information 55 and decoration position information 57 are centrally managed in the database 31, and the decoration application unit 11 obtains this decoration type information and decoration position information from the database 31 and applies the decoration 3. This makes it easier to suppress confusion of decoration information 54.

[0126] Furthermore, in database 31, various types of information are assigned and stored in data tables as shown in Figure 10. Therefore, it is easy to centrally manage various types of information related to decoration.

[0127] The decoration application system 1 in this embodiment includes at least one sewing machine 10 and a server 30. At least one sewing machine 10 is equipped with a decoration application unit 11. The server 30 is equipped with a database 31 and is connectable to at least one sewing machine 10 via a network 5. Furthermore, when the server 30 receives identification information 50 from any of the at least one sewing machine 10, it transmits decoration type information 55 and decoration position information 57 associated with the identification information 50 to the sewing machine 10 that sent the identification information 50.

[0128] Therefore, the server 30 transmits the decoration type information 55 and decoration position information 57, which are associated with the identification information 50, to the sewing machine 10, the source of the identification information 50. This makes it easier to suppress confusion of the decoration information 54.

[0129] In this embodiment, the database 31 stores frame-specific information 62, which is unique information set for each embroidery frame 40 that holds the object to be decorated 2, in a state where it is further linked to the identification information 50. When the decoration application unit 11 applies decoration 3 to the object to be decorated 2 held in the embroidery frame 40, it applies decoration 3 to the object to be decorated 2 based on decoration type information 55 and decoration position information 57 which are mutually linked with the frame-specific information 62 of the embroidery frame 40.

[0130] Therefore, the frame-specific information 62 of the embroidery hoop 40 that currently holds the object to be embroidered 2 by the sewing machine 10 is mutually linked with the decoration type information 55 and decoration position information 57 of the decoration 3 to be applied to the object to be embroidered 2. As a result, confusion of decoration information 54 when applying decoration 3 is easily suppressed.

[0131] In this embodiment, the embroidery hoop 40 is provided with an identification marker 42 having hoop-specific information 62, and the database 31 stores the identification information 50 with the hoop-specific information 62 obtained from the hoop image 73, which is a captured image including the identification marker 42, linked to it.

[0132] The frame-specific information 62, which can be linked to the identification information 50, is possessed by the identification marker 42 provided on the embroidery hoop 40 on which the object to be decorated 2 is held. Therefore, the decoration system 1 can be used without separately providing an identification tag that can be linked to the identification information 50 on the object to be decorated 2, or separately providing a device to which such an identification tag is attached. In other words, the object that can be used as the object to be decorated 2 is not limited to a predetermined garment or the like that is equipped with an identification tag that can be linked to the identification information 50. Therefore, compared to a system that can be applied to, for example, a garment with a special tag attached, or a garment hanging on a hanger with a special tag attached, the decoration system 1 has a wider range of objects to which the system can be applied.

[0133] The decoration application system 1 in this embodiment comprises a plurality of sewing machines 10 and a server 30. Each sewing machine 10 is an embodiment of a decoration application device and includes a decoration application unit 11. The server 30 includes a database 31 and can connect to the plurality of sewing machines 10 via a network 5. Each sewing machine 10 further includes a frame information acquisition unit 14 capable of acquiring frame-specific information 62. When the server 30 receives information including frame-specific information 62 from any of the plurality of sewing machines 10, it transmits decoration type information 55 and decoration position information 57, which are mutually linked with the frame-specific information 62 via identification information 50, to the sewing machine 10 that transmitted the frame-specific information 62.

[0134] Therefore, the server 30 transmits decoration type information 55 and decoration position information 57, which are mutually linked with the frame-specific information 62, to the sewing machine 10 that has transmitted the frame-specific information 62. As a result, it is possible to suppress the sewing machine 10 from acquiring decoration type information 55 and decoration position information 57 related to the orders of other users, thus making it easier to suppress confusion of decoration information 54.

[0135] Furthermore, a database 31 capable of storing various types of information in a linked state is provided within the server 30. In other words, all types of information are centrally managed by the server 30. Therefore, for example, this configuration improves the efficiency of system maintenance compared to a configuration where each sewing machine 10 has its own database 31. If each sewing machine 10 has its own database 31, maintenance would need to be performed on each sewing machine 10, which could make maintenance complicated. Therefore, by centralizing the management of complex processes and various types of data on the server 30, the complexity of maintenance can be eliminated, and the efficiency of system maintenance can be improved.

[0136] In this embodiment, the order information 52 is created by the server 30 for each order. The order information 52 is then stored together with the user information 51 as identification information 50, linked to the decoration type information 55 and the decoration position information 57.

[0137] Therefore, the creation of order information 52 is done collectively by the server 30. As a result, even if there are multiple orders, confusion between orders is less likely to occur, and consequently, confusion of decorative information 54 is easier to suppress.

[0138] The decoration application system 1 in this embodiment comprises a plurality of terminals 20, a plurality of sewing machines 10, and a server 30. The sewing machine 10 is an embodiment of a decoration application device and comprises a decoration application unit 11. The server 30 comprises a database 31 and can connect to the plurality of terminals 20 and the plurality of sewing machines 10 via a network 5. The database 31 stores identification information 50 received from the terminals 20, with decoration type information 55 and decoration position information 57 linked to them.

[0139] Therefore, information regarding the type and location of decoration 3 specified by the user is centrally managed and linked to each identification information 50 transmitted from each terminal 20 used by the user. This makes it easier to prevent confusion with decoration information 54 related to orders received from different users.

[0140] Furthermore, a database 31 capable of storing various types of information in a linked state is provided within the server 30. In other words, various types of information are centrally managed on the server 30. Therefore, the efficiency of system maintenance can be improved compared to a configuration where, for example, a database 31 is provided on each terminal 20. In particular, it is assumed that the models and operating systems of each terminal 20 used by users are all different. Therefore, if a database 31 is provided on each terminal 20, maintenance can become complicated. By centralizing the main functions of complex processing and various data management on the server 30, the complexity of maintenance can be eliminated, and the efficiency of system maintenance can be improved.

[0141] In this embodiment, the server 30 further includes a decoration position calculation unit 36 ​​that calculates decoration position information 57. The database 31 stores identification information 50 received from a source terminal, which is one of the multiple terminals 20, along with decoration type information 55 received from the source terminal 20 and decoration position information 57 calculated by the decoration position calculation unit 36, in a linked state.

[0142] Therefore, since the calculation process for the decorative position information 57 is performed on the server 30, the maintenance targets related to this process can be consolidated on the server 30. As a result, compared to a configuration in which maintenance related to this process is performed on each terminal 20 or sewing machine 10, maintenance efficiency can be improved by centralizing the main body of the complex processing on the server 30.

[0143] Furthermore, while the sewing machine 10 applies the decoration 3, the server 30 can simultaneously calculate the decoration position information 57 for the next order. In other words, because the entity executing the embroidery execution process and the entity executing the order process are different, the embroidery execution process and the order process can proceed simultaneously. Therefore, compared to a configuration in which, for example, the sewing machine 10 calculates the decoration position information 57, the operational efficiency of the decoration application system 1 can be improved, and the efficiency of applying the decoration 3 can be improved.

[0144] In this embodiment, the server 30 further includes a decoration position calculation unit 36 ​​that calculates decoration position information 57. The database 31 stores the identification information 50 received from the source terminal, which is one of at least one of the terminals 20, in a state in which the decoration type information 55 and the decoration position information 57 calculated by the decoration position calculation unit 36 ​​are linked together.

[0145] Therefore, even if the source that sends the decoration type information 55 to the server 30 is different from the source terminal 20, the server 30 can store the decoration type information 55 and the decoration location information 57 linked to the identification information 50. This makes it easier to prevent the decoration type information 55 and decoration location information 57 from being confused with decoration type information and decoration location information related to other orders.

[0146] Furthermore, even when the linking process for associating the decoration type information 55 with the identification information 50 is performed outside of server 30, the decoration type information 55 and the decoration position information 57 can be stored in server 30 in a state where they are linked to the identification information 50. This makes it easier to prevent the decoration type information 55 and decoration position information 57 from being confused with decoration type information and decoration position information related to other orders.

[0147] In this embodiment, the decoration position calculation unit 36 ​​calculates decoration position information 57 based on the position of the position marker 4 placed on the object to be decorated 2.

[0148] A position marker 4, which can be placed on the object to be decorated 2, is used as the reference for the decoration position. Since the position marker 4 can be easily placed at the desired position by the user, the operation of determining the decoration position is easy.

[0149] In this embodiment, the decoration position calculation unit 36 ​​calculates decoration position information 57 based on the position of the position marker in the target image 71, which is a photographed image of the object to be decorated 2 on which the position marker 4 is placed.

[0150] The decorative position information 57 is calculated by performing image processing on the target image 71 to determine the position of the position marker on the target image 71. Therefore, compared to configurations that calculate the decorative position information 57 using, for example, a laser, this method does not require special processing or equipment.

[0151] In this embodiment, the superimposed image creation unit 37 creates a superimposed image 72 by superimposing the image of decoration 3 onto the target image 71, using the position of the position marker 4 on the target image 71 received from the terminal 20 as a reference. The server 30 then transmits the superimposed image 72 to the terminal 20 that sent the target image 71. The terminal 20 displays the superimposed image 72 received from the server 30.

[0152] Therefore, the server 30 transmits the superimposed image 72 to the terminal 20 that is the source of the target image 71. The terminal 20 then displays the superimposed image 72, which is the image with the decoration 3 superimposed on the target image 71, on the display unit 24. As a result, it is easy for the user to visually confirm the image with the decoration applied. Also, because it is possible for the user to verify it, even if there is confusion in the decoration information 54, it is easy to detect before the decoration 3 is applied.

[0153] In this embodiment, the decoration 3 is applied to the object to be decorated 2 by embroidery. In the case of embroidery, since the needle is passed through the object to be decorated 2, if the decoration 3 is applied while the decoration information 54 is mixed in, it becomes difficult to repair and restore the object to be decorated 2 to its original state afterward. Therefore, applying the decoration application system 1 in this embodiment has the advantage of suppressing the occurrence of such problems.

[0154] The decoration application system 1 in this embodiment includes a linking processing unit 38 that links decoration type information 55 and decoration position information 57 to identification information 50.

[0155] Therefore, by linking the decoration type information 55 and decoration position information 57 to the identification information 50, the linking processing unit 38 makes it easier to prevent the decoration type information 55 and decoration position information 57 from being confused with decoration type information and decoration position information related to other orders.

[0156] The decoration application in this embodiment operates the server 30 and the sewing machine 10 used in the decoration application system 1. The decoration application stores in the server 30's database 31, linked to identification information 50, decoration type information 55, which is information about the type of decoration 3 to be applied; decoration position information 57, which is information about the location where the decoration 3 is applied; and frame-specific information 62, which is unique information set for each embroidery frame 40. The decoration application causes the sewing machine 10 to acquire the frame-specific information 62 and to send the information including the acquired frame-specific information 62 to the server 30. Based on the received information, the decoration application causes the server 30 to send the decoration type information 55 and the decoration position information 57 to the sewing machine 10.

[0157] Therefore, the decoration type information 55 and decoration position information 57 are centrally managed by the server 30 while being linked to the identification information 50. As a result, confusion of decoration information 54 can be suppressed even if there are multiple orders. In addition, the decoration application causes the server 30 to transmit the decoration type information 55 and decoration position information 57, which are mutually linked with the frame-specific information 62, to the sewing machine 10 that is the source of the frame-specific information 62. As a result, confusion of decoration information 54 is easily suppressed.

[0158] In this embodiment, the decoration application further operates the terminal 20 used in the decoration application system 1. The decoration application causes the terminal 20 to acquire identification information 50 and to transmit the acquired identification information 50 to the server.

[0159] Therefore, information regarding the type and location of decoration 3 specified by the user is centrally managed and linked to each identification information 50 transmitted from each terminal 20 used by the user. This makes it easier to prevent confusion with decoration information 54 related to orders received from different users.

[0160] In this embodiment, the decoration application causes the terminal 20 to further transmit decoration type information 55 to the server 30. The decoration application causes the server 30 to calculate decoration position information 57 and stores the decoration type information 55 and decoration position information 57 linked to the identification information 50.

[0161] Therefore, since the calculation process for the decorative position information 57 is performed on the server 30, the maintenance targets related to this process can be consolidated on the server 30. As a result, compared to a configuration in which maintenance related to this process is performed on each terminal 20 or sewing machine 10, maintenance efficiency can be improved by centralizing the main body of the complex processing on the server 30.

[0162] Furthermore, while the sewing machine 10 applies the decoration 3, the server 30 can simultaneously calculate the decoration position information 57 for the next order. In other words, since the decoration application has different entities perform the embroidery execution process and the order processing, the decoration application system 1 can perform the embroidery execution process and the order processing simultaneously. Therefore, compared to a configuration in which, for example, the sewing machine 10 calculates the decoration position information 57, the operational efficiency of the decoration application system 1 can be improved, and the efficiency of applying the decoration 3 can be improved.

[0163] In this embodiment, the decoration application causes the server 30 to acquire decoration type information 55. The server 30 calculates decoration position information 57 and stores the decoration type information 55 and decoration position information 57 linked to the identification information 50.

[0164] Therefore, even if the server 30 obtains decoration type information 55 from a source different from the terminal 20, centralizing the calculation process on the server 30 can improve maintenance efficiency and decoration application efficiency.

[0165] Furthermore, even when the linking process of the decoration type information 55 to the identification information 50 is performed outside of server 30, the decoration type information 55 and the decoration position information 57 can be stored in server 30 in a state linked to the identification information 50. This improves the efficiency of decoration application.

[0166] In this embodiment, the decoration application causes the superimposed image creation unit 37 to create a superimposed image 72 by superimposing the image of decoration 3 onto the target image 71, using the position of the position marker 4 as a reference. The decoration application then causes the server 30 to transmit the superimposed image 72 to the terminal 20. The decoration application also causes the decoration position calculation unit 36 ​​to calculate decoration position information 57 based on the position of the position marker 4 on the target image 71 and the position of the superimposed image of decoration 3. Furthermore, the decoration application causes the display unit 24 of the terminal 20 to display the superimposed image 72 received by the terminal 20.

[0167] Therefore, the server 30 transmits the superimposed image 72 to the terminal 20 that is the source of the target image 71. The terminal 20 then displays the superimposed image 72, which is the image with the decoration 3 superimposed on the target image 71, on the display unit 24. As a result, it is easy for the user to visually confirm the image with the decoration applied. Also, because it is possible for the user to verify it, even if there is confusion in the decoration information 54, it is easy to detect before the decoration 3 is applied.

[0168] The sewing machine 10 in this embodiment includes a frame information acquisition unit 14, a sewing machine transmission unit 17, a sewing machine reception unit 18, and a decoration application unit 11. The frame information acquisition unit 14 acquires frame-specific information 62, which is unique information set for each embroidery hoop 40, from an identification tag 44 provided on the embroidery hoop 40 that holds the object to be decorated 2. The sewing machine transmission unit 17 transmits information including the frame-specific information 62 to the server 30. The sewing machine reception unit 18 receives from the server 30 decoration type information 55, which is information regarding the type of decoration 3 to be applied to the object to be decorated 2, and decoration position information 57, which is information regarding the position on the object to be decorated 2 where the decoration 3 is applied. The decoration application unit 11 applies decoration to the object to be decorated 2 by embroidery based on the decoration type information 55 and the decoration position information 57. The decoration type information 55 and the decoration position information 57 are mutually linked with the frame-specific information 62.

[0169] Therefore, the frame-specific information 62 of the embroidery hoop 40, which currently holds the object to be decorated 2, and the information regarding the type and position of the decoration 3 to be applied to the object to be decorated 2 are mutually linked. The sewing machine 10 then receives the decoration type information 55 and decoration position information 57 linked to the acquired frame-specific information 62, and applies the decoration 3 to the object to be decorated 2 by embroidery based on the received information. As a result, the sewing machine 10 can suppress confusion of decoration information 54 even if there are multiple orders for embroidery.

[0170] In this embodiment, the frame-specific information 62 that can be linked to the identification information 50 is contained in the identification tag 44 provided on the embroidery hoop 40 that holds the object to be decorated 2. Therefore, decoration can be applied by the sewing machine 10 without having to separately provide an identification tag that can be linked to the identification information 50 on the object to be decorated 2, or a separate device to which such an identification tag is attached. In other words, the object that can be used as the object to be decorated 2 is not limited to a predetermined garment or the like that is equipped with an identification tag that can be linked to the identification information 50.

[0171] In this embodiment, the decoration position calculation unit 36 ​​calculates second decoration position information 57B, which is decoration position information based on the frame coordinate system, and the database 31 stores the second decoration position information 57B and frame-specific information 62 linked to each other. Therefore, the decoration position is calculated based on the embroidery frame 40. As a result, the decoration position can be reliably calculated regardless of the shape and material of the object to be decorated 2, the state in which the object to be decorated 2 is held in the embroidery frame 40, etc., and the accuracy of decoration application can be improved.

[0172] In this embodiment, the frame coordinate setting unit 35 sets the frame coordinate system based on the position of the coordinate determination marker 43 on the corrected frame image 73. It is conceivable that the frame image 73 may be tilted with respect to the plane orthogonal to the optical axis of the shooting unit 22 due to camera shake or the like that that occurred when shooting with the shooting unit 22. Even in such a case, the frame coordinate setting unit 35 can correct the tilt based on the gradient of the embroidery frame 40 on the frame image 73 and set the frame coordinate system using the corrected frame image 73. Therefore, even if the frame image 73 is tilted, it is not necessary for the user to take another picture. In other words, since the process related to adding decorations can be continued without acquiring another image, a decrease in efficiency when using the system can be avoided.

[0173] In this embodiment, the frame coordinate setting unit 35 deforms the image of the decoration 3 based on the gradient of the embroidery frame 40 on the frame image 73. Then, the superimposed image creation unit 37 superimposes the deformed image of the decoration 3 onto the frame image 73 at a position based on the position of the position marker 4 on the frame image 73 to create a frame superimposed image 74.

[0174] Therefore, it is conceivable that the frame image 73 may be tilted relative to the plane perpendicular to the optical axis of the shooting unit 22 due to camera shake or other factors that occur during shooting by the shooting unit 22. Even in such cases, the frame coordinate setting unit 35 can deform the decoration 3 based on the gradient of the embroidery hoop 40 on the frame image 73 and superimpose the image of the deformed decoration 3 to create a frame superimposed image 74. The frame superimposed image 74 can then be used to set the frame coordinate system. As a result, even if the frame image 73 is tilted, the frame coordinate system can be set without the user having to take another picture. In other words, since the process related to decoration application can continue without acquiring another image, a decrease in efficiency when using the system can be avoided.

[0175] In this embodiment, the sewing machine 10 is an example of a decoration application device, the terminal 20 and server 30 are examples of a processing device, the database 31 is an example of a storage unit, the embroidery hoop 40 is an example of a holder, the embroidery hoop and the object to be decorated are examples of an object to be photographed, and the frame body 41a is an example of a holder body. Also, the frame information 60 is an example of holder information, the frame-specific information 62 is an example of holder-specific information, the frame information acquisition unit 14 is an example of a holder information acquisition unit, the identification marker 42 is an example of a holder marker, the frame image 73 is an example of a holder image, the frame superimposed image 74 is an example of a holder superimposed image, the frame coordinate system is an example of a holder coordinate system, and the frame coordinate setting unit 35 is an example of a holder coordinate setting unit.

[0176] Furthermore, in this embodiment, the frame coordinate setting unit 35 and / or the superimposed image creation unit 37 correspond to the correction processing unit, and the terminal 20 and / or the server 30 correspond to the correction device. And, in this embodiment, the terminal 20 having the shooting unit 22 corresponds to the shooting device. However, the shooting device is not limited to this, and any device having a shooting function is acceptable.

[0177] Regarding other embodiments, the decoration system 1, decoration application, sewing machine 10, etc. of the embodiment described above can be modified in various ways without departing from the spirit of this disclosure.

[0178] In this embodiment, a configuration in which the decoration 3 is applied to the object to be decorated 2 by embroidery is illustrated, but the means for applying the decoration 3 are not limited to this. For example, the decoration 3 on the object to be decorated 2 may be applied by printing. In this case, the object to be decorated 2 may be, for example, a printable paper product, a synthetic resin product, clothing, etc. Also, in this embodiment, a sewing machine 10 is illustrated as an example of a decoration application device, but the decoration application device is not limited to this, and may be, for example, a printer, etc.

[0179] In this embodiment, the decoration system 1 is exemplified as having a configuration with multiple sewing machines 10 and multiple terminals 20, but the number of sewing machines 10 and terminals 20 may be one each. Also, the sewing machines 10 may be installed in places other than stores T1 and T2 (for example, each user's home, etc.).

[0180] In this embodiment, it has been illustrated that multiple types of embroidery hoops 40 may be used in the decoration system 1, but only one type of embroidery hoop 40 may be used in the decoration system 1.

[0181] In this embodiment, images captured by the shooting unit 22 of terminal 20 are used as the target image 71 and the frame image 73, but this is not the only way to use them. The shooting unit that captures the target image 71 and the frame image 73 may be a device other than terminal 20. Furthermore, the terminal 20 that communicates with the server 30 in the decoration application system 1 and the terminal 20 that captures the target image 71 and the frame image 73 may be different terminals.

[0182] In this embodiment, a configuration in which order information 52 is created by the linking processing unit 38 is illustrated, but the entity responsible for creating order information 52 is not limited to this. For example, order information 52 may be created by the terminal 20 and sent from the terminal 20 to the server 30 at the same time as the terminal 20 sends user information 51 and decoration type information 55 to the server 30. Furthermore, the entities responsible for assigning the order information ID and retrieving the order history may be different. Also, order information 52 may be held by, for example, the position marker 4.

[0183] In this embodiment, an example is given in which the identification information 50 includes user information 51 and order information 52. However, the embodiment is not limited to this, and for example, the identification information 50 may include only one of either the user information 51 or the order information 52. That is, either the user information 51 or the order information 52 may be linked to various types of information and stored in a data table.

[0184] Furthermore, various types of information do not necessarily have to be linked to the identification information 50. That is, it is sufficient if the decoration type information 55, decoration position information 57, frame-specific information 62, etc., are assigned to and stored in a data table so that these pieces of information are linked to each other. In this case, the server 30 does not need to acquire the identification information 50.

[0185] Furthermore, various types of information do not necessarily have to be linked to the identification information 50. That is, it is sufficient if the decoration type information 55, decoration position information 57, frame-specific information 62, etc., are assigned to and stored in a data table so that these pieces of information are linked to each other. In this case, the server 30 does not need to acquire the identification information 50.

[0186] In this embodiment, the position marker 4, identification marker 42, and coordinate determination marker 43 are AR markers as an example, but these markers are not limited to AR markers and other types of markers may be used as long as they can perform the functions required for each marker. For example, the position marker 4, identification marker 42, and coordinate determination marker 43 may be markers called two-dimensional codes. This type of marker has unique information for each marker and can detect the position of the marker on the frame image 73, for example. Alternatively, the position marker 4 and coordinate determination marker 43 may be markers that can detect their position on the image but do not have individual information for each marker, such as markers with distinctive shapes or colors that are highly identifiable on the image. Furthermore, the identification marker 42 may be a marker that has frame-specific information 62 for each embroidery frame, such as a barcode.

[0187] In this embodiment, the position marker 4 is configured to be attached to the surface of the object to be decorated 2, but the position marker 4 is not limited to this configuration. For example, the position marker 4 may be sewn to the object to be decorated 2 for temporary fastening, or it may simply be placed on the object to be decorated 2.

[0188] In this embodiment, an embroidery hoop 40 having four coordinate determination markers 43, including two vertical markers 43a and two horizontal markers 43b, is illustrated, but the number of coordinate determination markers 43 in the embroidery hoop 40 is not limited to four. For example, the embroidery hoop 40 may have three or fewer, or five or more, coordinate determination markers 43. Furthermore, each coordinate determination marker 43 may be provided in a different size or shape from one another, for example, to make it easier to distinguish between the front, back, left, and right directions of the embroidery hoop 40.

[0189] In this embodiment, a configuration in which the identification marker 42 and the coordinate determination marker 43 are different markers is illustrated, but the embodiment is not limited to this, and for example, at least one of the coordinate determination markers 43 may also serve as the identification marker 42.

[0190] In this embodiment, an example configuration is provided in which the identification marker 42 has frame-specific information 62 including a frame-specific ID, and at least one of the coordinate determination markers 43 has frame-type information 61 including a frame-type ID, but the embodiment is not limited to this. For example, the identification marker 42 may include both the frame-specific ID and the frame-type ID, and the coordinate determination marker 43 may not include these IDs.

[0191] In this embodiment, the frame information acquisition unit 14 is an RFID reader and is configured to be able to communicate wirelessly with the identification tag 44, which is an RFID tag. However, the configuration of the frame information acquisition unit 14 is not limited to this. For example, the frame information acquisition unit 14 may be a mechanism capable of reading an AR marker. In this case, by making the AR marker having frame-specific information 62 the identification tag 44, the frame information acquisition unit 14 can communicate wirelessly with the identification tag 44 to acquire the frame-specific information 62.

[0192] Furthermore, the frame information acquisition unit 14 may be incorporated into the CPU 13 as a control mechanism. In this case, the frame information acquisition unit 14 may be configured to acquire frame-specific information 62 based on information input by the user via, for example, the input unit 15. With such a configuration, there is no need to provide an identification tag 44, and the number of parts required in the decoration application system 1 can be reduced.

[0193] In this embodiment, an example is provided in which the sewing machine 10 transmits information including frame-specific information 62 to the server 30, and the server 30 transmits decoration type information 55 and decoration position information 57, which are mutually linked with the frame-specific information 62 via identification information 50, to the sewing machine 10, which was the source of the frame-specific information 62. The information that the sewing machine 10 transmits to the server 30 is not limited to this. For example, the sewing machine 10 may transmit identification information 50 to the server 30, and the server 30 may transmit decoration type information 55 and decoration position information 57, which are linked to the identification information 50, to the sewing machine 10. In this case, the sewing machine 10 may acquire the identification information 50, for example, by inputting the identification information 50 from a user via the input unit 15. However, the method by which the sewing machine 10 acquires the identification information 50 is not limited to this.

[0194] In step S33 of the embroidery execution process, the extraction unit 39 may further extract the frame overlay image 74. The server 30 may then send the frame overlay image 74 to the sewing machine 10, which is the source of the decoration data request information. In this case, the sewing machine 10 may display the received frame overlay image 74 on the display unit 16. With this configuration, the user can easily check the frame overlay image 74 displayed on the display unit 16 and confirm whether or not the desired decoration 3 can be applied to the object to be decorated 2.

[0195] In the above embodiment, the server 30 may transmit the frame overlay image 74 to the terminal 20 based on the identification information 50, and the terminal 20 may display the received frame overlay image 74 on the display unit 24. With this configuration, the user can easily check the frame overlay image 74 displayed on the display unit 24 and confirm whether or not the desired decoration 3 can be applied to the object to be decorated 2.

[0196] In this embodiment, when the server 30 creates a frame superimposed image 74 in which the embroidery hoop 40 is tilted with respect to the optical axis of the imaging unit 22, two methods are illustrated: one is to correct the frame image 73 to a reference orientation before superimposing the image of the decoration 3, and the other is to deform the image of the decoration 3 to be superimposed according to the gradient (blur) of the embroidery hoop 40 on the frame image 73. In this disclosure, either of the illustrated methods can be adopted, and regardless of which method is used, the frame superimposed image 74 can be created without re-shooting and reacquiring the frame image 73.

[0197] Furthermore, in this embodiment, the processing in steps S23 to S24 is disclosed as an example of the modification process, but the modification process is not limited thereto. For example, the modification process may involve deforming the image of the decoration 3 superimposed on the frame image 73 according to the gradient of the embroidery frame 40, which is performed by the superimposed image creation unit 37, and superimposing the deformed image of the decoration 3 based on the position of the position marker 4 on the frame image 73.

[0198] In step S01 of the order processing, the process by which the server 30 obtains decoration type information 55 from the terminal 20 was disclosed. Alternatively, the server 30 may obtain decoration type information 55 from a source different from the terminal 20.

[0199] Furthermore, when the server 30 acquires information, the decoration type information 55 may already have identification information 50 attached to it, and the server 30 may acquire the decoration type information 55 together with the identification information 50.

[0200] In step S02 of the order processing, the linking processing unit 38 links the decoration type information 55 to the identification information 50, and the database 31 stores the identification information 50 and the decoration type information 55 in a mutually linked state. Alternatively, for example, the linking process of linking the decoration type information 55 to the identification information 50 may be performed outside the server 30.

Claims

1. A decoration application system for applying decoration to an object to be decorated, comprising: a decoration position calculation unit that calculates a decoration position, which is a position for applying decoration to the object to be decorated, based on a holder image, which is a captured image of a holder capable of holding the object to be decorated; and a correction processing unit that applies a correction process to the holder image, wherein the decoration position calculation unit calculates the decoration position based on the holder image after the correction process has been applied; and the correction processing unit calculates the gradient of the object to be photographed on the holder image with respect to a plane orthogonal to the optical axis of the photographing unit of the photographing device that photographed the holder image, and applies the correction process to the holder image based on the calculated gradient.

2. A decoration application system according to claim 1, wherein the holder is provided with at least one coordinate determination marker, the correction processing unit sets a holder coordinate system which is a coordinate system based on the holder based on at least one of the coordinate determination markers on the holder image that has been corrected, and the decoration position calculation unit calculates the position of the decoration in the holder coordinate system as the decoration position.

3. A decoration application system according to claim 2, wherein the holder is configured to be attachable to a decoration application device capable of applying decoration to an object to be decorated held by the holder, and at least one coordinate determination marker includes a pair of vertical markers facing each other in the vertical direction, which is the front-to-back direction, and a pair of horizontal markers facing each other in the horizontal direction, which is the left-to-right direction, when the decoration application device to which the holder is attached is viewed from directly above, and the modification processing unit sets a first line connecting the pair of vertical markers on the modified holder image as the vertical axis of the holder coordinate system, sets a second line connecting the pair of horizontal markers on the holder image as the horizontal axis of the holder coordinate system, and sets the intersection of the first line and the second line as the origin of the holder coordinate system.

4. A decoration application system according to claim 3, wherein the decoration position calculation unit calculates the decoration position based on the position of a position marker on the holder image.

5. A decoration system according to any one of claims 1 to 4, wherein the decoration is applied to the object to be decorated by embroidery.

6. A decoration application system according to claim 5, comprising at least one decoration application device and a processing device that can be connected to at least one of the decoration application devices via a network, wherein the processing device comprises the decoration position calculation unit and the correction processing unit.

7. A decoration application system according to any one of claims 1 to 6, wherein the modification process includes a process of modifying the holder image to a orientation perpendicular to the optical axis based on the gradient.

8. A decoration application system according to any one of claims 1 to 7, wherein the modification process includes a process of deforming the image of the decoration to be applied to the holder image based on the gradient, and superimposing the deformed image of the decoration onto the holder image.

9. A decoration application system according to claim 8, wherein the deformed image of the decoration is superimposed with reference to the position of a position marker on the holder image.

10. A holder that can be attached to a decoration application device, comprising: a holder body for holding an object to be decorated; and a plurality of markers provided on the holder body, wherein the plurality of markers include at least one coordinate determination marker used for setting a holder coordinate system which is a coordinate system based on the holder; and at least one identification marker having holder-specific information set for each holder.

11. A holder according to claim 10, wherein at least one of the coordinate determination markers includes a pair of vertical markers facing each other in the front-to-back direction and a pair of horizontal markers facing each other in the left-to-right direction when the decoration device to which the holder is attached is viewed from directly above.

12. A holder according to claim 10 or claim 11, wherein the decoration-applying device is a sewing machine, and the holder is an embroidery hoop that can be attached to the sewing machine.

Images