Sewing machine and method for using same

The sewing machine integrates a data collection system and neural network to improve seam accuracy and user interaction by adapting to materials and user skills, enhancing overall performance.

JP2024543337AInactive Publication Date: 2024-11-21SINGER SOURCING LTD LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024525970
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-11
Filing Date
2022-11-10
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing sewing machines lack advanced control systems that can adapt to various materials and user interactions, leading to suboptimal seam formation and user experience.

Method used

A sewing machine equipped with a data collection system, neural network, and processor that collects data on the machine, environment, materials, and user interactions to control sewing operations, adjust parameters, and provide real-time feedback and guidance.

Benefits of technology

Enhances seam accuracy, user interaction, and overall sewing performance by adapting to different materials and user skills, providing real-time feedback and guidance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543337000001_ABST
    Figure 2024543337000001_ABST
Patent Text Reader

Abstract

11. A method for calibrating one or more optical sensors on a sewing machine, the method comprising: collecting data for one or more features of one or more predefined areas associated with the sewing machine; processing the data through one or more neural networks, wherein the one or more neural networks detect and recognize the one or more features of the one or more predefined areas from the data; calculating one or more accuracy indicators of the one or more features from the data compared to one or more trained features from the one or more neural networks; comparing the value of the one or more accuracy indicators to one or more indicator thresholds; and adjusting one or more parameters of one or more optical sensors based on a comparison between the one or more accuracy indicators and the one or more indicator thresholds.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] [Cross reference to related items] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 278,286, entitled “Sewing Machine and Methods of Using the Same,” filed November 11, 2021 (Attorney Docket No. 31982.04247), which is incorporated by reference in its entirety herein.

[0002] The present invention relates generally to sewing machines, and more particularly to control systems thereof. [Background technology]

[0003] Sewing machines can be used to create seams in single materials and to sew together a variety of materials. A particular sewing machine can be used to create seams in a workpiece having a particular shape, to cut and sew the edges of a workpiece, to attach decorative elements to a workpiece, to cut and hem the edges of a workpiece, to attach decorative stitching and embroidery patterns to a workpiece that is mounted in an embroidery frame, or to cut a workpiece during the sewing operation. Sewing machines can also cut, fold, roll, or otherwise process a workpiece in addition to or separate from the sewing procedure. The workpiece is moved under a needle so that a seam can be created in the fabric. A user configures a sewing machine for a particular application by adjusting various parameters of the machine and by attaching a variety of different tools or accessories to the machine. Summary of the Invention

[0004] SUMMARY OF THE DISCLOSURE Exemplary embodiments of a sewing machine, a control system for a sewing machine, and a method of using a sewing machine are disclosed herein.

[0005] The exemplary sewing machine includes a sewing head attached to an arm suspended above the sewing bed by a post, a needle bar extending from the sewing head toward the sewing bed, a needle held by the needle bar, a motor connected to the needle bar for moving the needle bar in a reciprocating motion to move the needle and sewing thread through a workpiece during a sewing operation, and a user interface for receiving instructions from a user of the sewing machine and providing feedback information to the user. The exemplary sewing machine also includes a data collection device, a data storage device, and a processor. The data collection device is for collecting data regarding at least one of the sewing machine, an environment surrounding the sewing machine, sewing materials, a sewing operation performed by the sewing machine, and one or more interactions of the user with the sewing machine. The data storage device is for storing the data collected by the data collection device as collected data and for storing data regarding the neural network. The neural network is composed of a plurality of nodes. Each node of the neural network has an input connection for receiving input data, a node parameter, a calculation unit for calculating an activation function based on the input data and the node parameter, and an output connection for sending output data. The processor is configured to process the collected data through the neural network to generate processed data, and to control, based on the processed data, at least one of a user interface for interacting with a user, a data storage device for storing the processed data, and a motor for modifying the sewing operation.

[0006] An exemplary method for controlling a sewing machine includes collecting data, storing the collected data in a data storage device, processing the collected data through a neural network, and controlling a user interface, a data storage device, and a motor based on the processed data. The collecting data includes collecting data related to at least one of the sewing machine, an environment surrounding the sewing machine, sewing materials, a sewing operation performed by the sewing machine, and one or more interactions of a user with the sewing machine. The neural network in the processing stage has a plurality of nodes, each node including an input connection for receiving input data, a node parameter, a calculation unit for calculating an activation function based on the input data and the node parameter, and an output connection for sending output data. During the control stage, the processor controls the user interface to interact with the user, controls the data storage device to store the processed data, and / or controls the motor to modify the sewing operation.

[0007] An exemplary control system for a sewing machine includes a data collection device, a data storage device, and a processor. The data collection device is for collecting data regarding at least one of the sewing machine, an environment surrounding the sewing machine, sewing materials, a sewing operation performed by the sewing machine, and one or more interactions of a user with the sewing machine. The data storage device is for storing the data collected by the data collection device as collected data and for storing data regarding the neural network. The neural network is composed of a plurality of nodes. Each node of the neural network has an input connection for receiving input data, a node parameter, a calculation unit for calculating an activation function based on the input data and the node parameter, and an output connection for sending output data. The processor is configured to process the collected data through the neural network to generate processed data, store the processed data in the data storage device, and control at least one of a user interface for interacting with a user and a motor for modifying the sewing operation based on the processed data.

[0008] An exemplary method for calibrating one or more optical sensors on a sewing machine includes collecting data of one or more features of one or more predefined areas associated with the sewing machine; processing the data through one or more neural networks, where the one or more neural networks detect and recognize the one or more features of the one or more predefined areas from the data; calculating one or more accuracy indicators of the one or more features from the data compared to one or more trained features from the one or more neural networks; comparing the values ​​of the one or more accuracy indicators to one or more indicator thresholds; and adjusting one or more parameters of one or more optical sensors based on a comparison between the one or more accuracy indicators and the one or more indicator thresholds.

[0009] An exemplary sewing machine has a sewing head attached to an arm suspended above a sewing bed by a post, a needle bar extending from the sewing head toward the sewing bed, the needle bar holding a needle, a presser bar having a presser foot extending away from the sewing head toward the sewing bed, one or more optical sensors positioned to collect data from one or more features of one or more predefined areas associated with the sewing machine, and one or more processors for processing the data collected by the one or more optical sensors through one or more neural networks. The one or more processors are configured to receive the data from the one or more optical sensors; process the data through the one or more neural networks, where the one or more neural networks detect and recognize the one or more features of the one or more predetermined regions from the data; calculate one or more accuracy indicators of the one or more features from the data compared to trained features from the one or more neural networks; compare the values ​​of the one or more accuracy indicators to one or more indicator thresholds; and adjust one or more parameters of the one or more optical sensors based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds.

[0010] An exemplary sewing machine includes a sewing head attached to an arm suspended above a sewing bed by a post, a needle bar extending from the sewing head toward the sewing bed, the needle bar holding a needle, a presser bar having a presser foot extending away from the sewing head toward the sewing bed, one or more data collection devices associated with the sewing machine and positioned to collect data from one or more features of one or more predefined areas associated with the sewing machine, and one or more processors for processing the data collected by the one or more data collection devices through one or more neural networks. The one or more processors are configured to receive the data from the one or more data collection devices; process the data through the one or more neural networks, where the one or more neural networks detect and recognize the one or more features of the one or more predetermined regions from the data; calculate one or more accuracy indicators of the one or more features from the data compared to trained features from the one or more neural networks; compare the values ​​of the one or more accuracy indicators to one or more indicator thresholds; and adjust one or more parameters of the one or more data collection devices based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds.

[0011] A further understanding of the nature and advantages of the present invention is set forth in the following description and claims, particularly when considered in conjunction with the accompanying drawings in which like parts bear like reference numerals and in which: [Brief description of the drawings]

[0012] In order to further clarify various aspects of the embodiments of the present disclosure, a more particular description of certain embodiments will be made by reference to various aspects of the accompanying drawings. It is understood that these drawings show only typical embodiments of the present disclosure and therefore are not to be considered as limiting the scope of the present disclosure. Also, while the drawings may be drawn to scale for some embodiments, the drawings are not necessarily drawn to scale for all embodiments. The embodiments and other features and advantages of the present disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0013] [Figure 1] 1 shows a diagram of an exemplary sewing machine and its system. [Diagram 2] 1 shows a diagram of an exemplary sewing machine and its system. [Diagram 3] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 4] 1 shows a diagram of an exemplary sewing machine and its system. [Diagram 5] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 6] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 7] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 8] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 9] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 10] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 11] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 12] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 13] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 14] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 15]1 shows a diagram of an exemplary sewing machine and its system. [Figure 16] 1 shows diagrams of an exemplary sewing machine and its system. [Figure 17] 1 shows a diagram of an exemplary sewing machine and its system. [Figure 18] 1 shows diagrams of an exemplary sewing machine and its system.

[0014] [Figure 19] This shows diagrams related to artificial intelligence and neural networks. [Figure 20] This shows diagrams related to artificial intelligence and neural networks. [Figure 21] This shows diagrams related to artificial intelligence and neural networks. [Figure 22] 1 shows a flowchart relating to artificial intelligence and neural networks. [Figure 23] This shows diagrams related to artificial intelligence and neural networks. [Figure 23B] 1 shows a flowchart relating to artificial intelligence and neural networks.

[0015] [Figure 24] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Diagram 25] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Figure 26] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Figure 27] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Figure 28] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Figure 29] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Diagram 30] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Diagram 31] 1 shows a diagram of an exemplary sewing machine stitch adjustment. [Diagram 32]1 shows a diagram of an exemplary sewing machine stitch adjustment. [Diagram 33] 1 shows a diagram of an exemplary sewing machine stitch adjustment.

[0016] [Diagram 34] A diagram of machine vision technology is shown. [Diagram 35] A diagram of machine vision technology is shown. [Diagram 36] A diagram of machine vision technology is shown. [Figure 37] A diagram of machine vision technology is shown. [Figure 38] A diagram of machine vision technology is shown. [Figure 39] A diagram of machine vision technology is shown.

[0017] [Diagram 40] 1 illustrates a diagram of an exemplary sewing projection feature of an exemplary sewing machine. [Diagram 41] 1 illustrates a diagram of an exemplary sewing projection feature of an exemplary sewing machine. [Diagram 42] 1 illustrates a diagram of an exemplary sewing projection feature of an exemplary sewing machine.

[0018] [Diagram 43] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Diagram 44] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Diagram 45] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Diagram 46] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 47] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 48] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 49]1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 50] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 51] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 52] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Diagram 53] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 54] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 55] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 56] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 57] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 58] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 59] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 60] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 61] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 62] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 63] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 64] 1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine. [Figure 65]1 shows a diagram of an example fabric and thread compatibility monitoring feature of an example sewing machine.

[0019] [Figure 66] 1 shows a diagram of an example sewing quality monitoring feature of an example sewing machine. [Figure 67] 1 shows a diagram of an example sewing quality monitoring feature of an example sewing machine. [Figure 68] 1 shows a diagram of an example sewing quality monitoring feature of an example sewing machine. [Figure 69] 1 shows a diagram of an example sewing quality monitoring feature of an example sewing machine. [Figure 70] 1 shows a diagram of an example sewing quality monitoring feature of an example sewing machine.

[0020] [Figure 71] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 72] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 73] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 74] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 75] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 76] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 77] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 78] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 79] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 80] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine. [Figure 81] 1 illustrates a diagram of exemplary object recognition features of an exemplary sewing machine.

[0021] [Figure 82] 1 illustrates a diagram of an exemplary tactile feedback feature of an exemplary sewing machine. [Figure 83] 1 illustrates a diagram of an exemplary tactile feedback feature of an exemplary sewing machine. [Figure 84] 1 illustrates a diagram of an exemplary tactile feedback feature of an exemplary sewing machine.

[0022] [Figure 85] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine. [Fig. 86(A)] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine. [Fig. 86(B)] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine. [Fig. 86(C)] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine. [Figure 87] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine. [Figure 88] 1 illustrates a diagram of exemplary machine diagnostic features for an exemplary sewing machine.

[0023] [Figure 89] 1 illustrates a cross-sectional view of an exemplary suture sensor.

[0024] [Figure 90] 1 illustrates a perspective view of an exemplary sewing machine.

[0025] [Figure 91] FIG. 91 shows a front view of the sewing machine of FIG.

[0026] [Figure 92] FIG. 91 shows a bottom-left-front perspective view of the sewing machine of FIG. 90.

[0027] [Figure 93] 91 shows a bottom-left-rear perspective view of the sewing machine of FIG. 90;

[0028] [Figure 94] 93 shows a detailed view of area 92 in FIG. 92.

[0029] [Figure 95] 93. A detailed view of area 93 in FIG. 93 is shown.

[0030] [Figure 96] 3 illustrates an exemplary process for controlling a sewing machine.

[0031] [Figure 97] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 98] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 99] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 100] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 101] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 102] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 103] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 104] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 105] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 106] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 107] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Figure 108] 3 shows a flow chart detailing the operation of an exemplary sewing machine. [Fig. 109] 3 shows a flow chart detailing the operation of an exemplary sewing machine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] The following description refers to the accompanying drawings which illustrate certain embodiments of the present disclosure. Other embodiments having different structure and operation do not depart from the scope of the present disclosure. Exemplary embodiments of the present disclosure are directed to sewing machines and accessories for use with sewing machines.

[0033] As described herein, when one or more components are described as being connected, joined, fixed, coupled, attached, or otherwise interconnected, such interconnection may be direct between the components or may be indirect, such as through the use of one or more intermediate components. Also, as described herein, references to a "member," "component," or "portion" should not be limited to a single structural member, component, or element, but may include an assembly of components, members, or elements. Also, as described herein, the terms "substantially" and "about" are defined as at least close to (and including) a given value or condition (preferably within 10%, more preferably within 1%, and most preferably within 0.1%).

[0034] 1-18 and 90-95, various views and diagrams of exemplary sewing machines and portions thereof are shown. An exemplary sewing machine, such as sewing machine 100 shown in FIG. 1, includes a sewing bed or base 104 having a post 106 extending upward from one end to support an arm that extends horizontally above the sewing bed. A sewing head 102 is attached to the end of the arm and may include one or more needle bars 108 for moving one or more needles 110 up and down to sew a workpiece on the sewing bed 104 under the sewing head 102. The sewing bed includes a needle plate or sewing plate located below the sewing head having an opening for the needle or needles 110 to pass through when making or forming a seam in the workpiece. In some sewing machines, a bobbin located below the needle plate assists in seam formation and distributes a bobbin thread that is stitched with a top thread that is fed through the workpiece from above by the needle. In other sewing machines, such as an overlock or serger, the bobbin thread is distributed by a looper. A user may interact with the sewing machine 100 via a wide variety of buttons, knobs, switches, and other user interface elements. A touch screen display 112 may also be used to both present a software-based user interface to the user and to receive input from the user. A projector 114 located on the sewing head 102 may be used to project one or more user interface elements onto the sewing bed 104, or a workpiece placed thereon. One or more cameras 116 located on the sewing head 102, or elsewhere around the sewing machine 100, collect information from the workpiece and the environment surrounding the sewing machine 100 that may be used to improve the performance of the sewing machine 100 and the user experience.

[0035] As used herein, "sewing machine" refers to a device that forms one or more stitches in a workpiece with a reciprocating needle and a length of sewing thread. As used herein, "sewing machine" includes, but is not limited to, sewing machines for forming specific stitches (e.g., sewing machines configured to form double stitches, chain stitches, overlock stitches), embroidery machines, quilting machines, overlock or serger machines, and the like. It should be noted that various embodiments of sewing machines and accessories are disclosed herein, and any combination of these options may be made unless specifically excluded. In other words, individual components or parts of the disclosed devices may be combined unless they are mutually exclusive or otherwise physically impossible.

[0036] A "stitch" means a loop formed with one or more sewing threads, at least one of which passes through a hole formed in a workpiece. The machine parts of a sewing machine, such as needles, hooks, loopers, tensioning devices, and feed mechanisms, work together to form a stitch in one or more workpieces. One iteration of this complex machine movement may form a stitch or a pattern of stitches in the workpiece. The "stitch length" of a repeat or pattern refers to the distance the workpiece is moved when the repeat is performed. Stitch length measurements vary for different types of repeats and patterns and may encompass one or more stitches in a workpiece.

[0037] A pressure bar having a presser foot also extends downward from the sewing head to press the workpiece against the sewing bed and against feed dogs that move back to front and optionally side to side to move the workpiece. The feed dogs move the workpiece in coordination with the presser foot and at a speed that may be fixed or variably controlled by the user, for example with a foot pedal. A wide variety of presser feet and other types of accessories, such as a buttonhole foot, may be attached to the pressure bar to assist in forming particular types of stitches or features in the workpiece. An accessory platform may also extend below the sewing head to hold special tools or accessories on or above the sewing bed.

[0038] The speed or frequency at which the needle bar is moved up and down is controlled by the user as described above. The needle bar typically moves up and down in a cyclical motion to create a stitch in the workpiece, but the needle bar can also move side to side simultaneously to create different stitches, such as a zigzag stitch or a tapered stitch, or to change the width of the stitch. The type and pitch of the stitch performed by the machine can be selected by the user through a manual interface including buttons, knobs, levers, or the like, through a user interface presented by a computer on a touch screen, or through a voice controlled interface.

[0039] Different types of sewing machines may include additional components to form seams in or otherwise process the workpiece during the sewing process. For example, in a serger, one type of sewing machine that may be used to form the edges of a workpiece, among other functions, a needle called a looper operates under the sewing bed to feed a bobbin thread to form various stitches. Sergers may also include two, three, or more needles above the needle plate and a knife for cutting the edges of the workpiece. Sewing machines may also be used to create embroidery patterns on a workpiece by including a holder for an embroidery hoop on the sewing bed (e.g., FIG. 6). The embroidery hoop holder may be actuated in at least two axes so that the sewing machine's controller can move the embroidery frame so that the needle can trace the embroidery pattern on the workpiece.

[0040] The sewing thread used during sewing is held in various locations on the sewing machine, such as within a bobbin (FIGS. 13-15) or on a spool held by a spool holder that is part of or extends onto the arm of the sewing machine (FIGS. 10-12). The sewing thread is pulled from a sewing thread source (e.g., a bobbin or spool) through various other elements of the sewing machine that are positioned to redirect the sewing thread so that it is smoothly pulled and fed to the workpiece with minimal damage to the sewing thread (FIGS. 7-9). The tension of the sewing thread may also be changed by various tensioning devices positioned along the sewing thread path or within the sewing thread source. Tensioning and distribution devices ensure that only the desired amount of sewing thread is dispensed and that the sewing thread that forms the stitch on the workpiece is properly tightened. A loose sewing thread may allow the stitch to unravel and a tight sewing thread may cause the stitch to be formed incorrectly. The sewing thread tension for the top and bobbin threads may also be adjusted to ensure that the top and bottom tensions are balanced to properly form a stitch along the desired sewing path on the workpiece.

[0041] 16, a block diagram of a computer-based control system for a sewing machine 100 is shown. The sewing machine includes one or more data collection devices 118, one or more data storage devices 120, a processor 122, a user interface 124, and motors and actuators 126. The sewing machine 100 may also include a network interface that interfaces with the processor 122 and is used to connect the sewing machine 100 to a cloud system and / or other sewing machines or devices via a wireless network. The data collection devices 118 include a wide variety of digital, analog, active, passive sensors, and software components, as described in more detail below. These sensors collect data about the sewing machine itself, the workspace or environment around the sewing machine, the sewing operation being performed by the sewing machine, the user's interaction with the sewing machine, and the sewing material (e.g., fabric and sewing thread) on which the sewing machine is operating. The data storage devices 120 include one or more computer memory chips for storing data collected by the data collection devices 118, and the operating software of the sewing machine 100. The structure, various functions, and parameters of the one or more neural networks 128 are also stored by the data storage device 120. The processor 122 accesses the data stored in the data storage device 120 and executes operating software to provide functionality to the sewing machine 100. A user interface 124 is presented to the user via the touch screen display 112 and via physical controls, such as buttons, levers, dials, lights, speakers, and actuators. Motors and actuators 126 include electromechanical actuators, motors, and general purpose mechanical components that are controlled by the control system to cause movement of the various moving parts of the sewing machine, i.e., the needle bar, feed dogs, bobbin, and looper, etc.For example, the speed of the motor may be controlled directly by input from a foot pedal actuated by a user, or may be controlled via a computer that receives and interprets the foot pedal input before sending a signal to one or more motor controllers that control the sewing machine motors.

[0042] As used herein, a "computer" or a "processor" includes, but is not limited to, any programmed or programmable electronic device or collaborative device capable of storing, retrieving, and processing data, and may be a processing unit or a distributed processing configuration. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), floating point units (FPUs), reduced instruction set computing (RISC) processors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), and the like. A single microprocessor and / or one or more cores of multiple microprocessors, each having one or more cores, may be used to perform the operations described as being performed by a processor herein. A processor may also be a processor dedicated to training neural networks and other artificial intelligence (AI) systems. The processor may be located locally on the sewing machine or provided at a remote location that may be accessed via a network interface.

[0043] As used herein, a "network interface" or "data interface" includes, but is not limited to, any interface or protocol for transmitting and receiving data between electronic devices. A network or data interface may refer to a connection to a computer via a local network or through the Internet, and may also refer to a connection to a portable device, such as a mobile device or a USB thumb drive, via a wired or wireless connection. A network interface may be used to form a network of computers to facilitate distributed and / or remote computing (i.e., cloud-based computing). "Cloud-based computing" refers to computing implemented on a network of computing devices remotely connected to a sewing machine via a network interface.

[0044] As used herein, "logic," which is synonymous with "circuitry," includes, but is not limited to, hardware, firmware, software, and / or combinations of each, for performing one or more functions or actions. For example, based on a desired application or need, logic may include software-controlled processors, discrete logic such as application specific integrated circuits (ASICs), programmed logic devices, or other processors. Logic may also be fully embodied as software. As used herein, "software" includes, but is not limited to, one or more computer-readable and / or executable instructions that cause a processor or other electronic device to perform a function, action, process, and / or behave in a desired manner. Instructions may be embodied in various forms, such as routines, algorithms, modules, or programs, including separate applications or code from a dynamically linked library (DLL). Software may also be implemented in various forms, such as a stand-alone program, a web-based program, a function call, a subroutine, a servlet, an application, an app, an applet (e.g., a Java applet), a plug-in, instructions stored in a memory, part of an operating system, or other types of executable instructions or interpreted instructions from which executable instructions are created.

[0045] As used herein, a "data storage device" refers to a device for non-transitory storage of code or data, e.g., a device having a non-transitory computer readable medium. As used herein, a "non-transitory computer readable medium" refers to any suitable non-transitory computer readable medium for storing code or data, e.g., magnetic media, e.g., fixed disks in external hard drives, fixed disks in internal hard drives, and floppy disks, optical media, e.g., CD disks, DVD disks, and other media, e.g., ROM, PROM, EPROM, EEPROM, Flash PROM, external flash memory drives, etc.

[0046] The user interface of the sewing machine may include a wide variety of input devices and means of communication with the user, such as, for example, buttons, knobs, switches, lights, displays, speakers, touch interfaces, and lights. The user interface for the sewing machine may be graphically presented to the user via one or more displays, including a touch screen display 112 that includes a touch-sensitive overlay for detecting the location of the user's fingers touching the display. The user may thus interact with the user interface by directly touching the screen with a hand 150 at a particular location and by performing touch gestures, such as the touch 152, touch 152 and hold 154, touch 152 and pinch or spread 156, and touch 152 and move 158 gestures shown in FIG. 2. The presence, position, and movement of the user's hands, fingers, or eyes may also be detected via analysis of data from optical sensors (e.g., cameras) or by sound, light, infrared radiation, or electromagnetic field disturbances via proximity sensors (e.g., FIG. 3). The graphical user interface may also be projected by one or more projectors on the sewing machine onto the sewing bed 104, the workpiece, an adjacent surface such as a wall or table, or any other suitable surface. Alternatively, the sewing machine 100 may be operated without a graphical user interface via voice commands and audible feedback in the form of specific sounds and / or computer voices. Haptic feedback may also be provided via actuators that vibrate various parts of the machine, such as the feedback portion 160, upon actuation or in response to a wide variety of conditions of the workpiece or the machine. Audible and tactile interaction with the sewing machine is particularly useful for visually impaired users.

[0047] As seen in FIG. 17, the sewing machine 100 may provide notifications and feedback to the user through visual, audio, and tactile means. For example, an indication that an incorrect accessory is installed on the machine may be presented to the user through a user interface on the sewing machine's display while a notification sound, e.g., a beep or computer voice, is sent to the user through the sewing machine's speaker. Notification may also be sent to the user through haptic or tactile feedback through vibration of the feedback portion 160 of the sewing machine 100 that is touched by the user. That is, the sewing bed 104 may be vibrated by the sewing machine to provide a warning to the user that the machine is not properly configured for the particular sewing task selected by the user. The user feels the vibration under his or her fingers in contact with the workpiece and the sewing bed, which prompts the user to look at the display for further information. The illumination lights of the sewing machine may also be controlled to alert the user, for example, by changing the color of the flash when an incorrect accessory is installed to prompt the user to look at the display for further information.

[0048] A projector 114 is also provided on the sewing head 102 and may be directed downwardly toward the sewing bed 104 and the workpiece, as shown in FIGS. 1, 4, 92 and 94. The projector 114 is positioned to project useful information onto the workpiece to aid the user in using the sewing machine. For example, the projector 114 may project needle penetration points onto the fabric so that the user can see the needle position before making a stitch. Lines or other guides may also be projected onto the workpiece to aid the user in sewing in a straight line or along a desired path. Similar to guides, the projector 114 may project a selected stitch pattern onto the workpiece to represent a planned stitch. The projector 114 may also project an image onto the workpiece to display the selected embroidery pattern on the workpiece so that the user can position the embroidery pattern on the workpiece at a desired location. Information projected by the projector 114 may also include feedback to the user about the status of the machine or a particular sewing operation. For example, the projector 114 may project a warning notice onto the workpiece that the wrong needle has been installed for the type of material being used as the workpiece. The projector 114 may also assist the user by providing visual instructions, such as still or animated images instructing the user how to change the needle, thread the machine, or rotate the workpiece. In other words, the projector 114 may be used by the computer as another means of providing feedback and instructions to the user. It should also be noted that the images projected by the projector 114 may also be detected by an optical sensor such that the sewing machine can respond to the user's interaction with the projected images, such as by touching the projected images of a button or series of buttons.

[0049] A wide variety of data collection devices 118, i.e., digital sensors, analog sensors, active sensors, passive sensors, and software components, may be used by the sewing machine 100 to acquire data regarding the sewing machine itself, the workspace or environment surrounding the sewing machine, the sewing material on which the sewing machine is operating (e.g., the fabric workpiece and the sewing thread used to form the seams), the sewing operations performed by the sewing machine, and the user's interactions with the sewing machine. A non-exhaustive list of types of sensors for the sewing machine 100 includes acoustic, sound, vibration, chemical, biometric, sweat, breath, fatigue detection, gas, smoke, retina, fingerprint, fluid velocity, speed, temperature, light (e.g., camera), light, infrared, ambient brightness, color, RGB color (or another color space sensor such as one using CMYK or grayscale color spaces), touch, tilt, motion, metal detector, magnetic field, humidity, moisture, imaging, photon, pressure, force, density, proximity, ultrasonic, load cell, digital accelerometer, motion, translation, friction, compressibility, voice, microphone, voltage, current, impedance, barometer, gyroscope, Hall effect, magnetometer, GPS, electrical resistance, tension, strain, and many others. The software-based data collection device 118 may include various data logs that are entered as the sewing machine 100 is used. For example, a user activity log may record events involving input from a user via the user interface 124, and a system event log may record software events that occur during normal use of the sewing machine 100, which may be used for machine learning or diagnostic purposes.

[0050] Sensors may be located at a wide variety of locations on the machine and may be used by the sewing machine in a wide variety of ways. For example, the sewing machine may include touch and proximity sensors 170 (e.g., proximity sensor 170 shown in FIG. 3) to provide touch control of a user interface presented on the sewing machine's display. Similar touch or proximity sensors may also be provided at other locations on the sewing machine, such as on the arm or sewing bed. Touch sensors at these other locations may be used in conjunction with the user interface presented to the user, or may be used to monitor the position of the user's hand (or other foreign object, such as the user's hair or a loose sewing pin) on the machine during the sewing process for safety purposes. The sewing machine may also include eye tracking sensors incorporating optical sensors, such as a camera or other detection means for tracking eye position and / or the user's line of sight. One or more optical sensors on the machine may be used to collect data not only about the user of the machine, but also about the workpiece, other sewing materials, such as sewing thread, and the sewing machine itself. Additional examples of the use of sensors and sensor data by sewing are provided throughout this disclosure.

[0051] Many of the sensors used in the sewing machine require calibration after installation to ensure that the data collected by the sensors and provided to the neural network is accurate. Sensor calibration may also be updated in the field periodically or when specified by the user. Sensors may be calibrated in any suitable manner. For example, camera calibration may be performed using the techniques described in U.S. Pat. No. 8,606,390, the entirety of which is incorporated herein by reference. Cameras and other sensors may also be calibrated with techniques that use neural networks to identify characteristics of the sewing machine, for example, when calibrating the cameras.

[0052] The sewing machine's optical sensor or sensors may be located at a wide variety of locations around the sewing machine. As used herein, "optical sensor" means a sensor capable of collecting data from electromagnetic radiation (see FIG. 18), and may include, but is not limited to, sensors for detecting ultraviolet radiation, visible light, infrared radiation, and the like. Particular optical sensors may be tuned to particular wavelengths of electromagnetic radiation, such as, for example, particular wavelengths of laser light. One particular optical sensor that may be used in an exemplary sewing machine is a camera. The camera may include a lens to focus or otherwise redirect light to a sensor that receives the light data and transmits the light data to another device for processing.

[0053] One or more optical sensors may be placed in the sewing machine to observe the workpiece during the sewing process, such as the camera 116 shown in Figs. 1, 4, 6, 93 and 95. The optical sensor or sensors looking at the workpiece may be used to determine the color of the workpiece, the material of the workpiece, the position of the workpiece, the orientation of the workpiece, the magnitude and direction of the movement of the workpiece, etc. The same optical sensors may also be used to detect objects in the sewing area, such as the user's hands, hazards (e.g., hair, the user's clothing, sewing pins, etc.), the type of needle installed in the sewing machine, or the type of presser foot. The optical sensors may also monitor whether the needle, presser foot, sewing plate, or accessories are properly installed and remain properly installed during use. Additional optical sensors or similar sensing devices may be placed on the machine facing the user to provide the sewing machine's computer with information about the user, such as the user's eye position and line of sight, so that the sewing machine can determine which part of the machine the user is looking at. Tracking the user's eyes and current gaze, for example, allows the sewing machine to determine where best to illuminate the sewing bed or present useful information to the user so that important notifications or warnings are not missed.

[0054] The sewing machine may include various security features to restrict access to the sewing machine and to prevent theft of the sewing machine. When the machine is powered on or awakened from a sleep mode, for example, the user may be presented with a prompt requesting that the user prove his or her identity. The user may then enter a predefined code to prove that he or she is an authorized user to access and use the sewing machine. In addition to or instead of the predefined code, the user may provide biometric information as proof of identity, for example, via a fingerprint sensor or facial recognition. A fingerprint sensor may be included on the sewing bed or another location where a user would typically place their hands to use the machine. One or more user-facing cameras allow the sewing machine to identify the user using facial recognition technology for purposes of providing access to the machine.

[0055] A user may also associate another device with their account on the sewing machine and use that device to unlock the sewing machine. For example, an app on a smartphone or tablet may be associated with a user account so that the sewing machine may be unlocked via an app or by holding the smartphone or tablet within a predetermined range of the sewing machine. Any of these means of authenticating a user may be used individually or together to provide two-factor authentication. A phone number capable of receiving text messages may also be associated with the user's account so that a code may be sent for use in two-factor authentication. These other devices or phones may also receive alerts from the sewing machine when other attempts to access the machine have failed, for example, generated after a predetermined number of attempts to access the sewing machine. If the sewing machine is suspected to have been stolen, these other devices may be used to determine the location of the sewing machine via a GPS sensor in the sewing machine or via other means such as a local network detected by the sewing machine. Additionally, alerts resulting from the machine being moved from its normal location or from failed attempts to access the machine may include the location of the sewing machine as determined by the built-in GPS sensor to facilitate recovery of the sewing machine if relevant.

[0056] To process and act on the wide variety of data provided to computers located inside and / or outside the sewing machine via the above sensors, various artificial intelligence ("AI") tools and techniques are used that enable analysis of very large structured or unstructured and changing data sets, deductive or inductive reasoning, complex problem solving, and computer learning based on historical patterns, expert input, and feedback loops (see, for example, FIG. 19). As used herein, "artificial intelligence" refers to a broad range of tools and techniques in the field of computer science that enable computers to learn and improve over time. FIG. 19 shows a non-exhaustive overview of these tools, such as, for example, symbolic artificial intelligence, machine learning, and evolutionary algorithms. As seen in FIG. 19, artificial neural networks can be used for a variety of machine learning applications and may use a variety of learning methods, including but not limited to statistical learning, deep learning, supervised learning, unsupervised learning, and reinforcement learning. Artificial intelligence allows the sewing machine to adapt to situations that are not anticipated or precisely predicted by the software programmer, facilitating sophisticated yet intuitive ways of interacting with the sewing machine to achieve desired results. That is, the AI ​​tools and techniques described herein are used by a computer integrated into or external to the sewing machine to make decisions that support or benefit the user based on data provided to the computer via sensors described herein. Although specific artificial intelligence tools (e.g., neural networks) are described below, other artificial intelligence tools may be used for the same tasks, and the description of one tool or technique should not be seen as limiting the application to only that tool or technique unless otherwise indicated herein.

[0057] Diagrams of neural networks and associated processes are shown in Figures 20-23. As used herein, "neural network" includes, but is not limited to, multiple interconnected software nodes or neurons arranged in multiple layers, such as an input layer, a hidden layer, and an output layer as seen in Figure 20. Figure 20 shows a diagram of a neural network 128 including nodes 130 arranged in various layers. Like neurons in the human brain, each node 130 may have one or more input connections 132 and output connections 138 to create a many-to-many relationship with other nodes 130 in the network. That is, the output of a single node may be connected to the inputs of many different nodes, and a single node may receive as input the outputs of many different nodes.

[0058] Each node 130 of the network is configured to perform calculations on data from other nodes and to calculate output data in conjunction with node parameters that are adjusted during the training process of the neural network (FIG. 21). That is, a node of the network is a computation unit having one or more input connections for receiving input data from a node in a previous layer of the network and one or more output connections for sending output data to a node in a subsequent or next layer of the network. Each node 130 includes a computation unit 136 for calculating the input data received via the input connections, the input parameters associated with each input connection, and the result of an activation function that may incorporate any function parameters 134 to calculate output data that may be further modified by the output parameters. For example, the input data from each input connection may be modified by the associated input parameters of that input connection, e.g., weight parameters, to provide the relevant weight of the input connection. The result of the activation function that may be modified by any function parameters is sent as output data via the output connections to a node in a subsequent layer of the neural network. Any function parameter may be, for example, a threshold, such that the calculated result of the activation function is sent as other node output data only if the combined and weighted input data exceeds a threshold set by the threshold.

[0059] All forms of data available to the sewing machine, such as from sensors, software, data storage devices, and user input via the software, can be processed through the neural network. The information to be processed first encounters an input layer that performs initial processing of the input data and outputs the results to one or more hidden layers to process the output values ​​from the input layer. The information processed through the hidden layers is presented in the output layer as a confidence probability for a given result, such as the location of a detected object in an image and the classification of that object. The software in the sewing machine's computer can receive information from one of the layers of the neural network and take action accordingly to adjust the sewing machine's parameters and / or notify the user based on the results of the neural network processing (FIG. 22).

[0060] During training of the neural network 128, the node parameters (i.e., at least one of the input parameters, function parameters, and output parameters) of each node in the neural network are adjusted via a back-propagation algorithm until the output of the neural network corresponds to the desired output of the set of input data. Referring now to FIG. 21, a process for training a neural network is shown. The neural network begins the training process with node parameters that may be randomized or transferred from an existing neural network. The neural network is then presented with data from a sensor to process. For example, an object may be presented to an optical sensor to provide visual data to the neural network. The data is processed by the neural network and the output is tested such that the node parameters of the various nodes of the neural network may be updated to increase the confidence probability of the detection and classification performed by the neural network. For example, when a presser foot is presented to the optical sensor for identification by the neural network, the neural network presents a confidence probability that the object presented to the optical sensor is located within the coordinates of the image and may be classified as a particular presser foot. During the training process, the node parameters of the neural network's nodes are adjusted to increase the neural network's confidence that a particular answer is correct, so that when presented with particular visual data, the neural network "understands" that a particular answer is the most correct answer, even if that data is not exactly the same as what it "saw" before.

[0061] A neural network is considered "trained" when the decisions made by the network reach a desired level of accuracy. A trained neural network may be characterized by a set of node parameters that were adjusted during the training process. The set of node parameters may be sent to other neural networks with the same node structure, so that those other neural networks process data in the same manner as the originally trained network. Thus, the neural network stored in the data storage device of a particular sewing machine may be updated by downloading new node parameters, as shown in FIG. 23. It should be noted that the node parameters of a neural network, such as input weight parameters and thresholds, tend to occupy significantly less storage space than the image library used for comparison with images or visual data collected by an optical sensor. As a result, the neural network's file and other important files may be updated quickly and efficiently over the network. For example, the structure of the neural network, i.e., the map of connections between the nodes and the activation functions calculated at each node, may also be updated in this way.

[0062] The neural network may also be trained continuously such that the node parameters are periodically updated based on feedback provided from various data sources. For example, the locally or externally stored node parameters of the neural network may be periodically updated based on data collected from sensors that may or may not match the output of the neural network. These adjusted node parameters may also be uploaded to a cloud-based system and shared with other sewing machines such that the neural networks of all sewing machines improve over time. The input data of the neural network may also be shared with a server or cloud-based system to provide further training information for the neural network. A large amount of data from the sewing machines in the field may improve the accuracy of the predictions made by the neural network through training.

[0063] 96, an exemplary process 200 for controlling a sewing machine 100 is shown. The process 200 includes steps 202 of collecting data, 204 of storing the collected data in a data storage device, 206 of processing the collected data through a neural network, and 208 of controlling the sewing machine based on the processed data. The collected data relates to at least one of the sewing machine, the workspace or environment surrounding the sewing machine, sewing materials (e.g., sewing thread and workpiece), sewing operations performed by the sewing machine, and user interactions with the sewing machine (e.g., recorded by a user interface or by other sensors). The neural network includes a number of nodes, each of which includes input and output connections, node parameters, and computational units. Based on the processed data, the user interface may be controlled to interact with the user (e.g., by presenting alerts and / or prompts), the data storage device may be controlled to store the processed data (e.g., as a separate record or by updating neural network parameters), and the adjustable components may be adjusted to change the current state of the sewing machine (e.g., change the motor speed, activate a light, move a needle, or any other action that changes the sewing machine or the sewing operation performed by the sewing machine). As seen in FIG. 22, the data collected by the sewing machine's sensors may be processed locally on the sewing machine or via an external processor in a cloud-based neural network. The locally stored neural network may be pre-trained or may be a continuously updating neural network. The data processed by the neural network, whether locally or remotely, is then used by the sewing machine's software to make decisions that result in machine and / or user interaction.

[0064] Cameras and other data collection devices (e.g., sensors) may also be calibrated with techniques that employ one or more neural networks or other artificial intelligence tools. An exemplary camera calibration method is shown in FIG. 23B. Camera calibration may be performed at any relevant time, such as during machine startup, during machine use, and at any time determined by the user. Camera calibration may be performed automatically without requiring any input from the user. Camera calibration may also be manually initiated by the user. The user may also fine-tune the location of the calibration points.

[0065] To perform camera calibration, the camera collects data from a predefined area or areas on an object or objects associated with the sewing machine. The data may be any relevant data that may be used for camera calibration, such as, for example, visual or image data related to the geometry, color, contrast, or reflectance of the predefined area or areas or parts thereof. The predefined area or areas on the object or objects used for calibration have known characteristics, such as known geometric characteristics (e.g., distance and angle) and / or known color and contrast characteristics (e.g., hue, saturation, and brightness). The color and contrast standards are used to calibrate the camera's image settings (i.e., saturation, white balance, temperature, ...). The geometry standards are used to calibrate the camera, for example, for light collection.

[0066] The camera may collect data associated with any suitable object or objects, such as, for example, one or more of a needle bar, a presser foot, a presser foot articulation, a sewing plate, a needle, a paper or plastic sheet, or other sewing machine features and / or accessories (e.g., fabric, projected images, and any movable objects associated with the sewing machine). Calibration may use a single image or multiple images, including images showing movement of the object in two or three dimensional directions, to collect the data.

[0067] The surface fiducial pattern has a defined geometry and location and may include any surface variation detectable by the camera, which may include, but is not limited to, holes, edges, lines, and shapes engraved, stamped, embossed, debossed, etched, cut, or painted on the sewing machine or sewing accessory. The surface fiducial pattern may or may not be used in conjunction with the area being used for calibration. For example, in some cases, the object or objects being used for calibration (e.g., presser foot) may already have an inherent topology that provides sufficient information, so a surface fiducial pattern may not be necessary. If such an accessory is used, color and contrast fiducials may be obtained simultaneously, if necessary, from another location, for example, on the needle or sewing plate. In this case, multiple fiducials may be used for robustness.

[0068] The data collected by the one or more cameras from the one or more predefined areas is sent to one or more computers on the sewing machine or to another processing unit or units associated with the sewing machine. The one or more computers process the data through one or more predictive algorithms, for example in a hidden layer of one or more trained neural networks, or in another important methodology. The one or more neural networks are trained to detect and recognize one or more objects by known feature or features (e.g., geometry, color, contrast, topology, etc.) of the one or more predefined areas used.

[0069] The predictor algorithm groups object features, including future reference patterns that are selected and assigned for use in camera calibration, and identifies feature intersections, including radial and tangential intersections that either already intersect or are otherwise presumed to intersect. Potential line regions of interest may be pre-trained in the neural network. For example, if lines on a sewing board are used as calibration features, finding these lines and estimating them if necessary can be accomplished relatively quickly, since the entire image does not need to be computed, and only predefined regions of interest are processed.

[0070] The one or more predictor algorithms provide a confidence probability for the accuracy of the known features. In other words, the one or more predictor algorithms compare the known features from the data provided by the one or more cameras with previously trained known features of the one or more neural networks and calculate one or more accuracy prediction ratios that quantify the similarity between the features of the one or more data provided by the one or more cameras and the trained features.

[0071] The threshold level of acceptability can be set to any desired level. If the accuracy prediction rate (i.e., probability) is calculated to be lower than the threshold level of acceptability, fine adjustments to the settings of one or more cameras are made, the camera or cameras collect more data, and the process of running the algorithm again on the new data is repeated. This process can be repeated many times until an acceptable probability is achieved. If an acceptable probability is not achieved, an alert signal or message can be sent to the user requesting that the user ensure that the object or objects to be recognized are entirely within the field of view of the camera or cameras, that the lighting is acceptable, there are no obstructions in between, and that there is nothing that would distort the appearance of the associated camera calibration surface that is unacceptable. If an acceptable probability is still not achieved after the above steps are taken, a service alert message can be sent to the user, the workshop, and the manufacturer (e.g., lens cleaning is required, or other issues that can only be addressed by a maintenance technician).

[0072] If one or more features from the data are determined by the algorithm to be within an acceptable threshold of the accuracy prediction rate, the one or more features are used to estimate parameters for the calibration of the camera lens and image sensor, and adjustments may be made via software to modify the final lens and sensor image quality. For example, adjustments may be made to various parameters associated with one or more cameras, such as, but not limited to, light collection, image format, focal length, skew, distortion, image center, color / intensity, exposure, temperature, and brightness.

[0073] As an example of a calibration procedure, at start-up, a camera may capture an image of the presser foot currently installed on the sewing machine. The image is sent to a neural network that recognizes the currently installed presser foot based on its geometry and color coding. The color (e.g., orange) detected on the presser foot by the neural network is different from the color (e.g., red) recognized on the same presser foot during multiple previous sewing sessions where the image settings and predicted presser foot ID were deemed acceptable. The camera settings may then be modified so that the color from the image data appears red.

[0074] 24-33 and 104-105, various diagrams and charts are shown relating to the use of artificial intelligence in a sewing machine to control the location of stitches made on a fabric during sewing. The location of stitches is typically left to the user during normal sewing operations. That is, the user may be provided with various visual aid guides, such as guides on the needle plate, projected guides, or markings on the workpiece, and it is up to the user to maintain the sewing path in the correct position. However, the visual aids do not control the position of the workpiece, so the final location of any stitch is dependent on the user's skill in holding and advancing the fabric in the proper orientation. However, the sewing machines described herein can use one or more optical sensors and depth perception systems (described in more detail below and which may include optical sensors, projectors, ultrasonic, and thermal vision systems) to find the desired path on the workpiece, and can manipulate the lateral position of the needle bar and workpiece by the feed dog so that the stitches are located along the desired path even if the user moves the workpiece off-path. Feed dogs used to influence the feed direction of the workpiece may include linear translation feed dogs, linear translation feed dogs in combination with circular rotation feed dogs, and multi-piece feed dogs having two or more independently moving parts, such as left and right parts that translate at different distances and / or speeds (similar to tank treads) to rotate the workpiece during feed. The two or more parts of the feed dog may also be positioned at different heights to accommodate sewing together fabrics having different thicknesses.

[0075] In quilting, it is common to need to create a stitch line along a seam already formed between two or more fabrics, as seen in FIG. 24. Sewing along a groove is commonly known as creating a "groove seam" because the two fabrics tend to lift away from the seam, making the seam look like a long, low groove with a relatively deep wedge shape between the two fabrics. Creating a groove seam helps hide the seam in the finished quilt. It is very difficult to maintain consistent placement of the stitch along the groove, since the groove is moving and is a very narrow target. A sewing thread that matches the color of the surrounding quilt pieces to some degree, or even a transparent thread, may be used to try to hide the stitch if the stitch is damaged.

[0076] Referring now to FIG. 25, an optical sensor or the field of view of the optical sensor and the depth perception system of the sewing machine are shown superimposed on the image of FIG. 24. As seen in FIG. 104, the user starts sewing and activates the "groove stitch" feature. As the workpiece is placed on the sewing bed, data is continuously collected from the sensor and processed through a neural network. The collected data includes data collected from a camera pointed toward the sewing area upstream and / or downstream of the needle drop location, data regarding the sewing operation (e.g., stitch type and parameters), sewing thread data (e.g., thread tension), and collected data regarding the sewing material (e.g., feed speed, motion vectors, and workpiece topology). The data is processed through a neural network trained to detect and recognize grooves formed between two or more pieces of fabric. That is, the neural network provides a confidence probability of the location of the groove and details about its appearance. When a user begins sewing and moves a workpiece into position under the needle, for example by depressing a foot pedal, pressing a button, giving a voice command, etc., the sewing machine detects and recognizes the groove and controls the position of the stitch being formed on the workpiece by oscillation of the needle bar and / or lateral feeding of the workpiece so that the stitch is formed in the groove according to the groove. The actuators shown in Figures 26-33 may be used to change the lateral position of the needle bar so that the needle communicates with the groove to position the stitch. As the workpiece is moved, the feed dogs may also be controlled to move the workpiece somewhat laterally during back and forth feeding of the workpiece using the actuators shown in Figures 30-33. That is, the feed dogs are capable of moving in two axes so that the direction of the sewing path may be changed in addition to the feed rate of the workpiece, which is typically controlled by the feed dogs.

[0077] Referring now to FIG. 105, a process similar to the "groove stitch" feature can be used to form stitches at a predetermined offset distance or tolerance from a workpiece feature. As the workpiece is placed on the sewing bed, data is continuously collected from sensors and processed through a neural network. The collected data includes data collected from cameras pointed toward the sewing area upstream and / or downstream of the needle drop location, data related to the sewing operation (e.g., stitch type and parameters), sewing thread data (e.g., thread tension), and collected data related to the sewing material (e.g., feed speed, motion vectors, and workpiece topology). The data is processed through a neural network trained to detect and recognize workpiece features such as, for example, edges, seams, "grooves" between two fabrics, buttonholes, or pockets. That is, the neural network provides confidence probabilities of the location of the feature and details about its appearance. When a user begins sewing and moves a workpiece into position under the needle, for example by depressing a foot pedal, pressing a button, giving a voice command, etc., the sewing machine detects and recognizes the feature and controls the location of the stitch formed on the workpiece by oscillation of the needle bar and / or lateral feeding of the workpiece so that the stitch is formed a predetermined distance from the identified feature. For example, a user can specify a seam tolerance of one-half inch (1.27 centimeters) and begin sewing a workpiece having an edge located within the range of lateral movement of the needle bar. As the workpiece is moved through the sewing machine, the edge of the workpiece is detected and the stitch is formed one-half inch (1.27 centimeters) away from the edge without the user having to precisely follow an edge guide.

[0078] Referring now to Figures 34-39, diagrams of various computer vision systems are shown. In addition to the visual data provided by the optical sensors, the neural network can receive input from a depth perception system to provide a more accurate calculation of the groove location and three-dimensional topography. The depth perception system can use any of the stereoscopic and other computer vision techniques, including but not limited to those shown in Figures 34-39, which use two optical sensors, two optical sensors and a projector, a projector and a single optical sensor, an optical sensor and a laser light source, a thermal vision system, or an ultrasonic vision system to calculate the distance to the workpiece. Referring now to Figure 34, a passive stereo depth perception system is shown that uses two cameras (similar to human stereoscopic vision) to determine the distance to a target object based on a comparison between two images captured by the cameras. The active stereo vision system shown in Figure 35 is similar, but includes a projector to project a graphic onto the target object to improve the measurement of distance by the two cameras. A projector is also used in the structured light vision system of FIG. 36 to project a line or some other visual pattern onto the target object that can be viewed by the camera to determine the distance of the target object from the camera. Another depth perception system is shown in FIG. 37, which calculates the distance to the target object using a camera that measures the time it takes for light from a laser to travel from the laser to the target object and back to the camera. This distance information can also be provided to a neural network that processes the visual data of the workpiece from the optical sensor to calculate the location of the groove while the workpiece is moved under the sewing head to form a seam in the workpiece. That is, the line of points on the workpiece that are located furthest from the sewing head can be identified as the groove in the fabric.

[0079] It should also be noted that the optical sensors and depth perception systems described above may have a wide variety of applications. That is, one or more optical sensors and depth perception configurations may be used to recognize the topology of fabrics and sewing threads in three dimensions to identify the type of fabric material and type of sewing thread already used on the workpiece. The density and type of fabric material may also be determined using an ultrasonic or thermal vision system that may be part of the depth perception system. That is, denser materials respond differently to ultrasonic pulses than lighter materials. A laser, infrared radiation, or some other heat source may be used to heat a portion of the workpiece, which may be detected by a thermal vision system including, for example, an infrared sensor. The thermal conductivity of the fabric may then be measured and compared to values ​​known for different types of fabrics. This feature may be particularly useful in the embroidery process when working on a workpiece with existing stitches. The information provided by these systems may also be used to identify the type of fabric and sewing thread used on the workpiece to automatically adjust the sewing machine to sew that type of material and to recommend to the user specific needles or other accessories that may be installed on the machine for use with that workpiece. Automatic dimming can be performed to allow the user and the sewing machine's sensors to view the workpiece material in a manner that is particularly suitable for sewing (i.e., lower brightness improves visibility of highly reflective fabrics). Additionally, as described in more detail below, the sewing machine can provide recommendations and even warnings to the user based on the identified combination of sewing thread and fabric type. The 3D topology of the workpiece can also be used to determine when to release pretension on the presser foot to more easily climb multiple layers of fabric, such as when sewing a hem.

[0080] Processing the distance information along with the visual data through the neural network further improves the accuracy of the information, since the neural network can be trained to consider both the appearance and shape of the workpiece when determining the location of the groove. The neural network used to process the visual and distance data may be trained elsewhere, and the node parameters and other necessary information sent to the sewing machine via the sewing machine's computer (FIG. 23) and / or a cloud connection with the neural network may be trained during the sewing process. For example, the same or additional optical sensors may be used to observe the stitches formed on the workpiece and identify stitches that have missed the groove. Because the computer knows the sewing pitch, control data for a particular missed stitch may be determined and used to adjust the node parameters of the neural network to reduce the likelihood of a missed stitch. The computer on the sewing machine may work in conjunction with the cloud-based neural network, which may provide additional computing power to process the data provided to the neural network and to train the neural network while the sewing machine is operating, so that the neural network is a continuously learning neural network.

[0081] The techniques implemented to accurately form the above-mentioned "groove stitch" may have broader applications in sewing in a wide variety of contexts to form the "perfect stitch". That is, data from one or more optical sensors and a depth perception system may be processed through a neural network to provide control data to one or more motors and actuators of the sewing machine to accurately and precisely form any type of desired stitch at any particular location on the workpiece. In addition to using visual data from the optical sensors and depth perception data from the depth perception system, the perfect stitch control system may take into account data from a sewing thread tension sensor, a needle position sensor, a needle force sensor, a fabric feed rate sensor, the speed and frequency of needle bar movement, the pressure applied by the presser foot, and the feed rate of the feed dog, etc. Data from these sensors may be processed through a neural network to predict whether an incorrect stitch is likely to be made and may instruct the control system to adjust various parameters accordingly to compensate for various factors likely to be causing the error. Upon processing the data provided by these sensors, the decisions from the neural network can be used by the sewing machine's computer to adjust a wide variety of sewing parameters, such as thread tension, needle position, force, speed, and timing; stitch length and type; motor speed; and fabric feed settings, to actively achieve ideal stitch precision and accuracy. All of these features can be combined to correlate machine performance with the user's skill level; that is, the sewing machine can learn to work with beginner, intermediate, and advanced users, adapting machine speed, presenting corrections and alerts, recommending guides or assistance, and so on.

[0082] As with the groove stitch example above, the sewing machine can also check for errors in stitches that have already been formed; that is, each completed stitch can be actively monitored for quality purposes. If data collected by the sewing machine's sensors indicates that a less than perfect stitch is being formed (e.g., stitches are skipped or misaligned), the output data generated by the neural network can be used to make decisions regarding adjustments that can be made to the sewing machine's parameters. These adjustments can be made and the resulting stitches monitored until the stitches being formed are perfect and error-free. Sensors can also be used to detect thread breaks so that sewing can be stopped and the sewing thread replaced. Thus, the quality of the stitches can be improved over time as the neural network is continually trained. For example, a zigzag stitch can be controlled to maintain a particular width on either side of a seam in the fabric so that a subsequent stitch is formed in the opposite fabric. Alternatively, when making a simple straight stitch, the tension of the upper and lower threads can be controlled to avoid the stitch pulling to one side of the work piece. The optical sensor can also identify a pattern line that is painted, superimposed, drawn, or projected onto the fabric, or that is pre-existing (e.g., woven or printed into the fabric) as part of the pattern on the workpiece, so that the seam is formed along that line or at a fixed offset distance from the line. That is, the optical sensor can be used to detect the edge of the fabric and assist the user in sewing along the edge of the fabric with a fixed seam tolerance. Two or more materials may have edges that the user attempts to align while sewing, and the sewing machine can detect a misaligned workpiece and recommend a correction to the user.

[0083] An example of a process for sewing error detection and adjustment is shown in FIG. 106. While the user sews, data regarding the sewing operation (e.g., stitch type and parameters), sewing thread data (e.g., thread tension), and sewing material (e.g., workpiece feed speed, motion vectors, and topography) are continuously collected from a camera pointed toward the sewing area upstream and / or downstream from the needle drop location. Input data may also be provided from a database of known sewing errors and their causes (e.g., a pinched seam may be caused by an imbalance in the tension of the top and bobbin threads). The data is processed through a neural network trained to identify sewing errors, and the identified errors may be recorded and reported to the user along with contextual information, such as the machine parameters at the time of the error or the movement of the workpiece when the error occurred. The recorded information may be used to update local and remote neural networks to improve error detection and prediction. The sewing machine may also control actuators or motors in response to an identified sewing error to correct the error in the next stitch or prevent the recurrence of a similar error. A non-exhaustive list of sewing errors includes skipped stitches, uneven stitches, misaligned stitches, seam shrinkage, variation in stitch density, broken bobbin, broken looper, broken needle, melted thread, broken needle, stuck needle, needle hitting the throat plate, thread cut by needle, inconsistent thread tension, wavy seams, unthreaded needle, loose needle holder, loose presser foot, misplaced presser foot, stationary needle, stationary work piece, bunched work piece, bunched thread, thread knots, loose stitches, tangled threads, frayed threads, torn threads, variation in work piece feed, bent needle, damaged looper, damaged stitch fingers, misplaced looper, misplaced stitch fingers, and dull fabric knife.

[0084] A continuously trained neural network, i.e. one that is trained and can be adjusted during the sewing process, can end up adjusting numerous parameters of the sewing process in unpredictable ways to compensate for unforeseen problems that would be very difficult or impossible to anticipate and address by the user via traditional control software or by adjusting the sewing machine settings. For example, the sewing machine can adjust the feed speed and stitch pitch in response to the user applying an external force to the sewing machine that would otherwise cause the workpiece to move and become dislodged. Adjustments to the tension of the sewing thread or the presser foot can also be determined by the neural network to be useful. That is, the sewing machine can learn to compensate for and even resist erroneous movements by the user to further ensure that the stitches being made are accurate and precise.

[0085] The sewing machine's projector can be used in conjunction with the artificial intelligence techniques described herein to improve the location of the image projected onto the workpiece. For example, as shown in FIG. 107, a neural network can be used to identify features of the workpiece so that a sewing guide can be projected at the location of the feature or at a predetermined distance from the feature. As the workpiece is placed on the sewing bed, data is continuously collected from the sensor and processed through the neural network. The collected data includes data collected from a camera pointed toward the sewing area upstream and / or downstream of the needle drop location, data regarding the sewing operation (e.g., stitch type and parameters), sewing thread data (e.g., thread tension), and collected data regarding the sewing material (e.g., feed speed, motion vectors, and workpiece topology). The data is processed through the neural network, which is trained to detect and recognize features of the workpiece, such as, for example, edges, seams, "grooves" between two fabrics, buttonholes, or pockets. That is, the neural network provides a confidence probability of the location of the feature and details about its appearance. When a user begins sewing and moves a workpiece into position under the needle, for example by depressing a foot pedal, pressing a button, giving a voice command, etc., the sewing machine detects and recognizes the feature and controls the projector to project a sewing guide, such as a straight line in the feed direction, at the location of the feature or a predetermined offset distance from the identified feature. For example, a user may specify a seam tolerance of one-half inch (1.27 centimeters) and activate a sewing guide that projects a line one-half inch (1.27 centimeters) away from the edge of the workpiece that moves with the workpiece as it is moved by the user so that the user can modify the lateral position of the workpiece to form a seam in a desired location.

[0086] 40-42, various diagrams and charts are shown relating to the use of artificial intelligence in a sewing machine to predict the path of a stitch being formed and project an image of the predicted stitch onto the fabric in front of the needle position to inform and guide the user. As with the stitch adjustment and control functions described above, an optical sensor collects visual data from the fabric workpiece and provides the data to a computer. The computer processes the data through a neural network trained to predict the sewing path based on the visual data regarding the stitches already formed and sewing machine parameters such as needle position and speed, fabric position and speed, feed dog speed, force applied by the presser foot, tension of the upper and lower threads, and speed selected by the user. The neural network processes the data and provides the predicted sewing path to the sewing machine computer, which then projects a series of stitches along the predicted path in front of the needle. The type of stitch selected by the user via the user interface (FIG. 40) is incorporated into the projected image (FIG. 41) so that the user can see the shape of the stitches being formed along the predicted sewing path. As the workpiece is moved, either by the user or by the fabric-translating portion of the sewing machine, the projected path of the stitch also moves so that the path appears at a constant location on the workpiece. (In an embroidery machine, the projected embroidery pattern may move with the workpiece as the embroidery frame moves.) The predicted path may also be adjusted to suggest a path the user can follow to return to a deviated pattern. The projected path of the stitch may also begin at the needle drop point and run in a straight or curved line in the feed direction that does not move as the workpiece is rotated or translated.

[0087] Referring now to FIG. 42, the predicted stitch lines are shown as projected in black, and the formed stitches are shown in blue. The projected stitches appear to be consumed by the actual stitches being formed on the workpiece. The prediction distance can also be set by the user so that only some predicted stitches are shown, or stitch lines that extend to the limits of the projector's range are shown. Embroidery patterns can also be projected in a similar manner, such that the projected stitches disappear once the pattern is formed on the workpiece. As the workpiece held by the embroidery frame is moved, the projected image also moves to track the workpiece, such that the needle follows the projection of the predicted sewn patch.

[0088] Projecting the predicted stitch path along the workpiece in front of the needle has many advantages. In some situations, the user may want to place a smooth curve of the stitch that terminates near or at a certain distance from an existing feature of the workpiece. Or, the user may want to avoid contacting or overlapping an existing feature of the workpiece. In these cases, a predicted sewing path that moves with the workpiece makes it easier to create the desired seam in a single pass. Additional information beyond the predicted sewing path may also be provided. For example, the color of the projected stitch may be changed if the projected path is predicted to encounter or get too close to a workpiece feature, such as a pin, button, another seam, buttonhole, decorative element, or fabric edge, that the user has designated as an object to avoid or that the sewing machine identifies and predicts the user will want to avoid. The projected stitch may also flash in these scenarios and can be combined with other notifications, such as audio or haptic feedback as discussed in this disclosure. Alternatively, the projected path may be automatically changed by the sewing machine to guide the user around an obstacle, with the original path and the new, modified path projected in different colors and / or with motion cues that clearly indicate that the path has been changed, such as flashing or otherwise animated arrows near the path.

[0089] Such attention signals and warnings may also be sent if the user's fingers are moved into the predicted sewing path or into the path of another component of the sewing machine, such as the presser foot or an attached accessory. It is noted that the projector is not limited to projecting only the predicted sewing path, but can also project many other symbols and / or wording near the projected path to notify and alert the user of changes in the path or obstacles to watch out for. For example, the neural network can identify the button on the fabric and provide the computer system with button location and size data so that the computer can instruct the projector to project a button outline around a button on a workpiece, thereby drawing the user's attention to that feature.

[0090] As a last resort, the sewing machine may be stopped completely if the needle is about to hit an obstacle and the user does not respond to disable the warning, for example, via a touch screen interface or via a voice control system, to avoid the obstacle. FIG. 108 shows a flow diagram illustrating the use of a neural network to avoid harm to a user or damage to a sewing machine during a sewing operation. When a user begins sewing on a sewing machine, data is collected from a camera pointed toward the sewing area, and data may also be collected from other sensors, such as one or more microphones listening to the environment to capture voice cues or other expressions made by the user. The collected data is processed through a neural network trained to detect foreign objects that may be harmed by or cause damage to the sewing machine. For example, the neural network may identify a user's finger under the presser foot or in the path of the needle. Voice data may also be useful to determine if the user is engaged in a conversation and is distracted, thereby increasing the probability of an unintended finger or hand position. Once an object is identified, the sewing machine can alert the user and stop sewing operations or lower the presser foot to avoid harm to the user and the sewing machine. The foreign object may not be directly in the sewing path, but may be near the path, which will cause the sewing machine to generate an alert. For example, the sewing machine may audibly alert the user or project a warning onto the workpiece as described above. If no foreign object is detected, the sewing operation proceeds in the normal manner.

[0091] In addition to compensating for deviations from the desired sewing path, data collected by the sewing machine during sewing performed by the user may be analyzed via a neural network to detect the user's level of expertise. For example, frequent deviations from the desired sewing path may indicate that the user is a beginner, while a low number of deviations may indicate that the user is an expert. Instructions and training exercises may then be suggested by the sewing machine to the user for improvement. Feedback may be shared via any single or combined means, including audio, text, video, image projection, and augmented reality configurations, from the sewing machine or a connected device. Adjustments to the settings of the sewing machine may also be recommended to improve sewing for novice sewers and to improve the efficiency of expert sewers. The sewing machine may also suggest new opportunities and challenges for advanced users to help them further improve and expand their skill set.

[0092] An exemplary flow chart for using a neural network to detect sewing problems is shown in FIG. 109. When a user uses the sewing machine in any manner, data is collected from a user-facing sensor such as a camera, the user's real-time interactions with the user interface, a log of historical interactions with the sewing machine, and in some cases, information about the current sewing task. The collected data is processed through a neural network trained to detect the user's skill level, as described above. If the neural network assesses the user's skill level, the sewing machine may proceed to warn the user that the difficulty of the task is beyond the detected skill level, or may provide a helpful tip or prompt, as appropriate. As described above, the sewing machine may also provide recommended training based on the detection of the user's skill level.

[0093] The analysis of the user's level of expertise may also be applied to the interaction between the user and the sewing machine. That is, the sewing machine may detect, via neural network analysis, that the user is struggling to properly use a feature of the sewing machine, and may suggest a tutorial video or instructions, and may provide on-screen prompts to assist the user in knowing which user interface control to interact with next. The user interaction data may include the user-facing camera data described above, and may also include timing information from the user interface that indicates the speed at which the user interacts with the settings of the sewing machine. The timing of the user's interaction with the sewing machine may be one indicator of the user's skill level (i.e., a user who selects menu items in the user interface more quickly is more likely to know the sewing machine well) and may be combined with other data to help the sewing machine identify the user's estimated skill level. As an example, after a feature is activated, the sewing machine may highlight a button and present a pop-over message encouraging the user to take the next step and use the activated feature. Input from the user-facing camera and facial recognition technology provide further input about the user's emotional state when interacting with the sewing machine. That is, when the user appears frustrated or confused, graphical and auditory prompts may be provided, or the sewing machine may best support and coach the user on whatever problem they are trying to solve, without presenting further prompts that may be perceived as irritating and unhelpful.

[0094] Based on data collected from monitoring the use of the sewing machine, the sewing machine can also provide useful recommendations of additional products or accessories. Product advertising can be done through any single or combined means, including audio, text, video, image projection, and augmented reality configurations from the sewing machine or connected device. In recommending products, the sewing machine or external processor collects and monitors data through real-time or retrospective data analysis, specifically, for example, the frequency and preferences of the user's selection of sewing accessories, programs, and machines. For example, the sewing machine can constantly track how much of each type of sewing thread is used to understand typical purchases of sewing thread, and can recommend purchasing more of that sewing thread when supplies are estimated to be low. Another example is when a user uses a particular presser foot for a particular purpose, but a more suitable presser foot exists, and the sewing machine can recommend purchasing the more suitable option if the user has not entered it in the list of sewing accessories currently owned. The list of sewing accessories can be stored in one or both of the sewing machine and the app on the connected device. This data can be sent back to the manufacturer to enable engineering, marketing, and customer service groups to improve the quality of their sewing machine and other product offerings.

[0095] 43-65, there are shown various diagrams and charts relating to the use of artificial intelligence in a sewing machine to identify the thread used in the machine and the textile material of the workpiece, adjust the sewing parameters, and provide the user with information regarding the thread and fabric combination identified by the machine. 43-49, there is shown a portion of a sewing machine illustrating the path that the thread may follow from a spool mounted on the machine (FIGS. 43-45) and from a bobbin mounted below the needle plate (FIGS. 46-49) to the sewing needle.

[0096] The sewing machine may include various sensors along these thread paths to detect the type of thread installed on the machine by the user. These sensors may include, but are not limited to, RGB sensors, light sensors, optical sensors, such as cameras, etc. Specific sensors may also be provided for sources of illumination and magnifiers. For example, an optical thread sensor may be included on the machine arm and behind where the spool is attached, as shown in FIG. 50. An exemplary thread sensor 140 is shown in FIG. 87, including a tube-shaped housing 142 through which the thread 141 passes. The tube-shaped housing 142 blocks ambient light from impinging on the thread 141, so that a light source 144 is provided that illuminates the thread 141 for detection by an optical sensor 146, such as a camera or RGB sensor, used to collect the thread data. The sewing machine may also include sensors for detecting parameters of the thread, and mechanisms for adjusting same. The tube-shaped geometry of the sensor assembly provides a known background for illumination of the thread, thereby increasing the accuracy and precision of the thread information collected by the RGB or other sensor. Sensors provided in the tunnel housing can detect light, sound, or other parameters of the sewing thread to determine the color, density or weight, surface quality, material or fiber type, and overall quality of the sewing thread, i.e., RGB or other sensors can be used to detect the inherent characteristics of the sewing thread as it passes through the sensor housing.

[0097] The data collected by the sewing thread sensor may be transmitted to the sewing machine's computer and compared to a thread information database that contains information on a wide variety of thread types and colors. Thus, the sewing machine may identify the thread and present information to the user that may not be known to the user. If a particular thread can be identified from information about the spool (manually entered by the user or detected by the machine), the detected thread characteristics of the thread may be compared to stored thread characteristics from the thread information database. The sewing machine may then detect threads that differ significantly from the stored thread characteristics, which may indicate a faulty spool of thread, and may present an alert indicating the same to the user. Information about the spool of thread may be collected by optical or other sensors located near the spool pin to which the spool is attached during sewing. Spool information may also be collected from the spool when the user holds the spool in front of an optical or other sensor located in the sewing head or another location, such as, for example, a camera in the sewing head or one or more cameras facing the user. To build a history of thread usage on a sewing machine, the time and date of thread identification may be stored and associated with the projection, stitch type, etc.

[0098] The sewing machine may also include sensors for detecting the current state of the sewing thread as it is being manipulated by the machine and may include mechanisms for regulating it. For example, the sewing machine may include a thread tension sensor (FIGS. 51-54), a thread distribution unit (FIGS. 55-58), and a thread tension unit (FIGS. 59-62). The sensors for detecting the inherent characteristics of the sewing thread and the current state of the sewing thread are positioned to collect data about the quality and condition of the sewing thread as it passes from the spool or bobbin, through a tensioner, around a hook or other element of the sewing machine, and finally through the needle. As described in more detail below, optical sensors may also be used in conjunction with neural networks to detect the type of presser foot and / or needle assembled to the machine.

[0099] The sewing machine also includes optical or other sensors that may be used in conjunction with neural networks to detect the type, color, reflectivity, pattern, weave direction, orientation (i.e., right side and wrong side), and topology of the material or fiber of the fabric used in the workpiece. An exemplary sensor for collecting data regarding the fabric of the workpiece includes a source of radiation (e.g., a visible or infrared light source) provided on the sewing head and directed downward toward the workpiece. A radiation meter, such as an optical light sensor or infrared sensor, is provided above the sewing bed, i.e., below the workpiece. It should be noted that the arrangement of the emitter and receiver may be reversed, i.e., the emitter may be provided on the sewing bed and the receiver may be provided on the sewing head. In that way, the amount or portion of the transmitted radiation (e.g., visible or infrared light) that passes through the workpiece, and consequently, the amount of radiation reflected by the top surface of the workpiece, may be detected and measured. To provide a means for determining the density of the fabric more accurately than other techniques, the ultrasonic emitter and receiver may be arranged in a similar manner, i.e., the emitter may be located on the sewing head and the receiver may be located on the sewing bed. These emitters and detectors, i.e., for light (IR, camera), color (RGB), ultrasonic, etc., can be used individually or together to determine the material or fiber type, density, and reflectivity of the workpiece material. Additional depth perception techniques described herein may also be used to detect the topology of the workpiece.

[0100] The data collected by the fabric sensor can be sent to the sewing machine's computer or any connected external processor and compared to a fabric information database that contains information about a wide variety of fabric types with various colors and patterns. Thus, the sewing machine can identify the fabric of the workpiece and present information to the user that may not be known to the user. If a particular fabric can be identified from information about a batch of fabric (manually entered by the user or detected by the machine), the detected fabric characteristics of the fabric can be compared to stored fabric characteristics from the fabric information database. The sewing machine can then detect fabrics that are significantly different from the stored fabric characteristics, which may indicate a defective fabric, and can present an alert to the user accordingly. The workpiece identification data can be used in combination with the stitch data to train a neural network to associate workpiece characteristics with different stitches. As a result, the sewing machine can alert the user that a stitch is being formed on the wrong side of a workpiece facing in the wrong direction.

[0101] As seen in FIG. 63, the data collected by the various sensors and other devices described above is processed by the sewing machine's computer through a neural network. The neural network is trained to provide recommendations, alerts, and warnings to the user based on the input data. That is, the neural network is trained to recognize compatible and incompatible combinations of thread, fabric, presser foot, and needle type. For example, FIG. 64 shows a fabric and thread table showing whether heavy or light fabrics are compatible with thick or thin threads. If the sewing machine detects that there may be a problem with the thread and fabric combination, the user may be provided with a recommendation on the display accompanied by an audible or tactile notification. If the potential compatibility issue is more serious, the user may be alerted or even warned. In some scenarios, the sewing machine may stop all together and provide a combination of audible, tactile, and visual warnings. In addition to notifying the user of potential compatibility issues, the sewing machine may make adjustments such as thread tension, presser foot pressure, or stitch type and speed to improve sewing performance when using heavy or light threads and / or fabrics. Even if the correct type of thread is selected for a given fabric, the color of the thread may not be aesthetically pleasing given the color and / or pattern of the selected fabric, so the neural network can also be trained to advise the user on the suitability of various threads and fabric colors and patterns, as seen in FIG.

[0102] An exemplary flow diagram for identifying a workpiece and potential problems with the workpiece using a neural network is shown in FIG. 102. When a user begins sewing on a sewing machine, data is collected from a camera pointed toward the sewing area, the sewing operation, an optical sewing thread sensor, a feed speed sensor, a database of known workpiece or fabric materials, and a log of previously identified workpiece materials. The collected data is processed through a neural network trained to detect workpiece compatibility issues, damage, and other sewing thread quality issues. If the neural network identifies a workpiece and that the workpiece is incompatible with the current sewing operation and other sewing materials (e.g., a light sewing thread is more likely to break when used with a thicker or heavier workpiece fabric), the sewing machine will alert the user and can continue sewing if the user chooses to disable or ignore the notification. The user is also alerted if damage or other quality issues with the workpiece are identified. The sewing machine can optionally prevent further sewing if the damage is sufficient and requires user intervention, for example, to replace or repair the workpiece.

[0103] 66-71 and 101, various diagrams and charts are shown relating to the use of artificial intelligence in a sewing machine to identify degradation in thread quality, adjust sewing parameters, and provide the user with information regarding the quality of the thread being used. As noted above, the sewing machine may include various sensors along one or more paths that the thread in the sewing machine follows from the thread source to the sewing head, such as those shown in FIGS. 43-49. These sensors may include, but are not limited to, RGB sensors, light sensors, and optical sensors, such as cameras. These sensors are positioned to collect data about the quality and condition of the thread as it passes from the spool or bobbin, through the tensioner, around the hook or other elements of the sewing machine, and finally through the needle. Additional sensors may be included, such as thermal sensors to monitor the temperature of various components involved with the thread and that may cause damage to the thread.

[0104] 66 and 67, examples of the appearance and characteristics of high and low quality sewing threads are shown. A sewing thread that is considered to be of high quality or good condition has qualities including tight and stable fibers, consistent diameter, consistent color, consistent reflectance, and consistent friction quality. A sewing thread that is considered to be of low quality or poor condition has qualities including loose and frayed fibers, inconsistent diameter, inconsistent color, inconsistent reflectance, inconsistent friction quality, and poorly bonded. For example, an additional light may be provided in or near a sensor, such as the tube-shaped sensor housing 142 described above, to provide a consistent light source when observing the sewing thread so that the sewing thread is not misdiagnosed based on color changes in varying lighting conditions such as daylight, bluish white, horizon light, and incandescent light. As seen in FIG. 68, one or more optical sensors on the sewing machine may also detect the accumulation of debris in areas of the sewing machine where debris is known to accumulate when low quality sewing threads are used.

[0105] Referring now to FIG. 69, a flow diagram is shown for an exemplary situation in which the quality of the sewing thread used by the sewing machine is indicated. In the illustrated scenario, a sensor collects data regarding the condition of the sewing thread being used in the sewing machine. The data is processed through a neural network, previously trained or continually trained, to determine whether the sewing thread exhibits any of the markers of a low-quality sewing thread. When a low-quality sewing thread is detected, the user is notified via notification means disclosed herein, such as, for example, a user interface, a computer-generated voice, an indicator light, or haptic feedback (FIG. 70). The user can then look at the sewing machine's display for further details regarding the quality of the sewing thread or request an audible description thereof. The user can choose to disable the warning or take action, after which the user continues sewing. In a sewing machine with multiple spools of sewing thread, the sewing machine can also track the sewing thread parameters of each spool of sewing thread and, in some cases, notify the user as to which spool contains a low-quality sewing thread.

[0106] Another flow diagram for using a neural network to detect thread problems is shown in FIG. 101. When a user begins sewing on a sewing machine, data is collected from a camera pointed toward the sewing area, the sewing operation, an optical thread sensor, other thread sensors to measure thread tension, feed rate, and allocation, a database of known thread materials, and a log of previously identified thread materials. The collected data is processed through a neural network trained to detect thread compatibility issues, damage, and other thread quality issues. If the neural network identifies a thread and that the thread is incompatible with the current sewing operation (e.g., the thread is likely to break when used in a particular stitch), the sewing machine will alert the user and can continue sewing if the user chooses to disable or ignore the notification. The user is also alerted if damage or other quality issues with the thread are identified. The sewing machine can optionally prevent further sewing if the damage is sufficient and requires user intervention, for example to replace or repair the thread.

[0107] The sewing machine may also include multiple suture quality sensors, such as the one or more sensors 140 provided in the tubular housing 142 described above, along the sewing thread path to determine if the quality of the suture changes along the path. For example, if a decrease in suture quality is observed after a particular feature of the sewing thread path, the sewing machine may recommend changes to the sewing parameters to reduce the possibility that the sewing machine is causing damage to the suture. Monitoring suture quality at multiple locations along the sewing thread path may also provide an opportunity for the sewing machine to recommend inspection of various components that may need to be repaired or replaced, such as guides that may have sharp edges that are causing the suture to fray. Such monitoring may also enable the sewing machine to identify improper threading of the sewing machine based on where the suture appears to deviate from the intended suture path through the sewing machine.

[0108] 71-81, various diagrams and charts are shown regarding the use of artificial intelligence in a sewing machine to recognize and identify objects placed in view of the sewing machine's optical sensors and provide the user with information about the object's characteristics and its relationship to the sewing machine. The optical sensor, e.g., a camera, may be one that is pointed toward the sewing area or may be a front or user-facing sensor that allows the user to hold an object up in front of the user-facing sensor to detect the component. A library or database of identified objects is stored to allow the sewing machine to build an inventory of known objects, such as components of the sewing machine or accessories for use with the sewing machine. For example, the sewing machine may recognize the type of needle attached to the sewing machine and whether the needle is properly attached (FIGS. 71-72), the type of presser foot attached to the sewing machine and whether the presser foot is properly attached (FIG. 73), the type and characteristics of the embroidery frame attached to the sewing machine and whether the work piece is properly attached within the embroidery frame (FIG. 74), and the user's fingers and hands and whether there is a safety risk to the user during the current operation (FIG. 75). With respect to an embroidery hoop, the sewing machine can, for example, recognize whether the clamping mechanism for securing the embroidery hoop is secured, whether the workpiece is lying flat in the hoop, and whether all of the edges of the fabric are outside the hoop. The quality of the components can also be identified, i.e., the sewing machine can also detect whether the components are damaged, rusted, bent, worn, incorrectly threaded (in the case of needles and loopers), or otherwise altered from the acceptable quality standards for the components. In each of these examples, neural networks are used to process visual data collected by one or more optical sensors of the sewing machine.

[0109] Referring now to FIG. 76, data collected by the various sensors of the sewing machine is processed through a neural network by the sewing machine's computer to determine if a particular object has been detected by the sewing machine and if the object should be there. For example, as seen in FIG. 77, optical data may be captured encompassing an area including the presser foot of the sewing machine. Visual data of the image is processed through a neural network to determine if a presser foot is present, what type of presser foot is present, and if the presser foot is properly installed. A similar determination may be made for the needle installed on the sewing machine. Once the presser foot and needle are identified, the corresponding needle translation range of the presser foot is stored and the user is notified if the needle and presser foot combination is not recommended. The user may then choose to disable the warning, such as by selecting an "expert mode" that includes an alert about possible safety risks associated with selecting "expert mode". The selected stitch may also be compared to the installed presser foot and needle to determine if the installed presser foot and needle are appropriate and compatible for the selected stitch or series of stitches in the projection. If the presser foot or needle is not installed, a presser foot and needle may be recommended by the sewing machine. Upon installation of the presser foot and / or needle, the sewing machine may again inspect the presser foot and needle to ensure that the proper presser foot and / or needle have been installed and that the needle and / or presser foot are properly installed. The sewing machine may also identify incompatibilities, for example, between the needle and the sewing plate, between the presser foot and the selected stitch pattern, and between the needle and the selected stitch pattern. For example, incompatibilities between stitch types and presser feet may be provided in a table or database of incompatibilities or may be learned over time by monitoring sewing errors in relation to the identity of the various components and the sewing operation being performed.

[0110] Another flow diagram for using a neural network to detect objects and identify compatibility and installation issues is shown in FIG. 100. When a user begins sewing on a sewing machine, data is collected from a camera pointed toward the sewing area, the sewing operation, a database of known components and accessories previously used with the sewing machine, and a database of known components and accessories that are compatible with the sewing machine. The collected data is processed through a neural network trained to detect components and accessories, classify those components and accessories, determine whether the components and accessories are installed correctly, and determine whether there are any mismatches or other problems resulting from the combination of components and accessories with the selected sewing operation. If the neural network identifies that a component is not compatible with the sewing operation or may cause a problem, the user is alerted and given the opportunity to disable the alert (e.g., as in the "expert mode" described above). The neural network identifies whether the components and accessories are installed correctly. If not, the user is alerted and the sewing machine may be prevented from operating until the component is removed or installed correctly.

[0111] A similar determination may be made regarding the embroidery hoop that may be mounted on the sewing bed. Once the type and size of the embroidery hoop has been determined, the sewing machine may notify the user if the selected embroidery pattern extends beyond the limits of the embroidery frame. The sewing machine may also inspect the edges of the fabric held within the embroidery frame to detect improper mounting of the fabric within the hoop. If a problem with the fabric mounting or embroidery hoop size is detected, the user may be notified via any of the notification means described herein, such as a visual display of information on the sewing machine's display, an audible notification, or tactile feedback.

[0112] During identification and inspection of the embroidery frame, or if specified by the user, a camera pointed toward the sewing bed can be used to capture an image of the workpiece mounted in the embroidery frame. The entire workpiece can be captured in a single image, or the embroidery frame can be moved to capture multiple images of the workpiece that are sewn together to form a single image of the entire workpiece. Data collected during the scanning process can be used as input to a neural network trained to recognize and predict color. This pre-trained color calibration facilitates more accurate color predictions over time as the neural network learns to correctly recognize colors. The scan data can also be used as input for a neural network trained to detect translational skips or other movement anomalies so that the embroidery frame's motion system can be controlled to correct for the anomalies.

[0113] Other accessories may also be identified if attached to the sewing machine, and the sewing machine may provide feedback as to whether the accessory is properly loaded and whether the machine is configured to operate properly with that accessory. For example, if a user attaches an accessory to the machine that is used to attach ribbon to a workpiece, the sewing machine may display information about the accessory on the screen to assist the user in the proper use of the accessory. The functionality of the sewing machine may also be limited to those that are compatible with the accessory, unless such limitations are overridden by the user. The sewing machine may also display information on the screen regarding materials that may be used with the accessory, and may recommend other accessories to the user.

[0114] 78-81, various views and diagrams of exemplary presser feet, sewing needles, and other components are shown that include features designed to make the presser feet and needles more easily recognized via object recognition techniques, such as through the use of neural networks, or by sensors, including magnetic sensors, configured to detect the features of the components. The presser feet, sewing needles, needle plates, or other sewing machine components may include various markers that improve the robustness of optical or other sensor-based object recognition systems. For example, the markers or markings may include patterns of two or more geometric shapes (FIGS. 78-81), stripes of color in specific locations (FIG. 80), the overall shape of the component including recognizable protrusions or recesses, filled color codes and other color treatments, reflective finishes, bar codes, QR codes, and other surface treatments that enable UV, IR, or other optical sensing techniques. The markers may include unique patterns of etched rings or lines, etched, debossed, or embossed shapes or patterns in the surface of the sewing machine components. The marker may also be configured with different areas of the surface of the sewing machine component having surface finishes of varying reflectivity, i.e., the marker may include a first area having a first surface finish and a second area having a second surface finish. The marker may use electronic identification technologies such as Near Field Communication (NFC) devices and Radio Frequency Identification (RFID) devices.

[0115] Additional alternative identification information may be based on markers with magnetic field line profiles or polarity profiles for each component that can be detected by a sensor when the component is attached to the sewing machine. For example, a needle may include a magnet to create a specific magnetic field that is only detected when the needle is inserted into the needle bar. Similar techniques are applied to embroidery frames to improve recognition of such frames via neural networks or other object recognition techniques.

[0116] 82-84, various diagrams and charts relating to providing haptic feedback to a user in connection with use of a sewing machine are shown. Haptic feedback is feedback provided to a user through a means that can be felt. For example, a control part with which the user is interacting (e.g., a knob, button, pedal, lever, or slider, e.g., the components shown in FIG. 83) may vibrate slightly when a particular position is reached, and even there may be resistance to further movement of the controlled object. Micro-vibrations may also be provided through a surface of the sewing machine, such as the sewing bed, on which the user's hands and fingers may rest during use. Haptic feedback may be used to alert the user to a particular condition of the sewing machine or workpiece, or may be used as further reinforcement to the user that an action taken by the user has been received. For example, the sewing bed may vibrate under the workpiece and the user's hand when the user deviates from the desired sewing path. Alternatively, a knob or button may vibrate to indicate that a button has been pressed or that the knob has reached a particular position. Haptic feedback may also be replaced with mechanical features that provide similar feedback, such as a detent on a knob that indicates that a particular position around the knob has been reached. Haptic feedback can be used for any sewing machine surface. Vibrotactile feedback via piezoelectric sensors can be used for any surface of the sewing machine and can be used to replace the mechanical user interface. Piezoelectric and capacitive sensors formed to replace the traditional plastic sewing machine cover can be arranged in an array under an OLED or similar type screen. The presence of a user's finger on or near the OLED interface engages menus that are activated based on the user request or current sewing action and associated user interface needs, such as finger taps, slides, and scroll movements for threading, adjusting thread tension, and activating or deactivating sewing accessories. Other forms of haptic feedback can include force, electrotactile, ultrasonic, air vortex ring, and thermal haptic feedback.

[0117] Referring now to FIG. 83, a flow diagram for an example situation in which haptic feedback may be used is shown. In the illustrated scenario, a user attaches a presser foot to the machine that is recognized through neural network processing of visual data received from the machine's optical or other sensors. The user then selects a particular pattern to perform or stitch to sew. The sewing machine then determines whether the combination of the particular presser foot and the selected operation is a valid combination, i.e., whether the attached presser foot can be used with the particular selected stitch, and provides haptic and other feedback to the user if the combination is not valid. This haptic feedback may be provided at the location of the last action, for example, via a touch screen when the user selects the stitch or other operation to perform. At the same time, a visual and audible alert may be provided to the user that the presser foot and the selected operation are not compatible, prompting the user to install the correct presser foot or select a compatible operation (FIG. 84). The interface may also provide an option for the user to disable the warning.

[0118] Referring now to Figs. 85-88, various diagrams and charts are shown regarding the use of artificial intelligence in a sewing machine to monitor the mechanical and electrical health of the sewing machine. Operating a sewing machine produces a wide variety of sounds and mechanical vibrations, as well as variations in the electrical signals that drive the motors and actuators of the sewing machine. An exemplary sewing machine includes sensors to monitor the sounds and noises, mechanical vibrations, and electrical signals to identify patterns related to the performance of the associated components. Sensors may also be provided on or near various components to measure component temperatures, an increase of which may indicate excessive wear. The sensors are located at a wide variety of locations on the sewing machine, as seen in Fig. 85. The sensors may be continuously active or may be turned on to collect data during specific times, such as during start-up procedures, idle conditions, active conditions, and shutdown procedures.

[0119] The collected data may be processed through neural networks trained to detect performance issues in the particular sewing machine component in question. For example, a particular sound may be associated with two components rubbing together, which in turn indicates that a bushing or strut needs to be replaced. Or the voltage required to run a motor at a particular speed may be higher when the motor performance is degraded, as compared to a motor running at nominal conditions. Motor performance may be monitored to determine when a problem occurs, for example, when performing a particular task or working with a particular fabric or sewing thread material. These conditions may also be recognized via an increase in heat generated by the machine's components and a corresponding increase in the temperature of the particular component. More importantly, the sensors used by the sewing machine may be significantly more sensitive to changes in sounds or other parameters generated by the sewing machine components, and thus may be able to make predictions earlier than would otherwise be possible, such as those made by a trained maintenance technician. Furthermore, these performance issues may be correlated with other information from the sewing machine, such as, for example, the sewing operation being performed at the time the performance issue was detected and identified. In this way, specific performance problems can be associated with specific uses of the sewing machine, and information about that relationship can be provided to engineers and maintenance technicians to better identify causes for repairs and improve future designs. As with other data collected by the sewing machines and generated by the neural network, the data can be transmitted to the cloud for sharing with other sewing machines to improve the training of the neural networks of all sewing machines in the network.

[0120] 86(A)-86(C), a flow chart is presented illustrating various ways in which diagnostic information may be generated by the sewing machine and used by a user. When the neural network of the sewing machine's computer identifies that a correction to the machine is needed, an incident is logged and the user is notified. The user may then be instructed to perform a specific task to correct the problem, such as removing the sewing thread or moving the sewing machine to a firmer table. After taking action, the sewing machine may be used normally while running diagnostics at regular intervals to see if additional corrective measures are needed. If the user does not take corrective action, the motor or other actuator may be calibrated to attempt to correct the problem. If calibration does not correct the problem, the user is notified, an incident is logged, and a service request may be sent to the service provider. Calibration may also be set to run every certain number of cycles of the motor or other components as preventative maintenance. Calibration may also be run when changing the sewing thread used and the fabric being worked on, or for any task performed by the sewing machine.

[0121] Once a potential problem has been diagnosed, the sewing machine can notify the user of the problem by a variety of means, such as those described in this disclosure. In particular, the sewing machine can present an alert to the user via a user interface, audibly alert the user, speak to the user via a computer voice, and / or send an email to the user via a network connection. For example, as shown in FIG. 87, the sewing machine can present an indication to the user that service is needed and prompt the user to schedule a service request with a service dealer. Alternatively, as shown in FIG. 88, the sewing machine can suggest modifying the operating environment to improve the performance of the sewing machine, as described in more detail below. If a change in the operating environment or a repair action is deemed necessary or recommended, the user can be alerted and guided to modify the user's work environment by using instructional illustrations, animations, and videos on the touch screen, or localized guiding lighting or other 2D or 3D static or dynamic light projections, placing the sewing machine on a firmer table to reduce vibration, or guiding the user through a simple repair or through a technician through a complex repair of the machine.

[0122] In the case of a software problem, the update may be installed automatically, so that the user may not be aware of the update. Alternatively, the user may be guided through the software update process and may contact a customer service representative through a user interface to provide support and fixes for the software problem. Now referring to FIG. 99, an exemplary flow chart for using a neural network with automatic software updates, as described above, is shown. If the sewing machine has not been used for a predetermined period of time, i.e., the user has been inactive, data may be collected from the user's interaction or activity log, a camera and microphone facing the user, a network interface connected to a software update server, and a clock providing the current date and time. If a software update is available, the neural network provides an indication as to whether the user will typically be away from the sewing machine for a long enough time to install the software update before returning to the machine. If sufficient time is available, the software update is allowed to be installed if the user has enabled automatic updates. A similar process may be followed for the calibration of various sensors, motors, and actuators.

[0123] The sewing machine may also include light sources, such as LED lights, located near various components known to wear out during use, and a particular component may be illuminated, for example, with a yellow, orange, or red light, to indicate that the component is underperforming and may need to be serviced or replaced. These lights may be activated when the machine is placed in a maintenance or service mode and may provide a quick status of the overall health of the machine.

[0124] Information related to the health of the sewing machine may be stored in a health log and, if necessary, transmitted to a remote customer service representative or service technician to assist the remote worker in determining what maintenance may need to be performed on the machine and whether the sewing machine needs to be sent to a service center for repair. Sewing machine health data may also be automatically sent, with permission from the user, to a dealer, service center, and / or manufacturer so that the recipient of the data can take proactive steps to order replacement parts and notify customers that certain components of the sewing machine may soon require replacement. In a commercial environment, a sewing machine owner may choose to subscribe to a maintenance plan in which such replacement parts are provided or service calls are automatically scheduled so that the sewing machine maintains a certain number of uptimes.

[0125] The historical data recorded in the health log can be particularly useful when diagnosing the cause of a sewing machine failure. For example, historical temperature data can include both ambient temperature readings and temperature readings at various points throughout the machine. Ambient temperature history can reveal that the sewing machine was exposed to excessive heat that damaged the sewing machine. Point temperature readings, i.e., temperature readings at specific locations within the sewing machine, can aid the technician in determining the root cause of the sewing machine's damage, such as wear between damaged components. Similarly, historical vibration or acceleration data can be used. Acceleration data can also indicate whether the machine experienced a drop or fall that caused the damage.

[0126] As described above, optical sensors can be used in conjunction with neural networks to detect when a user's finger or any other foreign object is in the way of the sewing head and may cause injury to the user or damage to the machine. Similarly, the neural network can be trained to recognize whether a user's finger or other foreign object is in the way of the presser foot, cutting accessory, or any other moving component of the sewing machine that may cause harm to the user while using the machine. When a finger or other foreign object is detected, the sewing machine can control the needle and other components to avoid the object, or can prevent further sewing if avoidance is not possible or the potential for harm is great enough to warrant prohibition of further operation of the sewing machine. For example, the sewing machine can prevent lowering the presser foot when a finger is detected under the presser foot. Alternatively, the sewing machine can prevent further sewing if a finger or a user's hand is detected in the sewing path. If the detected foreign object is a pin inserted into the seam, the sewing machine can adjust the feed speed or other sewing parameters to avoid the needle striking the pin.

[0127] The neural network can also take into account the orientation of the sewing machine (via an accelerometer and / or pressure sensor on the base) so that if the sewing machine is tilted, tilted enough to tip over and potentially injure the user, the sewing machine can be turned off or prevented from starting. The accelerometer can also be active when the sewing machine is in sleep or standby mode to detect machine movement and prevent the machine from powering up if it is moved or lifted or tipped over. Thermal data from a temperature sensor can be fed into the neural network so that the machine automatically turns off to prevent overheating of components, or because heat buildup can be a symptom of an electrical anomaly.

[0128] A user-facing proximity sensor (e.g., infrared sensor) and / or camera can be used to monitor the presence of a user of the sewing machine so that the sewing machine can be automatically turned off to conserve energy after the user has been absent for a predefined period of time. These user-facing sensors can also prevent activation of the sewing machine after determining, via a neural network or other means, that an unauthorized person is attempting to access the machine. For example, a neural network can be trained to recognize a child attempting to access the sewing machine. In response, the computer can prevent activation of the sewing machine and notify an authorized user of the attempted access by generating an audible sound or by sending a notification to the user via an internet connection, text message, or smartphone app. An exemplary flow diagram of a child safety feature is shown in FIG. 98. Child safety analysis can be triggered for a wide variety of reasons, such as after a failed attempt to access the sewing machine or when the user may wish to step away from the machine during a long embroidery stitch. Data is then collected from the user-facing camera, microphone, and various user interface elements, such as touch screens, buttons, and knobs. If the neural network determines that a child is attempting to access the sewing machine or approaching an operating component of the sewing machine, the sewing machine may issue an audible alert and send an alert to a mobile device assigned to an authorized user. If the child does not respond to the alert, the sewing machine may repeat the alert and stop the sewing operation to prevent harm. If the neural network determines that the failed attempt was not made by a child, the sewing machine may still issue an audible alert and send a message to an authorized user. As another example, the sewing machine may periodically monitor the environment around the sewing machine to identify the presence of a person, such as a user or a child. This periodic monitoring may be performed, for example, when a longer embroidery stitch is in progress and there is a possibility of disturbance of the embroidery workpiece or harm to a person if a child gets too close to the sewing operation in progress.If a child is detected, the sewing operation may be stopped and an alert may be sent to the user to inform them that the sewing operation has been stopped and why.

[0129] When the sewing machine is used, the profile, user preferences, graphical user interface settings, feedback settings, object recognition preferences, tutorial preferences, etc. set by the user are monitored and stored. Beyond the machine settings, all interactions between the user and the sewing machine can be recorded and stored. This collection of data related to the interactions between the user and the sewing machine is processed through a neural network so that the sewing machine can learn how the user prefers to interact with the sewing machine and anticipate what the user may prefer in new situations. That is, setting changes can be related to projections, stitch types, thread types, material types, etc. detected by the sewing machine or provided by the user. This collection of data allows the sewing machine to assist the user, for example, by suggesting feed speed settings for a stitch that the user has not sewn based on the characteristics of the new stitch and the feed speeds the user has set for other stitch patterns. As another example, the sewing machine can remind the user about settings that are usually set, taking into account the current context, i.e., by suggesting a certain feed speed or stitch pitch for thinner materials and a different feed speed or stitch pitch for thicker materials. An exemplary workflow for recommending setting changes using a neural network is shown in FIG. 97. When a setting is changed, data is collected from the user interaction log, the current real-time interactions with the user, from other sensors and neural networks regarding the sewing material, and from data related to the current sewing operation. If the neural network identifies that the user typically makes the same change in a similar context, the sewing machine prompts the user to determine whether the default setting should be changed. If a change in the setting is not typically made in a similar context, the sewing machine can prompt the user to confirm that the change was intended. The neural network can also identify other settings that may typically be changed in a similar context and suggest those other changes to the user.

[0130] The sewing machine may also suggest from time to time that the user take a break during use of the machine or perform exercises to improve the user's ergonomic health. The timing of suggestions and the types of exercises and suggested rest times are based on an analysis of the machine's use by a neural network trained to monitor the user's health. The user's posture may also be detected via neural network analysis of data from one or more user-facing cameras so that suggested exercises may be further customized for the user's benefit.

[0131] Conditions of the user's workspace may also be detected by the sewing machine and analyzed by the neural network. An ambient light sensor may enable the neural network to take into account the lighting conditions in the sewist's room and the lighting of the workspace to reduce or soften the contrast between the work area and the room. For example, the sewing machine may suggest brightening the room light to reduce eye strain caused by the contrast between the bright work surface of the sewing machine and the dark room. The sewing machine may also connect to the workspace and room lighting system, for example, through a Wi-Fi network, to automatically manage brightness adjustments. A camera facing the user may be used to determine the height of the work surface, the position of the user's chair, and other environmental conditions. For example, if the machine has access to an active control surface, such as a workbench with a controllable height, the sewing machine may make suggestions and adjustments to improve the ergonomics of the work environment. FIG. 103 shows an exemplary flow diagram demonstrating how the sewing machine may reduce strain on the user by monitoring the environment around the sewing machine. During use of the sewing machine, data collected from the accelerometer, photo sensor, user-facing camera, and history logs from previous sessions can be processed through a neural network to identify health issues in the user. For example, the neural network can identify when the user tends to sit in poor posture and can recommend changes such as adjusting the user's chair. The neural network can also identify when the work surface is unstable by monitoring vibration and acceleration data and recommend adjustments to the work surface so that it is flat and less prone to moving during use of the sewing machine.

[0132] As mentioned above, data collected by the various sensors on the sewing machine and data generated by monitoring how the sewing machine is being used can be stored in a database on the machine and transmitted to a remote server. The data transmitted to the various remote servers can be collected in a central database and used to analyze the performance of the sewing machine and the sewing behavior of the user across a much larger data set. So-called "big data" analysis can reveal patterns that are not otherwise detectable in smaller data sets. The results of this analysis can be fed back to the neural network of the sewing machine, or to a remote neural network operating to support the operation of the sewing machine, thereby improving the quality of the results determined by the neural network. Big data analysis can also help research and development teams improve quality control processes in factories and testing of various components performed in laboratory environments. For example, failure modes that could not have been predicted during the initial development of the machine can be identified via big data analysis, and future parts and processes can be modified in response.

[0133] Although various inventive aspects, concepts, and features of the present disclosure may be described and illustrated herein as embodied in combination in exemplary embodiments, these various aspects, concepts, and features may be used in many alternative embodiments, either individually or in various combinations and subcombinations thereof. Unless expressly excluded herein, all such combinations and subcombinations are intended to be within the scope of the present application. Still further, although various alternative embodiments of various aspects, concepts, and features of the present disclosure may be described herein, such as alternative materials, structures, configurations, methods, devices, and components, form, fit, and function alternatives, such as alternatives, such descriptions are not intended to be a complete or exhaustive list of available alternative embodiments, whether currently known or later developed. Those skilled in the art may readily employ one or more of the inventive aspects, concepts, or features into additional embodiments and applications within the scope of the present application, even if such embodiments are not expressly disclosed herein.

[0134] Moreover, although some features, concepts, or aspects of the present disclosure may be described herein as being preferred configurations or methods, such description is not intended to imply that such features are required or necessary, unless expressly indicated as such. Still further, while example or representative values ​​and ranges may be included to aid in understanding the present application, such values ​​and ranges should not be construed in a limiting sense, and are intended to be critical values ​​or ranges only when expressly indicated as such.

[0135] Also, although various aspects, features, and concepts may be expressly identified herein as being inventive or forming part of the present disclosure, such identification is not intended to be exclusive; rather, there may be aspects, concepts, and features of the invention that are fully described herein without being expressly identified as such or as part of a particular disclosure, which disclosure is instead set forth in the appended claims. The description of an exemplary method or process is not limited to including all steps as required in all cases, nor should the order in which the steps are presented be construed as required or necessary unless expressly indicated as such. The words used in the claims have their full ordinary meaning and are not limited in any way by the description of the embodiments in the specification.

Claims

1. 1. A method for calibrating one or more optical sensors on a sewing machine, the method comprising: collecting data for one or more characteristics of one or more predefined areas associated with the sewing machine; processing the data through one or more neural networks, wherein the one or more neural networks detect and recognize the one or more features of the one or more predetermined regions from the data; calculating one or more accuracy indicators of the one or more features from the data compared to one or more trained features from the one or more neural networks; comparing the value of the one or more accuracy indicators to one or more indicator threshold values; and adjusting one or more parameters of one or more optical sensors based on a comparison between the one or more accuracy indicators and the one or more indicator thresholds. A method comprising:

2. 2. The method of claim 1, wherein if the value of the one or more accuracy indicators is less than the one or more indicator thresholds, the method further comprises collecting additional data for the features, processing the additional data through the one or more neural networks, calculating additional accuracy indicators, and comparing the value of the additional accuracy indicator to the one or more indicator thresholds.

3. 2. The method of claim 1, wherein if the value of the one or more accuracy indicators is greater than the one or more indicator thresholds, the method further comprises setting the one or more parameters of the one or more optical sensors to an adjusted state as calibrated parameters of the one or more optical sensors based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds.

4. 10. The method of claim 1, wherein the method is performed automatically at one or more of start-up, while the sewing machine is in use, and during any user-defined time while the sewing machine is powered on.

5. The method of claim 1 , wherein one or more of the predefined regions are located on a component or accessory associated with the sewing machine, either attached or detached.

6. 2. The method of claim 1, wherein the one or more predefined areas are located on one or more of a needle bar, a presser foot, a presser foot articulation, a sewing plate, a needle, a paper or plastic sheet, and a fabric.

7. The method of claim 1 , wherein the data is visual or image data relating to at least one of geometry, color, contrast, or reflectance of the one or more predefined regions.

8. The method of claim 7 , wherein the data is collected from a plurality of images.

9. The method of claim 1 , wherein the one or more accuracy indicators comprise a confidence probability for accuracy of the one or more features from the data.

10. 10. The method of claim 1, further comprising transmitting an alert signal or message requesting a user to ensure that the entirety of the one or more predetermined areas is within the field of view of the one or more optical sensors.

11. A sewing machine comprising: a sewing head attached to an arm suspended above the sewing bed by a post; a needle bar extending from said sewing head toward said sewing bed, said needle bar carrying a needle; a presser bar having a presser foot extending away from said sewing head toward said sewing bed; one or more optical sensors positioned to collect data from one or more characteristics of one or more predefined areas associated with the sewing machine; and one or more processors for processing the data collected by the one or more optical sensors through one or more neural networks, wherein the one or more processors: receiving the data from the one or more optical sensors; processing the data through the one or more neural networks, wherein the one or more neural networks detect and recognize the one or more features of the one or more predetermined regions from the data; calculating one or more accuracy indicators of the one or more features from the data compared to trained features from the one or more neural networks; comparing the value of the one or more accuracy indicators to one or more indicator threshold values; and adjusting one or more parameters of the one or more optical sensors based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds. It is configured to A sewing machine equipped with

12. 12. The sewing machine of claim 11, wherein if the value of the one or more accuracy indicators is less than the one or more indicator thresholds, the one or more processors are further configured to collect additional data for the one or more features, process the additional data through the one or more neural networks, calculate additional accuracy indicators, and compare the value of the additional accuracy indicators to the one or more indicator thresholds.

13. 12. The sewing machine of claim 11, wherein if the values ​​of the one or more accuracy indicators are greater than the one or more indicator thresholds, the one or more processors are further configured to set the one or more parameters of the one or more optical sensors to an adjusted state as calibrated parameters of the one or more optical sensors based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds.

14. The sewing machine of claim 11 , wherein one or more of the predefined regions are located on an attached or detached component or accessory associated with the sewing machine.

15. 12. The sewing machine of claim 11, wherein the one or more predefined areas are located on at least one of a needle bar, a presser foot, a presser foot articulation, a sewing plate, a needle, a paper or plastic sheet, or a fabric.

16. The sewing machine of claim 11 , wherein the data is visual or image data relating to at least one of geometry, color, contrast, or reflectance of the one or more predefined areas.

17. The sewing machine of claim 16 , wherein the data is collected from a plurality of images.

18. The sewing machine of claim 11 , wherein the one or more accuracy indicators include one or more confidence probabilities for accuracy of the one or more features from the data.

19. 12. The sewing machine of claim 11, wherein if the value of the one or more accuracy indicators is less than the one or more indicator threshold values, the one or more processors are further configured to send an alert signal or message requesting a user to ensure that the entirety of the one or more predetermined areas is within the field of view of the one or more optical sensors.

20. 12. The sewing machine of claim 11, wherein the one or more neural networks are associated with the sewing machine and configured to share the data with one or more additional neural networks associated with one or more different sewing machines or one or more parent neural networks to train the additional neural networks associated with the one or more different sewing machines or the one or more parent neural networks.

21. A sewing machine comprising: a sewing head attached to an arm suspended above the sewing bed by a post; a needle bar extending from said sewing head toward said sewing bed, said needle bar carrying a needle; a presser bar having a presser foot extending away from said sewing head toward said sewing bed; one or more data collection devices associated with the sewing machine and positioned to collect data from one or more characteristics of one or more predefined areas associated with the sewing machine; and one or more processors for processing the data collected by the one or more data collection devices through one or more neural networks, wherein the one or more processors: receiving the data from the one or more data collection devices; processing the data through the one or more neural networks, wherein the one or more neural networks detect and recognize the one or more features of the one or more predetermined regions from the data; calculating one or more accuracy indicators of the one or more features from the data compared to trained features from the one or more neural networks; comparing the value of the one or more accuracy indicators to one or more indicator threshold values; and adjusting one or more parameters of the one or more data collection devices based on the comparison between the one or more accuracy indicators and the one or more indicator thresholds. It is configured to A sewing machine equipped with