A control method, sewing machine, medium and product

Abstract

The application discloses a control method, a sewing machine, a medium and a product. It relates to the technical field of sewing machines, and the method comprises the following steps: acquiring continuous image frames, wherein the continuous image frames comprise a hand; determining a target control instruction according to the continuous image frames and a current state of the sewing machine; and executing the target control instruction. According to the technical scheme of the application, the sewing quality, efficiency and safety can be improved.

Description

Technical Field

[0001] This invention relates to the field of sewing machine technology, and more particularly to a control method, a sewing machine, a medium, and a product. Background Technology

[0002] Currently, both industrial and household sewing machines generally rely on foot pedals, physical buttons, knobs, or touchscreens to achieve core control functions such as start / stop, speed adjustment, needle position positioning, thread tension adjustment, and thread trimming. In actual production scenarios, sewing operators need to continuously use both hands to support and guide the fabric, while frequently looking down to confirm the needle position and thread path. If it is necessary to adjust equipment parameters, one hand must be freed from the fabric operation interface, which leads to the following problems:

[0003] Manual interaction interrupts the sewing process, disrupts production continuity, and significantly reduces work efficiency;

[0004] Briefly removing your hand from the fabric can cause instability in the feeding guide, which can easily lead to quality defects such as misaligned stitches, uneven stitch spacing, and fabric wrinkling.

[0005] Frequent hand movements between the fabric and the control interface increase the safety risks of accidental needle contact and needle pricks. Summary of the Invention

[0006] The present invention provides a control method, a sewing machine, a medium, and a product that can solve at least one of the above problems.

[0007] According to one aspect of the present invention, a control method is provided for a sewing machine, the sewing machine including an image acquisition module for acquiring continuous image frames, the control method comprising:

[0008] Acquire consecutive image frames, wherein the consecutive image frames include a hand;

[0009] The target control command is determined based on the continuous image frames and the current state of the sewing machine;

[0010] Execute the target control command.

[0011] According to another aspect of the present invention, a sewing machine is provided, the sewing machine comprising:

[0012] At least one processor; and

[0013] A memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the control method described in any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the control method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer program product is provided, which, when executed by a processor, implements the control method as described in any of the embodiments of the present invention.

[0017] This invention addresses several issues: first, acquiring continuous image frames including those of the hand; then, determining a target control command based on the continuous image frames and the current state of the sewing machine; finally, executing the target control command. This solves the problems of low work efficiency caused by manual interaction interrupting the sewing process, disrupting production continuity, and causing instability in material feeding due to brief hand separation from the fabric, leading to quality defects such as thread deviation, uneven stitch spacing, and fabric wrinkling. It also addresses the safety risks of accidental needle contact and needle pricks due to frequent hand switching between the fabric and the operating interface. By controlling the sewing machine through changes in hand posture, this invention improves sewing quality, efficiency, and safety.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a control method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] Example 1

[0026] Figure 1 This is a flowchart illustrating a control method provided in an embodiment of the present invention. This embodiment is applicable to the control of a sewing machine. The method can be executed by a control device in this embodiment, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0027] S110, acquire consecutive image frames.

[0028] The continuous image frames include a hand.

[0029] In this embodiment, a continuous image frame is acquired using an image acquisition module. The image acquisition module can acquire images of the operating area to obtain continuous image frames including the hand.

[0030] In this embodiment, the image acquisition module can be a camera, such as any one of a monocular RGB camera, an RGB-D depth camera, a binocular camera, and an infrared camera. The image acquisition module can be installed below the front of the machine head facing the needle plate, or it can be installed above the worktable to view the needle area from above, so that the field of view covers the needle tip and a preset area around the presser foot (for example, the preset area around the presser foot can be a rectangular area centered on the needle tip). The needle area is the spatial range centered on the needle tip, including the presser foot, needle plate, and the working area of ​​the feed teeth.

[0031] In this embodiment, the image acquisition module can be pre-calibrated with intrinsic parameters to obtain the focal length, principal point, and distortion coefficients; extrinsic parameters can be acquired to establish a mapping between image coordinates and working plane coordinates; for top-view installation scenarios, a homography matrix can be used to map the needle plate plane into an approximate front view, thereby simplifying the calculation of safety boundaries and trajectory deviations. The working plane is the plane containing the upper surface of the needle plate.

[0032] Optionally, the sewing machine further includes: a sewing head, a needle bar, and a needle bar cover, wherein the needle bar is mounted on the front end of the sewing head, and the needle bar cover is disposed on the outside of the needle bar.

[0033] The image acquisition module includes at least one camera.

[0034] The camera is installed at the front end of the machine head or on the bracket above the workbench. The front end of the machine head includes any one of the following: the side of the front end of the machine head, the lower edge of the front end of the machine head, the front of the needle bar cover, the side of the needle bar cover, and the lower edge of the needle bar cover.

[0035] Optionally, the clamp between the optical axis of the camera and the needle plate plane is related to the installation position of the camera. If the camera is installed at the front end of the machine head, the clamp between the optical axis of the camera and the needle plate plane is the first clamp. If the camera is installed on the support above the workbench, the clamp between the optical axis of the camera and the needle plate plane is the second clamp, and the second clamp is larger than the first clamp.

[0036] In this embodiment, the image acquisition module can be a camera, and the installation method of the image acquisition module can be as follows: the camera is installed at an angle below the front of the machine head, and the installation point of the camera is located at the lower edge of the front end of the machine head or near the needle bar cover. The angle θ between the optical axis of the camera and the plane of the needle plate can be 20° to 60°; the vertical height H from the camera to the needle tip projection point O can be 80 to 250 mm; the field of view (FOV) can be 70° to 120°, so that the shooting area of ​​the camera covers the needle tip, presser foot, and the main activity area of ​​the operator's hand.

[0037] In this embodiment, the image acquisition module can also be installed as follows: the camera is mounted above the workbench in a top-down view, with the camera located on a bracket above the workbench, and the optical axis of the camera approximately perpendicular to the needle plate plane (θ close to 90°). This installation method facilitates the measurement of planar distances and trajectory deviations; it is suitable for scenarios with high requirements for measuring suture trajectory deviations.

[0038] In this embodiment, a multi-camera combination installation can also be adopted. The sewing machine is equipped with at least two cameras: one overhead view for trajectory / edge detection and one tilt view for gesture / key point recognition. Multi-camera time synchronization can be achieved through hardware triggering or software timestamp alignment. Example imaging parameter range: resolution ≥ 1280×720, frame rate ≥ 30 fps (optional 60~120 fps to reduce latency and improve hand tracking stability during high-speed sewing); supports automatic exposure and setting the maximum exposure time to avoid blurring during high-speed motion. In workstations with unstable lighting, the illumination unit can be configured as visible light supplementary lighting or infrared supplementary lighting (e.g., 850nm / 940nm), and white balance / brightness normalization can be performed on the image processing.

[0039] Optionally, the sewing machine further includes a presser foot and a needle, the needle being connected to the lower end of the needle bar, the presser foot being located below the needle bar cover and to the side of the needle, and the camera being mounted on the side of the machine head, in front of the presser foot and the needle, with the camera lens facing the needle area at an angle downwards.

[0040] S120, determine the target control command based on the continuous image frames and the current state of the sewing machine.

[0041] In this embodiment, the method for determining the target control command based on the continuous image frames and the current state of the sewing machine can be as follows: Recognize the continuous image frames containing the hand to obtain the target gesture type and the corresponding confidence level; determine the target control command based on the target gesture type, the corresponding confidence level, and the current state of the sewing machine. Alternatively, the method can be as follows: Recognize the continuous image frames containing the hand and the needle tip to obtain the target gesture type, the corresponding confidence level, and a first distance, where the first distance is the distance between the hand and the needle tip; determine the target control command based on the target gesture type, the corresponding confidence level, the current state of the sewing machine, and the first distance. The method for determining the target control command based on the continuous image frames and the current state of the sewing machine can be as follows: Identify the continuous image frames containing the hand, needle tip, and fabric to obtain the target gesture type, the confidence level corresponding to the target gesture type, a first distance, and the fabric state, where the first distance is the distance between the hand and the needle tip; determine the target control command based on the target gesture type, the confidence level corresponding to the target gesture type, the current state of the sewing machine, the fabric state, and the first distance. Alternatively, the method can be as follows: Acquire continuous image frames including at least the hand; preprocess the image frames and extract key points of the hand; output the gesture type and confidence level based on the key points of the hand; fuse the gesture type with the current state of the sewing machine to generate a control intent; confirm or filter the control intent based on a false touch suppression strategy; and convert the confirmed control intent into a target control command. The false touch suppression strategy includes one or more of the following: confidence threshold judgment, temporal consistency judgment, and action cooldown time judgment. It should be noted that for high-risk control commands, a dual-condition confirmation mechanism can be configured.

[0042] Optionally, the target control command is determined based on the continuous image frames and the current state of the sewing machine, including:

[0043] The continuous image frames containing the hand are identified to obtain the target gesture type and the confidence level corresponding to the target gesture type.

[0044] In this embodiment, the method for identifying the continuous image frames containing the hand to obtain the target gesture type and the confidence level corresponding to the target gesture type can be as follows: input the continuous image frames into the gesture recognition model to obtain the target gesture type and the confidence level corresponding to the target gesture type. The gesture recognition model is trained based on a first training sample set, which includes: frame samples and the gesture types corresponding to the frame samples.

[0045] The target control command is determined based on the target gesture type, the confidence level corresponding to the target gesture type, and the current state of the sewing machine.

[0046] In this embodiment, the method for determining the target control command based on the target gesture type, the confidence level corresponding to the target gesture type, and the current state of the sewing machine can be as follows: if the confidence level corresponding to the target gesture type is greater than a confidence threshold, then query the initial control command corresponding to the target gesture type; obtain the disabled control command corresponding to the current state of the sewing machine; if the initial control command corresponding to the target gesture type is different from the disabled control command, then use the initial control command corresponding to the target gesture type as the target control command. In this embodiment, the method for determining the target control command based on the target gesture type, the confidence level corresponding to the target gesture type, and the current state of the sewing machine can also be as follows: if the confidence level corresponding to the target gesture type is greater than a confidence threshold, then query the initial control command corresponding to the target gesture type; obtain the available control command corresponding to the current state of the sewing machine; if the initial control command corresponding to the target gesture type is an available control command, then use the initial control command corresponding to the target gesture type as the target control command.

[0047] In this embodiment, a set of configurable gesture libraries can be predefined, including but not limited to: gesture A (palm down, rapidly swinging left and right) corresponding to a pause / continue command; gesture B (thumb up / down) corresponding to an acceleration / deceleration command; gesture C (index finger drawing a circle) corresponding to an adjustment of the stitch length setting command; gesture D (two fingers together, swiping backward) corresponding to a reverse stitch command; gesture E (clenching a fist and holding for 1 second) corresponding to a needle stop upper / lower position switching command; and gesture F (five fingers spread close to the edge of the needle area) corresponding to a safety deceleration / stop command (which can also be used as an emergency gesture). By querying the gesture library, the initial control command corresponding to the target gesture type is obtained.

[0048] Optionally, the continuous image frames further include: a needle tip;

[0049] Determining target control instructions based on the continuous image frames and the current state of the sewing machine includes:

[0050] The continuous image frames containing the hand and the needle tip are identified to obtain the target gesture type, the confidence level corresponding to the target gesture type, and the first distance.

[0051] Wherein, the first distance is the distance between the hand and the needle tip. It should be noted that the first distance is the closest distance between the hand and the needle tip. For example, if the distance between the palm and the needle tip is L1 and the distance between the fingertip and the needle tip is L2, and L2 is less than L1, then L2 is taken as the first distance.

[0052] In this embodiment, the method for identifying the continuous image frames containing the hand and the needle tip to obtain the target gesture type, the confidence level corresponding to the target gesture type, and the first distance can be as follows: identify the continuous image frames containing the hand and the needle tip to obtain the distance between the hand and the needle tip, input the continuous image frames containing the hand and the needle tip into the gesture recognition model to obtain the target gesture type and the confidence level corresponding to the target gesture type.

[0053] The target control command is determined based on the target gesture type, the confidence level corresponding to the target gesture type, the current state of the sewing machine, and the first distance.

[0054] In this embodiment, the method for determining the target control command based on the target gesture type, the confidence level corresponding to the target gesture type, the current state of the sewing machine, and the first distance can be as follows: if the confidence level corresponding to the target gesture type is greater than a confidence threshold, then query the initial control command corresponding to the target gesture type; obtain the disabled control command corresponding to the current state of the sewing machine; if the initial control command corresponding to the target gesture type is different from the disabled control command, and the first distance is greater than or equal to a first distance threshold, then use the initial control command corresponding to the target gesture type as the target control command. Alternatively, the method for determining the target control command based on the target gesture type, the confidence level corresponding to the target gesture type, the current state of the sewing machine, and the first distance can be as follows: if the confidence level corresponding to the target gesture type is greater than a confidence threshold, then query the initial control command corresponding to the target gesture type; obtain the available control command corresponding to the current state of the sewing machine; if the initial control command corresponding to the target gesture type is an available control command, and the first distance is greater than or equal to a first distance threshold, then use the initial control command corresponding to the target gesture type as the target control command.

[0055] Optionally, the continuous image frames containing the hand and the needle tip are identified to obtain the target gesture type, the confidence level corresponding to the target gesture type, and the first distance, including:

[0056] The continuous image frames containing the hand and the needle tip are processed to obtain the key points of the hand and the position of the needle tip corresponding to each frame.

[0057] In this embodiment, the key points of the hand can be the fingertips or the finger joints.

[0058] The first distance is determined based on the key hand points and needle tip positions corresponding to each frame of the image.

[0059] It should be noted that the first distance can be determined based on the fingertip position and the needle tip position corresponding to each frame of the image. For example, the distance between the fingertip position and the needle tip position can be used as the first distance.

[0060] In this embodiment, the first distance can be the minimum distance between the fingertip position and the needle tip position in consecutive image frames.

[0061] Based on the hand key points corresponding to each frame of the image, the target gesture type and the confidence level corresponding to the target gesture type are determined.

[0062] In this embodiment, the method for determining the target gesture type and the confidence level corresponding to the target gesture type based on the hand key points corresponding to each frame image can be as follows: perform temporal smoothing on the hand key points corresponding to each frame image to obtain a sequence of hand key points for consecutive image frames, input the sequence of hand key points for consecutive image frames into the gesture recognition model, and obtain the target gesture type and the confidence level corresponding to the target gesture type.

[0063] Optionally, a target control command is determined based on the target gesture type, the confidence level corresponding to the target gesture type, the current state of the sewing machine, and the first distance, including:

[0064] If the confidence level corresponding to the target gesture type is greater than the confidence level threshold, then query the initial control command corresponding to the target gesture type.

[0065] In this embodiment, a confidence threshold can be preset, which can be 0.6 to 0.9. If the confidence level corresponding to the target gesture type is less than or equal to the confidence threshold, no trigger is triggered. If the confidence level corresponding to the target gesture type is greater than the confidence threshold, the initial control command corresponding to the target gesture type is queried.

[0066] It should be noted that if the confidence level corresponding to the target gesture type is less than or equal to the confidence level threshold, it means that the accuracy of the target gesture type is low, and therefore the gesture is discarded.

[0067] Obtain the disable control command corresponding to the current state of the sewing machine.

[0068] In this embodiment, the method for obtaining the disable control command corresponding to the current state of the sewing machine can be as follows: a correspondence between the sewing machine state and the disable control command is established in advance, the correspondence is queried, and the disable control command corresponding to the current state of the sewing machine is obtained.

[0069] In a specific example, if the sewing machine is currently running, the control command "lift presser foot" is disabled, and the control commands "accelerate / decelerate / pause" are available. If the sewing machine is currently stopped, the control commands "lift presser foot / cut thread / switch needle position" are available.

[0070] If the initial control command corresponding to the target gesture type is different from the disabled control command, and the first distance is greater than or equal to the first distance threshold, then the initial control command corresponding to the target gesture type is taken as the target control command.

[0071] In this embodiment, if the initial control command corresponding to the target gesture type is different from the disabled control command, and the first distance is greater than or equal to the first distance threshold, then the method of using the initial control command corresponding to the target gesture type as the target control command can be as follows: if the initial control command corresponding to the target gesture type is different from the disabled control command, the initial control command corresponding to the target gesture type is an available control command, and the first distance is greater than or equal to the first distance threshold, then the initial control command corresponding to the target gesture type is used as the target control command.

[0072] In this embodiment, a false touch suppression strategy can also be preset. The false touch suppression strategy may include: not triggering gesture control if the value is below a confidence threshold; requiring stable recognition of the same gesture type for multiple consecutive frames before gesture control can be performed (e.g., stable recognition of the same gesture type for N consecutive frames, where N can be 3-10 frames); setting context gating (e.g., allowing the "presser foot lift" gesture only when the sewing machine is in a stopped state); and setting a cooldown time (e.g., ignoring repeated triggering within a short time Δt after the same gesture is triggered, where Δt can be 0.3-2 seconds).

[0073] Optionally, the target control command is determined based on the continuous image frames and the current state of the sewing machine, including:

[0074] The continuous image frames and the current state of the sewing machine are input into the control command generation model to obtain the target control command. The control command generation model is trained based on a training sample set, which includes frame samples and corresponding annotation information. The annotation information corresponding to the frame samples includes the sewing machine state and control command, and the frame samples include hands.

[0075] In this embodiment, the training process of the instruction generation model includes: inputting the frame sample containing the hand and the sewing machine state corresponding to the frame sample into the model to be trained to obtain the predicted control instruction; training the parameters of the model to be trained according to the difference between the predicted control instruction and the control instruction corresponding to the frame sample to obtain the control instruction generation model.

[0076] In this embodiment, the continuous image frames further include a needle tip. The method for obtaining the target control command by inputting the continuous image frames and the current state of the sewing machine into the control command generation model can be: inputting the continuous image frames containing the hand and needle tip and the current state of the sewing machine into the control command generation model to obtain the target control command. Alternatively, the method for obtaining the target control command by inputting the continuous image frames and the current state of the sewing machine into the control command generation model can be: inputting the continuous image frames containing the hand and needle tip, the current state of the sewing machine, and a first distance into the control command generation model to obtain the target control command.

[0077] It should be noted that this embodiment also supports calibration when workers use it for the first time. The specific calibration method can be: collect several standard gesture samples, establish individualized parameters (hand size, habitual movement amplitude, lighting conditions), and adjust the threshold or update the model based on the above parameters, thereby improving the accuracy and stability of gesture recognition.

[0078] Optionally, the training process of the instruction generation model includes:

[0079] The frame samples containing the hand and the corresponding sewing machine states are input into the model to be trained to obtain predictive control commands.

[0080] In this embodiment, the frame sample containing the hand can be a continuous image frame sample containing the hand, and the sewing machine state corresponding to the frame sample can be the state of the sewing machine during the process of acquiring the continuous image frame sample containing the hand.

[0081] The parameters of the model to be trained are trained based on the difference between the predicted control command and the control command corresponding to the frame sample, so as to obtain the control command generation model.

[0082] Optionally, the frame sample further includes a needle tip, and the annotation information corresponding to the frame sample further includes: the distance between the hand and the needle tip;

[0083] The training process of the instruction generation model includes:

[0084] The frame samples containing the hand and needle tip, the sewing machine state corresponding to the frame samples, and the distance between the hand and the needle tip are input into the model to be trained to obtain predictive control commands.

[0085] The parameters of the model to be trained are trained based on the difference between the predicted control command and the control command corresponding to the frame sample, so as to obtain the control command generation model.

[0086] It should be noted that, when the input to the control command generation model includes: continuous image frames containing the hand and needle tip, the current state of the sewing machine, and a first distance, the training process of the control command generation model is as follows: input the frame samples containing the hand and needle tip, the sewing machine state corresponding to the frame samples, and the distance between the hand and the needle tip into the model to be trained to obtain the predicted control command; train the parameters of the model to be trained based on the difference between the predicted control command and the control command corresponding to the frame samples to obtain the control command generation model.

[0087] S130, execute the target control command.

[0088] In this embodiment, the target control command can be executed by controlling the corresponding actuator based on the target control command.

[0089] In this embodiment, the sewing machine can be any of a flatbed sewing machine, an overlock sewing machine, a coverstitch sewing machine, and a pattern sewing machine. If the sewing machine is a flatbed sewing machine, the control instructions may include one or more of the following: needle position control instructions, reverse stitch control instructions, and thread trimming control instructions. If the sewing machine is an overlock sewing machine, the control instructions may include: differential feed ratio adjustment instructions. If the sewing machine is a pattern sewing machine, the control instructions may include: pattern program selection instructions and / or single-step execution control instructions.

[0090] In this embodiment, control commands can also be sent to the sewing machine controller via at least one of serial port, CAN, RS-485, USB, and Ethernet.

[0091] Optionally, the continuous image frames further include fabric, and the control method further includes:

[0092] The actual stitch pattern is obtained by identifying the continuous image frames containing the fabric.

[0093] In this embodiment, the method for identifying the continuous image frames containing the fabric to obtain the actual stitch pattern can be as follows: Gaussian denoising and histogram equalization are performed on the acquired image frames containing the fabric and the sewn threads. The stitch points are identified through a model. Based on the inter-frame temporal relationship, the stitch points detected in multiple frames are spliced ​​and smoothed. The spliced ​​stitch point sequence is fitted into a continuous curve to obtain the actual stitch pattern.

[0094] The quality data is determined based on the stitch length corresponding to the actual stitch pattern and / or the theoretical stitch pattern.

[0095] In this embodiment, the quality data can be determined based on the stitch length corresponding to the actual stitch pattern and / or the theoretical stitch pattern by: comparing the stitch length corresponding to the actual stitch pattern with the theoretical stitch length to obtain the stitch length uniformity; determining the stitch deviation based on the actual stitch pattern and the theoretical stitch pattern; and using the stitch length uniformity and / or stitch deviation as the quality data.

[0096] In a specific example, the sewing reference can be at least one of the following: fabric edge, printed marking line, pre-sewn positioning line, tooling fixture edge, and pattern outline. The theoretical stitch curve B(t) is pre-acquired. The actual stitch curve S(t) is obtained by processing the acquired image frames containing the fabric and sewn threads. The deviation d(t) = dist(S(t), B(t)) is calculated; when d(t) exceeds the first threshold d1, a quality prompt is output (screen arrow / beep prompts the operator to correct the deviation); when d(t) exceeds the second threshold d2, automatic speed reduction or pause is triggered; the first threshold is less than the second threshold, with examples of thresholds: d1 = 0.5~2.0mm, d2 = 1.5~5.0mm. For models equipped with electronic feed / controllable differential feed, a feed compensation amount Δv is output to mitigate the expansion of the deviation.

[0097] It should be noted that the quality data can be visualized on a display screen.

[0098] The corresponding compensation instruction is generated based on the quality data, and the operating parameters of the sewing machine are adjusted based on the compensation instruction.

[0099] In this embodiment, the compensation command can be a speed reduction command, a pause command, etc.

[0100] In this embodiment, a correspondence list between quality data and compensation instructions can be pre-established. By querying the correspondence list, the compensation instructions corresponding to the quality data can be obtained.

[0101] Optionally, the sewing machine further includes a display module, and the control method further includes:

[0102] If the distance between the hand and the needle tip is less than the distance threshold, an alarm message will be generated.

[0103] In this embodiment, the distance between the hand and the needle tip can be determined in real time based on the collected continuous image frames. When the distance between the hand and the needle tip is detected to be less than the distance threshold, an alarm message is generated.

[0104] The display module displays the target gesture type, the confidence level corresponding to the target gesture type, the target control command, alarm information, and the current status of the sewing machine.

[0105] In this embodiment, while displaying alarm information through the display module, an audio-visual prompt can also be triggered.

[0106] In this embodiment, the initial region of interest (ROI) of the image acquisition module is based on the needle tip, presser foot, and needle plate opening. The ROI can be dynamically adjusted as the presser foot is raised or lowered and the fabric position changes. It can also set a restricted area (such as when the fabric edge shakes significantly or is severely obscured) to pause gesture triggering and only retain safe shutdown.

[0107] Optionally, the consecutive image frames may also include fabric;

[0108] Also includes:

[0109] The continuous image frames containing the fabric and the hand are identified to obtain the fabric state and the area of ​​the hand that is covered.

[0110] In this embodiment, the fabric state includes: fabric edge swaying, fabric offset, fabric wrinkles, etc.

[0111] In this embodiment, the method for identifying the continuous image frames containing the fabric and the hand to obtain the fabric state and the occlusion area of ​​the hand can be as follows: The continuous image frames containing the fabric and the hand are classified at the pixel level; the fabric region and the hand region are extracted; the edge contour of the fabric region is extracted; the contour curvature and flatness are calculated; based on the contour curvature and flatness, it is determined whether the fabric has wrinkles or shifts; the texture changes of the fabric region are analyzed; and based on the texture changes of the fabric region, it is determined whether the fabric is stretched or wrinkled, etc. The occlusion area of ​​the hand is obtained based on the number of pixels in the overlapping area between the hand and the fabric.

[0112] If any shutdown condition is met, a shutdown command is executed, wherein the shutdown conditions include:

[0113] The fabric condition meets the preset conditions.

[0114] In this embodiment, the preset conditions include at least one of the following: fabric edge offset is greater than an offset threshold; fabric flatness is less than a flatness threshold; and fabric stretch is greater than a stretch threshold.

[0115] The area obscured by the hand is greater than the area threshold.

[0116] In this embodiment, the area threshold can be a preset value.

[0117] The hand moves toward the needle tip at a speed greater than the speed threshold.

[0118] In this embodiment, if a hand is detected rapidly approaching the needle tip, or if the area covered by the hand exceeds the area threshold, an emergency shutdown is triggered, and a time log is recorded for management and traceability.

[0119] It should be noted that if the image acquisition module is an RGB-D or binocular camera, more accurate spatial determination can be achieved by further considering the Z-axis distance between the hand and the needle plate; if the image acquisition module is a monocular camera, an approximate depth estimation can be made by combining changes in hand size and the movement speed of key points.

[0120] Optionally, the sewing machine further includes a display module, and the control method further includes:

[0121] If the distance between the hand and the needle tip is less than the first distance threshold and greater than or equal to the second distance threshold, an audio-visual prompt is triggered, and an alarm message is displayed through the display module, wherein the first distance threshold is greater than the second distance threshold.

[0122] In this embodiment, if the distance between the hand and the needle tip is less than the first distance threshold and greater than or equal to the second distance threshold, the maximum rotation speed limit or the acceleration slope can also be limited.

[0123] If the distance between the hand and the needle tip is less than the second distance threshold, the rotation speed is reduced to the preset speed, or a stop command is executed.

[0124] In this embodiment, if the distance between the hand and the needle tip is less than the second distance threshold, the needle can be stopped to reduce fabric rebound and prevent accidental contact to continue sewing, or high-risk actions can be locked and released by a specific gesture or button, or speed can be limited.

[0125] The technical solution provided in this embodiment controls any one of the following based on the spatial relationship between the hand and the needle area: early warning, speed limit, or shutdown, which can reduce the risk of injury to personnel.

[0126] Optionally, the target control command includes at least one of the following: start / stop command, speed adjustment command, stitch length adjustment command, reverse stitch command, needle position control command, presser foot control command, and thread cutting command.

[0127] In this embodiment, the sewing machine includes a sewing machine body and a vision system. The sewing machine body includes: a sewing head, a needle mechanism, a presser foot mechanism, a fabric feeding mechanism, a drive motor, a sewing controller, an operation panel, and a display screen. The vision system includes at least two of the following: an image acquisition module, a lighting module, an image processing module, a gesture recognition module, a gesture analysis module, a gesture application module, and a human-machine interaction module. The human-machine interaction module can be reused with the display screen. The image processing module can be an embedded processor (e.g., a CPU, an AI accelerator, or a GPU) or an AI control board integrated with the sewing controller, and is electrically connected to the image acquisition module. The gesture application module has a built-in mapping table that binds the recognized gesture types with control commands and can be configured on the HMI interface. The system comprises the following modules: an image acquisition module for acquiring images of the needle area / workbench area, obtaining continuous image frames containing the hand and fabric; an illumination module (optional if sufficient brightness is available) for supplementing the work area with supplemental lighting or providing infrared illumination to enhance robustness; an image processing module for denoising, distortion correction, target detection (or target segmentation), key point extraction, and temporal smoothing of the image frames; a gesture recognition module for classifying gestures and outputting confidence scores based on the hand area or hand key point sequence; a gesture analysis module for fusing and analyzing gestures with sewing status, hand trajectory, and needle area safety boundaries to obtain at least one of the following: control intent, risk level, and quality prompts; a gesture application module for converting control intents into control commands executable by the sewing controller, controlling at least one or more sewing parameters / actions, including start / stop, speed, stitch length, reverse stitching, needle position (stop when needle goes up / down), presser foot lifting / lowering, thread cutting, and alarm prompts; and a human-machine interaction module for displaying recognition results, current mode, quality prompts, or risk alarms, and for configuring gesture libraries and performing individualized calibration.

[0128] It should be noted that the image acquisition module is installed at the front end of the machine head, above the needle plate, or above the worktable, so that its field of view covers at least the needle tip, presser foot, and fabric guiding area; the gesture application module is equipped with a mis-touch suppression strategy, which confirms or filters control commands based on at least one of confidence threshold, timing consistency, and context state (sewing in progress / stopped / presser foot lifted).

[0129] In a specific example, the image acquisition module acquires continuous image frames at a preset frame rate; the image processing module performs distortion correction, brightness normalization, and noise reduction on the image frames; the hand region ROI is obtained through a hand detection network or skin color / depth segmentation; key points of the hand (fingertips, joints, etc.) are extracted from the ROI and temporally smoothed; the sequence of N consecutive key points is input into the gesture recognition module, which outputs the gesture type and confidence level; the gesture analysis module determines the control intent by combining the current state of the sewing machine (running / stopping, speed, needle position, presser foot status, fabric status, etc.) and a first distance; the gesture application module encodes the control intent into control commands and sends them to the sewing controller, which then executes the control of the motor, presser foot, fabric feeding, and other actuators; the human-machine interaction module displays the currently recognized gesture, the executed action, and prompt information.

[0130] This embodiment uses a human-computer interaction module to intuitively perform gesture interactions, and with the help of screen prompts and personalized calibration, improves the usability and consistency of gesture control.

[0131] It should be noted that if the target control command is a high-risk control command, the control command can be determined using a "gesture + pause" or "gesture + secondary confirmation gesture". The high-risk control command can be a pre-set control command, such as a rapid start / stop or wire-cutting command.

[0132] In this embodiment, the end-to-end latency T from data acquisition to the issuance of control commands satisfies: T ≤ 120 ms (preferably ≤ 80 ms); and the computational load can be reduced by using a "lightweight network + inter-frame multiplexing + key point tracking" approach.

[0133] In a specific example, if the sewing machine is a flatbed sewing machine (single-needle lockstitch), the control commands can be at least one of the following: main motor speed adjustment command, needle stop up / down position control command, reverse sewing control command, thread trimming control command, and presser foot lift / lower control command. The corresponding gesture and command mapping library includes: gesture (thumb up / down) corresponds to speed setting ±1; gesture (palm horizontal swing) corresponds to pause / continue command; gesture (two fingers backward sweep) corresponds to reverse sewing command; and gesture (clenched fist) corresponds to needle position switching command. Quality monitoring is performed by detecting and alerting the user regarding fabric deviation based on the fabric edge.

[0134] In another specific example, if the sewing machine is an overlock / sealing machine (including differential feed), the control commands can be at least one of the following: main motor speed adjustment command, differential feed ratio adjustment command, presser foot lifting / lowering control command, and start / finish stitch mode command. The corresponding gesture and command mapping library includes: a gesture (index finger drawing a circle) corresponds to an instruction to adjust the differential feed ratio within a preset range; a gesture (palm pressing down and holding) corresponds to an instruction to enter slow fine sewing mode. Quality monitoring is performed by detecting fabric wrinkling trends (e.g., texture compression, edge fluctuation) and suggesting adjustments to the differential ratio or speed reduction.

[0135] In another specific example, if the sewing machine is a coverstitch / covering sewing machine, the control commands can be at least one of the following: speed adjustment command, presser foot pressure setting (if adjustable) adjustment command, and needle position / start / end command. Quality monitoring is achieved by detecting uneven stitch spacing caused by fabric shrinkage during stretching, triggering a speed reduction or prompting an adjustment of the tension method.

[0136] In another specific example, if the sewing machine is a pattern sewing machine / computerized sewing machine (programmed trajectory), the control commands can be at least one of the following: pattern program selection command, start / pause command, single-step / jog command, and return-to-origin command. The corresponding gesture and command mapping library includes: a gesture (three fingers spread) corresponds to a pattern sequence number switching command; a gesture (two fingers tapping) corresponds to a single-step execution command. The quality monitoring method is to combine tooling and fixture recognition to verify whether the workpiece is placed in place (starting is not allowed if it is not in place).

[0137] The technical solution provided in this embodiment can complete the input of control commands while ensuring that the worker's hands continuously guide the fabric, thereby improving the efficiency of continuous operation. In addition, by combining gestures with the sewing status for real-time prompts or adaptive control, defects such as deviation and unstable stitch length can be reduced. The technical solution of this embodiment first acquires continuous image frames including the hand; then determines the target control command based on the continuous image frames and the current state of the sewing machine; and finally executes the target control command. This can solve the problems of low work efficiency caused by manual interaction interrupting the sewing process, disrupting production continuity, and causing problems such as unstable feeding guidance due to the hand briefly leaving the fabric, which can easily lead to quality defects such as thread deviation, uneven stitch length, and fabric wrinkling. It also solves the safety risks of accidental needle contact and needle pricks due to the frequent switching of the hand between the fabric and the operating interface. The sewing machine can be controlled by changes in hand posture, thereby improving sewing quality, efficiency, and safety.

[0138] Example 2

[0139] Figure 2A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0140] like Figure 2 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0141] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as control methods.

[0143] In some embodiments, the control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the control method by any other suitable means (e.g., by means of firmware).

[0144] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0146] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0148] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0149] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0150] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0151] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the control method according to any embodiment of the invention.

[0152] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A control method, characterized in that, Applied to a sewing machine, the sewing machine includes an image acquisition module for acquiring continuous image frames, and the control method includes: Acquire consecutive image frames, wherein the consecutive image frames include a hand and a needle tip; Process consecutive image frames containing the hand and the needle tip to obtain the key points of the hand and the position of the needle tip corresponding to each frame; Based on the key hand points and needle tip position corresponding to each frame of the image, a first distance is determined, wherein the first distance is the distance between the hand and the needle tip; Based on the hand key points corresponding to each frame of the image, determine the target gesture type and the confidence level corresponding to the target gesture type; If the confidence level corresponding to the target gesture type is greater than the confidence level threshold, then query the initial control command corresponding to the target gesture type; Obtain the disable control command corresponding to the current state of the sewing machine; If the initial control command corresponding to the target gesture type is different from the disabled control command, and the first distance is greater than or equal to the first distance threshold, then the initial control command corresponding to the target gesture type is taken as the target control command. Execute the target control command.

2. The method according to claim 1, characterized in that, The target control command is determined based on the continuous image frames and the current state of the sewing machine, including: The system identifies consecutive image frames containing hands, and obtains the target gesture type and the corresponding confidence level for the target gesture type. The target control command is determined based on the target gesture type, the confidence level corresponding to the target gesture type, and the current state of the sewing machine.

3. The method according to claim 1, characterized in that, The target control command is determined based on the continuous image frames and the current state of the sewing machine, including: The continuous image frames and the current state of the sewing machine are input into the control command generation model to obtain the target control command. The control command generation model is trained based on a training sample set, which includes frame samples and corresponding annotation information. The annotation information corresponding to the frame samples includes the sewing machine state and control command, and the frame samples include hands.

4. The method according to claim 3, characterized in that, The training process of the instruction generation model includes: Input the frame samples containing the hand and the corresponding sewing machine state into the model to be trained to obtain the predictive control command. The parameters of the model to be trained are trained based on the difference between the predicted control command and the control command corresponding to the frame sample, so as to obtain the control command generation model.

5. The method according to claim 4, characterized in that, The frame sample also includes a needle tip, and the annotation information corresponding to the frame sample also includes: the distance between the hand and the needle tip; The training process of the instruction generation model includes: The frame samples containing the hand and needle tip, the sewing machine state corresponding to the frame samples, and the distance between the hand and the needle tip are input into the model to be trained to obtain predictive control commands. The parameters of the model to be trained are trained based on the difference between the predicted control command and the control command corresponding to the frame sample, so as to obtain the control command generation model.

6. The method according to claim 1, characterized in that, The sewing machine also includes: a machine head, a needle bar, and a needle bar cover; the needle bar is installed at the front end of the machine head, and the needle bar cover is installed on the outside of the needle bar. The image acquisition module includes at least one camera. The camera is installed at the front end of the machine head or on the bracket above the workbench. The front end of the machine head includes any one of the following: the side of the front end of the machine head, the lower edge of the front end of the machine head, the front of the needle bar cover, the side of the needle bar cover, and the lower edge of the needle bar cover.

7. The method according to claim 6, characterized in that, The clamp between the optical axis of the camera and the needle plate plane is related to the installation position of the camera. If the camera is installed at the front end of the machine head, the clamp between the optical axis of the camera and the needle plate plane is the first clamp. If the camera is installed on the support above the workbench, the clamp between the optical axis of the camera and the needle plate plane is the second clamp. The second clamp is larger than the first clamp.

8. The method according to claim 6, characterized in that, The sewing machine also includes a presser foot and a needle. The needle is connected to the lower end of the needle bar. The presser foot is located below the needle bar cover and to the side of the needle. The camera is installed on the side of the machine head, in front of the presser foot and the needle. The camera lens faces the needle area at an angle downwards.

9. The method according to claim 1, characterized in that, The continuous image frames also include fabric, and the control method further includes: Identify consecutive image frames containing fabric to obtain the actual stitch pattern; The quality data is determined based on the stitch length corresponding to the actual stitch pattern and / or the theoretical stitch pattern; The corresponding compensation instruction is generated based on the quality data, and the operating parameters of the sewing machine are adjusted based on the compensation instruction.

10. The method according to claim 2, characterized in that, The sewing machine also includes a display module, and the control method further includes: If the distance between the hand and the needle tip is less than the distance threshold, an alarm message will be generated; The display module displays the target gesture type, the confidence level corresponding to the target gesture type, the target control command, alarm information, and the current status of the sewing machine.

11. The method according to claim 1, characterized in that, The continuous image frames also include fabric; Also includes: Identify consecutive image frames containing fabric and hands to obtain the fabric state and the area of ​​hand occlusion; If any shutdown condition is met, a shutdown command is executed, wherein the shutdown conditions include: The fabric condition meets the preset conditions; The area obscured by the hand is greater than the area threshold. The hand moves toward the needle tip at a speed greater than the speed threshold.

12. The method according to claim 1, characterized in that, The sewing machine also includes a display module, and the control method further includes: If the distance between the hand and the needle tip is less than the first distance threshold and greater than or equal to the second distance threshold, an audio-visual prompt is triggered, and an alarm message is displayed through the display module, wherein the first distance threshold is greater than the second distance threshold; If the distance between the hand and the needle tip is less than the second distance threshold, the rotation speed is reduced to the preset speed, or a stop command is executed.

13. The method according to claim 1, characterized in that, The target control commands include at least one of the following: start / stop command, speed adjustment command, stitch length adjustment command, reverse stitch command, needle position control command, presser foot control command, and thread cutting command.

14. A sewing machine, characterized in that, The sewing machine includes electronic equipment, which includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the control method according to any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the control method according to any one of claims 1-13.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the control method according to any one of claims 1-13.

Images