Intelligent template machine adaptive control method based on multi-modal perception and related equipment

Abstract

The application provides a kind of intelligent template machine adaptive control method based on multi-modal perception, comprising: when the intelligent template machine is sewn to the garment to be sewn by preset sewing track, real-time acquisition is carried out to the multi-modal perception data and actual sewing track during sewing;Determine the deviation value between actual sewing track and preset sewing track;If the deviation value is greater than the first deviation value threshold value preset, start trajectory compensation mode, generate compensation sewing track by actual sewing track, and control the intelligent template machine to sew the garment to be sewn with compensation sewing track;If the deviation value is less than the first deviation value threshold value preset, and greater than the second deviation value threshold value preset, start parameter adjustment mode, determine dynamic sewing parameter by multi-modal perception data, and control the intelligent template machine to sew the garment to be sewn with dynamic sewing parameter on the basis of preset sewing.The application can realize accurate, efficient and adaptive sewing control, and improve the efficiency of garment sewing.

Description

Technical Field

[0001] This invention relates to the field of adaptive control technology for template machines, and in particular to an intelligent template machine adaptive control method and related equipment based on multimodal perception. Background Technology

[0002] In automated garment production, intelligent template machines are widely used due to their high efficiency and good consistency. Their working principle involves pre-setting a sewing trajectory (template path), and the machine automatically guides the fabric or seam allowance along this trajectory to complete the sewing.

[0003] Existing intelligent template machines, during actual sewing, experience deviations from the preset ideal trajectory due to factors such as fabric characteristics (e.g., elasticity, uneven thickness), relative slippage between multiple layers of fabric, fluctuations in thread tension, changes in presser foot pressure, minor errors in mechanical transmission, and environmental factors (e.g., temperature and humidity). Traditional template machines primarily rely on motor encoders or simple photoelectric sensors for position feedback, lacking real-time, comprehensive sensing capabilities for the multi-dimensional physical states of the sewing process (e.g., fabric deformation, tension, visual texture matching, pressure distribution). They cannot fully capture the root causes of trajectory deviations. Furthermore, existing systems typically employ fixed control parameters or simple threshold alarms (e.g., over-limit shutdown), lacking adaptive adjustment capabilities. For minor but not severe deviations, they cannot proactively optimize parameters to suppress deviation accumulation; for sudden, large deviations, they often require emergency stopping and manual intervention, severely impacting garment sewing efficiency. Summary of the Invention

[0004] This invention provides an adaptive control method for an intelligent template sewing machine based on multimodal perception, aiming to improve the garment sewing efficiency of the intelligent template sewing machine. By collecting multimodal perception data and the actual sewing trajectory in real time during the sewing process, the deviation value from the preset sewing trajectory is calculated. When the deviation value is large, a trajectory compensation mode is activated to correct the sewing trajectory. When the deviation value is neither too large nor too small, a parameter adjustment mode is activated to avoid deviation accumulation, thereby achieving precise, efficient, and adaptive sewing control and improving garment sewing efficiency.

[0005] In a first aspect, embodiments of the present invention provide an adaptive control method for an intelligent template machine based on multimodal perception, the method comprising:

[0006] When the intelligent template machine sews the garment to be sewn according to the preset sewing trajectory, it acquires multimodal perception data and the actual sewing trajectory in real time.

[0007] The deviation value between the actual sewing trajectory and the preset sewing trajectory is determined;

[0008] If the deviation value is greater than the preset first deviation value threshold, the trajectory compensation mode is activated, a compensation sewing trajectory is generated based on the actual sewing trajectory, and the intelligent template machine is controlled to sew the garment to be sewn using the compensation sewing trajectory.

[0009] If the deviation value is less than a preset first deviation value threshold and greater than a preset second deviation value threshold, then the parameter adjustment mode is activated, the dynamic sewing parameters are determined by the multimodal sensing data, and the intelligent template machine is controlled to sew the garment to be sewn based on the preset sewing and the dynamic sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold.

[0010] Optionally, the step of determining the deviation value between the actual sewing trajectory and the preset sewing trajectory includes:

[0011] The actual sewing trajectory is compared with the preset sewing trajectory to obtain a first deviation value;

[0012] The multimodal sensing data and the actual sewing trajectory are input into the trajectory prediction model for trajectory prediction processing to obtain the predicted sewing trajectory corresponding to the actual sewing trajectory.

[0013] The predicted sewing trajectory is compared with the preset sewing trajectory to obtain a second deviation value;

[0014] The first deviation value and the second deviation value are fused together using dynamic weighting to obtain the deviation value between the actual sewing trajectory and the preset sewing trajectory. The dynamic weighting value is determined based on the completion degree of the actual sewing trajectory.

[0015] Optionally, before the step of fusing the first deviation value and the second deviation value using dynamic weights, the method further includes:

[0016] Based on the number of completed stitches in the actual sewing trajectory and the total number of stitches in the preset sewing trajectory, the stitch completion ratio of the actual sewing trajectory is determined.

[0017] The average coordinate deviation between the actual coordinates of each stitch point in the completed stitches of the actual sewing trajectory and the preset coordinates of the corresponding stitch points of the preset sewing trajectory.

[0018] The attenuation coefficient is determined based on the average coordinate deviation value;

[0019] Based on the attenuation coefficient and the stitch completion ratio, the dynamic weights corresponding to the first deviation value and the second deviation value are determined.

[0020] Optionally, the step of generating a compensating sewing trajectory using the actual sewing trajectory includes:

[0021] Starting from the latest stitch point in the actual sewing trajectory, trace back the error distribution of the previous M stitch points.

[0022] Based on the error distribution, a dynamic compensation sequence for the last N needle points is generated;

[0023] The compensated sewing trajectory is obtained by compensating the N needle points following the needle point corresponding to the starting point in the preset sewing trajectory based on the dynamic compensation sequence.

[0024] Optionally, the step of generating the dynamic compensation sequence for the next N needle points based on the error distribution includes:

[0025] Extract the position error sequence of the first M needle points;

[0026] Based on the fabric material of the garment to be sewn, the target error propagation matrix is ​​determined, with different fabric materials corresponding to different error propagation matrices;

[0027] Based on the target error propagation matrix and the position error sequence, the dynamic compensation sequence for the last N needle points is determined.

[0028] Optionally, the step of determining dynamic sewing parameters using the multimodal sensing data includes:

[0029] Based on the multimodal sensing data, the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle are determined.

[0030] Based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle, dynamic sewing parameters are determined.

[0031] Optionally, the step of determining the dynamic sewing parameters based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle includes:

[0032] Based on the fabric material of the garment to be sewn, a target multimodal fusion neural network is determined, with different fabric materials corresponding to different multimodal fusion neural networks;

[0033] Based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle, a multimodal fusion feature vector is constructed.

[0034] The multimodal fusion feature vector is input into the target multimodal fusion neural network, and the dynamic sewing parameters are output.

[0035] Secondly, embodiments of the present invention also provide an adaptive control device for an intelligent template machine based on multimodal perception, the adaptive control device for the intelligent template machine based on multimodal perception comprising:

[0036] The acquisition module is used to acquire multimodal perception data and the actual sewing trajectory in real time when the intelligent template machine sews the garment to be sewn according to the preset sewing trajectory.

[0037] The processing module is used to determine the deviation value between the actual sewing trajectory and the preset sewing trajectory;

[0038] The first control module is used to activate the trajectory compensation mode if the deviation value is greater than the preset first deviation value threshold, generate a compensation sewing trajectory through the actual sewing trajectory, and control the intelligent template machine to sew the garment to be sewn with the compensation sewing trajectory.

[0039] The second control module is used to activate the parameter adjustment mode if the deviation value is less than a preset first deviation value threshold and greater than a preset second deviation value threshold, determine the dynamic sewing parameters through the multimodal sensing data, and control the intelligent template machine to sew the garment to be sewn based on the preset sewing and the dynamic sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold.

[0040] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the intelligent template machine adaptive control method based on multimodal perception provided in embodiments of the present invention.

[0041] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the intelligent template machine adaptive control method based on multimodal perception provided in the embodiments of the present invention.

[0042] In this embodiment of the invention, when the intelligent template machine sews the garment to be sewn according to a preset sewing trajectory, it acquires multimodal perception data and the actual sewing trajectory in real time; determines the deviation value between the actual sewing trajectory and the preset sewing trajectory; if the deviation value is greater than a preset first deviation value threshold, it activates the trajectory compensation mode, generates a compensation sewing trajectory through the actual sewing trajectory, and controls the intelligent template machine to sew the garment to be sewn according to the compensation sewing trajectory; if the deviation value is less than the preset first deviation value threshold and greater than a preset second deviation value threshold, it activates the parameter adjustment mode, determines dynamic sewing parameters through multimodal perception data, and controls the intelligent template machine to sew the garment to be sewn according to the preset sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold. This invention calculates the deviation from the preset sewing trajectory by collecting multimodal sensing data and the actual sewing trajectory in real time during the sewing process. When the deviation is large, the trajectory compensation mode is activated to correct the sewing trajectory. When the deviation is neither too large nor too small, the parameter adjustment mode is activated to avoid the accumulation of deviation, thereby achieving precise, efficient and adaptive sewing control and improving garment sewing efficiency. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of an intelligent template machine adaptive control method based on multimodal perception provided in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the structure of an intelligent template machine adaptive control device based on multimodal perception provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, Figure 1 This is a flowchart of an adaptive control method for an intelligent template machine based on multimodal perception, provided in an embodiment of the present invention. The adaptive control method for an intelligent template machine based on multimodal perception includes the following steps:

[0049] 101. When the intelligent template machine sews the garment to be sewn according to the preset sewing trajectory, it acquires multimodal perception data and the actual sewing trajectory in real time.

[0050] 102. Determine the deviation between the actual sewing trajectory and the preset sewing trajectory.

[0051] 103. If the deviation value is greater than the preset first deviation value threshold, the trajectory compensation mode is activated, and a compensation sewing trajectory is generated by the actual sewing trajectory. The intelligent template machine is then controlled to sew the garment to be sewn using the compensation sewing trajectory.

[0052] 104. If the deviation value is less than the preset first deviation value threshold and greater than the preset second deviation value threshold, the parameter adjustment mode is activated. The dynamic sewing parameters are determined through multimodal sensing data, and the intelligent template machine is controlled to sew the garment to be sewn based on the preset sewing parameters and the dynamic sewing parameters.

[0053] Among them, the first deviation threshold is greater than the second deviation threshold.

[0054] In this embodiment of the invention, the intelligent template machine is an automated sewing device equipped with a computer control system, a servo drive mechanism and a sensing system, which can automatically complete garment sewing operations according to a preset sewing trajectory.

[0055] The preset sewing trajectory can be a sequence of spatial coordinates (such as two-dimensional plane coordinates XY or three-dimensional coordinates XYZ) that is digitally generated based on the garment design drawing before sewing begins, describing the ideal movement path of the sewing needle.

[0056] Multimodal sensing data can be a collection of multi-dimensional real-time information reflecting the physical state of sewing, synchronously collected through a heterogeneous sensor array during the sewing process. It includes at least visual modal data, mechanical modal data, position / motion modal data, and distance data. When sewing begins, the control system triggers all sensors to collect timestamp-aligned data at a fixed sampling period (e.g., 10ms), thereby acquiring timestamp-aligned multimodal sensing data.

[0057] The visual modal data can be images / video streams of the sewing area acquired by an industrial camera or line scan camera. This visual modal data can be used to identify the spatial location of fabric texture, edges, marker points, and actual seams. Specifically, frame images can be acquired via an image acquisition card using an industrial camera / line scan camera mounted above the sewing area. Edge detection and feature point matching algorithms are then used to extract the real-time coordinates of the actual seams or fabric marker points from the images.

[0058] Mechanical modal data can be real-time physical quantities acquired through embedded force / torque sensors, tension sensors, and pressure sensors. This data can include suture tension, the vertical pressure of the presser foot on the fabric, interlayer friction of the fabric, and needle puncture resistance. The suture tension sensor can be integrated into the thread guide roller, outputting an analog voltage signal which is converted by an ADC to obtain the suture tension data. The presser foot pressure sensor is embedded in the presser foot drive mechanism, outputting a pressure value to obtain the vertical pressure of the presser foot on the fabric. Interlayer friction and puncture force sensors are placed below the needle plate or feed teeth to collect interlayer friction of the fabric and needle puncture resistance, respectively.

[0059] Position / motion modal data can be the real-time position coordinates (X, Y, Z) and motion velocity and / or acceleration of the seam head, obtained through a servo motor encoder, grating ruler, or laser displacement sensor. The position data of the seam head can be obtained by using a high-precision encoder built into the X / Y / Z axis servo motor and reading the position feedback in real time through a motion control card; or by using a separate grating ruler to provide closed-loop position feedback; or by using a laser displacement sensor to acquire the motion velocity and / or acceleration of the seam head.

[0060] Distance data can be collected using a rangefinder positioned above the sewing area.

[0061] The actual sewing trajectory mentioned above can be a sequence of actual motion path coordinates of the needle tip or key feature points of the fabric, reconstructed in real time based on position / motion modal data during the sewing process.

[0062] The deviation value can be a quantitative measure of the spatial difference between the actual sewing trajectory and the preset sewing trajectory at the same sewing progress point. It can be calculated using the Euclidean distance between the trajectories or the morphological similarity of the trajectories.

[0063] The first deviation threshold can be a preset critical value for severe deviation. When the deviation value exceeds the first deviation threshold, it indicates that the trajectory deviation may lead to sewing failure or serious defects. In this case, the trajectory needs to be corrected.

[0064] The second deviation threshold can be a preset lower limit for allowable fluctuations. When the deviation value is greater than the second deviation threshold but less than or equal to the first deviation threshold, it indicates that there is a cumulative small deviation that requires intervention. In this case, no correction to the trajectory is necessary.

[0065] The trajectory compensation mode is a control strategy that is triggered when the deviation value reaches a first deviation threshold. The trajectory compensation mode can bypass the original preset path and, based on the current actual position and subsequent preset points, generate a new spatial path (compensated sewing trajectory) in real time to guide the device directly to the target point.

[0066] The parameter adjustment mode can be set when the deviation value is greater than the second deviation threshold but less than or equal to the first deviation threshold. Alternatively, the parameter adjustment mode can maintain the original preset trajectory while dynamically calculating and adjusting process parameters affecting trajectory following accuracy based on multimodal sensing data.

[0067] Dynamic sewing parameters are variables that are optimized in real time under parameter adjustment mode, including at least the following parameters: servo motor PID gain, fabric feed speed, presser foot pressure, thread tension, needle height, etc.

[0068] When the intelligent template machine starts, it loads the preset sewing trajectory and begins sewing. Simultaneously, a multimodal sensor is activated to collect multimodal sensing data in real time. This multimodal sensing data includes at least visual modal data, mechanical modal data, and position / motion modal data. The actual sewing trajectory can be determined from either the visual modal data or the position / motion modal data. Specifically, the coordinate set of the actual sewing needle points can be extracted from the visual modal data using an image processing model. Alternatively, the coordinate set of the actual sewing needle points can be determined based on the actual coordinates of the seam head corresponding to the position / motion modal data. Arranging the coordinate set of the actual sewing needle points in chronological order of sampling time yields the actual sewing trajectory.

[0069] The preset sewing trajectory and the actual sewing trajectory can be aligned in the time dimension by aligning the starting needle points of the preset sewing trajectory and the actual sewing trajectory. After alignment, for each sampling time t, the preset trajectory point P corresponding to the current sewing progress is obtained in the preset sewing trajectory. t,pre In the actual sewing trajectory, obtain the synchronized actual trajectory point P. t,act Calculate the preset trajectory point P corresponding to sampling time t. t,pre With respect to the actual trajectory point P t,act The coordinate deviation values ​​between the sampling points are calculated, and the average coordinate deviation value is taken as the deviation value.

[0070] Alternatively, the actual sewing trajectory and the preset sewing trajectory can be projected onto the XY plane of the intelligent template machine. The needle point coordinates in the actual sewing trajectory are connected in chronological order of sampling time to form a two-dimensional trajectory corresponding to the actual sewing trajectory. Similarly, the needle point coordinates in the preset sewing trajectory are connected in chronological order of sampling time to form a two-dimensional trajectory corresponding to the preset sewing trajectory. Since the sampling time is the same as that of the actual sewing trajectory, the two-dimensional trajectory corresponding to the actual sewing trajectory and the two-dimensional trajectory corresponding to the preset sewing trajectory should completely overlap under the condition of no error. Considering the error in the actual sewing process, the similarity between the line images of the two-dimensional trajectory corresponding to the actual sewing trajectory and the two-dimensional trajectory corresponding to the preset sewing trajectory can be calculated. The deviation value is determined based on the similarity. The smaller the similarity, the larger the deviation value, and vice versa. Furthermore, the above deviation value can be 1 minus the similarity, where the similarity is a value between 0 and 1.

[0071] Alternatively, the needle point coordinates in the actual sewing trajectory can be vector-connected according to the sampling time sequence to obtain the first vector set corresponding to the actual sewing trajectory. a 1, a 2,…, a t ,…, a T-1 The above vector connection processing can be understood as a directional connection, where, a 1 represents the first actual trajectory point P. 1,act To the second actual trajectory point p 1,act Vectors between a t Let p be the t-th actual trajectory point. t,act To the (t+1)th actual trajectory point p t+1,act Vectors between a T-1 For the (T-1)th actual trajectory point p T-1,act To the Tth actual trajectory point p T,act The vectors between them. The needle point coordinates in the actual sewing trajectory are vector-connected according to the sampling time sequence to obtain the second vector set corresponding to the preset sewing trajectory. b 1, b 2,…, b t ,…, b T-1 ),in, b 1 represents the first preset trajectory point p. 1,pre To the second preset trajectory point p 1,pre Vectors between b t Let p be the t-th preset trajectory point t,pre To the (t+1)th preset trajectory point pt+1,pre Vectors between b T-1 For the (T-1)th preset trajectory point p T-1,pre To the T-th preset trajectory point p T,pre The vector between them. The above deviation values. E It can be calculated using the following formula:

[0072]

[0073] in, For vector length weights, The vector direction weight is determined by the ratio between the minimum and maximum vector lengths of the actual sewing trajectory, while the vector length weight is determined by the maximum cosine value among the cosine values ​​of two adjacent first vectors in the actual sewing trajectory. This formula considers deviations in both vector length and vector direction, improving the accuracy of the deviation values.

[0074] After calculating the deviation value, if the deviation value is greater than the first deviation value threshold, it indicates that the actual sewing trajectory deviation is large and trajectory compensation is required. If the deviation value is less than or equal to the first deviation value threshold and greater than the second deviation value threshold, it indicates that the actual sewing trajectory has a certain deviation and dynamic parameter adjustment is required. In this case, trajectory compensation is not required. If the deviation value is less than or equal to the second deviation value threshold, it indicates that the actual sewing trajectory deviation is small or there is no deviation and no adjustment is required.

[0075] In this embodiment of the invention, when the intelligent template machine sews the garment to be sewn according to a preset sewing trajectory, it acquires multimodal perception data and the actual sewing trajectory in real time; determines the deviation value between the actual sewing trajectory and the preset sewing trajectory; if the deviation value is greater than a preset first deviation value threshold, it activates the trajectory compensation mode, generates a compensation sewing trajectory through the actual sewing trajectory, and controls the intelligent template machine to sew the garment to be sewn according to the compensation sewing trajectory; if the deviation value is less than the preset first deviation value threshold and greater than a preset second deviation value threshold, it activates the parameter adjustment mode, determines dynamic sewing parameters through multimodal perception data, and controls the intelligent template machine to sew the garment to be sewn according to the preset sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold. This invention calculates the deviation from the preset sewing trajectory by collecting multimodal sensing data and the actual sewing trajectory in real time during the sewing process. When the deviation is large, the trajectory compensation mode is activated to correct the sewing trajectory. When the deviation is neither too large nor too small, the parameter adjustment mode is activated to avoid the accumulation of deviation, thereby achieving precise, efficient and adaptive sewing control and improving garment sewing efficiency.

[0076] It is understood that in the specific implementation of this application, multimodal perception data, sewing trajectory data and other related data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use and processing of related data, as well as the training and use of various models, must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0077] Optionally, in the step of determining the deviation value between the actual sewing trajectory and the preset sewing trajectory, the actual sewing trajectory can be compared with the preset sewing trajectory to obtain a first deviation value; multimodal sensing data and the actual sewing trajectory can be input into the trajectory prediction model for trajectory prediction processing to obtain the predicted sewing trajectory corresponding to the actual sewing trajectory; the predicted sewing trajectory can be compared with the preset sewing trajectory to obtain a second deviation value; the first deviation value and the second deviation value can be fused through dynamic weights to obtain the deviation value between the actual sewing trajectory and the preset sewing trajectory, wherein the dynamic weight value is determined according to the completion degree of the actual sewing trajectory.

[0078] In this embodiment of the invention, the first deviation value is the deviation between the actual sewing trajectory and the preset sewing trajectory. This can be understood as the actual deviation value or the deviation value that has already occurred. It can be calculated using any of the above-mentioned deviation value calculation methods to obtain the first deviation value between the actual sewing trajectory and the preset sewing trajectory. The second deviation value is the deviation between the predicted sewing trajectory and the preset sewing trajectory. This can be understood as the deviation value that will occur in the future without adjustment. After obtaining the predicted sewing trajectory, the predicted sewing trajectory and the preset sewing trajectory are calculated using any of the above-mentioned deviation value calculation methods to obtain the second deviation value between the predicted sewing trajectory and the preset sewing trajectory.

[0079] The predicted sewing trajectory described above can be obtained by using a trajectory prediction model to predict the actual sewing trajectory and multimodal sensing data. This trajectory prediction model can be constructed based on models capable of extracting spatiotemporal features, such as Long Short-Term Memory (LSTM) or Transformer models. The predicted sewing trajectory is the sequence of coordinates of the expected movement path of the needle or fabric feature points over a future period starting from the current moment, output by the trajectory prediction model based on the actual sewing trajectory and multimodal sensing data.

[0080] The trajectory prediction model described above can be trained using a dataset. The dataset and the model to be trained can be prepared. The model to be trained can be a multimodal perception module combined with a main model such as a Long Short-Term Memory (LSTM) model or a Transformer model. The multimodal perception module can be a convolutional neural network module used to extract data features from different modalities, thereby obtaining feature vectors corresponding to different modalities. These feature vectors are then input into the main model for further prediction processing, and the main model outputs the predicted trajectory. The dataset includes sample data and labeled data. The sample data consists of unadjusted or uncompensated multimodal perception data (also called sample multimodal perception data) and actual sewing trajectories (also called sample actual sewing trajectories) from a previous period. The labeled data consists of actual sewing trajectories (also called label actual sewing trajectories) from a later period. The sample actual sewing trajectories and the label actual sewing trajectories are continuous in sampling time, meaning they can be connected in the time dimension. During training, multimodal sensing data of samples and actual sewing trajectories are input into the model to be trained. The multimodal sensing module extracts features from the multimodal sensing data, obtaining feature vectors corresponding to each modality. These feature vectors, along with the actual sewing trajectories of the samples, are then input into the main model for further feature extraction. An attention mechanism is used to spatiotemporally fuse the extracted feature vectors, resulting in fused features. Finally, the fused features are decoded to output the predicted trajectory. A loss function is used to calculate the error loss between the predicted trajectory and the actual sewing trajectory of the label. Minimizing this error loss is the optimization objective. The model parameters in the model to be trained are adjusted using a backpropagation algorithm. This adjustment process is iterated until the error loss reaches a preset value or the number of iterations reaches a preset number, at which point training stops, and the trained model is obtained as the trajectory prediction model. The loss function can be either the cross-entropy loss function or the squared difference loss function.

[0081] In one possible embodiment, to further improve the prediction accuracy of the trajectory prediction model, the actual deviation value between the sample preset sewing trajectory and the actual sewing trajectory of the label can be added to the dataset. The time length of the sample preset sewing trajectory is the total sampling time length of the sample actual sewing trajectory and the label actual sewing trajectory. The accuracy of the predicted trajectory is improved by improving the loss function, which is as follows:

[0082]

[0083] in, For the total loss function, For trajectory position loss, Indicates the first digit in the predicted trajectory t + h The coordinates of the predicted needle point This indicates the first line in the actual sewing trajectory of the label. t + h +1 predicted needle point coordinates. It should be noted that, in t The previous image shows the actual sewing trajectory of the sample. H To predict the total number of needle points in the trajectory; For spatiotemporal consistency loss, Indicates the first digit in the predicted trajectory i The coordinates of each needle point Indicates the first digit in the predicted trajectory j The coordinates of each needle point This indicates the first line in the actual sewing trajectory of the label. i The coordinates of each needle point This indicates the first line in the actual sewing trajectory of the label. j The coordinates of each needle point This represents the time error coefficient, which can be set based on experience; the default value is 10. Indicates the loss of consistency due to deviation. Indicating the first step in the predicted sewing trajectory i The coordinates of each needle point E This represents the actual deviation value of the sample. For trajectory curvature loss, This represents the coordinates of the three needle points connected in the predicted trajectory. To compensate for the loss weights, iterative tuning is performed during the model parameter iteration process. Using the aforementioned loss function, the relative positional relationships of the needle points in the predicted trajectory can be maintained, suppressing the inconsistencies in trajectory segmentation prediction, while ensuring that the deviation characteristics are consistent, thus improving the accuracy of the predicted trajectory.

[0084] After obtaining the first and second deviation values, they can be fused using dynamic weights to obtain the deviation between the actual sewing trajectory and the preset sewing trajectory. The dynamic weight value is determined based on the completion degree of the actual sewing trajectory. The dynamic weights include a first dynamic weight value corresponding to the first deviation value and a second dynamic weight value corresponding to the second deviation value. The higher the completion degree of the actual sewing trajectory, the larger the first dynamic weight value and the smaller the second dynamic weight value; conversely, the lower the completion degree of the actual sewing trajectory, the smaller the first dynamic weight value and the larger the second dynamic weight value. In one possible implementation of chamfering, the sum of the first and second dynamic weight values ​​is 1. The weighted sum of the first and second deviation values ​​is calculated as the deviation between the actual sewing trajectory and the preset sewing trajectory.

[0085] Optionally, before fusing the first deviation value and the second deviation value using dynamic weights, the stitch completion ratio of the actual sewing trajectory can be determined based on the number of completed stitches in the actual sewing trajectory and the total number of stitches in the preset sewing trajectory; the average coordinate deviation value between the actual coordinates of each needle point in the completed stitches of the actual sewing trajectory and the preset coordinates of the corresponding needle points in the preset sewing trajectory can be determined; an attenuation coefficient can be determined based on the average coordinate deviation value; and the dynamic weights corresponding to the first deviation value and the second deviation value can be determined based on the attenuation coefficient and the stitch completion ratio.

[0086] In this embodiment of the invention, the number of completed stitches refers to the total number of needle points in the actual sewing trajectory, and the total number of stitches in the preset sewing trajectory refers to the total number of needle points in the preset sewing trajectory. Each needle point in the completed stitches can be aligned with the corresponding needle point in the preset sewing trajectory. After alignment, for each sampling time t, the preset needle point P corresponding to the current sewing progress is obtained in the preset sewing trajectory. t,pre Obtain the actual stitch point P that is synchronized from the completed stitches. t,act Calculate the preset needle point P corresponding to sampling time t. t,pre Compared with the actual needle point P t,act The coordinate deviation values ​​between the sampling points are calculated, and the average coordinate deviation value is calculated. The larger the average coordinate deviation value, the smaller the attenuation coefficient; the smaller the average coordinate deviation value, the larger the attenuation coefficient. The attenuation coefficient is multiplied by the stitch completion ratio to obtain the first dynamic weight value. The second dynamic weight value is obtained by subtracting the first dynamic weight value from 1.

[0087] By taking into account the sewing progress and the degree of attenuation accumulation through the above dynamic weight values, the calculation of attenuation values ​​becomes more accurate.

[0088] Optionally, in the step of generating the compensation sewing trajectory through the actual sewing trajectory, the latest stitch point in the actual sewing trajectory is taken as the starting point, and the error distribution of the previous M stitch points is traced backward; according to the error distribution, a dynamic compensation sequence for the next N stitch points is generated; compensation is performed on the N stitch points after the stitch point corresponding to the starting point in the preset sewing trajectory based on the dynamic compensation sequence, and the compensation sewing trajectory is obtained.

[0089] In this embodiment of the invention, the most recent stitch point is the coordinate point formed by the most recent needle drop in the actual sewing trajectory. The error distribution of the first M stitch points is E. M ={e m |e m =a m -b m} m∈M , m represents the m-th stitch among the first M stitches.

[0090] Multiple compensation sequences can be pre-set, with each sequence corresponding to a different error distribution. A mapping table is constructed and stored to establish this mapping. Once the error distribution is obtained, the corresponding compensation sequence is searched in the mapping table. The N points following the starting point in the compensation sequence are then extracted as the dynamic compensation sequence.

[0091] After obtaining the dynamic compensation sequence, compensation is performed on N needle points after the needle point corresponding to the starting point in the preset sewing trajectory to obtain the compensated sewing trajectory.

[0092] This embodiment can extract the error vector from the current point in reverse, avoiding the introduction of outdated information, capturing the propagation trend of sudden shifts, improving the timeliness and pertinence of error analysis, and thus improving the sensitivity of the dynamic compensation sequence.

[0093] Optionally, in the step of generating the dynamic compensation sequence of the last N needle points based on the error distribution, the position error sequence of the first M needle points can be extracted; the target error propagation matrix can be determined according to the fabric material of the garment to be sewn, with different error propagation matrices corresponding to different fabric materials; and the dynamic compensation sequence of the last N needle points can be determined based on the target error propagation matrix and the position error sequence.

[0094] In this embodiment of the invention, the fabric material may include cotton, silk, chemical fiber, elastic knitwear, etc., and the error propagation matrix can be understood as an N×M order transfer matrix characterizing the positional error propagation law of the fabric material. Error propagation matrix A mat Satisfying the dynamic compensation equation: C dyn =A mat ⋅E seq Among them, C dyn For a dynamically compensated sequence, E seq For the position error sequence, E seq ={e m |e m =d(p m,act p m,pre )} m∈M d() represents the coordinate distance between two needle points. Using the material-matrix mapping table pre-stored in the controller, the corresponding error propagation matrix can be found based on the fabric material of the garment to be sewn, and used as the target error propagation matrix.

[0095] The position error sequence is matrixed to obtain:

[0096]

[0097] After obtaining the target error propagation matrix and the matrix corresponding to the position error sequence, matrix multiplication is performed to obtain the dynamic compensation sequence C. dyn .

[0098] After obtaining the dynamic compensation sequence C dyn Then, the dynamic compensation sequence C dyn The needle point is compensated to the preset sewing trajectory to obtain the compensated sewing trajectory, and the intelligent template machine then executes the sinoatrial conduction block sewing trajectory.

[0099] This embodiment can calculate the corresponding dynamic compensation sequence according to different fabric materials, and adaptively compensate for the preset sewing, thereby improving the accuracy and efficiency of garment sewing.

[0100] Optionally, in the step of determining dynamic sewing parameters through multimodal sensing data, the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle can be determined based on the multimodal sensing data; and the dynamic sewing parameters can be determined based on the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle.

[0101] In this embodiment of the invention, the aforementioned real-time layer thickness represents the actual thickness of the multiple layers of fabric stacked at the current position of the sewing needle. The real-time layer thickness can be determined by the converted distance from each pixel in the visual modal data to the camera center, or by using distance data, specifically the distance from the sewing head to the rangefinder. The real-time layer thickness is obtained by subtracting the distance from the fabric to the camera center or the rangefinder from a preset calibrated distance when there is no fabric.

[0102] The above-mentioned real-time suture tension deviation indicates the relative deviation between the real-time suture tension measurement value and the theoretical setting value. The theoretical setting value is the suture tension value corresponding to the preset sewing trajectory. A negative value indicates slackness, and a positive value indicates excessive tightness.

[0103] The aforementioned real-time texture offset angle represents the angle between the plane of the main direction of the fabric surface texture and the actual feeding direction. Generally, the angle between the plane of the main direction of the fabric surface texture and the actual feeding direction is 0, and the main direction of the fabric surface texture is a pre-set texture direction that is the same as the feeding direction.

[0104] The aforementioned dynamic sewing parameters may include presser foot pressure, new thread tension setting, XY axis servo proportional gain, and feed speed compensation coefficient.

[0105] After obtaining the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle, these parameters can be encoded to obtain their coded values. A mapping table between coded values ​​and sewing parameters can be preset, with different coded values ​​corresponding to different sewing parameters. After obtaining the coded values ​​of the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle, the corresponding sewing parameter is found in the mapping table using this coded value as the dynamic sewing parameter.

[0106] In some possible embodiments, the relationship between historical layer thickness, historical suture tension deviation, historical texture offset angle and sewing parameters can also be analyzed to construct a sewing parameter generation model. The sewing parameters corresponding to the input data are generated by the sewing parameter generation model and used as dynamic sewing parameters. The input data are real-time layer thickness, real-time suture tension deviation and real-time texture offset angle.

[0107] In this embodiment, by determining the dynamic sewing parameters, the sewing parameters of the intelligent template machine can be adaptively adjusted, further improving sewing accuracy and reducing error accumulation based on the preset sewing trajectory.

[0108] Optionally, in the step of determining the dynamic sewing parameters based on real-time layer thickness, real-time seam tension deviation, and real-time texture offset angle, a target multimodal fusion neural network can be determined according to the fabric material of the garment to be sewn, with different multimodal fusion neural networks corresponding to different fabric materials; a multimodal fusion feature vector is constructed based on real-time layer thickness, real-time seam tension deviation, and real-time texture offset angle; the multimodal fusion feature vector is input into the target multimodal fusion neural network, and the dynamic sewing parameters are output.

[0109] In this embodiment of the invention, different multimodal fusion neural networks are trained for different fabric materials. After being lightweighted, the multimodal fusion neural networks can be stored in the storage module of the intelligent template machine.

[0110] In one possible embodiment, the aforementioned lightweighting process can be to compress the multimodal fusion neural network into one or more parameter matrices, thereby achieving a nonlinear mapping between real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle to sewing parameters. The intelligent template machine or its host computer is equipped with a processor and a graphics processor to support tensor operations on the matrix.

[0111] It should be noted that the various models mentioned above are trained on a server and deployed on the intelligent template machine or its host computer after training. When the model needs to be updated, it can be updated remotely via OTA (over-the-air).

[0112] Specifically, sample data and labeled data corresponding to different fabric materials can be collected. Each set of sample data includes a triplet of data on real-time layer thickness, real-time seam tension deviation, and real-time texture offset angle. Each set of labeled data includes a quadruple of data on presser foot pressure, new seam tension setting, XY axis servo proportional gain, and feed speed compensation coefficient. The above labeled data can be obtained through debugging or expert annotation. One set of sample data corresponds to one set of labeled data, and a dataset is constructed for each fabric material. The datasets corresponding to different fabric materials only include the sample data and labeled data corresponding to that specific fabric material.

[0113] A general-purpose trainable neural network is constructed, comprising a multimodal data fusion module and a main network. The multimodal data fusion module encodes real-time layer thickness, real-time seam tension deviation, and real-time texture offset angle into the same vector space and performs channel fusion to form a three-channel feature map. The main network performs convolution operations on the feature map and finally outputs a quadruple of predicted sewing parameters. For a given dataset, sample data is input into the trainable neural network, which outputs predicted sewing parameters (in quadruple form). The predicted sewing parameters are compared with the labeled data to calculate the loss value. The network parameters are adjusted to minimize the loss value of the sewing parameters. This adjustment process is iterated until the loss value between the predicted sewing parameters and the labeled data reaches a preset value, or the number of iterations reaches a preset number. Training then stops, resulting in the multimodal fusion neural network corresponding to that dataset, which is the multimodal fusion neural network for the fabric material of that dataset.

[0114] In one possible embodiment, after obtaining the multimodal fusion neural network, the multimodal fusion neural network is compressed into one or more parameter matrices. The one or more parameter matrices are then fine-tuned using the corresponding dataset. The fine-tuning process is the same as the network parameter adjustment process, and the loss function and optimization objective are also the same. The stopping condition is when the loss value between the predicted sewing parameters and the labeled data reaches a preset value, or when the number of iterations reaches a preset number. After fine-tuning, a parameter matrix is ​​obtained that can be stored in the intelligent template machine or its host computer. Different fabric materials correspond to different parameter matrices. When using them, it is only necessary to encode the real-time layer thickness, real-time seam tension deviation, and real-time texture offset angle into matrix form, and then perform matrix operations with the corresponding parameter matrix to obtain the dynamic sewing parameters.

[0115] In this embodiment, a multimodal fusion neural network is used to perform nonlinear processing on the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle to generate dynamic sewing parameters, thus obtaining adaptive dynamic sewing parameters. By compressing the multimodal fusion neural network into a parameter matrix, the computational and storage requirements can be reduced, and the generation speed of dynamic sewing parameters can be improved.

[0116] like Figure 2 As shown, this embodiment of the invention provides an adaptive control device for an intelligent template machine based on multimodal perception. This adaptive control device for an intelligent template machine based on multimodal perception includes:

[0117] The acquisition module 201 is used to acquire multimodal perception data and the actual sewing trajectory in real time when the intelligent template machine sews the garment to be sewn according to the preset sewing trajectory.

[0118] Processing module 202 is used to determine the deviation value between the actual sewing trajectory and the preset sewing trajectory;

[0119] The first control module 203 is used to activate the trajectory compensation mode if the deviation value is greater than the preset first deviation value threshold, generate a compensation sewing trajectory through the actual sewing trajectory, and control the intelligent template machine to sew the garment to be sewn with the compensation sewing trajectory.

[0120] The second control module 204 is used to activate the parameter adjustment mode if the deviation value is less than a preset first deviation value threshold and greater than a preset second deviation value threshold, determine the dynamic sewing parameters through the multimodal sensing data, and control the intelligent template machine to sew the garment to be sewn based on the preset sewing and the dynamic sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold.

[0121] Optionally, the processing module 202 is further configured to compare the actual sewing trajectory with the preset sewing trajectory to obtain a first deviation value; input the multimodal sensing data and the actual sewing trajectory into a trajectory prediction model for trajectory prediction processing to obtain a predicted sewing trajectory corresponding to the actual sewing trajectory; compare the predicted sewing trajectory with the preset sewing trajectory to obtain a second deviation value; and fuse the first deviation value and the second deviation value through dynamic weights to obtain a deviation value between the actual sewing trajectory and the preset sewing trajectory, wherein the dynamic weight value is determined based on the completion degree of the actual sewing trajectory.

[0122] Optionally, the processing module 202 is further configured to: determine the stitch completion ratio of the actual sewing trajectory based on the number of completed stitches in the actual sewing trajectory and the total number of stitches in the preset sewing trajectory; determine the average coordinate deviation between the actual coordinates of each stitch point in the completed stitches of the actual sewing trajectory and the preset coordinates of the corresponding stitch point in the preset sewing trajectory; determine the attenuation coefficient based on the average coordinate deviation; and determine the dynamic weights corresponding to the first deviation value and the second deviation value based on the attenuation coefficient and the stitch completion ratio.

[0123] Optionally, the first control module 203 is further configured to trace the error distribution of the previous M needle points backwards from the latest needle point in the actual sewing trajectory as the starting point; generate a dynamic compensation sequence for the next N needle points based on the error distribution; and compensate for the N needle points after the needle point corresponding to the starting point in the preset sewing trajectory based on the dynamic compensation sequence to obtain a compensated sewing trajectory.

[0124] Optionally, the first control module 203 is further configured to extract the position error sequence of the first M needle points; determine the target error propagation matrix according to the fabric material of the garment to be sewn, with different error propagation matrices corresponding to different fabric materials; and determine the dynamic compensation sequence of the next N needle points based on the target error propagation matrix and the position error sequence.

[0125] Optionally, the second control module 204 is further configured to determine the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle based on the multimodal sensing data; and to determine the dynamic sewing parameters based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle.

[0126] Optionally, the second control module 204 is further configured to determine a target multimodal fusion neural network based on the fabric material of the garment to be sewn, with different fabric materials corresponding to different multimodal fusion neural networks; construct a multimodal fusion feature vector based on the real-time layer thickness, the real-time seam tension deviation, and the real-time texture offset angle; input the multimodal fusion feature vector into the target multimodal fusion neural network, and output dynamic sewing parameters.

[0127] It should be noted that the intelligent template machine adaptive control device based on multimodal perception provided in this embodiment of the invention can be applied to computers, servers and other equipment that can perform intelligent template machine adaptive control methods based on multimodal perception.

[0128] The intelligent template machine adaptive control device based on multimodal perception provided in this embodiment of the invention can realize all the processes implemented by the intelligent template machine adaptive control method based on multimodal perception in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0129] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program for an intelligent template machine adaptive control method based on multimodal perception, stored in the memory 302 and executable on the processor 301, wherein:

[0130] The processor 301 is used to call the computer program stored in the memory 302 and perform the following steps:

[0131] When the intelligent template machine sews the garment to be sewn according to the preset sewing trajectory, it acquires multimodal perception data and the actual sewing trajectory in real time.

[0132] The deviation value between the actual sewing trajectory and the preset sewing trajectory is determined;

[0133] If the deviation value is greater than the preset first deviation value threshold, the trajectory compensation mode is activated, a compensation sewing trajectory is generated based on the actual sewing trajectory, and the intelligent template machine is controlled to sew the garment to be sewn using the compensation sewing trajectory.

[0134] If the deviation value is less than a preset first deviation value threshold and greater than a preset second deviation value threshold, then the parameter adjustment mode is activated, the dynamic sewing parameters are determined by the multimodal sensing data, and the intelligent template machine is controlled to sew the garment to be sewn based on the preset sewing and the dynamic sewing parameters, wherein the first deviation value threshold is greater than the second deviation value threshold.

[0135] Optionally, the step of determining the deviation value between the actual sewing trajectory and the preset sewing trajectory, performed by the processor 301, includes:

[0136] The actual sewing trajectory is compared with the preset sewing trajectory to obtain a first deviation value;

[0137] The multimodal sensing data and the actual sewing trajectory are input into the trajectory prediction model for trajectory prediction processing to obtain the predicted sewing trajectory corresponding to the actual sewing trajectory.

[0138] The predicted sewing trajectory is compared with the preset sewing trajectory to obtain a second deviation value;

[0139] The first deviation value and the second deviation value are fused together using dynamic weighting to obtain the deviation value between the actual sewing trajectory and the preset sewing trajectory. The dynamic weighting value is determined based on the completion degree of the actual sewing trajectory.

[0140] Optionally, before the step of fusing the first deviation value and the second deviation value using dynamic weights, the method executed by the processor 301 further includes:

[0141] Based on the number of completed stitches in the actual sewing trajectory and the total number of stitches in the preset sewing trajectory, the stitch completion ratio of the actual sewing trajectory is determined.

[0142] The average coordinate deviation between the actual coordinates of each stitch point in the completed stitches of the actual sewing trajectory and the preset coordinates of the corresponding stitch points of the preset sewing trajectory.

[0143] The attenuation coefficient is determined based on the average coordinate deviation value;

[0144] Based on the attenuation coefficient and the stitch completion ratio, the dynamic weights corresponding to the first deviation value and the second deviation value are determined.

[0145] Optionally, the step of generating a compensated sewing trajectory based on the actual sewing trajectory, performed by processor 301, includes:

[0146] Starting from the latest stitch point in the actual sewing trajectory, trace back the error distribution of the previous M stitch points.

[0147] Based on the error distribution, a dynamic compensation sequence for the last N needle points is generated;

[0148] The compensated sewing trajectory is obtained by compensating the N needle points following the needle point corresponding to the starting point in the preset sewing trajectory based on the dynamic compensation sequence.

[0149] Optionally, the step of generating the dynamic compensation sequence for the next N needle points based on the error distribution, performed by processor 301, includes:

[0150] Extract the position error sequence of the first M needle points;

[0151] Based on the fabric material of the garment to be sewn, the target error propagation matrix is ​​determined, with different fabric materials corresponding to different error propagation matrices;

[0152] Based on the target error propagation matrix and the position error sequence, the dynamic compensation sequence for the last N needle points is determined.

[0153] Optionally, the step of determining dynamic sewing parameters using the multimodal sensing data, performed by processor 301, includes:

[0154] Based on the multimodal sensing data, the real-time layer thickness, real-time suture tension deviation, and real-time texture offset angle are determined.

[0155] Based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle, dynamic sewing parameters are determined.

[0156] Optionally, the step of determining the dynamic sewing parameters based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle, performed by the processor 301, includes:

[0157] Based on the fabric material of the garment to be sewn, a target multimodal fusion neural network is determined, with different fabric materials corresponding to different multimodal fusion neural networks;

[0158] Based on the real-time layer thickness, the real-time suture tension deviation, and the real-time texture offset angle, a multimodal fusion feature vector is constructed.

[0159] The multimodal fusion feature vector is input into the target multimodal fusion neural network, and the dynamic sewing parameters are output.

[0160] It should be noted that the electronic device provided in the embodiments of the present invention can be applied to computers, servers and other devices that can perform intelligent template machine adaptive control methods based on multimodal perception.

[0161] The electronic device provided in this embodiment of the invention can implement all the processes of the intelligent template machine adaptive control method based on multimodal perception in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, further details are omitted here.

[0162] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the intelligent template machine adaptive control method based on multimodal perception provided in this invention, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0163] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The computer-readable storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0164] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A multi-modal perception based intelligent template machine adaptive control method, characterized in that, The method comprises the following steps: When the intelligent template machine sews the garment to be sewn through a preset sewing track, real-time multi-modal sensing data and an actual sewing track during sewing are acquired; A deviation value between the actual sewing track and the preset sewing track is determined; If the deviation value is greater than a preset first deviation value threshold, a track compensation mode is started, a compensation sewing track is generated through the actual sewing track, and the intelligent template machine is controlled to sew the garment to be sewn through the compensation sewing track; If the deviation value is less than the preset first deviation value threshold and greater than a preset second deviation value threshold, a parameter adjustment mode is started, dynamic sewing parameters are determined through the multi-modal sensing data, and the intelligent template machine is controlled to sew the garment to be sewn through the dynamic sewing parameters on the basis of the preset sewing track, wherein the first deviation value threshold is greater than the second deviation value threshold; The step of generating the compensation sewing track through the actual sewing track comprises: Taking the latest needle point in the actual sewing track as a starting point, error distribution of the previous M needle points is traced in reverse; According to the error distribution, a dynamic compensation sequence of the N subsequent needle points is generated; specifically comprising: extracting a position error sequence of the first M needle points; determining a target error propagation matrix according to the material of the garment to be sewn, different materials corresponding to different error propagation matrices; determining a dynamic compensation sequence of the N subsequent needle points based on the target error propagation matrix and the position error sequence; error propagation matrix A mat satisfies the dynamic compensation equation: C dyn =A mat E seq , wherein C dyn is the dynamic compensation sequence, E seq is the position error sequence, E seq ={e m |e m =d(p m,act , p m,pre )} m∈M , d() represents the coordinate distance between two needle points; According to the dynamic compensation sequence and N needle points after the needle point corresponding to the starting point in the preset sewing track, compensation is performed to obtain a compensation sewing track.

2. The multi-modal perception based intelligent template machine adaptive control method of claim 1, wherein, The step of determining the deviation value between the actual sewing track and the preset sewing track comprises: The actual sewing track is compared with the preset sewing track to obtain a first deviation value; The multi-modal sensing data and the actual sewing track are input into a track prediction model for track prediction processing to obtain a predicted sewing track corresponding to the actual sewing track; The predicted sewing track is compared with the preset sewing track to obtain a second deviation value; The first deviation value and the second deviation value are fused through a dynamic weight to obtain the deviation value between the actual sewing track and the preset sewing track, and the dynamic weight is determined according to the completion degree of the actual sewing track.

3. The multi-modal perception based intelligent template machine adaptive control method of claim 2, wherein, Before the step of fusing the first deviation value and the second deviation value through the dynamic weight, the method further comprises: According to the number of completed stitches of the actual sewing track and the total number of stitches of the preset sewing track, a stitch completion ratio of the actual sewing track is determined; According to the average coordinate deviation value between the actual coordinates of each needle point in the completed stitches in the actual sewing track and the preset coordinates of the corresponding needle point in the preset sewing track; According to the average coordinate deviation value, an attenuation coefficient is determined; Based on the attenuation coefficient and the stitch completion ratio, a dynamic weight corresponding to the first deviation value and the second deviation value is determined.

4. The multimodal perception based intelligent stencil machine adaptive control method according to any one of claims 1 to 3, characterized in that, The step of determining the dynamic sewing parameters through the multi-modal sensing data comprises: Based on the multi-modal sensing data, a real-time layer thickness, a real-time thread tension deviation, and a real-time texture offset angle are determined; Based on the real-time layer thickness, the real-time thread tension deviation, and the real-time texture offset angle, dynamic sewing parameters are determined.

5. The multi-modal perception based intelligent template machine adaptive control method of claim 4, wherein, The step of determining the dynamic sewing parameter based on the real-time layer thickness, the real-time thread tension deviation, and the real-time texture offset angle comprises: According to the fabric material of the garment to be sewn, a target multi-modal fusion neural network is determined, different fabric materials corresponding to different multi-modal fusion neural networks; Based on the real-time layer thickness, the real-time thread tension deviation, and the real-time texture offset angle, a multi-modal fusion feature vector is constructed; The multi-modal fusion feature vector is input into the target multi-modal fusion neural network, and a dynamic sewing parameter is output.

6. A multi-modal perception based intelligent stencil machine adaptive control apparatus, characterized in that, The intelligent template machine adaptive control device based on multi-modal perception comprises: An acquisition module is configured to acquire multi-modal perception data and an actual sewing trajectory in real time when the intelligent template machine sews a garment to be sewn according to a preset sewing trajectory; A processing module is configured to determine a deviation value between the actual sewing trajectory and the preset sewing trajectory; A first control module is configured to start a trajectory compensation mode if the deviation value is greater than a preset first deviation value threshold, generate a compensation sewing trajectory based on the actual sewing trajectory, and control the intelligent template machine to sew the garment to be sewn according to the compensation sewing trajectory; A second control module is configured to start a parameter adjustment mode if the deviation value is less than the preset first deviation value threshold and greater than a preset second deviation value threshold, determine a dynamic sewing parameter based on the multi-modal perception data, and control the intelligent template machine to sew the garment to be sewn according to the dynamic sewing parameter based on the preset sewing trajectory, wherein the first deviation value threshold is greater than the second deviation value threshold; The step of generating a compensation sewing trajectory based on the actual sewing trajectory comprises: Taking the latest needle point in the actual sewing trajectory as a starting point, the error distribution of the previous M needle points is traced in reverse; According to the error distribution, a dynamic compensation sequence of the N subsequent needle points is generated; specifically comprising: extracting a position error sequence of the first M needle points; determining a target error propagation matrix according to the material of the garment to be sewn, different materials corresponding to different error propagation matrices; determining a dynamic compensation sequence of the N subsequent needle points based on the target error propagation matrix and the position error sequence; error propagation matrix A mat satisfying the dynamic compensation equation: C dyn =A mat E seq , wherein C dyn is the dynamic compensation sequence, E seq is the position error sequence, E seq ={e m |e m =d(p m,act ,p m,pre )} m∈M , d() represents the coordinate distance between two needle points; According to the dynamic compensation sequence and the N needle points after the needle point corresponding to the starting point in the preset sewing trajectory, a compensation sewing trajectory is obtained.

7. An electronic device, comprising: It comprises: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the intelligent template machine adaptive control method based on multi-modal perception in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the intelligent template machine adaptive control method based on multi-modal perception in any one of claims 1 to 5.

Images