Textile fabric identification method and device based on multi-modal model, equipment and storage medium

Abstract

The invention discloses a textile fabric identification method and device based on a multi-modal model, equipment and a storage medium, and relates to the technical field of textile industry and computer vision. The multi-modal model-based textile fabric identification method comprises the following steps: acquiring a multi-view fabric image, a fabric language text corresponding to the multi-view fabric image, a fabric color and a fabric database; inputting the multi-view fabric image, the fabric language text and the fabric color into a target multi-modal model to obtain fusion features; calculating the similarity between the fusion features and the fusion features in the fabric database; reordering the fusion features in the fabric database according to the similarity to obtain a reordering result; and obtaining a fabric category according to the reordering result, and completing fabric identification. According to the method, the specific category of the input fabric can be quickly and accurately identified from the fabric database, and the accuracy and efficiency of textile fabric identification are improved.

Description

Technical Field

[0001] The present application relates to the field of textile industry and computer vision technology, and in particular to a textile fabric recognition method, device, equipment and storage medium based on a multimodal model. Background Art

[0002] With the development of automation and intelligence, textile fabric recognition technology has become increasingly important in the textile industry. This technology is not only related to production efficiency, but also directly affects product quality and market competitiveness. Traditional fabric recognition methods mainly rely on manual visual inspection, which is inefficient and easily affected by subjective factors, resulting in inconsistent recognition results. Therefore, the development of efficient and accurate fabric recognition methods has become an urgent need in the industry.

[0003] In recent years, fabric recognition methods based on computer vision and image processing technology have gradually emerged. These methods collect fabric images and use image processing algorithms to extract feature information to achieve automatic classification and recognition. For example, some deep learning networks are used to extract fabric recognition features and perform retrieval. However, these existing methods often rely on single-modal information and usually only use images from a single perspective for recognition.

[0004] Existing single-modality recognition methods perform poorly in complex environments, especially when the fabric material similarity is high or the detailed features are difficult to capture, and the recognition accuracy is limited. In addition, the existing approach lacks comprehensive consideration of the front and back features of the fabric and cannot fully utilize the information of multi-view images, thereby reducing the accuracy and robustness of recognition. Therefore, how to improve the accuracy and efficiency of textile fabric recognition has become an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The purpose of this application is to provide a textile fabric recognition method, device, equipment and storage medium based on a multimodal model, aiming to solve the technical problem of how to improve the accuracy and efficiency of textile fabric recognition.

[0007] To achieve the above objectives, the present application proposes a textile fabric recognition method based on a multimodal model, the method comprising:

[0008] Acquire multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and a fabric database;

[0009] Inputting the multi-view fabric image, the fabric language text, and the fabric color into a target multimodal model to obtain fusion features;

[0010] Calculating the similarity between the fused feature and the fused feature in the fabric database;

[0011] Reordering the fusion features in the fabric database according to the similarity to obtain a reordering result;

[0012] The fabric category is obtained according to the reordering result, and fabric identification is completed.

[0013] In one embodiment, the textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, wherein the fabric recognition device includes at least two detection cameras, and the focal lengths of the detection cameras are different;

[0014] The step of acquiring a multi-view fabric image comprises:

[0015] Acquire multiple groups of fabric images, each of which is captured by the detection camera, and each group of fabric images corresponds to each of the detection cameras one by one;

[0016] A multi-view fabric image is determined according to each of the fabric images.

[0017] In one embodiment, the textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, the fabric recognition device includes a first camera, a second camera, a third camera and a fourth camera, the focal length of the first camera is greater than the focal length of the second camera, the focal length of the second camera is greater than the focal length of the third camera, and the focal length of the third camera is greater than the focal length of the fourth camera;

[0018] The step of acquiring a multi-view fabric image comprises:

[0019] A multi-perspective fabric image is obtained according to the first-perspective fabric image, the second-perspective fabric image, the third-perspective fabric image and the fourth-perspective fabric image captured by the first camera, the second camera, the third camera and the fourth camera, wherein the first perspective is smaller than the second perspective, the second perspective is smaller than the third perspective, and the third perspective is smaller than the fourth perspective.

[0020] In one embodiment, the multi-view fabric image includes a front multi-view fabric image and a back multi-view fabric image, the target multimodal model includes a multimodal feature encoder and a multimodal feature fusion device, the multimodal feature encoder includes a feature embedding layer, an attention layer and a fully connected layer, and the step of inputting the multi-view fabric image, the fabric language text and the fabric color into the target multimodal model to obtain the fusion feature includes:

[0021] Inputting the front multi-view fabric image, the back multi-view fabric image, the fabric language text and the fabric color into the multi-modal feature encoder in sequence, and extracting the underlying features of the multi-view fabric image and the fabric language text through the feature embedding layer;

[0022] Downsampling the underlying features through the attention layer and reducing the feature dimension to obtain downsampled features;

[0023] Extracting high-level semantic features from the downsampled features through the fully connected layer to obtain deep features;

[0024] The deep features are fused through the multimodal feature fuser to obtain fused features.

[0025] In one embodiment, before the step of inputting the multi-view fabric image, the fabric language text, and the fabric color into the target multimodal model to obtain the fusion feature, the step further includes:

[0026] Obtain a target fabric dataset, and divide the target fabric dataset into a training set and a test set;

[0027] Using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model;

[0028] Testing and verifying the target detection model using the test set to obtain a test result;

[0029] When the test result does not meet the preset requirements, returning to the step of training and adjusting the initial multimodal model using the training set;

[0030] When the test result meets the preset requirement, the reference multimodal model is used as the target multimodal model.

[0031] In one embodiment, the step of obtaining the target fabric data set includes:

[0032] Obtaining initial fabric data set and historical fabric data;

[0033] Evaluating and screening the initial fabric data set to obtain data to be labeled;

[0034] Comparing the data to be labeled with the historical fabric data to obtain similarity;

[0035] The data to be labeled are sorted according to the similarity, and the data to be labeled whose similarity is lower than a preset similarity threshold is filtered to obtain a target fabric data set.

[0036] In one embodiment, the training set includes image data, category data, fabric description language text data, and fabric color data; the image data includes front-back multi-view image data, front multi-view image data, back multi-view image data, multi-view image data lacking the second and third views, and first-view front image data; the initial multimodal model includes a multimodal feature encoder, a multimodal feature fuser, and a multimodal feature classifier; the step of using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model includes:

[0037] Input the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third perspectives, or the first perspective front image data into the multimodal feature encoder to obtain the fabric multimodal depth feature;

[0038] The multimodal depth features of the fabric are fused by the multimodal feature fuser to obtain multimodal fusion features of the fabric;

[0039] Learning and distinguishing the multimodal fusion features of the fabric by the multimodal feature classifier to obtain the fabric category;

[0040] When the fabric category does not match the category data, the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier are adjusted, and the step of inputting the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third perspectives, or the first perspective front image data into the multimodal feature encoder is returned to obtain the fabric multimodal depth feature;

[0041] When the fabric category matches the category data, a reference multimodal model is obtained according to the multimodal feature encoder, the multimodal feature fuser and the multimodal feature classifier.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a textile fabric identification device based on a multimodal model, the device comprising:

[0043] A data acquisition module, used to acquire multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and a fabric database;

[0044] A model processing module, used for inputting the multi-view fabric image, the fabric language text and the fabric color into a target multimodal model to obtain a fusion feature;

[0045] A similarity calculation module, used for calculating the similarity between the fused feature and the fused feature in the fabric database;

[0046] A reordering module, used for reordering the fusion features in the fabric database according to the similarity to obtain a reordering result;

[0047] The fabric identification module is used to obtain the fabric category according to the reordering result and complete fabric identification.

[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a textile fabric identification device based on a multimodal model, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the textile fabric identification method based on the multimodal model as described above.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the textile fabric identification method based on the multimodal model as described above are implemented.

[0050] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the textile fabric identification method based on the multimodal model as described above.

[0051] One or more technical solutions proposed in this application have at least the following technical effects:

[0052] Obtain multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and fabric databases; input the multi-view fabric images, fabric language texts, and fabric colors into the target multimodal model to obtain fusion features; calculate the similarity between the fusion features and the fusion features in the fabric database; reorder the fusion features in the fabric database according to the similarity to obtain reordering results; obtain fabric categories according to the reordering results to complete fabric recognition. Multi-view images of the front and back of the fabric are obtained through multiple cameras to fully capture its features, and are input into the multimodal model. After processing through convolution, pooling, and fully connected layers, a fusion feature vector is generated, and visual information is converted into deep features, thereby improving recognition accuracy and efficiency. The system then calculates the similarity between the input fabric features and the database sample features, reorders the samples accordingly, and puts the most matching fabric in the front row, thereby quickly determining the category of the input fabric and completing the recognition process. This process can quickly and accurately identify the specific category of the input fabric from the fabric database, and maintain high accuracy even in complex environments or when facing similar fabrics. The entire fabric recognition process not only improves the accuracy and efficiency of textile fabric recognition, but also reduces the errors caused by human factors, providing technical support for the automated production process of the textile industry. In addition, it also has good flexibility and generalization capabilities, providing strong technical support for fabric quality control, inventory management and personalized customization. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0055] Figure 1 A flowchart diagram of a first embodiment of a textile fabric identification method based on a multimodal model of the present application is provided;

[0056] Figure 2 Schematic diagrams of the front and back of a textile fabric identification device provided in Embodiment 1 of the textile fabric identification method based on a multimodal model of the present application;

[0057] Figure 3 A schematic diagram of a multi-view fabric image provided in Example 1 of a textile fabric recognition method based on a multimodal model of the present application;

[0058] Figure 4A schematic diagram of front and back fabric images provided in Example 1 of the textile fabric recognition method based on a multimodal model of the present application;

[0059] Figure 5 A flow chart of a second embodiment of a textile fabric identification method based on a multimodal model of the present application;

[0060] Figure 6 A schematic diagram of fabric color recognition provided in Example 2 of the textile fabric recognition method based on a multimodal model of the present application;

[0061] Figure 7 A schematic diagram of the initial multimodal model structure provided for Embodiment 2 of the textile fabric identification method based on the multimodal model of the present application;

[0062] Figure 8 A schematic diagram of initial multimodal model training provided for Embodiment 2 of the textile fabric recognition method based on a multimodal model of the present application;

[0063] Fig. 9 This is a schematic diagram of the module structure of a textile fabric identification device based on a multimodal model according to an embodiment of the present application;

[0064] Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the textile fabric identification method based on the multimodal model in the embodiment of the present application.

[0065] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0066] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0067] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0068] With the advancement of automation and intelligence, textile fabric recognition technology is crucial to improving production efficiency, product quality and market competitiveness. Traditional manual inspection methods are inefficient and susceptible to subjective influences, so the industry urgently needs efficient and accurate recognition methods. Currently, computer vision-based fabric recognition technology is developing. Although it can automatically classify fabrics through image processing, most of them rely on single-view images and have limited accuracy in complex situations. In addition, these technologies fail to fully consider the front and back features of the fabric and multi-view information, resulting in less than ideal recognition results.

[0069] The main solution of the embodiment of the present application is to obtain multi-view images of the front and back of the fabric through multiple cameras to fully capture its features, and input them into the multimodal model, and generate a fused feature vector through convolution, pooling and fully connected layer processing, and convert visual information into deep features to improve recognition accuracy and efficiency. The system then calculates the similarity between the input fabric features and the database sample features, and reorders the samples accordingly, putting the most matching fabric in the front row, so as to quickly determine the category of the input fabric and complete the recognition process.

[0070] It should be noted that the execution subject of the embodiment of the present application can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a fabric identification device, a fabric identification system, etc. that can realize the above functions. The fabric identification system is taken as an example to illustrate this embodiment and the following embodiments.

[0071] Based on this, the embodiment of the present application provides a textile fabric identification method based on a multimodal model, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the textile fabric identification method based on the multimodal model of the present application.

[0072] In this embodiment, the textile fabric identification method based on the multimodal model includes steps S10 to S50:

[0073] Step S10, obtaining a multi-view fabric image, a fabric language text corresponding to the multi-view fabric image, a fabric color, and a fabric database;

[0074] It should be noted that multi-view fabric images refer to fabric images taken from different angles and distances (focal lengths). Specifically, they include using cameras with different focal lengths to collect large-view, medium-view, and small-view images of fabrics. These images correspond to different actual physical areas of the fabrics and contain content information of different ranges and details: large-view fabric images: corresponding to a larger actual physical area, containing a larger range of fabric content image information, suitable for capturing the overall pattern and structure of the fabric; medium-view fabric images: corresponding to the actual physical area of ​​the medium surface, containing a medium range of fabric content image information, used to supplement the information between large-view and small-view images; small-view fabric images: corresponding to a smaller actual physical area, containing detailed image information of the fabric, suitable for capturing the micro-texture and detailed features of the fabric; in addition, the front and back images of the fabric are also considered, because the front and back features of different fabrics may be quite different, and collecting the front and back images of the fabric at the same time can more accurately determine the category of the fabric. Fabric language text refers to fabric attribute information described in natural language, which includes detailed parameters such as fabric composition, texture, yarn state, and post-processing process. This information is usually provided by industry experts or system users based on the characteristics and classification standards of fabrics to explain in detail the physical and chemical properties of fabrics and help understand and identify the types and characteristics of fabrics. Different fabrics are classified according to the parameters of textile fabrics, including parameters such as: fabric composition categories, fabric composition subcategories, fabric textures, yarn states, post-processing processes, fabric composition, etc. The classification of fabric parameters requires strict industry and professional knowledge. The following are examples of some parameters: fabric composition categories, such as cotton, chemical fiber, linen, artificial leather, etc.; fabric composition subcategories, such as single silk cotton, golden cotton, washed cotton, etc.; fabric textures, such as woven-plain, woven-random, knitted-plain, knitted-mesh, etc.; yarn states, such as single yarn, double yarn, spliced ​​yarn, etc.; post-processing processes, such as surface sanding, fleece, hot stamping, printing, washing, etc.; fabric composition, such as cotton, polyester, cotton + modal, etc. A fabric can be described using text, for example: a picture of a synthetic satin double-sided silk, woven-satin, high-elastic, dyed, polyester fabric. Usually when searching, users only focus on some parameters of the fabric, such as cotton, washed, etc. Just enter the text that you are interested in. Fabric color refers to the specific value of the fabric color obtained using a colorimeter, usually expressed in Lab color space. Lab color space is a color space, where L represents lightness (brightness), A represents the component from green to red, and B represents the component from blue to yellow. These color values ​​can accurately describe the color characteristics of the fabric, providing a quantitative data basis for color matching and identification of fabrics.A fabric database refers to a data set that collects and organizes a large number of fabric samples. These samples include images of different fabrics and other related parameters. The establishment of a fabric database involves the following aspects: Collection of fabric samples: Collecting images of various fabrics, including images from different perspectives and front and back; Recording of fabric parameters: Recording various parameters of fabrics, such as major component categories, subcategories, textures, yarn states, yarn counts, spinning processes, finishing processes, fiber morphology, gloss, smoothness, fiber length, etc.; Fabric classification: Dividing fabrics into different categories according to their parameters, providing a basis for subsequent fabric identification and search and matching; Extraction of feature vectors: Using a multimodal model to extract feature vectors from fabric images in the database, and saving these feature vectors in the database for efficient search and matching.

[0075] It is understandable that the same fabric needs to be photographed with cameras of different focal lengths to obtain images of large, medium and small viewing angles. This is done to capture the overall pattern, medium-range details and micro-texture of the fabric, ensuring that the characteristics of the fabric can be fully recorded. At the same time, the front and back sides of the fabric are also photographed, because the characteristics of the front and back sides of the fabric may be different, which is crucial for accurately identifying the fabric category. In addition, each fabric image from each viewing angle corresponds to a detailed language text description, which covers key attributes such as the composition, texture, and yarn state of the fabric, providing rich semantic information for the fabric. In addition, the color of the fabric is obtained through a professional colorimeter and accurately recorded in the form of numerical values ​​in the Lab color gamut to ensure color accuracy and consistency. All of these data, including multi-view images, language text descriptions and color values, as well as the existing fabric database, together constitute a comprehensive dataset. This dataset not only contains the visual information of the fabric, but also includes the detailed attributes and historical data of the fabric, providing a solid foundation for subsequent model training and fabric recognition, making the entire recognition process more accurate and efficient.

[0076] As an example, the textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, and the fabric recognition device includes at least two detection cameras, and the focal lengths of the detection cameras are different; the step of obtaining a multi-perspective fabric image includes: obtaining multiple groups of fabric images, each of the fabric images is collected by the detection camera, and each group of fabric images corresponds one-to-one to each of the detection cameras; and determining a multi-perspective fabric image based on each of the fabric images.

[0077] Fabric recognition equipment refers to an automated machine or system specifically designed to identify different fabrics. This equipment usually contains multiple components, such as cameras, lighting systems, image processing units, and possible robotic arms or other operating components, which are used to capture fabric images and analyze and identify them. Detection cameras refer to cameras used to capture fabric images in fabric recognition equipment. These cameras are installed on the equipment to obtain visual data of fabrics for subsequent image processing and analysis. Focal length refers to the optical parameters of the camera lens, which determines how far the camera can "see" objects and the degree of image magnification. Different focal lengths affect the viewing angle range and detail level of the camera's captured images. For example, a wide-angle lens (short focal length) can capture a wider field of view, while a telephoto lens (long focal length) can capture images that are farther away or more detailed. Multiple sets of fabric images refer to multiple images of the same fabric taken by detection cameras with different focal lengths. These images capture the fabric from different perspectives and distances to ensure that the fabric's characteristics, including macroscopic patterns, textures, and microscopic details, can be fully recorded. Each set of fabric images corresponds to a specific detection camera, which ensures that visual information of the fabric is obtained from different angles and focal lengths, providing rich data input for the multimodal model.

[0078] First, each detection camera collects images of the same fabric to form multiple sets of fabric images. Each set of images corresponds to a specific camera, ensuring that different perspectives from wide angle to telephoto can be captured. Then, the images collected by these cameras with different focal lengths are combined to form multi-perspective fabric images, which contain the macro features and micro details of the fabric, providing rich visual data for the subsequent multimodal model, so that the model can more accurately identify and classify different textile fabrics. In this way, the fabric recognition device can fully capture the characteristics of the fabric and improve the accuracy and robustness of recognition.

[0079] As an example, the textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, and the fabric recognition device includes a first camera, a second camera, a third camera and a fourth camera, the focal length of the first camera is greater than the focal length of the second camera, the focal length of the second camera is greater than the focal length of the third camera, and the focal length of the third camera is greater than the focal length of the fourth camera; the step of obtaining a multi-perspective fabric image includes: obtaining a multi-perspective fabric image according to the first-perspective fabric image, the second-perspective fabric image, the third-perspective fabric image and the fourth-perspective fabric image collected by the first camera, the second camera, the third camera and the fourth camera, the first perspective is smaller than the second perspective, the second perspective is smaller than the third perspective, and the third perspective is smaller than the fourth perspective.

[0080] The first camera, the second camera, the third camera, and the fourth camera refer to the four cameras fixedly configured on the fabric recognition device, which are all at the same distance from the fabric but have different focal lengths. The first camera has the longest focal length, which is suitable for capturing the macro features of the fabric; the second camera has a slightly shorter focal length, which is used to capture the medium-range features of the fabric; the third camera has an even shorter focal length, which is used to capture more detailed features of the fabric; and the fourth camera has the shortest focal length, which is used to capture the smallest viewing angle range of the fabric, but with the richest details. The first-perspective fabric image, the second-perspective fabric image, the third-perspective fabric image, and the fourth-perspective fabric image refer to the fabric images collected by the above four cameras respectively, and each camera corresponds to an image of a perspective. Since all cameras are at the same distance from the fabric but have different focal lengths, the viewing angle range and degree of detail of the fabric images they capture are also different. The first-perspective fabric image shows the largest viewing angle range, which may include the overall pattern and structure of the fabric; the second-perspective fabric image shows a slightly smaller viewing angle range, which may include the medium-scale pattern and some details of the fabric; the third-perspective fabric image shows an even smaller viewing angle range, which may include the local pattern and more details of the fabric; the fourth-perspective fabric image shows the smallest viewing angle range, which may include the micro-texture and the richest details of the fabric. By combining these images from different perspectives, a comprehensive multi-perspective fabric image can be constructed, providing rich visual data for the multimodal model, thereby improving the accuracy and robustness of fabric recognition.

[0081] Please refer to Figure 2 , Figure 2 A schematic diagram of the front and back sides of a textile fabric recognition device provided in the first embodiment of the textile fabric recognition method based on a multimodal model of this application shows the configuration and operation interface of the textile fabric recognition device when shooting the front and back images of the fabric. The device is equipped with a double-sided shooting function (not to say that the front and back are shot at the same time), which can be combined with the front and back of the fabric for identification, ensuring that comprehensive information of the fabric is obtained from two directions. In addition, the device also provides a single-sided shooting option for shooting the front or back of the fabric separately for identification to meet different identification needs. The user can accurately control the shooting process through the manual shooting mode to ensure image quality. The device also has 2x and 5x image magnification functions, allowing the user to magnify the image as needed to observe the detailed features of the fabric more closely. The integration of these functions enables the fabric recognition device to flexibly adapt to various fabric recognition scenarios, improving the accuracy and efficiency of recognition. In the multimodal model training mentioned in the briefing book, these front and back images will be used as important input data to help the model learn and identify the characteristics of the fabric, so as to achieve efficient and accurate fabric recognition in practical applications.

[0082] Please refer to Figure 3 , Figure 3A schematic diagram of multi-view fabric images provided for the first embodiment of the textile fabric recognition method based on a multimodal model of this application shows multimodal images of the same fabric at different viewing angles, which identify fabric categories. The large-view image, i.e., the first viewing angle, provides the overall appearance of the fabric, which helps the model understand the macro texture and pattern of the fabric; the medium-view image, corresponding to the second and third viewing angles, reveals the medium-scale features of the fabric, such as the details of the texture and the structure of the yarn; and the small-view image, i.e., the fourth viewing angle, highlights the microscopic details of the fabric, such as the interweaving of fibers and the surface texture. The collection of these multi-view images enables the multimodal model to learn the characteristics of the fabric from multiple levels, thereby achieving higher accuracy in the fabric recognition task. By analyzing these images, the model can distinguish different fabric categories and work effectively even when the fabrics have similar appearance or texture. This multi-view approach enhances the model's ability to capture fabric details and improves the accuracy and efficiency of textile fabric recognition.

[0083] The fabric recognition device is equipped with four cameras with different focal lengths, namely the first, second, third and fourth cameras, of which the first camera has the longest focal length and the fourth camera has the shortest focal length. These cameras are installed at positions equidistant from the fabric to capture images of the fabric. The steps of obtaining multi-view fabric images are to first use the four cameras to simultaneously collect fabric images, and each camera captures fabric images with different viewing angles according to its focal length. The first camera, due to its longest focal length, captures fabric images with the smallest viewing angle, which mainly contains the detailed features of the fabric; the second camera captures fabric images with a slightly larger viewing angle, which contains more medium-range features; the third camera captures fabric images with a larger viewing angle, which contains a wider range of fabric features; and finally, the fourth camera, due to its shortest focal length, captures fabric images with the largest viewing angle, which mainly shows the overall pattern and structure of the fabric. By combining these four fabric images from different perspectives, a comprehensive multi-view fabric image can be obtained, which contains rich information from macro to micro, and provides detailed visual data for the subsequent multimodal model, so that different textile fabrics can be more accurately identified and classified.

[0084] Step S20, inputting the multi-view fabric image, the fabric language text and the fabric color into a target multimodal model to obtain a fusion feature;

[0085] It should be noted that the target multimodal model refers to a deep learning model designed to process and analyze multi-view fabric images. This model can receive fabric images collected by different cameras as input. These images include different viewpoints and possible different lighting conditions or detail levels. The core function of the target multimodal model is to understand and parse the rich information contained in these multi-view images and extract the key features of the fabric from them. The target multimodal model has strong zero-shot learning ability, multimodal understanding ability and generalization performance. By constructing data pairs consisting of fabric language and images, the multimodal model is trained using fabric language-image data pairs, so that the multimodal model can learn the relationship between language text and image. The multimodal model includes a fabric language-image contrast learning model and a multimodal fabric feature extraction model. The fabric language-image contrast learning model only uses a text feature extractor to convert language text into text features; the text features and fabric images are input into the multimodal fabric feature extraction model to obtain fused features, and the fused features are used for search. The fabric language-image contrast learning model includes an image feature extractor and a text feature extractor, both of which are built using a transformer neural network based on an attention mechanism. The fabric language-image contrast learning model learns the relationship between fabric images and description texts, combines visual information with language information, and has better generalization ability when new fabrics appear. When retrieving fabrics, the text feature extractor in the fabric language-image contrast learning model is used to obtain the deep features of the fabric description language text. As the input of the fabric feature extraction model, the text can guide the learning of the fabric feature extractor, so that more accurate search results can be obtained in the scene that focuses on the fabric language text. Usually, when users search, they only focus on some parameters of the fabric, such as cotton, washing, etc. When training the model, by shielding some texts, the fabric language-image contrast learning model can learn the relationship between fabric parameters and images.

[0086] Fusion features refer to the comprehensive feature representation obtained by analyzing multi-view fabric images through the target multimodal model. These features are not simply linear superposition of image features from different perspectives, but through complex calculations and learning processes within the model, the fabric features captured from different perspectives are effectively integrated to obtain a feature vector that can represent the essential properties of the fabric. This feature vector captures the multi-dimensional information of the fabric, including color, texture, structure, details, etc., providing strong data support for the accurate identification and classification of fabrics. The generation of fusion features is the process of multimodal models learning the features of different fabric samples and performing feature extraction, fusion and classifier prediction. The final output fusion feature vector can be used for fabric similarity comparison and identification.

[0087] It can be understood that, first, the collected multi-view fabric images, the corresponding fabric language texts, and the fabric color data are used as input data sets. These images capture the appearance characteristics of the fabric from different angles, the language texts provide detailed attribute descriptions, and the color data are obtained through a colorimeter and expressed in the Lab color gamut to ensure accurate color recording. Secondly, these multimodal data are input into the target multimodal model. The model first uses a multimodal feature encoder to process the image to extract the visual features of the fabric. At the same time, the text feature extractor processes the language text to extract the semantic features of the fabric. Then, the feature fusion device fuses these visual features and semantic features to form a comprehensive fusion feature. This fusion feature not only contains the visual appearance information of the fabric, but also integrates the detailed attributes and color information of the fabric. Finally, this fusion feature is used for fabric recognition and classification, so that the model can understand and identify the fabric more accurately, improving the accuracy and efficiency of fabric recognition. The effect of this method is that even when the fabrics are similar in appearance or texture, different fabrics can be accurately distinguished, providing strong technical support for the automatic recognition and retrieval of textile fabrics.

[0088] As an example, the multi-view fabric image includes a front multi-view fabric image and a back multi-view fabric image, the target multimodal model includes a multimodal feature encoder and a multimodal feature fusion device, the multimodal feature encoder includes a feature embedding layer, an attention layer and a fully connected layer, and the step of inputting the multi-view fabric image, the fabric language text and the fabric color into the target multimodal model to obtain fused features includes: inputting the front multi-view fabric image, the back multi-view fabric image, the fabric language text and the fabric color into the multimodal feature encoder in sequence, and extracting the underlying features of the multi-view fabric image and the fabric language text through the feature embedding layer; downsampling the underlying features through the attention layer and reducing the feature dimension to obtain downsampled features; extracting high-level semantic features in the downsampled features through the fully connected layer to obtain deep features; and fusing the deep features through the multimodal feature fusion device to obtain fused features.

[0089] Front multi-view fabric image and back multi-view fabric image refer to multi-view images collected from the front and back of the fabric respectively. Each view is taken by a camera with a different focal length to capture the overall characteristics of the fabric. Both the front and back images are used for recognition because the front and back of the fabric may have different textures and structural characteristics. Please refer to Figure 4 , Figure 4A schematic diagram of front and back fabric images provided for the first embodiment of the textile fabric recognition method based on the multimodal model of this application shows the front and back images of the same fabric, which are used as input data for fabric category recognition of the multimodal model. The front image presents the main texture and color of the fabric, while the back image reveals the possible different surface features and subtle color changes, which are crucial for the accurate classification of the fabric. In the application of the multimodal model, these front and back images enable the model to analyze fabric features from multiple angles and improve the accuracy of recognition. By learning the features of these images, the model can distinguish the types and properties of fabrics, and can work effectively even when the fabrics are similar in appearance or texture. In this way, the model can utilize the multi-dimensional information of the fabric, enhance the ability to capture fabric details, and improve the accuracy and efficiency of textile fabric recognition.

[0090] The multimodal feature encoder is a part of the target multimodal model that is responsible for converting the input fabric image into a series of feature representations. It usually contains multiple layers, such as feature embedding layers, attention layers, and fully connected layers. The multimodal feature fuser is another part of the model that further fuses the deep features extracted by the encoder to obtain a comprehensive feature vector that represents the multi-dimensional information of the fabric. Feature embedding layers, attention layers, and fully connected layers are different types of neural network layers that make up the multimodal feature encoder. The feature embedding layer is responsible for extracting local features in the image, such as edges, textures, etc. The attention layer downsamples the features output by the feature embedding layer, that is, reduces the spatial dimension of the features while retaining the most important information, which helps to reduce the amount of computation and improve the generalization ability of the model. The fully connected layer maps the features output by the attention layer to a higher dimensional space to extract more abstract high-level semantic features. The low-level features refer to the local features of the fabric image initially extracted by the feature embedding layer. Downsampling is the operation of the attention layer, which reduces the dimension of the features by reducing the size of the feature map. Feature dimension refers to the spatial size of the feature, such as width and height in an image. Downsampled features refer to features processed by the attention layer, whose dimensions are reduced. High-level semantic features refer to features extracted by the fully connected layer, which represent more complex and abstract concepts or patterns in the image. Deep features refer to features processed by the fully connected layer, which are high-level, abstract representations of fabric images and contain rich fabric information.

[0091] First, the multi-view fabric images of the front and back sides and the corresponding fabric language text and color data are sequentially input into the multimodal feature encoder. These input data first pass through the feature embedding layer, which is responsible for extracting the visual features of the image and the semantic features of the text, providing the underlying features for subsequent processing. Next, these underlying features are passed to the attention layer, which reduces the spatial dimension of the features through a downsampling process while retaining the most critical information, which helps to reduce computational complexity and prevent overfitting. Then, the fully connected layer further processes these downsampled features to extract higher-level semantic features to form deep features, which contain rich attribute information of the fabric. Finally, the multimodal feature fuser integrates these deep features to form a comprehensive fused feature, which comprehensively represents the multi-dimensional information of the fabric and provides a solid foundation for accurate fabric identification and classification. The effect of the whole process is to improve the accuracy and efficiency of fabric identification, ensuring that different fabrics can be accurately distinguished even when the fabrics are similar in appearance or texture.

[0092] Step S30, calculating the similarity between the fused feature and the fused feature in the fabric database;

[0093] It should be noted that the fused features are generated by the target multimodal model and represent the key features of the input fabric image, including the depth information extracted and fused from the front and back multi-view fabric images. The fused features in the fabric database are the feature vectors of fabric samples previously processed by the same multimodal model and saved in the database, representing the key features of each fabric sample in the database.

[0094] Understandably, this process uses specific algorithms, such as cosine similarity or Euclidean distance, to quantify how similar two feature vectors are in a high-dimensional space. Specifically, the system compares the fused feature vector of the input fabric with the fused feature vector of each fabric sample in the database, and calculates a similarity score between 0 and 1, where 1 means exactly the same and 0 means completely different. This score reflects the similarity between the input fabric and each sample in the database. The system then sorts the fabric samples in the database according to the similarity score and finds the sample that is closest to the input fabric features. This process allows accurate identification and classification of fabrics even when there are many types of fabrics with subtle differences in features, providing an efficient method for rapid retrieval and matching of fabrics.

[0095] Step S40, reordering the fusion features in the fabric database according to the similarity to obtain a reordering result;

[0096] It should be noted that reordering means that according to these similarity scores, the system will rearrange the fabric samples in the fabric database. The order is based on the similarity score. The samples with the highest similarity to the input fabric will be ranked first, and the samples with lower similarity will be ranked at the back. The reordering result refers to the result list of the reordering of the fabric samples in the fabric database according to the calculated similarity scores.

[0097] It is understandable that the system will reorder according to the similarity scores between the fused features of the input fabric and the fused features of each sample in the fabric database calculated by the multimodal model. This process involves sorting all fabric samples in the database from high to low according to their similarity with the input fabric, and generating a reordered result list in which the samples most similar to the input fabric features are placed at the front, and the samples with lower similarity are arranged in sequence. The reordered result is equivalent to a sorted fabric sample index, which guides the user or automated system to quickly locate the fabric option that is most likely to match, thereby achieving efficient retrieval and identification. This method is particularly suitable for situations where the fabric library is large and the fabric features are complex. It can significantly improve the speed and accuracy of fabric search, bringing convenience to fabric selection, procurement, design and production.

[0098] Step S50, obtaining the fabric category according to the reordering result, and completing fabric identification.

[0099] It should be noted that fabric categories are the result of detailed classification based on different parameters of fabrics, including major categories of fabric components, subcategories of fabric components, fabric texture, yarn state, yarn count, spinning process, finishing process, fiber morphology, gloss, smoothness, fiber length, etc. These detailed parameters help to divide the textile fabrics on the market into different categories. Each category contains fabrics with the same parameters, while different parameters are defined as new categories. Each fabric sample in the fabric database is marked with these detailed parameters, forming a detailed fabric category system. When the fusion features of the input fabric are compared with the fusion features of the fabric samples in the database for similarity, the system will determine the most matching fabric samples based on the re-ranking results of the similarity. The categories to which these most matching samples belong are the fabric categories predicted for the input fabric.

[0100] It can be understood that the re-ranking result refers to a list of fabric samples in the database sorted from high to low according to the similarity after comparing the similarity of the fusion features of the input fabric with the fusion features of each sample in the fabric database through a multimodal model. In this list, the fabric sample at the top is the most similar to the input fabric. According to this re-ranking result, the system can directly read the category label of the fabric sample at the top (or several most similar samples), which combines the detailed parameters of the fabric composition, texture, yarn state, spinning process, color, etc., and represents the specific classification of the fabric. The system uses these category labels as the predicted category of the input fabric, thereby completing the automatic identification of the fabric. This process not only improves the accuracy and efficiency of recognition, but also provides strong technical support for the intelligent upgrading of the textile industry, making the classification and management of fabrics more accurate and efficient.

[0101] This embodiment provides a textile fabric recognition method based on a multimodal model, which obtains a multi-view fabric image, a fabric language text corresponding to the multi-view fabric image, a fabric color, and a fabric database; inputs the multi-view fabric image, the fabric language text, and the fabric color into a target multimodal model to obtain a fusion feature; calculates the similarity between the fusion feature and the fusion feature in the fabric database; reorders the fusion feature in the fabric database according to the similarity to obtain a reordering result; obtains a fabric category according to the reordering result to complete fabric recognition. First, a multi-view image of the fabric is captured by multiple cameras, including the front and back sides of the fabric, to ensure that the fabric features are fully captured from macro to micro, and to provide detailed visual data for recognition. Then, these multi-view images and the corresponding fabric language text and fabric color are input into a multimodal model, which extracts features through a feature embedding layer, performs downsampling through an attention layer, and generates a fusion feature vector containing multi-dimensional information of the fabric through a fully connected layer, which helps to convert the visual information of the fabric into deep features that can be understood by the machine, thereby improving the accuracy and efficiency of textile fabric recognition. Subsequently, the system calculates the similarity score between the fusion features of the input fabric and the fusion features of each sample in the fabric database. This step evaluates the similarity between fabrics in a quantitative way and provides a scientific basis for fabric matching and identification. Based on these similarity scores, the system reorders the fabric samples in the database and puts the samples most similar to the input fabric in the front. This operation helps to quickly screen out the fabric samples that are most likely to match, significantly improving the efficiency of the identification process. Finally, the system determines the predicted category of the input fabric based on the category label of the top-ranked sample in the reordering result and completes the fabric identification. This process can quickly and accurately identify the specific category of the input fabric from the fabric database, even in complex environments or when facing similar fabrics. It can maintain a high accuracy rate. The entire fabric identification process not only improves the accuracy and efficiency of textile fabric identification, but also reduces the errors caused by human factors, providing technical support for the automated production process of the textile industry. In addition, it also has good flexibility and generalization ability, providing strong technical support for areas such as fabric quality control, inventory management and personalized customization.

[0102] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 5 , Figure 5 This is a flow chart of a second embodiment of a textile fabric identification method based on a multimodal model of the present application. Before step S20 of the textile fabric identification method based on a multimodal model, steps S01 to S05 are also included:

[0103] Step S01, obtaining a target fabric dataset, and dividing the target fabric dataset into a training set and a test set;

[0104] It should be noted that the target fabric dataset refers to a large number of fabric images and their related feature data collected for training and evaluating multimodal fabric recognition models. This dataset contains images of various fabrics. Each image is taken from different perspectives and focal lengths, and includes the front and back of the fabric to ensure that the dataset can fully cover various fabric features. These images and their labels (such as fabric composition, texture and other classification information) are used to train the model to enable it to recognize and classify different fabrics. The training set is a data subset used to train the model. It contains a series of fabric images with known labels. During the training process, the model learns how to recognize and classify different fabric features by analyzing these images and their corresponding labels. The role of the training set is to enable the model to "learn" the fabric features and corresponding classification relationships. The test set is a data subset used to evaluate the performance of the model. The fabric images it contains have not been used in the model training process. The purpose of the test set is to verify the recognition ability and accuracy of the model when processing unseen fabric images. The performance on the test set can be used to evaluate the generalization ability of the model, that is, the model's ability to recognize new fabric data.

[0105] It is understandable that the target fabric dataset is first collected, which involves using multiple cameras to capture the front and back of the fabric at different angles and focal lengths to capture the comprehensive characteristics of the fabric. Then, these images are annotated to record their detailed properties, such as composition, texture, etc. Next, in order to train and evaluate the model, this dataset needs to be divided into a training set and a test set. The division is usually carried out in a certain proportion, such as 80% of the data is used as a training set and the remaining 20% ​​is used as a test set. This division can be achieved by random selection or using specific data segmentation techniques, with the aim of ensuring that the samples in the training set and test set can represent the diversity of the entire dataset. The training set is used for model learning and parameter adjustment so that the model can recognize the characteristics and classification of fabrics. The test set is used to evaluate its performance on unseen data after the model training is completed, thereby verifying the accuracy and generalization ability of the model. Doing so can train a model that performs well on the training data and can accurately identify new and unseen fabric data, thereby improving the accuracy and efficiency of textile fabric recognition.

[0106] As an example, the step of obtaining a target fabric data set includes: obtaining an initial fabric data set and historical fabric data; evaluating and screening the initial fabric data set to obtain data to be labeled; comparing the data to be labeled with the historical fabric data to obtain similarity; sorting the data to be labeled according to the similarity, and filtering the data to be labeled whose similarity is lower than a preset similarity threshold to obtain a target fabric data set.

[0107] Initial fabric dataset refers to a newly collected collection of fabric images that have not been labeled or identified. Historical fabric data refers to fabric data that has been collected and labeled before. These data already contain detailed information about the fabric, such as composition, texture, etc., and have been used to train or verify the model. Evaluation and screening refers to the process of quality control and relevance evaluation of images in the initial fabric dataset. The purpose of this step is to determine which images are valuable and worthy of further processing and annotation. The evaluation may be based on factors such as image clarity, whether they contain enough fabric details, and whether the images are representative. Data to be labeled refers to a collection of images that need to be further labeled after evaluation and screening. These images are selected because they provide enough information to describe the characteristics of the fabric, but there is no specific label or classification information yet. Similarity refers to the use of specific algorithms (such as feature matching, pattern recognition, etc.) to quantify the similarity between two sets of data. In fabric identification, similarity usually refers to the degree of closeness in visual features between the fabric image to be labeled and the existing images in the historical fabric data. The preset similarity threshold is a pre-set numerical standard used to determine the degree of acceptance of similarity between fabric images. After the similarity between the fabric image to be annotated and the image in the historical fabric database is calculated, only those images with a similarity higher than this threshold will be retained as part of the target fabric dataset. This threshold is set to filter out fabric images with low similarity to the existing dataset. The purpose of this is to ensure that the images in the dataset have high uniqueness and reduce data redundancy, so that subsequent fabric category identification is more accurate. For example, if the preset similarity threshold is set to 70%, then only fabric images to be annotated with a similarity of more than 70% to existing images in the database will be retained for subsequent annotation and model training, which can ensure that the model can learn more different fabric features and improve recognition accuracy.

[0108] First, it is necessary to collect an initial fabric dataset, which usually involves taking or scanning a series of new fabric sample images, and also using existing historical fabric data, that is, a collection of fabric images that have been annotated before. Next, the images in these initial fabric datasets are evaluated and screened for quality, and those unclear or unrepresentative images are eliminated, leaving the images to be annotated with higher quality. Then, using image processing and pattern recognition techniques, these images to be annotated are compared with the images in the historical fabric dataset to calculate the similarity between them. This step usually involves extracting image features and using algorithms such as cosine similarity to quantify the similarity between images. According to the calculated similarity, the data to be annotated are sorted, and then a preset similarity threshold is set, such as 80%, and only those images with a similarity with historical data higher than this threshold are retained, while images with lower similarity are filtered out. In this way, the final target fabric dataset obtained only contains fabric images that are highly similar to the existing data, thereby ensuring the diversity and novelty of the dataset, and laying a solid foundation for training an accurate and efficient fabric recognition model. Through this series of steps, the amount of data to be processed later can be reduced and the accuracy of fabric recognition can be improved.

[0109] Step S02, using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model;

[0110] It should be noted that the initial multimodal model refers to the basic multimodal model before training begins. This model usually contains multiple modules that can process fabric images from different perspectives and front and back sides. The initial model may be pre-designed based on previous research or existing model architectures. It has the basic ability to extract and fuse features from multi-perspective fabric images, but has not yet been trained and parameter adjusted on a specific data set. The reference multimodal model is a multimodal model that has been trained and adjusted using a training set. During the training process, the parameters of the initial model are optimized based on the fabric images and their labels in the training set so that the model can more accurately capture and fuse the features of the fabric and improve the recognition accuracy. After this process, the model becomes more suitable for handling specific fabric recognition tasks and can be used as a reference or benchmark for subsequent testing, comparison or further optimization. The reference multimodal model can be used to evaluate performance on the test set or as a basis for further research and development.

[0111] It is understandable that a large number of collected fabric images and their corresponding detailed labels are first input into the initial model. This initial model has the ability to handle multi-view and front and back fabric images, but it needs to be further optimized through actual data. The training set consists of screened fabric images, including various features and classification information of the fabric. During the training process, the parameters of the model are constantly adjusted to minimize the difference between the predicted labels and the actual labels. This process involves updating the weights and biases of the model, with the aim of improving the accuracy of the model in identifying fabric features and classification. By repeatedly iterating the data on the training set, the initial model gradually learns how to extract key information from the image and make accurate classifications. Finally, the trained and adjusted model becomes a reference multimodal model, which can more accurately reflect the characteristics of the fabric and provide an optimized benchmark for the fabric recognition task. This reference model not only improves the recognition accuracy, but also lays the foundation for subsequent testing and application.

[0112] As an example, the training set includes image data, category data, fabric description language text data and fabric color data, the image data includes front-back multi-perspective image data, front multi-perspective image data, back multi-perspective image data, multi-perspective image data missing the second and third perspectives, and first perspective front image data, the initial multimodal model includes a multimodal feature encoder, a multimodal feature fusion device and a multimodal feature classifier, and the step of using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model includes: inputting the fabric description language text data, the fabric color data, the front-back multi-perspective image data, the front multi-perspective image data, the back multi-perspective image data, the multi-perspective image data missing the second and third perspectives, or the first perspective front image data into the multimodal feature encoder to obtain fabric multimodal depth features; through the multimodal feature fusion device The device fuses the multimodal depth features of the fabric to obtain the multimodal fusion features of the fabric; the multimodal fusion features of the fabric are learned and distinguished by the multimodal feature classifier to obtain the fabric category; when the fabric category does not match the category data, the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier are adjusted, and the step of inputting the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third view or the first view front image data into the multimodal feature encoder to obtain the multimodal depth features of the fabric; when the fabric category matches the category data, a reference multimodal model is obtained according to the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier.

[0113] Fabric description language text data refers to text information that describes fabric properties in natural language, including fabric composition, texture, yarn state, finishing process, etc. For example, "a picture of a fabric of chemical fiber satin double-sided silk, woven-satin, high elasticity, dyed, polyester". This description provides the key features of the fabric and can be used to train the model to identify and retrieve fabrics. Fabric color refers to the color presented on the surface of the fabric. Fabric color data is the specific numerical value of the fabric color obtained by a colorimeter. It is usually represented by the Lab color gamut in the textile industry, where L represents lightness, A represents the component from green to red, and B represents the component from blue to yellow. These color data help the model consider color matching when retrieving fabrics. Front-back multi-view image data includes multi-view images of the front and back of the fabric. Each view is collected by cameras with different focal lengths to capture the characteristics of the large, medium and small view angles of the fabric. This data enables the model to understand the appearance and structure of the fabric from different angles. Front multi-view image data (missing back modal images): When only multi-view images of the front side of the fabric are available, the model needs to be able to handle single-side data, which requires the model to be able to identify and retrieve fabrics based only on the features of the front side image. Back multi-view image data (missing front modal images): Corresponding to the front multi-view image data, this situation involves multi-view images of only the back side of the fabric, and the model needs to be able to handle this data and identify the features of the fabric. Missing second and third perspective multi-view image data (missing medium perspective modal images): In some cases, medium perspective images may be missing, and the model needs to be able to adapt to this situation and perform effective fabric identification and retrieval even when some perspective data is missing. First-view front image data (missing back modal images, medium perspective and small perspective modal images): This refers to the situation where only the first-view image of the front side of the fabric is available, and the images of the back side, as well as medium and small perspectives are missing. The model needs to be able to handle this incomplete data situation and identify the fabric as accurately as possible. Image data refers to fabric images captured from multiple perspectives, including the front and back sides of the fabric, which constitute the main body of the training set. Category data refers to the label information corresponding to these images, such as the type, composition, texture and other classification identifiers of the fabric, which are used to tell the model the correct category of each image. The multimodal feature encoder, multimodal feature fusion device and multimodal feature classifier are the key components of the initial multimodal model. The multimodal feature encoder is responsible for extracting deep features from the input fabric image. The multimodal feature fusion device further integrates these deep features to form a more comprehensive fabric feature representation. The multimodal feature classifier uses these fused features to predict the category of the fabric. Fabric multimodal deep features refer to the fabric features obtained after the multimodal feature encoder processes the image data. These features contain the deep information of the image, such as texture, color, shape, etc., which are the basis of fabric recognition.Fabric multimodal fusion features refer to the features obtained after the multimodal feature fuser further processes the deep features. These fusion features integrate fabric information from different perspectives and levels, providing a more comprehensive basis for classification. Fabric category refers to the specific category into which fabrics are classified based on their physical properties and visual features, such as "cotton plain weave" or "silk jacquard". During the training process, the goal of the model is to accurately predict this category label for the input fabric image.

[0114] First, a training set containing a large number of fabric images and their corresponding category labels is used. The image data in the training set captures the multi-view and front and back features of the fabric, while the category data provides information such as the type of fabric each image belongs to. The initial multimodal model consists of three main parts: a multimodal feature encoder, a multimodal feature fuser, and a multimodal feature classifier. At the beginning of training, the image data is input into the multimodal feature encoder, which extracts the multimodal deep features of the fabric through a series of deep learning operations. These features contain the key visual information of the fabric. Subsequently, these deep features are fed into the multimodal feature fuser, which fuses the information at different viewpoints and feature levels to form a more comprehensive multimodal fusion feature of the fabric. Next, the multimodal feature classifier learns and classifies these fused features to predict the category of the fabric. If the category predicted by the classifier does not match the actual category data in the training set, the parameters of the encoder, fuser, and classifier are adjusted, and then the process is repeated to optimize the model performance. When the category predicted by the model matches the actual category data, it means that the model has been able to accurately learn and distinguish the characteristics of different fabrics. At this time, a fully trained and adjusted reference multimodal model is obtained, which can accurately identify and classify fabrics, providing strong support for fabric recognition tasks in practical applications.

[0115] Please refer to Figure 6 , Figure 6The fabric color recognition schematic diagram provided for the second embodiment of the textile fabric recognition method based on the multimodal model of this application shows the process of fabric color recognition, which includes the specific values ​​of the color measurement data, which represent the different dimensional characteristics of the fabric color. In the schematic diagram, three key data points can be seen: 81.48, 8.11 and 0.11, which may correspond to parameters such as the brightness, saturation and hue of the color respectively. These parameters are the key features used in the multimodal model to identify and distinguish different fabric colors. By inputting the fabric image into the multimodal feature encoder, the model can extract these color features and combine them with other visual features (such as texture, pattern) through the multimodal feature fuser, and finally classify them by the multimodal feature classifier to achieve accurate recognition of the fabric color. This color recognition method helps to improve the accuracy of fabric classification, especially in automated textile production and quality control, and can ensure the consistency and accuracy of fabric color to meet specific design and production requirements.

[0116] Please refer to Figure 7 , Figure 7 The schematic diagram of the initial multimodal model structure provided for the second embodiment of the textile fabric identification method based on the multimodal model of the present application shows the structure of the initial multimodal model, which is designed for processing and analyzing the multimodal data of textile fabrics. The model includes three main parts: a multimodal feature encoder, a multimodal feature fusion device and a multimodal feature classifier. The multimodal feature encoder is responsible for extracting deep features from multi-view images of fabrics, which include the front and back of the fabrics and large, medium and small viewing angles of different focal lengths. Subsequently, the multimodal feature fusion device integrates these deep features with the features extracted from the language text description of the fabric to form a comprehensive fabric feature representation. Finally, the multimodal feature classifier uses these fused features to identify and classify the fabric and output the category of the fabric. This structure enables the model to capture the features of the fabric from multiple dimensions, including visual information and language description, thereby improving the accuracy and robustness of the retrieval. Through this structure, the model can effectively process and compare fabric images, text descriptions and color data, realize efficient and accurate fabric retrieval, and meet the needs of the textile industry in terms of intelligent upgrading, quality control and personalized customization.

[0117] Please refer to Figure 8 , Figure 8 The schematic diagram of the initial multimodal model training provided in the second embodiment of the textile fabric identification method based on the multimodal model of the present application corresponds to Figure 7Fabric images and image feature encoders, which show a schematic diagram of the initial multimodal model training process, where the front and back images of the fabric are captured at large, medium, and small viewing angles, respectively, to fully obtain the fabric feature information. These images first pass through the multimodal feature encoder, which is responsible for extracting deep features from the images at each view. These features are then fed into the multimodal feature fusion, where the features from different views are integrated to form a comprehensive fabric feature representation. Finally, the fused features are fed into the multimodal feature classifier, where the model learns to distinguish between different fabric categories. The entire process is automated and aims to optimize the model's parameters through deep learning techniques to ensure that it can accurately identify and classify a variety of fabrics, even when the fabrics have similar appearances or textures. In this way, the model is able to learn the characteristics of fabrics at different scales and angles, allowing for more accurate fabric classification and identification in practical applications.

[0118] Step S03, testing and verifying the target detection model through the test set to obtain a test result;

[0119] It should be noted that test validation refers to the use of an independent test set to evaluate the performance of the object detection model. The test set contains fabric images and their category labels that have not been used in the model training process. During the test validation process, the model will make predictions on the images in the test set, trying to identify and classify fabrics without using the actual label information of these images. This step is to test the accuracy and generalization ability of the model when processing new data, ensuring that it does not just "remember" the labels of the training set, but has truly learned how to identify the characteristics of fabrics. The test results are the performance of the model on the test set, including various performance indicators such as accuracy, precision, recall, and F1 score, which show the degree of match between the categories predicted by the model and the actual category labels in the test set. The test results can tell us how the model may perform in actual applications, whether it can accurately identify the category of fabrics, and its stability and reliability in different situations. If the test results meet the predetermined performance standards, then the model can be considered valid and can be used for actual fabric recognition tasks. If the test results are not ideal, the model may need to be further adjusted or retrained.

[0120] It can be understood that the model processes the images in the test set and uses the rules learned in the training phase to predict the fabric category of each image. After the prediction is completed, the model's prediction results are compared with the true labels in the test set, and the performance of the model is measured by calculating performance indicators such as precision, recall, and F1 score. These indicators reflect the accuracy and consistency of the model in identifying fabric categories. The final test results can indicate whether the model can reliably identify fabrics and whether it is suitable for deployment in actual applications. If the test results do not meet the expected standards, further adjustments and optimizations may be required to the model. This process ensures the generalization ability of the model on new data and provides an important reference for actual deployment.

[0121] Step S04, when the test result does not meet the preset requirements, returning to the step of training and adjusting the initial multimodal model using the training set;

[0122] It should be noted that preset requirements refer to a series of standards or thresholds set to evaluate model performance before model testing and verification. These requirements usually include the minimum accuracy, recall, precision or other related indicators that the model should achieve on the test set. For example, if the preset requirement is that the accuracy of the model needs to reach more than 95%, then the test results need to meet this condition for the model to be considered valid.

[0123] It is understandable that when the test results show that the performance of the model does not meet these preset requirements, such as the accuracy rate is less than 95%, this indicates that the model may need further training or adjustment to improve its recognition ability. In this case, it is necessary to return to the step of training and adjusting the initial multimodal model using the training set, which may include adjusting model parameters, adding training data, performing data enhancement, adjusting network structure or optimizing algorithms, in order to improve the performance of the model to meet the preset requirements. This process may require multiple iterations until the model's performance on the test set reaches a satisfactory level.

[0124] Step S05: when the test result meets the preset requirement, the reference multimodal model is used as the target multimodal model.

[0125] It is understandable that when the test results meet the preset requirements, such as an accuracy rate of more than 95%, it means that the model can accurately perform the recognition task on unseen fabric images and shows good generalization ability. In this case, the reference multimodal model, that is, the trained and adjusted model, is confirmed to be valid and used as the target multimodal model for the actual fabric recognition task. This model has proven its reliability and accuracy in processing diverse fabric images, so it can be deployed in textile production, quality control or fabric recommendation systems to automate and optimize the fabric classification and recognition process.

[0126] In this embodiment, a target fabric dataset is obtained, and the target fabric dataset is divided into a training set and a test set; the initial multimodal model is trained and adjusted using the training set to obtain a reference multimodal model; the target detection model is tested and verified using the test set to obtain a test result; when the test result does not meet the preset requirements, the step of training and adjusting the initial multimodal model using the training set is returned; when the test result meets the preset requirements, the reference multimodal model is used as the target multimodal model. First, the target fabric dataset is obtained by collecting multi-view images of different fabrics and their attribute labels, and the dataset is carefully divided into a training set and a test set. This is done to ensure that the model can learn the characteristics of the fabric in the training stage, and at the same time, the model's recognition ability of new data can be verified in the test stage. Then, the initial multimodal model is trained and adjusted using the training set, with the aim of allowing the model to learn a large number of fabric images and corresponding labels, so as to master how to accurately extract the characteristics of the fabric and classify it. Then, the trained model is tested and verified through the test set, and the test results are obtained. This step is to evaluate the generalization ability of the model and ensure that it can accurately identify fabric images that have not been seen. If the test results do not meet the preset performance requirements, such as the accuracy rate does not exceed the set threshold, the model will return to the training stage for further optimization and adjustment, which will help improve the robustness of the model and ensure that it can work stably under various conditions. Finally, when the test results meet the preset requirements, the reference multimodal model is confirmed to be valid and used as the target multimodal model for actual fabric recognition tasks. Through this process, a high-performance fabric recognition model is developed, which can improve the accuracy and efficiency of textile fabric recognition, reduce human errors, improve production efficiency, and provide strong technical support for automated and intelligent textile production processes. In addition, this method also enhances the adaptability and flexibility of the model, enabling it to cope with a variety of different fabrics and environmental conditions, providing technical guarantees for innovation and development in the textile industry.

[0127] This application also provides a textile fabric identification device based on a multimodal model, please refer to Fig. 9 , the textile fabric identification device based on the multimodal model comprises:

[0128] A data acquisition module 10 is used to acquire multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and a fabric database;

[0129] A model processing module 20, configured to input the multi-view fabric image, the fabric language text, and the fabric color into a target multimodal model to obtain a fusion feature;

[0130] A similarity calculation module 30 is used to calculate the similarity between the fused feature and the fused feature in the fabric database;

[0131] A reordering module 40, used for reordering the fusion features in the fabric database according to the similarity to obtain a reordering result;

[0132] The fabric identification module 50 is used to obtain the fabric category according to the reordering result and complete fabric identification.

[0133] In one embodiment, the data acquisition module 10 is also used to acquire multiple groups of fabric images, each of which is collected by the detection camera, and each group of fabric images corresponds to each detection camera one by one; and a multi-view fabric image is determined based on each of the fabric images.

[0134] In one embodiment, the data acquisition module 10 is also used to obtain a multi-perspective fabric image based on the first-perspective fabric image, the second-perspective fabric image, the third-perspective fabric image and the fourth-perspective fabric image captured by the first camera, the second camera, the third camera and the fourth camera, wherein the first perspective is smaller than the second perspective, the second perspective is smaller than the third perspective, and the third perspective is smaller than the fourth perspective.

[0135] In one embodiment, the model processing module 20 is also used to input the front multi-view fabric image, the back multi-view fabric image, the fabric language text and the fabric color into the multimodal feature encoder in sequence, and extract the underlying features of the multi-view fabric image and the fabric language text through the feature embedding layer; downsample the underlying features through the attention layer and reduce the feature dimension to obtain downsampled features; extract high-level semantic features from the downsampled features through the fully connected layer to obtain deep features; and fuse the deep features through the multimodal feature fuser to obtain fused features.

[0136] In one embodiment, the model processing module 20 is further used to obtain a target fabric data set and divide the target fabric data set into a training set and a test set; use the training set to train and adjust the initial multimodal model to obtain a reference multimodal model; test and verify the target detection model through the test set to obtain a test result; when the test result does not meet the preset requirements, return to the step of training and adjusting the initial multimodal model using the training set; when the test result meets the preset requirements, use the reference multimodal model as the target multimodal model.

[0137] In one embodiment, the model processing module 20 is also used to obtain an initial fabric data set and historical fabric data; evaluate and screen the initial fabric data set to obtain data to be labeled; compare the data to be labeled with the historical fabric data to obtain similarity; sort the data to be labeled according to the similarity, and filter the data to be labeled whose similarity is lower than a preset similarity threshold to obtain a target fabric data set.

[0138] In one embodiment, the model processing module 20 is further used to input the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third view or the first view front image data into the multimodal feature encoder to obtain the fabric multimodal depth feature; fuse the fabric multimodal depth feature through the multimodal feature fuser to obtain the fabric multimodal fusion feature; learn and distinguish the fabric multimodal fusion feature through the multimodal feature classifier to obtain the fabric category; when the fabric category does not match the category data When the fabric category matches the category data, the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier are adjusted, and the step of inputting the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third views, or the first view front image data into the multimodal feature encoder to obtain the multimodal depth features of the fabric is returned; when the fabric category matches the category data, a reference multimodal model is obtained according to the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier.

[0139] The textile fabric identification device based on a multimodal model provided by the present application adopts the textile fabric identification method based on a multimodal model in the above-mentioned embodiment, which can solve the technical problem of how to improve the accuracy and efficiency of textile fabric identification. Compared with the prior art, the beneficial effects of the textile fabric identification device based on a multimodal model provided by the present application are the same as the beneficial effects of the textile fabric identification method based on a multimodal model provided by the above-mentioned embodiment, and the other technical features of the textile fabric identification device based on a multimodal model are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0140] The present application provides a textile fabric identification device based on a multimodal model, and the textile fabric identification device based on a multimodal model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the textile fabric identification method based on the multimodal model in the above-mentioned embodiment 1.

[0141] Reference below Fig.10 , which shows a schematic diagram of the structure of a textile fabric identification device based on a multimodal model suitable for implementing the embodiment of the present application. The textile fabric identification device based on a multimodal model in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The textile fabric identification device based on the multimodal model shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0142] like Fig.10As shown, the textile fabric identification device based on the multimodal model may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the textile fabric identification device based on the multimodal model are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the multimodal model-based textile fabric recognition device to communicate wirelessly or wired with other devices to exchange data. Although the multimodal model-based textile fabric recognition device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0143] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The textile fabric identification device based on a multimodal model provided by the present application adopts the textile fabric identification method based on a multimodal model in the above-mentioned embodiment, which can solve the technical problem of how to improve the accuracy and efficiency of textile fabric identification. Compared with the prior art, the beneficial effects of the textile fabric identification device based on a multimodal model provided by the present application are the same as the beneficial effects of the textile fabric identification method based on a multimodal model provided by the above-mentioned embodiment, and the other technical features in the textile fabric identification device based on a multimodal model are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0145] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the textile fabric identification method based on the multimodal model in the above-mentioned embodiment.

[0146] The computer-readable storage medium provided in the present application can be a USB flash drive, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination thereof. More specific examples include, but are not limited to: electrical connection (one or more wires), a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. A computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. The program code can be transmitted via a medium such as a wire, an optical cable, a radio frequency (RF), or any suitable combination thereof.

[0147] The computer-readable storage medium may be included in the textile fabric identification device based on the multimodal model; or may exist independently without being assembled into the textile fabric identification device based on the multimodal model.

[0148] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a textile fabric recognition device based on a multimodal model, the textile fabric recognition device based on the multimodal model: obtains a multi-perspective fabric image, a fabric language text, a fabric color and a fabric database corresponding to the multi-perspective fabric image; inputs the multi-perspective fabric image, the fabric language text and the fabric color into a target multimodal model to obtain a fusion feature; calculates the similarity between the fusion feature and the fusion feature in the fabric database; reorders the fusion features in the fabric database according to the similarity to obtain a reordering result; obtains a fabric category according to the reordering result to complete fabric recognition.

[0149] Computer program code for performing the operations of the present application may be written in one or more programming languages, including object-oriented languages ​​such as Java, Smalltalk, C++, and procedural languages ​​such as C. The code may be executed in whole or in part on a user's computer, as a stand-alone software package, or in part on a user and a remote computer. Where a remote computer is involved, it may be connected via a local or wide area network, or directly to an external computer, such as through an Internet service provider.

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart and block diagram represents a module or code portion, containing instructions to implement a specific logical function. The order of function execution may vary, such as parallel or reverse execution, depending on the functional requirements. Boxes and combinations thereof can be implemented with dedicated hardware or a combination of hardware and software.

[0151] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0152] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned textile fabric identification method based on a multimodal model, and can solve the technical problem of how to improve the accuracy and efficiency of textile fabric identification. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the textile fabric identification method based on a multimodal model provided in the above-mentioned embodiment, and will not be repeated here.

[0153] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned textile fabric identification method based on the multimodal model.

[0154] The computer program product provided by the present application can solve the technical problem of how to improve the accuracy and efficiency of textile fabric identification. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the textile fabric identification method based on the multimodal model provided by the above embodiment, which will not be described in detail here.

Claims

1. A textile fabric recognition method based on a multimodal model, characterized in that: The method comprises: Acquire multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and a fabric database; Inputting the multi-view fabric image, the fabric language text, and the fabric color into a target multimodal model to obtain fusion features; Calculating the similarity between the fused feature and the fused feature in the fabric database; Reordering the fusion features in the fabric database according to the similarity to obtain a reordering result; The fabric category is obtained according to the reordering result, and fabric identification is completed.

2. The method according to claim 1, characterized in that The textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, wherein the fabric recognition device comprises at least two detection cameras, and the focal lengths of the detection cameras are different; The step of acquiring a multi-view fabric image comprises: Acquire multiple groups of fabric images, each of which is captured by the detection camera, and each group of fabric images corresponds to each of the detection cameras one by one; A multi-view fabric image is determined according to each of the fabric images.

3. The method according to claim 1, characterized in that The textile fabric recognition method based on the multimodal model is applied to a fabric recognition device, the fabric recognition device comprises a first camera, a second camera, a third camera and a fourth camera, the focal length of the first camera is greater than the focal length of the second camera, the focal length of the second camera is greater than the focal length of the third camera, and the focal length of the third camera is greater than the focal length of the fourth camera; The step of acquiring a multi-view fabric image comprises: A multi-perspective fabric image is obtained according to the first-perspective fabric image, the second-perspective fabric image, the third-perspective fabric image and the fourth-perspective fabric image captured by the first camera, the second camera, the third camera and the fourth camera, wherein the first perspective is smaller than the second perspective, the second perspective is smaller than the third perspective, and the third perspective is smaller than the fourth perspective.

4. The method according to claim 1, characterized in that The multi-view fabric image includes a front multi-view fabric image and a back multi-view fabric image, the target multi-modal model includes a multi-modal feature encoder and a multi-modal feature fusion device, the multi-modal feature encoder includes a feature embedding layer, an attention layer and a fully connected layer, and the step of inputting the multi-view fabric image, the fabric language text and the fabric color into the target multi-modal model to obtain the fusion feature includes: Inputting the front multi-view fabric image, the back multi-view fabric image, the fabric language text and the fabric color into the multi-modal feature encoder in sequence, and extracting the underlying features of the multi-view fabric image and the fabric language text through the feature embedding layer; Downsampling the underlying features through the attention layer and reducing the feature dimension to obtain downsampled features; Extracting high-level semantic features from the downsampled features through the fully connected layer to obtain deep features; The deep features are fused through the multimodal feature fuser to obtain fused features.

5. The method according to claim 1, characterized in that Before the step of inputting the multi-view fabric image, the fabric language text and the fabric color into the target multimodal model to obtain the fusion feature, the step further includes: Obtain a target fabric dataset, and divide the target fabric dataset into a training set and a test set; Using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model; Testing and verifying the target detection model using the test set to obtain a test result; When the test result does not meet the preset requirements, returning to the step of training and adjusting the initial multimodal model using the training set; When the test result meets the preset requirement, the reference multimodal model is used as the target multimodal model.

6. The method according to claim 5, characterized in that The step of obtaining the target fabric data set comprises: Obtaining initial fabric data set and historical fabric data; Evaluating and screening the initial fabric data set to obtain data to be labeled; Comparing the data to be labeled with the historical fabric data to obtain similarity; The data to be labeled are sorted according to the similarity, and the data to be labeled whose similarity is lower than a preset similarity threshold is filtered to obtain a target fabric data set.

7. The method according to claim 5, characterized in that The training set includes image data, category data, fabric description language text data and fabric color data, the image data includes front-back multi-view image data, front multi-view image data, back multi-view image data, multi-view image data missing the second and third view, and first view front image data, the initial multimodal model includes a multimodal feature encoder, a multimodal feature fuser and a multimodal feature classifier, and the step of using the training set to train and adjust the initial multimodal model to obtain a reference multimodal model includes: Input the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third perspectives, or the first perspective front image data into the multimodal feature encoder to obtain the fabric multimodal depth feature; The multimodal depth features of the fabric are fused by the multimodal feature fuser to obtain multimodal fusion features of the fabric; Learning and distinguishing the multimodal fusion features of the fabric by the multimodal feature classifier to obtain the fabric category; When the fabric category does not match the category data, the multimodal feature encoder, the multimodal feature fusion device and the multimodal feature classifier are adjusted, and the step of inputting the fabric description language text data, the fabric color data, the front-back multi-view image data, the front multi-view image data, the back multi-view image data, the multi-view image data missing the second and third perspectives, or the first perspective front image data into the multimodal feature encoder is returned to obtain the fabric multimodal depth feature; When the fabric category matches the category data, a reference multimodal model is obtained according to the multimodal feature encoder, the multimodal feature fuser and the multimodal feature classifier.

8. A textile fabric recognition device based on a multimodal model, characterized in that: The device comprises: A data acquisition module, used to acquire multi-view fabric images, fabric language texts corresponding to the multi-view fabric images, fabric colors, and a fabric database; A model processing module, used for inputting the multi-view fabric image, the fabric language text and the fabric color into a target multimodal model to obtain a fusion feature; A similarity calculation module, used for calculating the similarity between the fused feature and the fused feature in the fabric database; A reordering module, used for reordering the fusion features in the fabric database according to the similarity to obtain a reordering result; The fabric identification module is used to obtain the fabric category according to the reordering result and complete fabric identification.

9. A textile fabric recognition device based on a multimodal model, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the textile fabric identification method based on a multimodal model as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the textile fabric identification method based on a multimodal model as described in any one of claims 1 to 7 are implemented.

Images