A method and system for manufacturing enterprise customer demand identification and profiling

Abstract

The present application belongs to the technical field of intelligent manufacturing and natural language processing, and discloses a manufacturing enterprise customer demand identification and portrait generation method and system: obtaining the text and product image of customer demand, establishing seven demand categories of product selection, material performance, specification customization, process and surface treatment, quality improvement, delivery cycle, after-sales and technical service; after being encoded by an image encoder and a text encoder respectively, the demand category probability distribution is output by image feature querying text feature and text feature querying image feature through the interaction of a bidirectional cross-modal attention module; the model is trained by using weighted cross-entropy loss, and the hyperparameters are optimized by adaptive coordinate search; the trained model is packaged as a local tool for controlled calling of a large language model, and a structured customer demand portrait is generated under the category constraint of the tool output. The complementary evidence of image and text can be fully utilized, the demand identification consistency is improved, and the structured customer portrait which can be checked and traced is generated.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent manufacturing, natural language processing, computer vision and customer relationship management, and in particular relates to a method and system for identifying and generating customer profiles for manufacturing enterprises based on multimodal data and large language models. Background Technology

[0002] Accurately identifying customer needs and creating customer profiles that can be reused in sales, design, production, and service processes is a crucial foundation for manufacturing enterprises to respond quickly to the market and implement flexible production. With the digitalization of customer interaction channels, demand information typically enters enterprises in multimodal forms, including inquiry texts, instant messaging content, email descriptions, product photos, sample images, or drawing screenshots. The same demand often simultaneously includes information on product appearance, materials, specifications, processes, quality, delivery, and service. Textual expressions are characterized by colloquialisms, abbreviations, and the mixing of industry terminology, while images may contain structures, surface conditions, and application scenarios not explicitly stated in the text.

[0003] To address the multimodal intent recognition problem, existing research attempts to jointly utilize textual and visual information through different encoders and fusion structures. Kiela et al. proposed a supervised multimodal bidirectional Transformer, fusing the representations of the text encoder and image encoder for image-text classification. Kruk et al. established intent classification data for multimodal documents composed of images and accompanying text, and verified that jointly using images and text can improve intent recognition. Zhang et al. proposed the MIntRec multimodal intent recognition dataset and baseline to verify the complementary roles of textual, visual, and linguistic cues in open-scene intent recognition.

[0004] To enhance the interaction between vision and language, Lu et al. proposed the ViLBERT two-stream structure, enabling the visual and text streams to interact through a collaborative attention layer; Tan et al. proposed LXMERT, which sets up an object relation encoder, a language encoder, and a cross-modal encoder to learn intramodal relations and image-text alignment relations. Regarding the collaboration between large language models and external tools, Schick et al. proposed Toolformer, which enables the language model to learn when to call external interfaces, what parameters to pass, and how to utilize tool results; Yao et al. proposed ReAct, which, by alternately generating inference trajectories and task actions, allows the language model to interact with external knowledge sources or the environment.

[0005] The aforementioned research provides a foundation for image-text classification, cross-modal representation, and the invocation of large language model tools. However, its tasks are mostly geared towards general content understanding, social media, or digital assistant intent, and have not yet established a unified category system that takes into account product, material, specification, process, quality, delivery, and service needs for manufacturing enterprise customers. Existing multimodal methods, when using feature splicing or general pre-trained structures, struggle to maintain the business binding relationship between image-text samples in manufacturing scenarios and explicitly utilize bidirectional complementary evidence. Directly using large language models to generate customer profiles suffers from issues such as unstable category definitions, missing fields, and difficulty in auditing the processing. Therefore, a technical solution is needed that addresses the needs of manufacturing enterprise customers, can perform bidirectional cross-modal recognition, and generates structured profiles in a controlled manner by combining local classification tools and large language models. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for identifying and generating customer profiles for manufacturing enterprises based on multimodal data and large language models, so as to solve the problems of diverse sources of customer needs, insufficient use of graphic and textual evidence, inconsistent category definitions, incomplete profile fields, and difficulty in auditing the generation process.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for identifying and generating customer profiles for manufacturing enterprises includes the following steps:

[0009] Step S1: Obtain the text of the customer's requirements and the corresponding product image, perform preprocessing, and maintain the binding relationship between the text and images;

[0010] Step S2: Establish seven requirement categories: product selection, material performance, specification customization, process and surface treatment, quality improvement, delivery cycle, after-sales and technical services. Use a visual language model to assist in obtaining candidate categories and confirm them manually.

[0011] Step S3: Divide the labeled valid samples into training set, validation set and test set according to category, and generate image patch sequence and character sequence as model input;

[0012] Step S4: The image sequence and text sequence are encoded by the image encoder and text encoder respectively. After interaction by the bidirectional cross-modal attention module, the text features are queried by image features and the image features are queried by text features. After fusing the category labels of the two modalities, the probability distribution of the seven requirement categories is output.

[0013] Step S5: Train the model using weighted cross-entropy loss, calculate the class weights based on the sample size of each class, and optimize the model hyperparameters sequentially through adaptive coordinate search;

[0014] Step S6: The trained demand identification model is encapsulated into a local tool for the large language model to call to obtain the main category, confidence level and candidate categories; the controller verifies the tool call and actively supplements the call when the large language model does not form a valid call; the large language model generates a structured customer demand profile under the category constraints of the tool output, and the controller verifies the fields of the profile.

[0015] Furthermore, the preprocessing described in step S1 includes performing decodeability verification, color mode conversion, size check, content validity check, and near-duplicate detection on the product image; for near-duplicate images, the main sample is retained and associated samples are recorded; for damaged, too small, abnormal channel, or obviously irrelevant images, they are marked as invalid; after cleaning, the correspondence between the image and the text remains unchanged.

[0016] Furthermore, in step S2, when the same sample contains multiple demand claims, the demand that plays a dominant role in the current sales or production decision is selected as the main category, and the other demands are recorded as profile attributes or candidate categories and retained.

[0017] Furthermore, the method for generating the image block sequence in step S3 is as follows: the standardized image is divided into multiple image blocks, each image block is flattened and projected onto the embedding space by linear mapping, and combined with the image category label and position code to form an initial image sequence; the method for generating the character sequence is as follows: the client text is converted into a character index sequence, and after being mapped by the character embedding function, it is combined with the text category label and position code to form an initial text sequence.

[0018] Furthermore, the bidirectional cross-modal attention module in step S4 includes a visual query branch and a text query branch; the visual query branch uses image features as the query vector and text features as the key vector and value vector, and is used to select descriptive information related to the image region from the text; the text query branch uses text features as the query vector and image features as the key vector and value vector, and is used to select visual evidence related to the textual claims from the image; the cross-modal outputs of the two branches are updated by residual connection, layer normalization and feedforward network, respectively, to obtain the updated image category label representation and text category label representation.

[0019] Furthermore, in step S5, the category weights are determined based on the number of samples in each category in the training set, and the category weights are inversely proportional to the number of samples in that category; the weighted cross-entropy loss is the average of the negative log-likelihood of each sample multiplied by the corresponding category weight; the adaptive coordinate search adjusts each hyperparameter sequentially on the validation set, changing only the value of one parameter at a time, selecting the value that makes the evaluation index optimal and fixing it before continuing to optimize the next parameter.

[0020] Furthermore, the model hyperparameters mentioned in step S5 include one or more of the following: target image size, image patch size, maximum length of text sequence, embedding dimension, number of attention heads, number of encoder layers, number of cross-modal interaction layers, training batch size, and learning rate.

[0021] Furthermore, the structured customer demand profile described in step S6 includes multiple items from customer identifier, demand category, original text, standardized demand, key attributes, intent description, explicit constraints, implicit constraints, information to be supplemented, visual evidence summary, text evidence summary, confidence level, candidate category list, knowledge entity, downstream retrieval question, and audit information. When the controller fails to validate the fields of the profile, it performs a limited number of retries. If it still fails, it uses a preset fallback template to generate a usable profile.

[0022] A customer demand identification and profile generation system for manufacturing enterprises, used to execute the aforementioned customer demand identification and profile generation method, includes: a multimodal data access module for acquiring customer demand text and corresponding product images from an enterprise information system; a data cleaning and labeling module for validating and cleaning product images and labeling samples based on a seven-category demand system; a dataset construction module for hierarchically dividing valid samples into training, validation, and test sets according to categories and generating model input sequences; a multimodal demand identification module, including an image encoder, a text encoder, and a bidirectional cross-modal attention module, for encoding images and text and performing cross-modal interaction, outputting the probability distribution of the seven demand categories; a training and parameter search module for training the model using weighted cross-entropy loss and determining model hyperparameters through adaptive coordinate search; a controlled tool invocation and profile generation module for encapsulating the trained demand identification model into a local tool, verifying the tool invocation of the large language model through a controller and actively supplementing the invocation, enabling the large language model to generate a structured customer demand profile under the constraints of the tool output; and a result output and auditing module for writing the structured profile into the target system and saving processing records.

[0023] Furthermore, the multimodal data access module connects to the enterprise's inquiry and quotation system, customer relationship management system, instant messaging records, or email system via interfaces or batch files; the multimodal demand identification module and the large language model are deployed on the enterprise intranet or in a controlled remote inference service environment; the result output and auditing module also feeds back the modification records of the profile fields by business personnel to the data cleaning and labeling module, which serves as the data source for subsequent model updates and verification labeling, forming a closed-loop process of data access, demand identification, profile application, and feedback updates.

[0024] The method and system for identifying and generating customer profiles for manufacturing enterprises according to the present invention have the following advantages:

[0025] This invention addresses the mixed text and image customer needs of manufacturing enterprises, maintaining a one-to-one correspondence between images, text, and sample labels. It organizes business tags using a unified seven-category system, improves the consistency of demand identification through bidirectional cross-modal attention of visual and text queries, and uses local classification tools to constrain a large language model to generate verifiable, reversible, and traceable structured customer profiles. This enhances the automation of customer demand processing and provides directly reusable data for sales assignment, process review, knowledge retrieval, and after-sales service. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0027] Figure 2 A flowchart for multimodal data acquisition, cleaning, and labeling.

[0028] Figure 3 This is a diagram illustrating the overall structure of the multimodal demand identification model.

[0029] Figure 4 This is a structural diagram of a bidirectional cross-modal attention module.

[0030] Figure 5 This is a flowchart of the model training and adaptive coordinate search process.

[0031] Figure 6 A timeline diagram for the collaborative generation of profiles by local recognition tools and large language models.

[0032] Figure 7 System architecture diagram for customer needs identification and profile generation. Detailed Implementation

[0033] To better understand the purpose, structure, and function of this invention, the following detailed description of a method and system for identifying and generating customer needs for manufacturing enterprises, in conjunction with the accompanying drawings, is provided.

[0034] like Figure 1 As shown, the present invention provides a method for identifying and generating customer profiles for manufacturing enterprises, comprising the following steps:

[0035] Step S1: Multimodal data acquisition and preprocessing

[0036] Obtain customer request texts and corresponding product images from the enterprise's inquiry and quotation system, customer relationship management system, instant messaging records, emails, or manual data collection channels. Create a sample identifier for each customer request and record the image path, text line identifier, customer identifier, collection time, and source channel.

[0037] The product images undergo decodeability verification, color mode conversion, size checking, content validity checking, and near-duplicate detection. Damaged files, excessively small images, images with abnormal channels, or obviously irrelevant images are marked as invalid; for near-duplicate images, the master sample is retained and related samples are recorded. The cleaning process does not alter the binding relationship between the original line numbers and image files to prevent mismatches between images and customer text.

[0038] Step S2: Label the seven requirement categories

[0039] Establish a unified system of requirement categories, numbered sequentially as follows: 1. Product selection requirements; 2. Material performance requirements; 3. Specification customization requirements; 4. Process and surface treatment requirements; 5. Quality improvement requirements; 6. Delivery cycle requirements; 7. After-sales and technical service requirements. Each category can be used as a model output category or as a standardized requirement type in the customer profile.

[0040] Customer requirement text and corresponding images are input into a visual language model to obtain candidate categories and judgment criteria. These candidate categories serve only as weak priors. Reviewers confirm the final category based on category definitions, positive and negative example boundaries, and conflict handling rules. When the same sample contains multiple requests, the requirement that dominates the current sales or production decision is selected as the primary category, and other requests are recorded as profile attributes or candidate categories.

[0041] Step S3: Dataset Construction

[0042] The verified valid samples are stratified into training, validation, and test sets according to category. The training set is used for parameter learning, the validation set is used for model selection and hyperparameter comparison, and the test set is used only for final performance evaluation. On the text side, a character vocabulary is built based solely on the training set. Unregistered characters are represented by unknown character tags, and fixed-length character sequences are obtained through truncation or padding.

[0043] Suppose the input image is segmented after standardization. The image patch, the first Each image patch is flattened into a vector. After linear mapping Projected to 3D space, and associated with image category labels and position encoding Combine them to obtain the initial image sequence. :

[0044] (1)

[0045] Suppose the customer text is converted to a length of character sequence to The character embedding function is Text category is marked as The text location is encoded as The initial text sequence is obtained. :

[0046] (2)

[0047] Step S4: Multimodal demand feature extraction and bidirectional interaction

[0048] Image encoder pairs The text encoder performs multi-head self-attention operations. Perform multi-head self-attention computation. The attention function is based on the query matrix. Key matrix Sum matrix Calculate the correlation, where Indicates the dimension of the key vector:

[0049] (3)

[0050] In the bidirectional cross-modal attention module, the visual query branch uses image features as the query and text features as the key and value to select material, specification, or process descriptions related to the image region from the text; the text query branch uses text features as the query and image features as the key and value to select structural, appearance, or defect evidence related to the textual claim from the image. The cross-modal increment of the two branches can be expressed as:

[0051] (4)

[0052] The cross-modal increment is obtained as an intermediate representation through a dropout layer, residual connections, and a first-layer normalization; then, it is obtained as an updated representation through its respective feedforward network, a second residual connection, and a second-layer normalization. The calculation process is as follows:

[0053] (5)

[0054] (6)

[0055] (7)

[0056] Represent the updated image category labels and text category tags represent Concatenate into a fusion vector After classification layer Bias The Softmax function yields the probability distributions of the seven demand categories. :

[0057] (8)

[0058] Step S5: Model Training and Adaptive Coordinate Search

[0059] To reduce the bias caused by imbalanced class distribution, class weights are calculated based on the size of the training set. Let the total number of training samples be... The total number of categories is , No. The number of samples in each class is Then the category weight for:

[0060] (9)

[0061] Let the first The true class of each sample is The model predicts the probability of this category as follows: The weighted cross-entropy loss is then:

[0062] (10)

[0063] Constrained adaptive coordinate search is used to determine the model's hyperparameters. During the search, a baseline configuration is set, and candidate values ​​for each coordinate are changed sequentially while keeping other parameters constant. The optimal value for the current coordinate is selected using predefined evaluation metrics on the validation set, and then written back to the baseline configuration for subsequent coordinates. Searchable coordinates include target image size, image patch size, text length, embedding dimension, number of attention heads, number of encoding layers, number of cross-modal layers, batch size, and learning rate.

[0064] Step S6: Controlled tool usage and customer needs profile generation

[0065] The trained demand recognition model is packaged into a local tool. The tool receives image paths and customer text, and outputs the main demand category, category confidence score, and previous demand information. There are several candidate categories. The large language model requests and invokes the tool according to preset system instructions; the controller verifies the tool name, parameters, and return structure. When the large language model fails to form a valid invocation, the controller proactively completes the invocation and fills the tool result back into the context of the large language model.

[0066] The large language model generates a structured customer requirement profile under the category constraints output by the local tool. The profile includes multiple elements from customer identifier, requirement category, raw text, standardized requirements, key attributes, intent, explicit constraints, implicit constraints, information to be supplemented, visual evidence, textual evidence, confidence level, candidate categories, knowledge entities, downstream retrieval questions, and audit information. The controller validates the fields and data types of the profile; if parsing fails, it performs a limited number of retries; if it still fails, it uses a fallback template to generate a usable profile.

[0067] This invention also provides a system for implementing the above method, including a multimodal data access module, a data cleaning and labeling module, a dataset construction module, a multimodal demand identification module, a training and parameter search module, a controlled tool invocation and profile generation module, and a result output and auditing module. Each module can be deployed on the same server or distributed across an enterprise intranet.

[0068] Example:

[0069] This embodiment provides a method for identifying and generating customer profiles for manufacturing enterprises based on multimodal data and large language models, including the following steps:

[0070] Step S1: Acquire and standardize multimodal data on manufacturing enterprise customer needs. Specifically, obtain customer requirement text and corresponding product images from the enterprise's inquiry and quotation system, customer relationship management system, instant messaging records, emails, or manual access points. Establish a sample identifier for each requirement and record the image path, text line identifier, customer identifier, collection time, and source channel. Figure 2 The process shown performs decodeability verification, color mode conversion, size check, content validity check, and near-duplicate detection on the image, while maintaining a one-to-one correspondence between the image file name, text line identifier, and processing status.

[0071] In one specific data processing scenario, a total of 2004 original images were processed. After content fingerprinting and perceptual feature comparison, 224 near-duplicate images were identified. Further checks on decodeability, size, channels, and content removed 87 abnormal or irrelevant images, resulting in 1693 valid images. The cleaning record retained the original filenames, corresponding text lines, and processing reasons. Processing statistics are shown in Table 1.

[0072] Table 1 Statistics on multimodal image data preprocessing

[0073]

[0074] Step S2: Establish seven customer requirement categories and create graphic and textual annotations. The seven requirement categories are numbered sequentially as follows: 1. Product selection requirement; 2. Material performance requirement; 3. Specification customization requirement; 4. Process and surface treatment requirement; 5. Quality improvement requirement; 6. Delivery cycle requirement; 7. After-sales and technical service requirement. Input the customer requirement text and corresponding images into the visual language model to obtain candidate categories, judgment criteria, and uncertainty explanations. Then, the reviewers confirm the final category based on the category definition, positive and negative example boundaries, and conflict handling rules. When the same sample contains multiple requirements, select the requirement that plays a dominant role in the current sales or production decision as the main category, and retain the remaining requirements as profile attributes or candidate categories.

[0075] 1660 image and text labeled samples that can be used for classification were generated from the valid data. Category 1 to Category 7 contain 169, 280, 284, 197, 269, 368 and 93 samples respectively. The category distribution is shown in Table 2.

[0076] Table 2 Sample Distribution of Seven Demand Categories

[0077]

[0078] For example, if the customer text contains both the product name and the delivery time, and the core requirement is to complete the delivery before the specified date, then the main category is marked as a 6-delivery cycle requirement, and the product name is retained in the profile as a key attribute.

[0079] Step S3 involves constructing training, validation, and test sets, and generating the model input sequence. The 1660 samples are divided into 1162 training samples, 249 validation samples, and 249 test samples according to a stratified category approach. The training set is used for model parameter learning, the validation set for model selection and hyperparameter comparison, and the test set for final performance evaluation. On the text side, a character vocabulary is built based solely on the training set, forming fixed-length character sequences using start markers, end markers, unknown character markers, and padding markers. On the image side, standardized images are cut into fixed-size blocks, and image category labels and positional encodings are added.

[0080] Step S4: Build and run a bidirectional cross-modal customer demand recognition model. In one feasible configuration, the input image is scaled to... Pixels and press Image blocks are segmented into pixels, the maximum length of client text is set to 160, the unified embedding dimension is set to 256, and the number of attention heads is set to 8. The image encoder is set to 4 layers, the text encoder to 2 layers, and the bidirectional cross-modal attention module to 2 layers. The above parameters can be adjusted according to deployment resources and sample size.

[0081] like Figure 3 As shown, the image patch sequence and the text character sequence are input into the bidirectional cross-modal attention module after being encoded intramodally. Figure 4 As shown, the visual query branch uses image features as the query and text features as the key and value, while the text query branch uses text features as the query and image features as the key and value. The cross-modal increments obtained from the two branches are updated through residual connections, layer normalization, and feedforward networks, respectively, and then the category labels of the two modalities are fused to output the probability distribution of seven demand categories.

[0082] For a material performance sample, the text contains semantics such as "high temperature resistance" and "corrosion resistance," while the image shows the corresponding component structure. The text branch provides material and performance constraints, while the image branch provides object and structural evidence. After bidirectional interaction, the main category is identified as two material performance requirements, while fields such as temperature range and corrosion environment, which require further confirmation, are retained to illustrate the combined use of image and text evidence.

[0083] Step S5: Train the customer demand identification model and determine the hyperparameters. Calculate the class weights based on the number of samples in each class in the training set, and train using weighted cross-entropy loss; in one training configuration, the batch size is 32, and the learning rate is 0.0005. During the training phase, save the current optimal model based on predefined evaluation metrics on the validation set, and load the optimal weights after training.

[0084] Parameter search according to Figure 5 The process shown compares the target image size, image patch size, text length, embedding dimension, number of attention heads, number of encoding layers, number of cross-modal layers, batch size, and learning rate sequentially. Each time, while keeping other coordinates constant, only the candidate value of one coordinate is changed, and the baseline configuration is updated. After selection, a one-time performance validation is performed on an independent test set.

[0085] To demonstrate that multimodal bidirectional interaction can achieve the intended functions, several representative models were selected for comparison, and the results are shown in Table 3. The method achieved an accuracy of 0.9398, a macro-average precision of 0.9212, a macro-average recall of 0.9183, and a macro-average F1 score of 0.9185 on the test set. These results are higher than those of the listed single-image model, single-text model, and text sequence combination model, indicating that joint modeling of image and text evidence can improve the overall recognition performance of the seven requirements.

[0086] Table 3 Comparison of representative demand identification models

[0087]

[0088] Step S6 involves encapsulating the model weights, character vocabulary, category mapping, and preprocessing parameters obtained in step S5 into a local demand identification tool, and generating a customer demand profile under controlled invocation conditions. For example... Figure 6 As shown, after receiving the profile generation task, the large language model requests to call the local demand recognition tool. The controller verifies the tool name, parameter format, and return structure, and fills the main category, confidence level, and candidate category back into the large language model context. The large language model combines the customer's original text, visual evidence, and tool results to generate a structured profile, and the controller then performs necessary field, data type, and category value verification.

[0089] When the large language model fails to generate effective tool calls, the controller supplements the calls to the requirement identification tool. When profile parsing fails or lacks necessary fields, a limited number of retries are performed. If the constraints are still not met after the retries, a fallback template is used. In a run containing 70 balanced sampling samples, 69 profiles were successfully parsed directly by the large language model, and 1 profile was completed through a controlled fallback process. Ultimately, all 70 profiles formed usable structured profiles.

[0090] For customer records with a main category of 6-delivery-cycle requirements, the profile can include: a standardized requirement of "deliver a specified quantity of products before the agreed date"; key attributes including product name, quantity, destination, and expected date; explicit constraints including the expected date, and implicit constraints including production capacity and logistics timeliness; supplementary information including the delivery address, whether partial delivery is allowed, and the tolerance for delays; and audit information recording the classification model version, tool call status, and rollback status. This profile retains the original requirement while providing a structured entry point for sales follow-up and production scheduling.

[0091] like Figure 7 As shown, the present invention also provides a system for implementing the above steps, including a multimodal data access module, a data cleaning and labeling module, a dataset construction module, a multimodal requirement identification module, a training and parameter search module, a controlled tool invocation and profile generation module, and a result output and auditing module. The data access module can access the enterprise system via an interface or batch files; the identification model and large language model can be deployed on the enterprise intranet, or controlled remote inference services can be used if data security requirements are met.

[0092] The output and auditing module writes the structured profile into the customer relationship management system or knowledge base, retaining the original input summary, model version, tool call results, parsing status, and processing time. When business personnel modify profile fields, the modification records can be used as the data source for subsequent annotation review and model updates, thus forming a closed loop of data access, requirement identification, profile application, and feedback updates.

[0093] The above specific embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; any modifications or equivalent substitutions made by those skilled in the art to the encoding method, attention structure, field set, deployment location or interface form without departing from the concept of the present invention shall not cause the essence of the corresponding technical solutions to depart from the protection scope of the present invention.

Claims

1. A method for identifying and generating customer profiles for manufacturing enterprises, characterized in that, Includes the following steps: Step S1: Obtain the text of the customer's requirements and the corresponding product image, perform preprocessing, and maintain the binding relationship between the text and images; Step S2: Establish seven requirement categories: product selection, material performance, specification customization, process and surface treatment, quality improvement, delivery cycle, after-sales and technical services. Use a visual language model to assist in obtaining candidate categories and confirm them manually. Step S3: Divide the labeled valid samples into training set, validation set and test set according to category, and generate image patch sequence and character sequence as model input; Step S4: The image sequence and text sequence are encoded by the image encoder and text encoder respectively. After interaction by the bidirectional cross-modal attention module, the text features are queried by image features and the image features are queried by text features. After fusing the category labels of the two modalities, the probability distribution of the seven requirement categories is output. Step S5: Train the model using weighted cross-entropy loss, calculate the class weights based on the sample size of each class, and optimize the model hyperparameters sequentially through adaptive coordinate search; Step S6: The trained demand recognition model is encapsulated as a local tool for the large language model to call to obtain the main category, confidence score and candidate category; the controller verifies the tool call and actively supplements the call when the large language model does not form a valid call; The large language model generates a structured customer demand profile under the constraints of the tool's output categories, and the controller performs field validation on the profile.

2. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The preprocessing described in step S1 includes performing decodeability verification, color mode conversion, size check, content validity check, and near-duplicate detection on the product image; for near-duplicate images, the main sample is retained and associated samples are recorded; images that are damaged, too small, have abnormal channels, or are obviously irrelevant are marked as invalid; after cleaning, the correspondence between the image and the text remains unchanged.

3. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, In step S2, when the same sample contains multiple demand claims, the demand that plays a dominant role in the current sales or production decision is selected as the main category, and the other demands are recorded as profile attributes or candidate categories and retained.

4. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The method for generating the image block sequence in step S3 is as follows: the standardized image is divided into multiple image blocks, each image block is flattened and projected into the embedding space by linear mapping, and then combined with the image category label and position code to form an initial image sequence; the method for generating the character sequence is as follows: the client text is converted into a character index sequence, and after being mapped by the character embedding function, it is combined with the text category label and position code to form an initial text sequence.

5. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The bidirectional cross-modal attention module described in step S4 includes a visual query branch and a text query branch. The visual query branch uses image features as the query vector and text features as the key and value vectors to select descriptive information related to the image region from the text. The text query branch uses text features as the query vector and image features as the key and value vectors to select visual evidence related to the textual claims from the image. The cross-modal outputs of the two branches are updated by residual connections, layer normalization, and feedforward networks to obtain the updated image category label representation and text category label representation.

6. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The category weights in step S5 are determined based on the number of samples in each category in the training set, and the category weights are inversely proportional to the number of samples in that category; the weighted cross-entropy loss is the average of the negative log-likelihood of each sample multiplied by the corresponding category weight; the adaptive coordinate search adjusts each hyperparameter sequentially on the validation set, changing only the value of one parameter at a time, selecting the value that makes the evaluation index optimal and fixing it before continuing to optimize the next parameter.

7. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The model hyperparameters mentioned in step S5 include one or more of the following: target image size, image patch size, maximum text sequence length, embedding dimension, number of attention heads, number of encoder layers, number of cross-modal interaction layers, training batch size, and learning rate.

8. The method for identifying and generating customer profiles for manufacturing enterprises according to claim 1, characterized in that, The structured customer demand profile described in step S6 includes multiple items from customer identifier, demand category, original text, standardized demand, key attributes, intent description, explicit constraints, implicit constraints, information to be supplemented, visual evidence summary, text evidence summary, confidence level, candidate category list, knowledge entity, downstream retrieval question, and audit information. When the controller fails to validate the fields of the profile, it performs a limited number of retries. If it still fails, it generates a usable profile using a preset fallback template.

9. A system for identifying and generating customer profiles for manufacturing enterprises, used to execute the method for identifying and generating customer profiles for manufacturing enterprises as described in any one of claims 1-8, characterized in that, include: The multimodal data access module is used to obtain customer requirement text and corresponding product images from the enterprise information system; The data cleaning and labeling module is used to verify and clean the product images and label the samples based on a seven-category requirement system. The dataset building module is used to hierarchically divide valid samples into training, validation, and test sets according to categories, and generate model input sequences; The multimodal demand recognition module includes an image encoder, a text encoder, and a bidirectional cross-modal attention module, which are used to encode images and text and perform cross-modal interaction, outputting the probability distribution of seven demand categories; The training and parameter search module is used to train the model using weighted cross-entropy loss and determine the model hyperparameters through adaptive coordinate search. The controlled tool invocation and profile generation module is used to encapsulate the trained demand identification model into a local tool, verify the tool invocation of the large language model through the controller and actively supplement the invocation, so that the large language model generates a structured customer demand profile under the constraints of tool output. The result output and audit module is used to write the structured profile into the target system and save the processing records.

10. The manufacturing enterprise customer demand identification and profile generation system according to claim 9, characterized in that, The multimodal data access module connects to the enterprise's inquiry and quotation system, customer relationship management system, instant messaging records, or email system via interfaces or batch files; the multimodal demand identification module and the large language model are deployed on the enterprise intranet or in a controlled remote inference service environment; the result output and auditing module also feeds back the modification records of the profile fields by business personnel to the data cleaning and annotation module, which serves as the data source for subsequent model updates and verification annotations, forming a closed-loop process of data access, demand identification, profile application, and feedback updates.

Images