Product customization method, device, medium, and system based on multi-modal interaction

Abstract

This application provides a product customization method, device, medium, and system based on multimodal interaction, belonging to the field of intelligent product customization technology. The method determines the product type and multiple product customization parameters of the product to be customized based on initial customization information; performs constraint verification on the product customization parameters based on a set of key parameters corresponding to the product type and basic physical constraints; matches the verified parameters with the set of key parameters, and generates interaction instructions based on mismatched parameter items; obtains multimodal interaction information from user feedback through a multimodal interaction module and updates the product customization parameters of the product to be customized; after the key parameter set matching is completed, generates a corresponding product recommendation scheme based on multiple product customization parameters and a product knowledge base, and visualizes the recommended customized product through a product rendering module. This method improves the accuracy of demand understanding in multi-round customization interactions, reduces invalid interaction rounds, and improves product customization efficiency.

Description

Technical Field

[0001] This application relates to the field of intelligent product customization technology, and in particular to a product customization method, device, medium and system based on multimodal interaction. Background Technology

[0002] With the rapid development of e-commerce and intelligent manufacturing technologies, consumer demand for personalized and customized products has exploded. Taking home furnishings, home appliances, wearable devices, and 3C products as examples, users want to customize the appearance (such as color and material), functional configuration (such as modular combination), size, and other parameters of products according to their own preferences.

[0003] Current personalized product customization typically uses web form selection or natural language processing-based question-and-answer systems to collect user configuration requirements and generate product solutions.

[0004] However, most existing personalized product customization solutions focus on processing single inputs or current commands, making it difficult to accurately identify ambiguous needs by combining multi-turn interaction contexts. This leads to problems such as fragmented interaction and misunderstanding of intent during the product customization process, which in turn affects the efficiency of product customization. Summary of the Invention

[0005] This application provides a product customization method, device, medium, and system based on multimodal interaction to solve the technical problem that most existing personalized product customization solutions process a single input or current instruction, making it difficult to accurately identify ambiguous needs by combining multiple rounds of interaction context. This leads to problems such as interaction fragmentation and intention misunderstanding during the product customization process, which in turn affects the efficiency of product customization.

[0006] In a first aspect, embodiments of this application provide a product customization method based on multimodal interaction, applied to an intelligent product customization system. The intelligent product customization system includes a multimodal interaction module and a product rendering module. The method includes:

[0007] Based on the initial customization information, the product type and multiple product customization parameters of the product to be customized are determined, and the multiple product customization parameters are constrained and verified based on the key parameter set corresponding to the product type and basic physical constraints. The initial customization information is used to represent the initial customization requirements input by the user through the multimodal interaction module. The key parameter set is a set of basic parameter items preset based on the product type of the product to be customized.

[0008] The validated product customization parameters are matched with the key parameter set, and interactive instructions are generated based on the parameter items in the key parameter set that do not match the product customization parameters.

[0009] The multimodal interaction module obtains multimodal interaction information from user feedback, and updates the product customization parameters of the product to be customized based on the multimodal interaction information.

[0010] After all parameters in the key parameter set are matched with the corresponding product customization parameters, a product recommendation scheme for the product to be customized is generated based on multiple product customization parameters and the product knowledge base. The recommended customized products corresponding to the product recommendation scheme are then visualized through the product rendering module.

[0011] In one possible implementation, the initial customization information includes at least one of text information, voice information, image information, and interface trigger information; based on the initial customization information, the product type of the product to be customized is determined, including:

[0012] The initial customization information is parsed and processed to obtain the product type of the product to be customized;

[0013] The parsing process includes at least one of the following:

[0014] Perform semantic recognition on text information;

[0015] Perform speech recognition and semantic analysis on speech information;

[0016] Image recognition is performed on image information;

[0017] The interface trigger information is parsed.

[0018] In one possible implementation, based on parameter items in the key parameter set that do not match the product customization parameters, an interactive instruction is generated, including:

[0019] Based on the parameter items in the key parameter set that do not match the product customization parameters, the corresponding missing parameter types are determined. The missing parameter types are used to characterize the parameter types in the key parameter set that do not match the product customization parameters.

[0020] Determine the target interaction template corresponding to the missing parameter type;

[0021] Generate interactive instructions based on the missing parameter type and the target interaction template.

[0022] In one possible implementation, based on the missing parameter type and the target interaction template, interaction instructions are generated, including:

[0023] Based on the missing parameter type, the corresponding interaction template is matched from the preset mapping relationship library, and the matched interaction template is determined as the target interaction template;

[0024] Fill the missing parameter type's parameter name into the preset fill position of the target interaction template to generate the interaction command;

[0025] The preset mapping library is used to store the correspondence between parameter types and interaction templates.

[0026] In one possible implementation, based on a set of key parameters corresponding to the product type and basic physical constraints, constraint verification is performed on multiple product customization parameters, including:

[0027] Match multiple product customization parameters with the set of key parameters corresponding to the product type, and determine the parameter items that match the set of key parameters as valid customization parameters;

[0028] Based on fundamental physical constraints, compliance verification is performed on multiple valid customized parameters;

[0029] The fundamental physical constraints include at least one of the following:

[0030] Size compliance constraints;

[0031] Material property constraints;

[0032] Performance threshold constraints.

[0033] In one possible implementation, the product knowledge base includes module information, parameter configuration constraints, and module combination constraints for each product; based on multiple product customization parameters and the product knowledge base, a product recommendation scheme for the product to be customized is generated, including:

[0034] Based on multiple product customization parameters, the module information in the product knowledge base is matched to obtain multiple candidate module information. The module information is the basic information that constitutes each functional unit of the product.

[0035] Based on information from multiple candidate modules, multiple initial product recommendation schemes are generated by combining them.

[0036] Based on parameter configuration constraints and module combination constraints, a secondary constraint verification is performed on multiple initial product recommendation schemes to filter and obtain the product recommendation scheme.

[0037] In one possible implementation, the product knowledge base further includes: initial models and module rendering data for each product; after determining the product type and multiple product customization parameters of the product to be customized based on the initial customization information, the method further includes:

[0038] Determine the initial model corresponding to the product type, and visualize the initial model through the product rendering module;

[0039] The product rendering module visualizes the recommended customized products corresponding to the product recommendation scheme, including:

[0040] Determine the module rendering data corresponding to the product recommendation scheme;

[0041] Based on the module rendering data corresponding to the product recommendation scheme, the initial model is rendered to generate a visual model of the recommended customized products.

[0042] Secondly, embodiments of this application provide a product customization device based on multimodal interaction, comprising:

[0043] The processing module is used to determine the product type and multiple product customization parameters of the product to be customized based on the initial customization information, and to perform constraint verification on the multiple product customization parameters based on the key parameter set corresponding to the product type and basic physical constraints. The initial customization information is used to represent the initial customization requirements input by the user through the multimodal interaction module. The key parameter set is a set of basic parameter items preset based on the product type of the product to be customized.

[0044] The processing module is also used to match the validated product customization parameters with the key parameter set, and generate interactive instructions based on the parameter items in the key parameter set that do not match the product customization parameters.

[0045] The acquisition module is used to acquire multimodal interaction information from user feedback through the multimodal interaction module.

[0046] The processing module is also used to update the product customization parameters of the product to be customized based on multimodal interaction information.

[0047] The processing module is also used to generate a product recommendation scheme for the product to be customized based on multiple product customization parameters and the product knowledge base after all parameter items in the key parameter set have been matched with the corresponding product customization parameters, and to visualize the recommended customized products corresponding to the product recommendation scheme through the product rendering module.

[0048] In one possible implementation, the processing module is further configured to parse the initial customization information to obtain the product type of the product to be customized.

[0049] The parsing process includes at least one of the following:

[0050] Perform semantic recognition on text information;

[0051] Perform speech recognition and semantic analysis on speech information;

[0052] Image recognition is performed on image information;

[0053] The interface trigger information is parsed.

[0054] In one possible implementation, the processing module is further configured to determine the corresponding missing parameter type based on parameter items in the key parameter set that do not match the product customization parameters. The missing parameter type is used to characterize the parameter type in the key parameter set that does not match the product customization parameters.

[0055] The processing module is also used to determine the target interaction template corresponding to the missing parameter type.

[0056] The processing module is also used to generate interactive instructions based on the missing parameter type and the target interaction template.

[0057] In one possible implementation, the processing module is further configured to match the corresponding interaction template from a preset mapping relationship library based on the missing parameter type, and determine the matched interaction template as the target interaction template.

[0058] The processing module is also used to fill the parameter names of missing parameter types into the preset fill positions of the target interaction template to generate interaction instructions;

[0059] The preset mapping library is used to store the correspondence between parameter types and interaction templates.

[0060] In one possible implementation, the processing module is further configured to match multiple product customization parameters with a set of key parameters corresponding to the product type, and to determine the parameter items that match the set of key parameters as valid customization parameters.

[0061] The processing module is also used to perform compliance checks on multiple valid customized parameters based on fundamental physical constraints;

[0062] The fundamental physical constraints include at least one of the following:

[0063] Size compliance constraints;

[0064] Material property constraints;

[0065] Performance threshold constraints.

[0066] In one possible implementation, the processing module is further configured to match module information in the product knowledge base based on multiple product customization parameters to obtain multiple candidate module information, wherein the module information is the basic information constituting each functional unit of the product.

[0067] The processing module is also used to combine information from multiple candidate modules to generate multiple initial product recommendation schemes.

[0068] The processing module is also used to perform secondary constraint verification on multiple initial product recommendation schemes based on parameter configuration constraints and module combination constraints, and then filter to obtain the product recommendation scheme.

[0069] In one possible implementation, the processing module is further configured to determine an initial model corresponding to the product type and to visualize the initial model through the product rendering module.

[0070] The processing module is also used to determine the module rendering data corresponding to the product recommendation scheme.

[0071] The processing module is also used to perform module rendering on the initial model based on the module rendering data corresponding to the product recommendation scheme, and generate a visual model of the recommended customized products.

[0072] Thirdly, embodiments of this application provide an intelligent product customization system, which includes a multimodal interaction module, a product rendering module, and a controller;

[0073] The multimodal interaction module is used to obtain the initial customization information and multimodal interaction information input by the user; the product rendering module is used to render the recommended customized products in the product recommendation scheme.

[0074] The controller is used to execute the methods described in the first aspect and various possible implementations of the first aspect above to generate a product recommendation scheme.

[0075] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0076] The memory stores instructions that the computer executes;

[0077] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0078] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0079] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0080] The product customization method based on multimodal interaction provided in this application involves determining the product type and multiple product customization parameters of the product to be customized based on initial customization information; matching the multiple product customization parameters with a set of key parameters corresponding to the product type, and identifying the parameters that match the key parameter set as valid customization parameters; then performing compliance verification on the multiple valid customization parameters based on basic physical constraints; matching the verified product customization parameters with the set of key parameters, and determining the corresponding missing parameter types based on the parameter items in the set that do not match the product customization parameters; determining the target interaction template corresponding to the missing parameter type, and then... The system generates multimodal interaction commands from a target interaction template. It then acquires multimodal interaction information from user feedback via a multimodal interaction module and updates the product customization parameters based on this information. After all parameters in the key parameter set match their corresponding product customization parameters, it matches module information in the product knowledge base with multiple product customization parameters to obtain multiple candidate module information. Based on these candidate module information, it combines and generates multiple initial product recommendation schemes. Finally, based on parameter configuration constraints and module combination constraints, it performs secondary constraint verification on these initial product recommendation schemes, filters them to obtain the final product recommendation scheme, and uses a product rendering module to visualize the recommended customized products corresponding to the product recommendation schemes. This method parses multimodal inputs into unified structured parameters and continuously links multimodal input parsing, product type identification, key parameter completion, basic physical constraint verification, recommendation scheme generation, and rendering display to construct a closed-loop customization processing mechanism for the same product. This significantly improves the parsing accuracy of fuzzy commands, reduces the number of interaction rounds with users, and greatly improves interaction efficiency. By actively guiding interaction commands, it solves the problem of low interaction efficiency caused by single-round intent parsing. It not only improves the accuracy of demand understanding and the completeness of parameter acquisition in multi-round customization interactions, but also improves product customization efficiency and further enhances the user experience. Attached Figure Description

[0081] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0082] Figure 1 This is a flowchart illustrating the product customization method based on multimodal interaction provided in this application. Figure 1 ;

[0083] Figure 2 This is a flowchart illustrating the product customization method based on multimodal interaction provided in this application. Figure 2 ;

[0084] Figure 3This is a schematic diagram of the structure of the product customization device based on multimodal interaction provided in this application;

[0085] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application.

[0086] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0088] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0089] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0091] First, let me explain the terms used in this application:

[0092] Natural Language Understanding Model: Used to perform semantic parsing on natural language text input by users, identify the actual meaning, needs and implicit information expressed in the text, and realize the understanding and structured processing of natural language content.

[0093] Keyword extraction model: used to automatically extract keywords that can represent core content, needs or topics from natural language text, providing key feature information for subsequent processing such as intent recognition and parameter matching.

[0094] Intent classification model: used to determine and classify the user's current interaction intent based on the user's input, such as query intent, configuration intent, confirmation intent, cancellation intent, etc.

[0095] Semantic classification model: used to divide text content according to semantic categories, realize the semantic classification and intent refinement of user input, and distinguish the demand types under different semantic scenarios.

[0096] Image classification model: used to perform feature recognition and category determination on input image data, identify objects, scenes, states or identification information in the image, and realize structured parsing of image content.

[0097] Rule matching engine: Based on preset logical rules, matching conditions and judgment strategies, it performs rule retrieval and condition matching on input information, and quickly executes deterministic logical judgments and instruction distribution.

[0098] With the rapid development of e-commerce and intelligent manufacturing technologies, consumer demand for personalized customized products has exploded. Personalized product customization technology is widely used in e-commerce platforms, brand official stores, intelligent manufacturing front-end order receiving platforms, and digital interactive terminals in offline stores. It is especially suitable for product customization businesses with multi-parameter combination characteristics, such as smart wearable devices, home appliances, furniture, and cups and containers. Among them, product customization has gradually evolved from the traditional fixed model selection to an online configuration process that is oriented towards multiple categories, multiple parameters, and multiple rounds of interaction.

[0099] In the above scenarios, users typically do not provide all configuration requirements at once, but rather express their customization intentions gradually through various means such as text input, voice expression, uploading reference images, and clicking on interface options.

[0100] Taking home furnishings, home appliances, wearable devices, and 3C products as examples, users want to customize the appearance (such as color and material), functional configuration (such as module combination), size, and other parameters of products according to their own preferences. For example, a user might describe "I want a glass door refrigerator with a silent function" by voice, or select a specific material by uploading an image, and at the same time hope that the system can generate a 3D model in real time for them to confirm intuitively.

[0101] Existing personalized product customization solutions mainly include parameter selection based on web forms and simple question-and-answer methods based on natural language processing.

[0102] Specifically, the web form method relies on preset fixed controls such as drop-down boxes, radio buttons, and sliders to break down product parameters into several static options for users to select one by one, which is suitable for scenarios with clear requirements and fixed parameter ranges; the natural language question answering method parses user input through text or speech recognition, and then generates corresponding product parameters by combining preset vocabulary or intent recognition models.

[0103] However, existing product customization platforms mostly rely on fixed options or single-round semantic parsing. Users need to switch repeatedly between text forms, fixed option interfaces, and static images. The system has difficulty associating the context of multi-round dialogues, resulting in a cumbersome and inefficient customization process. Furthermore, it is difficult to accurately identify ambiguous needs by combining multi-round interaction contexts, which leads to problems such as interaction fragmentation, misunderstanding of intent, and reduced conversion efficiency in the product customization process, thus affecting the efficiency of product customization.

[0104] Furthermore, existing solutions often treat voice input, text input, image references, and interface clicks as separate data sources, failing to form a unified understanding of user needs. This makes it difficult for information submitted by users in different interaction channels to corroborate and supplement each other.

[0105] To address the aforementioned issues, this application proposes a product customization method based on multimodal interaction, applied to intelligent product customization systems. This method deeply integrates multimodal interaction, dialogue state tracking, and real-time 3D rendering technologies to construct a structured and collaborative personalized product customization process. By uniformly parsing multimodal inputs of user needs (text, voice, image, interface operation) into structured parameters, and maintaining the global context and process state through a dialogue state tracking module, it proactively guides users to complete product type selection and parameter feedback, generating multiple products to be customized. These products are then subjected to constraint verification to obtain compliant products that meet assembly and parameter constraints. Furthermore, real-time 3D rendering technology dynamically associates customization parameters with product models, providing visualization of compliant products determined based on user needs. This method, geared towards intelligent product customization scenarios, optimizes interaction efficiency by utilizing interactive commands to express user needs and guiding users to obtain customization requirements through product customization parameters. It solves the problem of low interaction efficiency caused by single-round intent parsing, reduces invalid interaction rounds, and not only improves the accuracy of requirement understanding and the completeness of parameter acquisition in multi-round customization interactions but also enhances product customization efficiency.

[0106] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0107] Figure 1 A flowchart illustrating the product customization method based on multimodal interaction provided in this application embodiment. Figure 1 The execution entity in this embodiment can be, for example, an intelligent product customization system equipped with a multimodal interaction module and a product rendering module. Figure 1 As shown in this embodiment, the product customization method based on multimodal interaction includes:

[0108] S101: Based on the initial customization information, determine the product type and multiple product customization parameters of the product to be customized, and perform constraint verification on the multiple product customization parameters based on the key parameter set corresponding to the product type and basic physical constraints.

[0109] The initial customization information is used to represent the initial customization requirements entered by the user in a multimodal manner.

[0110] In this embodiment of the application, the intelligent product customization system is used to carry out the execution of the entire product customization process. After the user inputs the initial customization requirements, the system can parse, verify, complete, recommend, and display the requirements.

[0111] Smart product customization systems are widely used in home appliances, wearable devices, 3C digital products, office supplies and consumer goods. Users typically need to complete customization based on multiple dimensions such as product type, appearance, material, size, functional modules, and capacity configuration. During the interaction process, recommended solutions and visualization results are provided in a timely manner (based on the product rendering module).

[0112] The intelligent product customization system includes a user interaction layer, which comprises a multi-module interaction module, an intelligent dialogue service interface, a product module selection interface, a multimodal input parsing module, and an interface operation parsing module. The multimodal interaction module is the core interaction execution unit of the user interaction layer. This multimodal interaction module is a functional unit used to receive, aggregate, and parse data from different input channels. The input channels corresponding to this multimodal interaction module include at least one of the following: text input channel, voice input channel, image input channel, and graphical interface operation input channel.

[0113] Specifically, the multimodal interaction module is used to coordinate the display logic of the intelligent dialogue service interface and the product module selection interface, and to receive the parsing results output by the multimodal input parsing module and the interface operation parsing module, thereby completing the unified processing and instruction distribution of user multimodal interaction behaviors. Furthermore, the intelligent dialogue service interface is associated with the multimodal input parsing module, and the user input data obtained by the intelligent dialogue service interface is parsed by the multimodal input parsing module; the product module selection interface is associated with the interface operation parsing module, and the user input data obtained by the product module selection interface is parsed by the interface operation parsing module. This application does not impose any special restrictions on the configuration of the multimodal interaction module.

[0114] Understandably, initial customization information refers to the demand description data provided by the user at the beginning of the smart product customization process, which is used to characterize the direction of the product to be customized at that time. This data can be, for example, a clear product name, or descriptive information with purpose, appearance, size preference, or component characteristics.

[0115] Product type is used to identify the category to which the product belongs in the subsequent parameter completion process, such as water cup, headphones, backpack, monitor, watch or office chair, etc. It is used to establish the basic constraints for the scope of subsequent parameter acquisition and recommendation rules, and is the basis for subsequent key parameter set calls, basic physical constraint matching and product knowledge base retrieval.

[0116] Product customization parameters are used to describe the specific configuration of the product to be customized, such as size, color, material, capacity, functional modules, style features or structural form, etc.

[0117] The key parameter set is a set of basic parameter items preset based on the product type, which is used to limit the specific parameter range that must be obtained for this type of product before generating a recommendation solution.

[0118] Basic physical constraints are a set of rules to ensure the physical feasibility of the product customization parameters input by the user, such as size compliance constraints, material property constraints, and performance threshold constraints.

[0119] The product knowledge base is used to store product types, key parameter items, parameter value relationships, combination rules, and data content required for subsequent recommendation and rendering.

[0120] For example, the intelligent product customization system can be deployed on an e-commerce platform server, a brand private customization server, or a store terminal and cloud collaborative architecture, where the processor calls program instructions in memory to execute this method.

[0121] The multimodal interaction module first receives the raw input data from the user's current session. When the user enters "I want to customize a blue water cup" through the input box, the intelligent product customization system performs word segmentation, entity recognition, and intent extraction on the text, identifying "water cup" as a product category candidate and "blue" as an appearance attribute candidate, thus determining the product type as a water cup, with the product customization parameters including "color - blue". When the user inputs voice through the microphone, the system first performs endpoint detection, noise reduction, and speech-to-text on the voice stream, and then performs semantic analysis on the transcribed text. When the user uploads a product reference image, the system performs object detection, contour recognition, component recognition, and similar category comparison on the image to extract structural features such as the cup body, watch strap, keyboard layout, and shell shape. When the user clicks on a product category entry, drags an appearance template, or selects a function tag through the interface, the system converts the interaction behavior into structured behavior events and maps them to corresponding requirement descriptions. After unified semantic mapping, the above analysis results are converted into a structured set of candidate parameters, which includes the product type and at least one product customization parameter.

[0122] For each product type acquired, the set of key parameters corresponding to that product type is retrieved from the product knowledge base. For example, the set of key parameters for a smartwatch may include dial size, strap material, screen type, battery capacity, and waterproof rating; the set of key parameters for a cup or kettle may include capacity, material, sealing structure, lid type, and temperature range; and the set of key parameters for furniture may include external dimensions, main material, load-bearing capacity, color scheme, and structural combination.

[0123] During the constraint verification phase, the system screens and verifies multiple product customization parameters extracted based on the key parameter set and fundamental physical constraints. This verification process can include parameter format standardization, unit unification, value range detection, logical consistency detection between parameters, and physical feasibility assessment. Taking size-related parameters as an example, the system can unify different units such as millimeters, centimeters, and inches into internal standard units before determining whether they fall within the range allowed by the product type. Taking material-related parameters as an example, the system can check whether the material is compatible with the product's intended use. For instance, whether the material of food contact containers meets temperature resistance requirements, and whether the strap material of smart wearable products meets flexibility and skin contact compatibility requirements. In other words, when a user customizes a smartwatch, if the screen size parameters are inconsistent with the case size parameters, or if there is a significant conflict between the high-brightness display requirement and the battery capacity constraint, the system can identify this in advance during this step.

[0124] If the verification passes, the system writes the valid parameters into the parameter status table of the current session and generates the structured parameter data required for subsequent matching. If there are conflicts or invalid parameters, the system can mark the conflict item as pending confirmation and record the source of the conflict, the type of conflict, and the associated parameters to guide the user to make corrections in subsequent steps.

[0125] Understandably, the embodiments of this application complete product type identification, parameter extraction, and constraint verification before parameters enter the completion and recommendation process. This enables the system to form a unified, structured, and physically feasible expression of user needs in multi-source input scenarios, reducing invalid customization links caused by parameter ambiguity, inconsistent units, or combination conflicts, thereby providing a stable data foundation for subsequent interactive completion and recommendation generation.

[0126] To ensure consistent processing of inputs from different modalities, the intelligent product customization system can set up a unified parsing unit to convert text semantic results, speech-to-text results, image recognition results, and interface event results into a unified semantic slot representation. Specifically, this can include fields such as "product category", "applicable scenario", "appearance preference", "structural features", and "adjustment intention". Then, based on the category confidence in the current recognition field, the product type and corresponding product customization parameters of the product to be customized are determined.

[0127] In one possible implementation, if multiple candidate categories exist in the same round of input, for example, if a user inputs "want a lightweight display device suitable for office use," the smart product customization system can combine keywords such as "office," "lightweight," and "display device," as well as the historical session context, to determine whether the customization target is closer to a portable monitor, laptop stand display device, or tablet product, and output the highest matching category based on a preset category determination model. This application does not impose any special limitations on the preset category determination model.

[0128] If a user uses vague expressions such as "bigger" or "lighter material", and a confirmed product context already exists in the current session, the intelligent product customization system first reads the confirmed product object in the current session state, and then attaches the vague expression to the current product object for interpretation, thereby avoiding misidentification of the vague description as a new product type.

[0129] When extracting multiple product customization parameters, the system maps the descriptions identified in the initial customization information to key or extended parameter items. For relative descriptions such as "lighter," "larger," and "lower profile," the system can combine the parameter semantic template of the product type and the mapping relationship in the product knowledge base to transform them into computable parameter constraints. For example, "lighter" can be mapped to a weight upper limit constraint or a tendency for lightweight materials; "larger" can be mapped to a size increment range or an increase in the lower limit of capacity; and "lower profile" can be mapped to style parameters such as lower color saturation, darker color scheme, or simplified appearance texture. For visual features extracted from images, the system can map them to appearance style parameters, component outline parameters, or candidate module identifiers. If a certain input content cannot be directly mapped to a unique parameter value, the system can first record it as a parameter intent and further confirm it in subsequent interactions.

[0130] In addition, if the initial customization information contains multiple objects, such as "I want to customize a cup and a cup sleeve", the smart product customization system can select the main object based on the constraints of the current interaction task, or trigger a task splitting process to establish a customization process for the object with stronger current customization needs of the user.

[0131] In one possible implementation, to improve the accuracy of product type identification, the intelligent product customization system can also introduce a semantic classification model, an image classification model, and a rule matching engine for joint determination. The rule matching engine is used to process common aliases, colloquialisms, and combined descriptions in the industry, such as mapping "thermos", "portable cup", and "sports water bottle" to sub-product types under the category of drinking containers.

[0132] In one possible implementation, the system can also correct the recognition results based on the user's account browsing history, purchased configuration records, or recent session content. For example, if a user has been browsing coffee equipment recently, "wanting a graduated one" can be interpreted more likely as a pour-over coffee cup rather than a regular measuring cup. In this case, the smart product customization system outputs a clear product type identifier and writes the identifier into the current customization session state, providing a basis for extracting the key parameter set corresponding to the product type in the future.

[0133] In addition, when generating interactive commands, the system prioritizes obtaining parameters that the user has previously focused on. For example, for users who have repeatedly focused on the "portability" and "leakage prevention" functions, the system prioritizes carrying methods, sealing structures, and weight ranges, and displays relevant options in the interactive interface to shorten the user's decision-making path.

[0134] In this embodiment of the application, this step transforms multi-source heterogeneous inputs into a structured requirement description and completes product type normalization identification in the session context. This enables the intelligent product customization system to stably enter the subsequent customization process even when the user's expression is not fixed and the information source is not singular. This reduces the identification deviation caused by dispersed entry points and ambiguous expressions, and improves the continuity of multi-round interactions and the accuracy of the initial judgment.

[0135] S102: Match the validated product customization parameters with the key parameter set, and generate interaction instructions based on the parameter items in the key parameter set that do not match the product customization parameters.

[0136] Among them, the key parameter set is a set of parameter items used to characterize product customization requirements.

[0137] Understandably, verified product customization parameters refer to the set of parameters that have passed the basic physical constraint verification and have been confirmed as being able to enter the subsequent process. The set of key parameters is the minimum complete parameter benchmark under the current product type.

[0138] The key parameter set refers to a set of parameters that are related to the product type and are used to support the decision-making process for the complete intelligent product customization. This set includes at least the core parameters that must be obtained for the current product customization, and may also include supplementary parameters, constraint parameters, and ranking parameters.

[0139] Interactive instructions refer to the guiding interactive information output by the intelligent product customization system to the user. This guiding interactive information can be text, voice broadcast, interface pop-up, parameter selection card, image annotation prompts, or a combination of multiple forms. Its form can be question prompts, option recommendations, conflict confirmation prompts, example display requests, or operational information such as guiding users to upload supplementary pictures, supplementary voice instructions, or click on interface controls. The purpose of these interactive instructions is to drive users to supplement product customization parameters that have not yet been fully provided.

[0140] In this step, the multimodal interaction module is not only responsible for outputting interaction commands to the user, but also for adapting the output method according to the device type. For example, it outputs text and options on the web page, card-style questions and answers and voice broadcast on the mobile terminal, and touch interface prompts with reference pictures on the store terminal.

[0141] In this embodiment, the intelligent product customization system further includes a core control layer, which comprises a dialogue state tracking module and a product knowledge base. The dialogue state tracking module is mainly used to track the completeness of the product customization parameters input by the user during the current product customization process. Specifically, at the initial stage of interaction, it generates corresponding interaction instructions based on parameter items in the key parameter set that do not match the currently acquired product customization parameters. The dialogue state tracking module is also used to perform completeness verification on the currently determined product customization parameters after completing at least one user interaction, determine whether it is necessary to continue interacting with the user to supplement the missing parameter items, and control the start and stop of the multimodal interaction process based on the judgment result.

[0142] The product knowledge base pre-stores the mapping relationship between product types and key parameter sets; the dialogue state tracking module can call the key parameter set corresponding to the current product type from the product knowledge base to perform integrity verification and interactive control of the currently acquired product customization parameters.

[0143] In this embodiment of the application, the intelligent product customization system establishes a key parameter matching table and compares each parameter item in the key parameter set with the identified parameters in the current parameter status table item by item; wherein, the comparison method can be parameter name mapping, alias normalization, thesaurus matching, and semantic slot alignment.

[0144] At least one product customization parameter that passes the current verification is matched with the key parameter set corresponding to the product type. If a parameter item in the key parameter set already has a corresponding valid parameter value, the parameter item is marked as matched. If there is only a vague intent or interval constraint but no clear executable parameter value, the parameter item is marked as partially matched. If there is no corresponding information at all, it is marked as not matched. Based on the parameter items in the key parameter set obtained this time that do not match the product customization parameters, the interaction template corresponding to the missing customization parameters is read, and the corresponding interaction instructions are generated based on this.

[0145] In one possible implementation, the smart product customization system can set different interaction strategies for matched, partially matched, and unmatched parameters to reduce redundant questioning and improve interaction efficiency. For example, for matched parameters, the system will not ask again but will display the current state on the interface; for partially matched parameters, it will generate confirmation-type interaction instructions, such as prompting the user to select from a specific size range for the description "larger"; for unmatched parameters, it will generate completion-type interaction instructions, such as asking for basic items such as capacity, material, or color.

[0146] For example, if the product type determined in this instance is a smartwatch, the parameter template stored in the product knowledge base includes fields such as: dial size, dial material, strap material, screen type, battery life, water resistance rating, function configuration, body color, compatible models, and weight specifications. Among these, dial size, dial material, and strap material are marked as mandatory parameters, body color and appearance texture can be marked as optional parameters, and water resistance rating and weight specifications can be marked as restrictive parameters. After reading the parameter template corresponding to the smartwatch, the system compares the known parameters in the current session state and identifies the parameter items corresponding to the currently acquired product customization parameters. For example, if the parameter item "black" obtained in the initial customization information indicates that it already corresponds to the body color parameter, then the body color item will no longer be the focus of the first round of follow-up questions. Instead, the mandatory parameters that have not yet been obtained will be placed with higher priority to generate the corresponding interaction commands.

[0147] If the system recognizes "lighter" or "larger screen" but has not yet confirmed the strap material and battery life, it can first generate an interaction command such as "more suited for sports or business use," because this question can simultaneously affect the inference of strap material, case style, and weight target. After the user selects sports use, it can then further inquire about the water resistance rating or strap material.

[0148] In one possible implementation, the intelligent product customization system generates corresponding interactive instructions based on the parameters to be completed for the customized product. If a single key parameter is missing (e.g., a required parameter), the system can generate a direct query instruction, such as "Please confirm the capacity range." If multiple key parameters are missing, the system can generate step-by-step guided instructions, issued sequentially according to structural constraints, usage scenario constraints, and appearance preference constraints. If ambiguous user input is detected, such as "lighter overall weight" or "larger capacity," the system can generate a clarifying instruction, mapping the ambiguous words to possible parameter dimensions and requesting confirmation, such as prompting the user to choose between weight, size, or capacity. If the system identifies a linkage between at least two parameters in the product knowledge base, such as a certain material only supporting a specific cup lid structure, the system can simultaneously display the optional range in the interactive instructions, thereby reducing subsequent conflicting modifications.

[0149] In one possible implementation, interactive instructions can be generated using a combination of template generation and intelligent rewriting. The template part ensures the accuracy of parameters, units, and business logic, while the intelligent rewriting part ensures that the prompts remain clear, concise, and contextually consistent across different terminals and user groups.

[0150] Meanwhile, the generation of interactive commands not only relies on the parameter templates themselves, but can also be dynamically adjusted based on user profiles and the stage of the interaction. For example, for users who have repeatedly focused on "portability" and "leakage prevention," the system will prioritize asking for parameters such as carrying method and sealing structure at the beginning of the interaction; for users operating on mobile devices, the system can prioritize generating short text questions with clickable tabs; for conversations with clearly defined usage scenarios such as in-vehicle, kitchen, and outdoor use, the system can prioritize parameters related to the scenario, first asking for parameters strongly related to the scenario such as heat resistance, drop resistance, and fixing method.

[0151] To ensure the consistency of multimodal interaction, the intelligent product customization system can output the same interactive intent to multiple channels simultaneously. For example, it can display a text question in a chat window, simultaneously display candidate option buttons in the interface, and read the question content through a voice channel, allowing users to answer using any input method.

[0152] In this embodiment, the intelligent product customization system can maintain a corresponding parameter identifier, parameter name, parameter data type, whether the parameter is required, set of optional parameter values, parameter value range, parameter dependency relationship, and parameter display template for each parameter item; therefore, the generation of interactive instructions can be based on automatically concatenating questions, selectors, and prompts from different fields.

[0153] For example, when the parameter data type is an enumeration type, the interface generates a radio button; when the parameter data type is a numerical range, the interface generates a slider with voice prompts; when the parameter depends on the result of the previous step, only sub-options compatible with the currently confirmed configuration are displayed; in addition, the system can also maintain the parameter completion status, such as not obtained, obtained, pending confirmation, conflict pending correction, etc., to determine the interaction content of the current round.

[0154] Understandably, this step establishes a mapping link between "product type - key parameter set - interaction command," enabling the system to automatically organize the acquisition path of missing parameter items in the key parameter set corresponding to the product type after identifying the product type and the determined product customization parameters. This eliminates the need for users to manually search for parameter entry points across multiple pages, effectively improving the orderliness and completeness of parameter completion and providing a structured and clearly defined input foundation for subsequent recommendation scheme generation.

[0155] In other words, the system can transform missing information in the set of key parameters into specific, actionable user interaction tasks, and make the interaction content revolve around the current product type and confirmed parameters, rather than using a static form to fill in all parameters. This significantly reduces the cognitive burden on users in the process of customizing complex products, gives the parameter completion process a clear direction, and avoids the system from entering the recommendation stage too early when the parameters are incomplete, which could lead to unstable results. At the same time, the system has the ability to proactively guide the completion of key parameters, thereby improving the continuity, relevance, and completeness of parameter collection in multi-round interaction processes.

[0156] S103: Obtain multimodal interaction information from user feedback through the multimodal interaction module, and update the product customization parameters of the product to be customized based on the multimodal interaction information.

[0157] In this embodiment, multimodal interaction information is used to represent the feedback content made by the user after receiving the interaction instruction. For example, it may include text reply, voice reply, uploading pictures, selecting interface options, dragging a slider, clicking a recommendation tag, or performing a modification operation on the system recommendation results.

[0158] The intelligent product customization system first receives multimodal interaction information from user feedback, preprocesses the data of different modalities, and integrates the results of this round of feedback with the parameter status table corresponding to the current product customization process, thereby realizing the update of product customization parameters for the product to be customized.

[0159] Understandably, when fusing the parameter status table based on the product customization parameters corresponding to the multimodal interaction information acquired in the current instance, the update rule of "explicit values ​​overriding explicit values, explicit corrections taking precedence over historical defaults, and the latest user confirmation taking precedence over system inferences" can be adopted to ensure that the final parameter status reflects the user's most recent valid expression. For example, if a user previously expressed a preference for metal watch straps through image uploads and later explicitly stated in text "change to silicone watch straps", the system should update the watch strap material parameter from metal to silicone and retain the parameter change trajectory in the status record.

[0160] For example, in a smartwatch customization scenario, after the system issues the interactive command "Please add strap material" to the user, the user can respond via voice "Change to fluoroelastomer, dark gray color". The system can first complete the voice transcription, then recognize the material and color parameters, then write the strap material into the parameter status table, update the color from the undetermined state to dark gray, and re-verify whether the material is compatible with the previously confirmed wearing scenario, case structure and price range.

[0161] In one possible implementation, the parameter update process can include five types of processing actions: adding, replacing, deleting, resetting, and deriving. Adding is used to write previously unmatched parameter items into the status table; replacing is used to update parameter values ​​when the user makes a clear modification to existing parameters; deleting is used to cancel the corresponding parameter when the user expresses a negative intention (such as "I don't want this style anymore," or "Remove the positioning module"); resetting is used to clear the historical values ​​of a parameter when there is a conflict and the user chooses to reselect it; and deriving is used to automatically deduce other related parameters based on the user's latest input. For example, if the user selects "outdoor use scenario," the system can automatically enhance the waterproof rating, drop-resistant structure, and battery life weight with the support of a knowledge base.

[0162] In this embodiment, the multimodal interaction module further includes a semantic understanding subunit, a context association subunit, a parameter standardization subunit, and a conflict detection subunit. The semantic understanding subunit is responsible for identifying parameter entities and intents from the input content. The context association subunit is responsible for binding pronouns, comparatives, degree words, and ellipsis expressions to the current product object and the current parameter dimension. The parameter standardization subunit is responsible for converting natural language expressions such as "medium size," "large dial," "light color scheme," "slightly matte texture," "thin and light," and "long battery life" into standard parameter ranges, standard enumeration values, or standard descriptive labels. The conflict detection subunit is responsible for detecting the consistency between new parameters and confirmed parameters, compatibility rules, and manufacturing constraints. Therefore, after parsing, the system can form a set of product-customized parameters and attach metadata such as source modality, parsing confidence, confirmation status, and timestamp to each parameter to track parameter validity during subsequent recommendation generation and rendering updates.

[0163] By continuously receiving and interpreting supplementary information from users within a unified session context, the system eliminates the need for users to repeatedly switch input methods or reiterate previous descriptions. It can reliably acquire executable product customization parameters around the current customization goal and correct ambiguities and conflicts in real time, thereby significantly improving the accuracy of multi-round requirement understanding and the completeness of parameter collection.

[0164] In one possible implementation, to avoid semantic conflicts between different modalities, the system can calculate the confidence level of each parameter value and sort them according to the certainty of the input source; for example, parameter values ​​directly selected from the interface usually have high certainty, followed by explicit text statements, while reference images and vague descriptions require further confirmation in conjunction with the context.

[0165] When new and old parameters conflict and cannot be automatically determined, the system can mark the parameter as pending confirmation and regenerate the interaction instructions for that parameter in the next round.

[0166] After the product customization parameters are updated, the system can re-execute local constraint verification instead of recalculating all parameters every time. Specifically, incremental verification is only performed on the parameter items involved in this round of update and their related parameters to improve the system response speed.

[0167] For example, when a user only modifies the color, there is no need to recalculate the structural dimension constraints; when a user modifies the capacity or module combination, it is necessary to simultaneously verify the size, weight, power consumption, or shell compatibility; after incremental verification confirms validity, the system refreshes the current parameter status table, key parameter matching table, and interaction context cache, and determines whether each parameter item in the key parameter set has obtained the corresponding valid product customization parameters; if there are still unmatched items, the system can continue to return to the interaction instruction generation logic; if all key parameters have been matched, the system proceeds to the next step of the solution generation stage.

[0168] In one possible implementation, after receiving multimodal interaction information from user feedback, the multimodal interaction module can attach a session number, timestamp, modality source identifier, and context round identifier to each feedback, so as to ensure that feedback from different sources can be attributed to the same product object to be customized.

[0169] In one possible implementation, when preprocessing different modal data, for text feedback, error correction, word segmentation, semantic disambiguation, and parameter value extraction can be performed; for voice feedback, noise reduction, endpoint detection, and speech recognition can be performed first, followed by the same semantic processing as for text; for image feedback, the main product, component structure, color matching, material texture, size proportion, or reference style label in the image can be identified; for interface trigger information, the selected parameter items and values ​​can be read directly, or user preferences can be inferred based on the swipe trajectory and click path.

[0170] S104: After all parameter items in the key parameter set are matched with the corresponding product customization parameters, a product recommendation scheme for the product to be customized is generated based on multiple product customization parameters and the product knowledge base. The recommended customized products corresponding to the product recommendation scheme are then visualized through the product rendering module.

[0171] The product recommendation scheme describes one or more executable customization results that meet the current product customization parameters. It may include module combination results, parameter details, appearance style description, price estimate, manufacturing feasibility indicator and delivery-related information.

[0172] In this embodiment, the product knowledge base also pre-stores compatibility relationships between different parameters, parameter value ranges, value formats, and constraint rules that need to be called during recommendation generation; the intelligent product customization system also includes a rendering and display layer, which includes a product rendering module and a product rendering interface; wherein, the product rendering module is a functional unit that generates two-dimensional or three-dimensional visualization effects based on the structural and appearance parameters in the recommended scheme, and can call model libraries, texture libraries, lighting configurations, animation engines, and view control components to achieve real-time display; the product rendering interface is a functional unit used to display the two-dimensional or three-dimensional visualization effects generated by the product rendering module.

[0173] Understandably, when the intelligent product customization system determines that all parameter items in the key parameter set have obtained the corresponding valid product customization parameters, it indicates that the product to be customized has met the minimum completeness conditions for entering the solution generation stage; that is, the intelligent product customization system has now obtained the required key parameter items in the parameter template corresponding to the currently determined product type.

[0174] If the system determines that the currently acquired product customization parameters are consistent with the parameters in the key parameter set, it will input the currently acquired product customization parameters into the product knowledge base, perform joint matching on the product customization parameters, filter out configuration combinations that meet the constraints, output one or more product recommendation schemes according to the preset sorting strategy, and use the product rendering module to visualize the recommended customized products in the product recommendation schemes acquired this time.

[0175] Specifically, the system can first retrieve candidate product modules and candidate configuration combinations from the product knowledge base based on multiple product customization parameters, thereby obtaining multiple candidate combinations of products. For each candidate combination, the system can calculate the corresponding comprehensive suitability, which can be determined by, for example, parameter matching degree, style consistency, physical feasibility, cost level, delivery status, and user preference weight. If the product knowledge base contains historical transaction data or user preference models, these can also be used as auxiliary ranking factors to make the final product recommendation scheme more in line with actual business needs.

[0176] After the product recommendation scheme is generated, the system sends the module identifiers, appearance parameters, and structural parameters in the scheme to the product rendering module. The product rendering module can first retrieve the initial model corresponding to the product type from the product knowledge base. The initial model is a standard appearance skeleton, basic structure template, or default component assembly model. Then, according to the module information determined in the recommendation scheme, the corresponding module rendering data is loaded, and the relevant parts in the model are replaced, combined, scaled, colored, material mapped, and textured to form a visual model consistent with the current product customization parameters.

[0177] For example, the retrieval methods that the system can choose include rule-based filtering, similarity-based recall, and constraint-based combination search. Rule-based filtering is used to quickly eliminate modules that are obviously inconsistent with the current parameters, such as candidates with mismatched materials, excessive size, or incompatible functions. Similarity-based recall is used to find candidate combinations that are similar to the user's reference images, historical preferences, or style descriptions, provided that the key parameters are met. Constraint-based combination search is used to find feasible configurations that meet the requirements of size, power consumption, load-bearing capacity, temperature resistance, assembly relationship, or appearance coordination among multiple modules. After determining the recommended solution for the current customized product (smartwatch), the initial watch model can be rendered at the module level according to the dial size, case shape, button layout, strap material, and color scheme to generate a corresponding 3D display model.

[0178] In one possible implementation, the product knowledge base includes a product module library, an attribute rule library, a compatibility rule library, a price rule library, and an inventory rule library. The module library records optional components and basic models for different product types. The attribute rule library records selectable values ​​for each parameter and their applicable conditions. The compatibility rule library records combination restrictions between different parameters, such as whether a certain material does not support a certain surface treatment. The price rule library calculates recommended prices based on configuration combinations. The inventory rule library determines whether the components corresponding to the current configuration can be produced or delivered immediately.

[0179] For example, after generating a product recommendation scheme, the product rendering module reads the model identifier, material identifier, color parameters, size parameters, and structural combination parameters in the corresponding scheme, and completes the product instantiation in the rendering engine.

[0180] If the product rendering module uses a 3D approach, it can load a basic model from a 3D model library, assemble components such as the dial, strap, case, screen, and bezel, and then map material textures, color maps, and surface process parameters to the corresponding mesh areas. After that, it sets the viewpoint, lighting, and shadows to generate the target view. If the user adjusts a parameter during the session, for example, changing the body color from silver to black or adjusting the dial size from 42mm to 46mm, the product rendering module only performs local updates on the affected components and size constraints to reduce rendering latency and improve real-time interaction.

[0181] If the system outputs multiple recommended options, the product rendering module can display the thumbnail views, main views, and partial detail views of different options side by side, allowing users to intuitively compare structural and appearance differences.

[0182] In one possible implementation, in addition to visual rendering, the product rendering interface can simultaneously display parameter summaries, compatibility specifications, estimated prices, and delivery information, facilitating user understanding and confirmation within the same interface. The product rendering interface can also support rotation viewing, zooming in on details, switching background scenes, and playing opening and closing animations to help users observe the performance of the recommended customized product under real-world usage conditions.

[0183] Understandably, this step, by jointly calculating the structured product customization parameters with the rules and resources in the product knowledge base after the parameter type meets the recommendation trigger condition, avoids the problem of prematurely generating incomplete solutions when key information is missing. At the same time, the product rendering module instantly transforms the recommendation results into visual content, allowing users to immediately observe the actual impact of configuration changes on product effects after supplementing parameters. This reduces interface jumps and cognitive gaps between user input, confirmation, and preview, and enables the simultaneous consideration of accurate understanding of requirements, complete parameter acquisition, and intuitive result display in complex products, multi-parameter customization, and multi-round negotiation interaction scenarios, thereby improving the execution efficiency of the overall customization chain.

[0184] This application provides a product customization method based on multimodal interaction. Based on initial customization information, the product type and multiple product customization parameters of the product to be customized are determined. Constraint verification is performed on the multiple product customization parameters based on a set of key parameters corresponding to the product type and basic physical constraints. The verified product customization parameters are matched with the set of key parameters, and interaction instructions are generated based on parameter items in the set of key parameters that do not match the product customization parameters. Multimodal interaction information from user feedback is obtained through a multimodal interaction module, and the product customization parameters of the product to be customized are updated based on this information. After all parameter items in the set of key parameters are matched with corresponding product customization parameters, a product recommendation scheme for the product to be customized is generated based on multiple product customization parameters and a product knowledge base. The recommended customized products corresponding to the product recommendation scheme are then visualized through a product rendering module. This method parses multimodal inputs into unified structured parameters and continuously links multimodal input parsing, product type identification, key parameter completion, basic physical constraint verification, recommendation scheme generation, and rendering display to construct a closed-loop customization processing mechanism for the same product. It significantly improves the parsing accuracy of fuzzy commands, reduces the number of interaction rounds with users, and greatly enhances interaction efficiency. By utilizing proactive guidance of interaction commands, it solves the problem of low interaction efficiency caused by single-round intent parsing, reduces invalid interaction rounds, and not only improves the accuracy of demand understanding and the completeness of parameter acquisition in multi-round customization interactions, but also enhances product customization efficiency.

[0185] Figure 2A flowchart illustrating the product customization method based on multimodal interaction provided in this application embodiment. Figure 2 .like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, the product customization method based on multimodal interaction is described in detail. The product customization method based on multimodal interaction shown in this embodiment includes:

[0186] S201: Based on the initial customization information, determine the product type and multiple product customization parameters of the product to be customized.

[0187] The initial customization information input by Tonghu obtained through the multimodal interaction module is parsed and processed to obtain the product type and multiple product customization parameters of the product to be customized corresponding to the current interaction process.

[0188] In one possible implementation, the initial customization information includes at least one of text information, voice information, image information, and interface trigger information; the initial customization information is parsed to obtain the product type of the product to be customized; wherein the parsing process includes at least one of the following: semantic recognition of text information; speech recognition and semantic parsing of voice information; image recognition of image information; and instruction parsing of interface trigger information.

[0189] Understandably, initial customization information is the user's original expression of their desire for customized products. This includes text information (natural language descriptions entered by the user in input boxes), voice information (audio clips captured by the user's microphone), image information (product illustrations, reference photos, or partial feature images uploaded by the user), and interface trigger information (interaction signals generated by the user through clicking, checking, swiping, or dragging on the interface). The system integrates information from different sources into the demand analysis module (multimodal input analysis module and interface operation analysis module), and then calls the corresponding recognition channel based on its modal attributes to obtain structured results that can be used for product classification.

[0190] For example, the multimodal interaction module performs joint processing of text, voice, image, and interface operations through a multimodal input parsing unit. When the initial customized information is text, the system performs semantic recognition on the text content. Semantic recognition can be achieved based on a natural language understanding model, keyword extraction model, or intent classification model, and extracts core semantics related to the product type from the text. When the initial customized information is voice, the system first converts the voice signal into text, and then performs semantic parsing on the converted text to identify the product name, category description, or function indication contained in the voice. When the initial customized information is image, the system uses an image recognition model to detect the main outline, appearance, local structure, or prominent markings in the image, thereby inferring the corresponding product category. When the initial customized information is interface trigger information, the system parses the user's trigger actions on the interface and maps the triggered control identifiers, option values, or interaction events to product type information.

[0191] Specifically, when a user uploads a "watch face image" and describes "anti-slip bezel" via voice, the system detects the watch face outline using an image recognition model and parses "anti-slip bezel" into the structured parameter "anti-slip bezel style" using a semantic classification model. At this point, the parsing result is written into the current session state through a unified semantic slot representation (such as "product type = smartwatch" or "bezel type = anti-slip") to ensure semantic consistency across different modal inputs.

[0192] Understandably, intelligent product customization systems can uniformly parse initial customization information from different modalities and convert unstructured inputs into product types for the products to be customized. This provides a foundation for subsequent product customization parameter completion, recommended solution generation, and visualization, thereby reducing recognition bias caused by different input formats, improving the accuracy and compatibility of product type determination, and reducing reliance on a single input method during multimodal interaction.

[0193] S202: Match multiple product customization parameters with the set of key parameters corresponding to the product type, and determine the parameter items that match the set of key parameters as valid customization parameters.

[0194] S203: Based on fundamental physical constraints, perform compliance verification on multiple valid customized parameters.

[0195] The basic physical constraints include at least one of the following: size compliance constraints; material property constraints; performance threshold constraints.

[0196] In this embodiment, dimensional compliance constraints are used to verify dimensional parameters in the valid customized parameters. The verification includes whether the length, width, height, diameter, thickness, or volume falls within a preset range, or whether it meets physical conditions such as assembly gap, shell space, and safe usage distance. Material property constraints are used to determine whether material parameters in the valid customized parameters meet requirements such as density, hardness, heat resistance, corrosion resistance, thermal conductivity, or flexibility, and to check whether the selected material is compatible with the product type, other component materials, or processing technology. Performance threshold constraints are used to determine performance parameters such as power consumption, load capacity, speed, battery life, output strength, or withstand voltage level, ensuring that they are not lower than or higher than preset thresholds to avoid configuration results that are unmanufacturable, unassembleable, or unusable.

[0197] Understandably, the key parameter set is a set of basic parameter items pre-configured for the product type of the product to be customized. Product customization parameters can come from text input, voice parsing results, image recognition results, or interface click results. After receiving multiple product customization parameters, the system first compares them item by item with the key parameter set corresponding to the product type of the current product to be customized. The parameter items among the multiple product customization parameters that can establish a corresponding relationship with any parameter item in the key parameter set are determined as valid customization parameters and enter the subsequent compliance verification stage.

[0198] After determining the product type and multiple product customization parameters of the product to be customized, the system can first retrieve the corresponding set of key parameters from the product knowledge base based on the product type, and then call the corresponding rule model based on the basic physical constraint library to perform item-by-item judgment on the valid customization parameters. When any parameter does not meet the corresponding constraint, it can be marked as a parameter to be corrected and fed back to the interaction module so that the user can re-enter or adjust it. This achieves the goal of filtering out invalid configurations before parameters enter the recommendation and rendering process, avoiding the generation of customized solutions that do not meet the actual production conditions, reducing the probability of invalid parameters entering the subsequent recommendation process, and reducing the risk of rework due to size conflicts, material incompatibility, or performance exceeding limits. At the same time, it improves the manufacturability, assemblability, and interactive confirmation efficiency of the customization results.

[0199] For example, effective customization parameters include configuration items directly related to the product type, such as size, material, weight, power, capacity, or color. Their matching relationship can be determined through parameter name, parameter encoding, synonym mapping, or semantic vector similarity.

[0200] S204: Match the validated product customization parameters with the set of key parameters.

[0201] S205: Based on the parameter items in the key parameter set that do not match the product customization parameters, determine the corresponding missing parameter type.

[0202] S206: Determine the target interaction template corresponding to the missing parameter type.

[0203] S207: Generate interactive instructions based on the missing parameter type and the target interaction template.

[0204] Understandably, the missing parameter type is used to characterize the parameter type in the set of key parameters that does not match the product customization parameter; that is, the missing parameter type refers to the parameter item that has not been confirmed or completed in the obtained user requirement information. Its parameter type is used to characterize the semantic category to which the parameter item belongs, such as size, material, color or functional module.

[0205] The target interaction template is used to pre-define the expression framework of the clarification interaction. It can contain parameter names, supplementary prompts, confirmation prompts, and interface display placeholders, which makes it easy to generate interactive content with consistent structure and clear semantics for different missing parameter types.

[0206] Interactive instructions are user-oriented interactive requests formed by filling in the missing parameter type information on the template. They can be output in at least one of the following ways: text prompts, voice broadcasts, interface highlights, and image annotations, to guide users to fill in the currently missing customized information.

[0207] In this embodiment, the missing parameter type is used to categorize and merge key parameters that have not yet been covered by product customization parameters. For example, unmatched items such as "color", "size", "material" or "functional module" are respectively categorized into the corresponding type labels so that the system can accurately identify the parameter domain where the current gap is located in subsequent interactions.

[0208] The validated product customization parameters are matched with the key parameter set. For parameter items in the key parameter set that do not match the product customization parameters, name parsing and semantic classification are performed. The missing parameter type is determined by combining the key parameter set corresponding to the product type, and this type is used as the index for subsequent template retrieval.

[0209] After determining the missing parameter type required for the current product customization, the target interaction template corresponding to the missing parameter type is retrieved according to the preset mapping relationship library; after matching the target interaction template, the parameter name of the missing parameter type is filled into the target interaction template to generate user-oriented interaction instructions.

[0210] In one possible implementation, based on the missing parameter type, a corresponding interaction template is matched from a preset mapping database, and the matched interaction template is determined as the target interaction template; the parameter name of the missing parameter type is filled into the preset filling position of the target interaction template to generate an interaction instruction.

[0211] The preset mapping library is used to store the correspondence between parameter types and interaction templates.

[0212] In this embodiment, the preset mapping relationship library can be constructed as a local database, a memory-mapped table, or a server-side index table. Internally, it establishes a one-to-one or many-to-one correspondence using parameter type as the retrieval key and interactive template as the retrieval value, enabling the system to quickly locate the appropriate interactive template based on the missing parameter type. The interactive template can be a pre-set text template, a voice broadcast template, a mixed text and image template, or an instruction template suitable for multimodal terminal presentation. Different parameter types can correspond to query-style, clarification-style, or recommendation-style templates. Pre-set fill positions in the template are used to carry the parameter name to be inserted, forming an interactive instruction with clear guiding semantics. Furthermore, the interactive template can also preset multiple fill positions for combined expressions, and the templates in the preset mapping relationship library can also be stored hierarchically according to product category. This application does not impose special limitations on the interactive template and the preset fill positions.

[0213] Understandably, the parameter name is the specific parameter identifier corresponding to the missing parameter type, used to place and replace it in the target interaction template to form interactive content that can be directly presented to the user.

[0214] After determining the required missing parameter type, a query is initiated to the preset mapping database based on the missing parameter type. If an interaction template corresponding to the type is found, the interaction template is determined as the target interaction template. If multiple templates correspond to the same parameter type, the final template can be further determined based on the terminal type, interaction scenario, or template version number to ensure that the interaction expression is consistent with the client's presentation. After determining the target interaction template, the parameter name of the missing parameter type is written into the preset fill position in the target interaction template to generate an interaction command that can be directly output to the user interface.

[0215] For example, when the missing parameter type is "material", "material" can be filled into the "Please supplement [] information" type template to form the guidance content of "Please supplement material information"; the instruction can carry text prompts, voice broadcast indicators and interface highlight indicators at the same time so that different modal channels can be presented synchronously; in addition, the interactive instruction generated at this time can be output in the form of text, voice broadcast or graphic combination to adapt to the interactive capabilities of different terminals. In practical applications, the interactive template can also select other template types, which are not limited in this embodiment.

[0216] Understandably, the intelligent product customization system completes template matching based on the missing parameter type, and then writes the parameter name into the preset filler in the template. This transforms the abstract missing item into a structured and perceptible interactive instruction, and ensures a stable correspondence between the template and parameter semantics through a preset mapping relationship library. Since there is a fixed mapping between the interactive template and the parameter type, the system does not need to regenerate the guidance statement every time, and can quickly output targeted completion prompts. This improves the accuracy and generation efficiency of guidance for missing parameter types, avoids follow-up question deviations or user misunderstandings caused by improper template selection, improves the efficiency of multimodal customization interaction, reduces irrelevant follow-up questions, and reduces recommendation deviations caused by parameter omissions. At the same time, it keeps the interactive instructions consistent with the multimodal display format, thereby improving the completeness of parameter completion and the continuity of interaction.

[0217] S208: Obtain multimodal interaction information from user feedback through the multimodal interaction module, and update the product customization parameters of the product to be customized based on the multimodal interaction information.

[0218] Step S208 is similar to step S103 above, and will not be described again here.

[0219] S209: After all parameter items in the key parameter set are matched with the corresponding product customization parameters, the module information in the product knowledge base is matched based on multiple product customization parameters to obtain multiple candidate module information.

[0220] S210: Based on information from multiple candidate modules, generate multiple initial product recommendation schemes by combining them.

[0221] S211: Based on parameter configuration constraints and module combination constraints, perform secondary constraint verification on multiple initial product recommendation schemes, filter to obtain product recommendation schemes, and visualize the recommended customized products corresponding to the product recommendation schemes through the product rendering module.

[0222] In this embodiment of the application, the product knowledge base includes module information, parameter configuration constraints, and module combination constraints for each product; that is, the product knowledge base is used to store modular configuration data corresponding to different product types; wherein, the module information includes the identifier, specification attributes, adaptation parameters, and interface characteristics of the functional unit, the parameter configuration constraints are used to limit the legal value range of a single module or module parameters, and the module combination constraints are used to limit the compatibility relationship and assembly relationship between different modules.

[0223] For example, taking a smart wardrobe product as an example, module information may include main cabinet frame module, door panel module, drawer module, smart lighting module, etc., specifically including functional unit name, specification range, interface type, compatible objects, and optional configuration items; parameter configuration constraints are the configuration parameter rules corresponding to each module, such as style parameters such as minimalist and fashionable corresponding to door panel module, material parameters such as glass door and wooden door, function parameters such as silent and normal corresponding to drawer module, and energy consumption level parameters corresponding to disinfection and drying module; module combination constraints are used to limit the assembly compatibility relationship between multiple modules, for example, sliding door panel module and external handle module are mutually exclusive and cannot be combined, quick-install snap-on interface module can only be assembled with modules of the same interface type, and modules of different interface types cannot be combined.

[0224] After all parameter items in the key parameter set are matched with corresponding product customization parameters, the module information in the product knowledge base is matched based on multiple product customization parameters. That is, the module information that matches each requirement item is selected by matching the parameter type, parameter value and parameter semantic tags in the product knowledge base, and multiple candidate module sets are established for each parameter item in the key parameter set. Then, according to the structural framework corresponding to the product type, the modules that can jointly constitute the whole machine are selected from each candidate module set and combined to form multiple initial product recommendation schemes containing different configuration combinations.

[0225] For the multiple initial product recommendation schemes determined in this instance, the parameters of each module are checked to see if they meet the range limits, dependency conditions, and conflict conditions based on parameter configuration constraints. The modules are also checked to see if they meet the assembly compatibility, connection topology, and functional coordination relationships based on module combination constraints. Schemes that do not meet the constraints will be eliminated or modified, and the schemes that can simultaneously meet the parameter configuration constraints and module combination constraints will be retained as the product recommendation schemes.

[0226] It is understandable that there may be multiple solutions that simultaneously satisfy both parameter configuration constraints and module combination constraints, or there may be only one. This application does not impose any special limit on the number of recommended product solutions.

[0227] For example, parameter configuration constraints include the value range, dependency conditions, and conflict conditions of each parameter. For instance, the dial size parameter must be between 38mm and 48mm, and the strap material parameter must be compatible with the surface treatment process parameter. Module combination constraints are used to limit the assembly compatibility of different functional modules. For instance, a specific case structure can only be matched with a corresponding type of dial module, and a specific interface specification can only be matched and connected with functional modules of the same specification. The intelligent product customization system iterates through the parameter values ​​and module combinations in the initial product recommendation scheme, checks in turn whether the parameter configuration constraints and module combination constraints are met, eliminates schemes that do not meet the conditions, and retains the schemes that pass the verification as the final recommended schemes.

[0228] In one possible implementation, module information can be indexed according to module categories to quickly locate candidate module information based on customized parameter information, thereby improving retrieval efficiency. Parameter configuration constraints and module combination constraints can be stored in the form of rule tables, logical expressions, or constraint matrices for direct retrieval during verification. Therefore, after user input, a complete and constraint-compliant recommendation result can be generated quickly, reducing the generation of invalid combinations and improving the accuracy and feasibility of customized solutions, thereby enhancing product recommendation efficiency and user experience.

[0229] In one possible implementation, the product knowledge base further includes: initial models and module rendering data for each product; after determining the product type and multiple product customization parameters of the product to be customized based on the initial customization information, determining the initial model corresponding to the product type, and visually displaying the initial model through the product rendering module; determining the module rendering data corresponding to the product recommendation scheme; and performing module rendering on the initial model based on the module rendering data corresponding to the product recommendation scheme to generate a visual model of the recommended customized product.

[0230] The initial model is used to represent the basic 3D appearance or 2D display model of the product to be customized before module configuration. Its data may include model mesh, skeletal structure, material texture, color parameters and view parameters. At the same time, the initial model is a basic visual carrier that corresponds one-to-one with the product type. Its structural data, appearance outline and display view are all pre-stored in the product knowledge base and can be directly called after the product type is determined.

[0231] Module rendering data is used to describe the assembly position, size boundaries, connection relationships and display attributes of each functional module on the target product. It is usually stored in the product knowledge base in a one-to-one correspondence with module information.

[0232] Understandably, the product rendering module can consist of a graphics processing unit, a model loading unit, and an interface output unit. Its function is to call the initial model that matches the product type from the product knowledge base after receiving the product type, and load the initial model into the rendering scene so that the basic appearance can be displayed in the user interface, so that users can form a preliminary understanding of the product outline, proportions and structure.

[0233] After at least one product recommendation scheme is generated, for any product recommendation scheme, the intelligent product customization system retrieves the module rendering data corresponding to the product recommendation scheme from the product knowledge base, and binds the module rendering data to the initial model. During the binding process, the rendering module overlays the corresponding module onto the preset position of the initial model according to the coordinates, rotation angle, scaling ratio and material mapping relationship defined by the module rendering data, forming a visual model containing the recommendation configuration.

[0234] Understandably, the visualized model can be output to the user interface in the form of a 3D rotating preview, a partial magnified preview, or a layered disassembly preview, so that users can intuitively view the appearance and structural combination relationship of the recommended customized product.

[0235] Initial model and module rendering data can be stored in a database or object storage unit, organized using JSON (JavaScript Object Notation), XML (Machine Learning), or binary model files to improve retrieval and rendering efficiency. This application does not impose any special restrictions on the storage of initial model and module rendering data.

[0236] In one possible implementation, after visually displaying the initial model corresponding to the product type, if the generated product recommendation solution is implemented through multiple rounds of interaction, after any round of interaction is completed, the initial model is simultaneously visualized based on the module rendering data supplemented by that round of interaction.

[0237] For example, when displaying the initial model, the product rendering module typically performs model format conversion and texture caching first, and then adaptively scales according to the terminal display resolution to ensure display clarity and loading speed. When generating a visual model of the recommended customized product, the rendering module can also correct the occlusion, splicing gaps, and color transitions between components according to the module combination rules, so that the final display result is consistent with the actual manufacturable solution. By displaying the initial model first after the product type is determined, and then overlaying the module rendering data corresponding to the recommended solution onto the initial model to generate the visual model, the system can complete the basic preview and solution preview in the same interactive interface, reducing the misunderstanding caused by interface switching.

[0238] Specifically, if the multimodal interaction module uses a semantic classification model to perform contextual analysis on the user's ambiguous input (such as "bigger"), and combines it with the currently confirmed product type (such as "smartwatch") and parameter dimensions (such as "dial size"), it can map it to specific structured parameters (such as "increase the dial size by one level"). At the same time, the product rendering module dynamically adjusts the size and material representation of the 3D model according to the updated parameters. For example, when the user selects "dial size 46mm", the model automatically expands the cup size and updates the material texture simultaneously, allowing the user to intuitively confirm the customization effect without relying on imagination, significantly reducing the decision-making burden.

[0239] In this embodiment of the application, the initial model and module rendering data in the product knowledge base are used to support basic display and solution display, respectively. The rendering module can quickly generate the corresponding appearance after the product type is determined, and output the configured visualization results in real time after the recommended solution is formed. This improves the display consistency and response speed in the customization process, reduces the user's judgment cost on the solution effect, and improves the confirmation efficiency of the recommended solution.

[0240] The product customization method based on multimodal interaction provided in this embodiment determines the product type and multiple product customization parameters of the product to be customized based on initial customization information; matches the multiple product customization parameters with a set of key parameters corresponding to the product type, and determines the parameter items that match the key parameter set as valid customization parameters; then performs compliance verification on the multiple valid customization parameters based on basic physical constraints; matches the verified product customization parameters with the set of key parameters, and determines the corresponding missing parameter types based on the parameter items in the set that do not match the product customization parameters; determines the target interaction template corresponding to the missing parameter type, and generates multiple... Modal interaction commands; The system acquires multimodal interaction information from user feedback through a multimodal interaction module, and updates the product customization parameters of the product to be customized based on this information; After all parameter items in the key parameter set are matched with corresponding product customization parameters, the system matches module information in the product knowledge base based on multiple product customization parameters to obtain multiple candidate module information; Based on the multiple candidate module information, multiple initial product recommendation schemes are generated by combining them, and then secondary constraint verification is performed on the multiple initial product recommendation schemes based on parameter configuration constraints and module combination constraints to filter and obtain the product recommendation scheme; Finally, the product rendering module visualizes the recommended customized products corresponding to the product recommendation schemes. This method parses multimodal inputs into unified structured parameters and continuously links multimodal input parsing, product type identification, key parameter completion, basic physical constraint verification, recommendation scheme generation, and rendering display to construct a closed-loop customization processing mechanism for the same product. This significantly improves the parsing accuracy of fuzzy commands, reduces the number of interaction rounds with users, and greatly enhances interaction efficiency. By actively guiding interaction commands, it solves the problem of low interaction efficiency caused by single-round intent parsing, reduces invalid interaction rounds, and not only improves the accuracy of demand understanding and parameter acquisition in multi-round customization interactions, but also improves product customization efficiency and further enhances the user experience.

[0241] Figure 3 A schematic diagram of the structure of the product customization device based on multimodal interaction provided in this application. Figure 3 As shown, this application provides a product customization device based on multimodal interaction. The product customization device 300 based on multimodal interaction includes:

[0242] The processing module 301 is used to determine the product type and multiple product customization parameters of the product to be customized based on the initial customization information, and to perform constraint verification on the multiple product customization parameters based on the key parameter set and basic physical constraints corresponding to the product type; the initial customization information is used to represent the initial customization requirements input by the user through the multimodal interaction module; the key parameter set is a set of basic parameter items preset based on the product type of the product to be customized.

[0243] The processing module 301 is also used to match the verified product customization parameters with the key parameter set, and generate interactive instructions based on the parameter items in the key parameter set that do not match the product customization parameters.

[0244] The acquisition module 302 is used to acquire multimodal interaction information from user feedback through the multimodal interaction module.

[0245] The processing module 301 is also used to update the product customization parameters of the product to be customized based on multimodal interaction information.

[0246] The processing module 301 is also used to generate a product recommendation scheme for the product to be customized based on multiple product customization parameters and the product knowledge base after all parameter items in the key parameter set have been matched with the corresponding product customization parameters, and to visualize the recommended customized products corresponding to the product recommendation scheme through the product rendering module.

[0247] In one possible implementation, the processing module 301 is further configured to parse the initial customization information to obtain the product type of the product to be customized.

[0248] The parsing process includes at least one of the following:

[0249] Perform semantic recognition on text information;

[0250] Perform speech recognition and semantic analysis on speech information;

[0251] Image recognition is performed on image information;

[0252] The interface trigger information is parsed.

[0253] In one possible implementation, the processing module 301 is further configured to determine the corresponding missing parameter type based on the parameter items in the key parameter set that do not match the product customization parameters. The missing parameter type is used to characterize the parameter type in the key parameter set that does not match the product customization parameters.

[0254] The processing module 301 is also used to determine the target interaction template corresponding to the missing parameter type.

[0255] The processing module 301 is also used to generate interactive instructions based on the missing parameter type and the target interaction template.

[0256] In one possible implementation, the processing module 301 is further configured to match the corresponding interaction template from a preset mapping relationship library based on the missing parameter type, and determine the matched interaction template as the target interaction template.

[0257] The processing module 301 is also used to fill the parameter name of the missing parameter type into the preset fill position of the target interaction template to generate the interaction instruction;

[0258] The preset mapping library is used to store the correspondence between parameter types and interaction templates.

[0259] In one possible implementation, the processing module 301 is further configured to match multiple product customization parameters with a set of key parameters corresponding to the product type, and determine the parameter items that match the set of key parameters as valid customization parameters.

[0260] The processing module 301 is also used to perform compliance verification on multiple valid customized parameters based on basic physical constraints;

[0261] The fundamental physical constraints include at least one of the following:

[0262] Size compliance constraints;

[0263] Material property constraints;

[0264] Performance threshold constraints.

[0265] In one possible implementation, the processing module 301 is further configured to match module information in the product knowledge base based on multiple product customization parameters to obtain multiple candidate module information, wherein the module information is the basic information constituting each functional unit of the product.

[0266] The processing module 301 is also used to generate multiple initial product recommendation schemes based on information from multiple candidate modules.

[0267] The processing module 301 is also used to perform secondary constraint verification on multiple initial product recommendation schemes based on parameter configuration constraints and module combination constraints, and to filter and obtain product recommendation schemes.

[0268] In one possible implementation, the processing module 301 is further configured to determine an initial model corresponding to the product type and to visualize the initial model through the product rendering module.

[0269] The processing module 301 is also used to determine the module rendering data corresponding to the product recommendation scheme.

[0270] The processing module 301 is also used to perform module rendering on the initial model based on the module rendering data corresponding to the product recommendation scheme, and generate a visual model of the recommended customized product.

[0271] The product customization device based on multimodal interaction provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0272] Figure 4 A schematic diagram of the structure of the electronic device provided in this application. Figure 4 As shown, the electronic device 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 400 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0273] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0274] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0275] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0276] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0277] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0278] This application also provides an intelligent product customization system, which includes a multimodal interaction module, a product rendering module, and a controller;

[0279] The multimodal interaction module is used to obtain the initial customization information and multimodal interaction information input by the user; the product rendering module is used to render the recommended customized products in the product recommendation scheme.

[0280] The controller is used to execute the methods described above to generate product recommendation schemes.

[0281] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0282] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0283] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0284] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0285] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0286] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0287] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0288] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0289] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0290] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A product customization method based on multimodal interaction, characterized in that, The method, applied to an intelligent product customization system, which includes a multimodal interaction module and a product rendering module, comprises: Based on the initial customization information, the product type and multiple product customization parameters of the product to be customized are determined, and the multiple product customization parameters are constrained and verified based on the key parameter set and basic physical constraints corresponding to the product type; the initial customization information is used to represent the initial customization requirements input by the user through the multimodal interaction module; the key parameter set is a set of basic parameter items preset based on the product type of the product to be customized; The product customization parameters that have passed the verification are matched with the set of key parameters, and interactive instructions are generated based on the parameter items in the set of key parameters that do not match the product customization parameters. The multimodal interaction module obtains the multimodal interaction information from the user feedback and updates the product customization parameters of the product to be customized based on the multimodal interaction information. After all parameter items in the key parameter set are matched with the corresponding product customization parameters, a product recommendation scheme for the product to be customized is generated based on multiple product customization parameters and the product knowledge base, and the recommended customized product corresponding to the product recommendation scheme is visualized through the product rendering module.

2. The method according to claim 1, characterized in that, The initial customization information includes at least one of the following: text information, voice information, image information, and interface trigger information; The process of determining the product type of the product to be customized based on the initial customization information includes: The initial customization information is parsed and processed to obtain the product type of the product to be customized; The parsing process includes at least one of the following: Perform semantic recognition on the text information; The speech information is subjected to speech recognition and semantic parsing; Perform image recognition on the image information; The interface trigger information is parsed.

3. The method according to claim 2, characterized in that, The step of generating interactive instructions based on parameter items in the key parameter set that do not match the product customization parameters includes: Based on the parameter items in the key parameter set that do not match the product customization parameters, the corresponding missing parameter type is determined. The missing parameter type is used to characterize the parameter type in the key parameter set that does not match the product customization parameters. Determine the target interaction template corresponding to the missing parameter type; The interaction instruction is generated based on the missing parameter type and the target interaction template.

4. The method according to claim 3, characterized in that, The step of generating the interaction instruction based on the missing parameter type and the target interaction template includes: Based on the missing parameter type, a corresponding interaction template is matched from a preset mapping relationship library, and the matched interaction template is determined as the target interaction template; The parameter name of the missing parameter type is filled into the preset fill position of the target interaction template to generate the interaction instruction; The preset mapping relationship library is used to store the correspondence between parameter types and interaction templates.

5. The method according to claim 1, characterized in that, The constraint verification of the multiple product customization parameters based on the key parameter set and basic physical constraints corresponding to the product type includes: The multiple product customization parameters are matched with the key parameter set corresponding to the product type, and the parameter items that match the key parameter set are determined as valid customization parameters; Based on the aforementioned fundamental physical constraints, compliance verification is performed on multiple of the valid customized parameters; The fundamental physical constraints include at least one of the following: Size compliance constraints; Material property constraints; Performance threshold constraints.

6. The method according to claim 1, characterized in that, The product knowledge base includes module information, parameter configuration constraints, and module combination constraints for each product; the process of generating a product recommendation scheme for the product to be customized based on multiple product customization parameters and the product knowledge base includes: Based on multiple product customization parameters, the module information in the product knowledge base is matched to obtain multiple candidate module information, which are the basic information of each functional unit that constitutes the product. Based on the information of the multiple candidate modules, multiple initial product recommendation schemes are generated by combining them. Based on the parameter configuration constraints and the module combination constraints, a secondary constraint verification is performed on the multiple initial product recommendation schemes to filter and obtain the product recommendation scheme.

7. The method according to claim 6, characterized in that, The product knowledge base also includes: initial models and module rendering data for each product; after determining the product type and multiple product customization parameters of the product to be customized based on the initial customization information, the method further includes: An initial model corresponding to the product type is determined, and the initial model is visualized through the product rendering module; The step of visualizing the recommended customized products corresponding to the product recommendation scheme through the product rendering module includes: Determine the module rendering data corresponding to the product recommendation scheme; Based on the module rendering data corresponding to the product recommendation scheme, the initial model is rendered to generate a visual model of the recommended customized product.

8. A smart product customization system, characterized in that, The intelligent product customization system includes a multimodal interaction module, a product rendering module, and a controller; The multimodal interaction module is used to acquire initial customization information and multimodal interaction information input by the user; The product rendering module is used to render the recommended customized products within the product recommendation scheme; The controller is used to perform the method as described in any one of claims 1-7 to generate the product recommendation scheme.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

Images