Customer style selection method and system based on three-dimensional multi-mode interaction

Abstract

The invention relates to a customer style selection method and system based on three-dimensional multi-modal interaction, and relates to the technical field of intelligent recommendation, and the method comprises the steps: collecting a detection text and historical detection parameters; obtaining a marked text based on the detection text; obtaining a detection model according to the historical detection parameters, and obtaining a marking key instruction based on the marking text and the detection model; obtaining an initial detection article according to the marking key instruction; acquiring image detection information in response to the detection signal; and generating and sending a target article through the image detection information and the initial detection article, and updating the detection model based on the target article. The method has the effect of improving the accuracy of style recommendation.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent recommendation, and in particular to a customer product selection method and system based on three-dimensional multimodal interaction. Background Technology

[0002] Intelligent recommendation is a technology system that uses algorithms to analyze user data (such as behavior, preferences, and context) and automatically predict and provide information, goods, or services that the user may be interested in.

[0003] In the process of recommending items for customers to choose from, the system first identifies the customer's search text or browsing on a platform where products are stored. The system records the search text and the styles the customer has historically selected, and pushes items of the same style type to the customer. Then, the items are sorted by sales volume or page views and displayed on the customer's system interface for the customer to choose from.

[0004] In the process of recommending items to customers, the final recommended styles are limited to the customer's search text and historical selections, requiring the customer to make further selections, which reduces the accuracy of style recommendations. Summary of the Invention

[0005] To improve the accuracy of style recommendations, this invention provides a customer style selection method and system based on three-dimensional multimodal interaction.

[0006] In a first aspect, the present invention provides a customer product selection method based on three-dimensional multimodal interaction, employing the following technical solution:

[0007] A customer product selection method based on three-dimensional multimodal interaction includes:

[0008] S10: Collect the detected text and historical detection parameters;

[0009] S11: Obtain labeled text based on detected text;

[0010] S12: Obtain the detection model based on historical detection parameters, and obtain the tagging key instructions based on the tagged text and the detection model;

[0011] S13: Obtain the initial inspection item based on the marked key instructions;

[0012] S14: Acquire image detection information in response to the detection signal;

[0013] S15: Generate and send target items using image detection information and initial detected items, and update the detection model based on the target items.

[0014] By adopting the above technical solution, the detection text, historical detection parameters, and image detection information are analyzed to generate and send target items and update the detection model. This allows for a more comprehensive understanding of customers' selection preferences, thereby improving the accuracy of style recommendations to customers.

[0015] Optionally, methods for generating the target item include:

[0016] S20: Identify environmental objects from image detection information;

[0017] S21: Determine the initial detection type based on the initial inspection item;

[0018] S22: Identify items of the initial detection type from environmental items as environmental detection items;

[0019] S23: Identify other features from environmentally inspected items;

[0020] S24: Update the initial detected item using other features, and use the updated initial detected item as the target item.

[0021] By adopting the above technical solution, image analysis of the user's physical environment is performed to identify environmental items that are related to the initially detected items, and other features are extracted to update the initially detected items. This reduces the situation where recommended items conflict with the user's existing environment and improves the accuracy and scene integration of style recommendations.

[0022] Optionally, methods for generating the target item also include:

[0023] S30: Obtain the associated item type based on the initial detection type;

[0024] S31: Obtain the initial usage environment by initially inspecting the items;

[0025] S32: Obtain associated usage behavior based on the type of associated item and the initial usage environment;

[0026] S33: Obtain associated usage characteristics based on associated usage behavior;

[0027] S34: Use associated features to further assist in filtering the initial detection items, and use the remaining initial detection items after filtering as target items.

[0028] By adopting the above technical solution, and by analyzing the user's associated usage behavior in the initial usage environment to update the initially detected items, it is possible to proactively filter out items that match the user's real life habits for recommendation.

[0029] Optional, also includes:

[0030] S40: Update image detection information and detection text based on the initial usage environment;

[0031] S41: Determine the scope of use of the item based on the updated image detection information and detection text;

[0032] S42: Based on image detection information and preset movable features, obtain movable objects around the object's usage range;

[0033] S43: Obtain the initial detection size based on the updated initial detection item;

[0034] S44: Obtain the initial detection range by using the initial detection size and the detection text;

[0035] S45: Compare the intersection of the initial detection range and the item usage range to update the initial detection item.

[0036] By adopting the above technical solution, and by guiding users to provide accurate images and descriptions of the initial usage environment, combined with the analysis of the initial detection dimensions, items whose size or shape does not match the intended placement space can be identified and eliminated in advance before the user places an order, thereby improving the certainty and reliability of style recommendations.

[0037] Optionally, methods for updating the initially detected items include:

[0038] S50: Compare the intersection of the initial detection range and the item's usage range to determine if the range is exceeded;

[0039] S51: Mark movable items located outside the range as movable items;

[0040] S52: Retrieve the actual distance of the marker for the movable object from the image detection information;

[0041] S53: Compare the moving distance of the marker with the actual distance of the marker to generate and send a placement image by combining the movable object of the marker with the image detection information;

[0042] S54: Collect user operation information;

[0043] S55: Update the initial detected items based on the placement image selected based on the operational information.

[0044] By adopting the above technical solution, the operation information and placement images are analyzed to calculate and demonstrate how to move obstacles to create sufficient space. The solution is then presented to the user in the form of placement images for confirmation. This allows items that were previously removed due to their movable nature to be retained, thereby improving the accuracy of style recommendations.

[0045] Optionally, methods for updating the initially detected items also include:

[0046] S60: Collects the user's delivery location;

[0047] S61: Obtain the conveying dimension based on the conveying location and detection text;

[0048] S62: Update the initial inspection items based on the conveyor dimensions.

[0049] Optional, also includes:

[0050] S70: Collect initial transport type based on initial inspection of items;

[0051] S71: Compare the initial transport type with the preset split type to obtain the split dimensions;

[0052] S72: Obtain the transport configuration based on the split dimensions and transport dimensions, or the initial inspection dimensions and transport dimensions;

[0053] S73: Update the operation information and use the selected transportation mode in the updated operation information as the marked transportation mode;

[0054] S74: Update the initial inspection item by marking the transportation pattern.

[0055] By adopting the above technical solution, and by informing users of different initial transportation types and updating operational information to filter initial inspection items, recommended items can be selected based on transportation conditions confirmed by the user and deemed feasible.

[0056] Optionally, methods for updating the initially detected items also include:

[0057] S80: Collects the user's historical target items;

[0058] S81: Obtain duplicate items based on historical target items and initial detected items;

[0059] S82: Obtain baseline weights based on historical target items;

[0060] S83: Obtain duplicate weights based on duplicate items and historical target items;

[0061] S84: Compare the duplicate weights with the baseline weights to eliminate duplicate items corresponding to duplicate weights that do not exceed the baseline weights, and update the initial detection items.

[0062] Secondly, this application provides a customer selection system based on three-dimensional multimodal interaction, employing the following technical solution:

[0063] A customer product selection system based on three-dimensional multimodal interaction includes:

[0064] The acquisition module is used to acquire the detected text, historical detection parameters, and image detection information;

[0065] A memory for storing a program for a customer selection method based on three-dimensional multimodal interaction;

[0066] The processor is used to load and execute programs stored in memory.

[0067] In summary, this application includes at least one of the following beneficial technical effects:

[0068] 1. By analyzing the detected text, historical detection parameters, and image detection information to generate and send target items and update the detection model, we can gain a comprehensive understanding of customers' selection preferences and improve the accuracy of style recommendations to customers;

[0069] 2. By analyzing users' associated usage behaviors in the initial usage environment to update the initially detected items, it is possible to proactively filter out items that match users' real-life habits for recommendation;

[0070] 3. By informing users of different initial transport types and updating operational information to filter initial inspection items, it is possible to select recommended items based on user-confirmed and feasible transport conditions. Attached Figure Description

[0071] Figure 1 This is a flowchart of a customer product selection method based on three-dimensional multimodal interaction, according to an embodiment of the present invention. Detailed Implementation

[0072] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0073] Reference Figure 1 This application discloses a customer product selection method based on three-dimensional multimodal interaction, including the following steps:

[0074] S10: Collect the detected text and historical detection parameters.

[0075] The detection text refers to the text entered by the user when selecting item styles on the system platform. It can be obtained by retrieving the text in the dialog box of the system platform where the item style selection is required.

[0076] Historical detection parameters refer to the information and corresponding item parameters entered by users when selecting item styles on the system platform in the past. Historical detection parameters can be retrieved from the system platform.

[0077] S11: Obtain labeled text based on detected text.

[0078] Tag text refers to a standardized text description that analyzes the intent of text input. Based on text fusion technology (pre-trained models such as BERT / RoBERTa), the detected text is deduplicated, completed, and semantically merged to generate unified tag text.

[0079] S12: Obtain the detection model based on historical detection parameters, and obtain the key instructions for marking based on the marked text and the detection model.

[0080] The detection model refers to the memory unit model formed by users' historical item search and selection. The detection model is obtained by analyzing historical detection parameters using user behavior sequence embedding models (such as Word2Vec and Doc2Vec).

[0081] Keyword tagging refers to text instructions used for searching item styles. It involves identifying keywords from the tagged text and combining these keywords with detection models (such as Prompt Engineering and LLMs like the GPT series) to generate user preference instructions as keyword tagging.

[0082] S13: Obtain the initial inspection item based on the marked key instructions.

[0083] The initial detection items refer to the items retrieved by the system platform based on the marked key instructions. Items whose features match the marked key instructions by performing similarity matching (cosine similarity) with the preset item feature database in the system platform are used as the initial detection items. In this embodiment, the initial detection items are sorted according to the comprehensive weight of their similarity.

[0084] The item feature database is a database formed by technicians who have pre-selected and recorded the features of multiple items, which will not be elaborated on here.

[0085] S14: In response to the detection signal, acquire image detection information.

[0086] The detection signal is a signal set by the technician that allows the user to photograph the surrounding environment.

[0087] Image detection information refers to the images captured when a user selects an item style on the system platform. When the detection signal is output, the system platform captures images of the user and their surrounding environment as image detection information.

[0088] S15: Generate and send target items using image detection information and initial detected items, and update the detection model based on the target items.

[0089] The target item refers to the item that the system platform ultimately recommends to the user. The target item is generated by analyzing the image detection information and the initial detection items and then sent to the system platform for display. The detection model is further improved by adding the target item selected by the user and its characteristics.

[0090] In this embodiment, each item displayed on the system platform is presented as a three-dimensional model, and the three-dimensional model can be controlled by the customer to change its posture so that the customer can view the item in multiple postures.

[0091] Methods for generating the target item include:

[0092] S20: Identify environmental objects from image detection information.

[0093] Environmental objects refer to objects in the user's surrounding environment. Environmental objects are identified from image detection information using image recognition technologies (such as YOLO and DETR).

[0094] S21: Obtain the initial detection type based on the initial detection item.

[0095] The initial detection type refers to the category of the item to which the initial detection item belongs. It is the initial detection type obtained from the preset item feature database (such as "sofa", "lamp", "decorative painting").

[0096] The item feature database stores the initial detection types corresponding to different initial detection items, which will not be elaborated here.

[0097] S22: Identify items of the initial detection type from environmental items as environmental detection items.

[0098] Environmental monitoring items refer to environmental items of the initial monitoring type. Referring to S21, the type of environmental item is first identified, and environmental items whose type is consistent with the initial monitoring type are used as environmental monitoring items.

[0099] S23: Identify other features from environmentally inspected items.

[0100] Other features refer to the attributes or characteristics of environmental monitoring items that influence the user's final choice. Multi-task learning algorithms are used to analyze environmental monitoring items to identify features such as color, style, material, and condition as other features.

[0101] S24: Update the initial detected item using other features, and use the updated initial detected item as the target item.

[0102] Referring to the other features identified by S23 for the initial detected items as initial other features, and using a collaborative filtering algorithm (Item-CF or SVD++) on the initial other features and other features, the initial detected items are filtered, sorted or replaced (for example, if the color tone of the user environment items is a mild color tone, then the initial detected items with a cool color tone are removed), and the remaining initial detected items are used as target items.

[0103] Other methods for generating target items include:

[0104] S30: Obtain the associated item type based on the initial detection type.

[0105] Associated item types refer to other item categories that frequently co-occur with or are functionally related to the initial detection type in typical use cases. Associated item types are matched from a preset item association lookup table based on the initial detection type.

[0106] The item association lookup table stores the associated item types corresponding to different initial detection types. The parameters in the item association lookup table are set in advance by those skilled in the art based on actual conditions and will not be elaborated here.

[0107] S31: Obtain the initial usage environment by initially inspecting the items.

[0108] The initial usage environment refers to the usage scenario environment of the initially tested item. Referring to the usage environment matched by the initial tested item and item association lookup table in S30, the initial usage environment is used as the reference, which will not be elaborated here.

[0109] S32: Obtain associated usage behavior based on the associated item type and initial usage environment.

[0110] Associated usage behavior refers to a user's actions when using associated item types in the initial usage environment. Associated usage behavior is defined as user behavior analyzed using inference techniques that link associated item types to initial usage environment behavior (Scene Graph Generation or rule-based knowledge graph inference). For example, if the initial detected item is a sofa or a table, identifying an ashtray (an item type associated with being placed on a table or sitting on a sofa) in the living room (the initial usage environment) indicates that the user smoked on the sofa in the living room, thus constituting associated usage behavior.

[0111] S33: Obtain associated usage characteristics based on associated usage behavior.

[0112] Associated use characteristics refer to the physical or functional attributes that a target item should possess in order to respond to or adapt to associated use behaviors. This is achieved by retrieving behavioral characteristics from associated use behaviors (e.g., smoking in S32 corresponds to high temperature, and Neo4j sets the function-attribute association), and using the functional characteristics (high temperature resistance) of the initial test item corresponding to the behavioral characteristics as associated use characteristics.

[0113] S34: Use associated features to further assist in filtering the initial detection items, and use the remaining initial detection items after filtering as target items.

[0114] Use associated features to filter initial detected items that contain those features as target items.

[0115] Also includes:

[0116] S40: Update image detection information and detection text based on the initial usage environment.

[0117] The system platform issues guidance instructions to users regarding the initial usage environment, prompting them to re-capture image detection information in their corresponding initial usage environment (such as the "living room sofa area"), and to re-enter or correct the descriptive text for that scene, thereby updating the image detection information and detection text.

[0118] S41: Obtain the item's usage range based on the updated image detection information and detection text.

[0119] The scope of use of an item refers to the specific and quantifiable physical space area where the target item is planned to be placed. By using image segmentation and depth estimation techniques (SegFormer, Mask R-CNN, and MiDaS), the updated image detection information is analyzed to identify the ground area where the user intends to place the item (such as a sofa). Combined with the location description in the detection text (such as "place against the wall"), the two-dimensional or three-dimensional boundary of the area is calculated as the scope of use of the item.

[0120] S42: Based on image detection information and preset movable features, obtain movable objects around the object's usage range.

[0121] Movable features are the characteristics of objects that can be moved, set by technicians, such as potted plants or tables and chairs.

[0122] Movable items refer to items that meet the movable characteristics in the updated image detection information in S40. New environmental items are identified from the updated image detection information, and these new environmental items that meet the movable characteristics are designated as movable items.

[0123] S43: Obtain the initial detection size based on the updated initial detection item.

[0124] The initial inspection size refers to the size parameters of the initial inspection item in S34, which are retrieved from the system platform as the initial inspection size.

[0125] In this embodiment, if an item has multiple sizes, then all sizes need to be extracted.

[0126] S44: Obtain the initial detection range by using the initial detection size and the detection text.

[0127] The initial detection range refers to the space occupied by the initial detection items after they are placed according to the detection text. The placement posture of the items is identified from the detection text, and the initial detection items of the initial detection size are placed in a three-dimensional simulation according to their placement posture (using Bullet or PyBullet). The space occupied by the simulated placement of the items is used as the initial detection range.

[0128] S45: Compare the intersection of the initial detection range and the item usage range to update the initial detection item.

[0129] The initial detection items are updated by analyzing the intersection of the initial detection range and the item usage range.

[0130] Methods for updating the initial detection items include:

[0131] S50: Compare the intersection of the initial detection range and the item's usage range to determine if the range is exceeded.

[0132] "Out of range" refers to the area within the initial detection range that does not intersect with the area where the item is used. By analyzing the intersection between the initial detection range and the area where the item is used, the area within the initial detection range that does not intersect with the area where the item is used is considered to be out of range.

[0133] S51: Mark movable items that are outside the range as movable items.

[0134] Marked movable items refer to movable items that are outside the designated area.

[0135] S52: Retrieve the actual distance of the marker for the movable object from the image detection information.

[0136] The actual distance of the marker refers to the actual distance that the marker movable object can move. It is determined by identifying the horizontal movement distance of the marker movable object from image detection information.

[0137] S53: Compare the moving distance of the marker with the actual distance of the marker to generate and send a placement image by combining the movable object of the marker with the image detection information.

[0138] The placement image refers to a composite image that displays the placement effect of items and suggests their movement to the user. By analyzing whether the marker movement distance exceeds the actual marker distance, if the marker movement distance does not exceed the actual marker distance, the placement image is not output, and the initially detected items whose marker movement distance does not exceed the actual marker distance are removed to update the initial detected items.

[0139] When the marker moves beyond its actual distance, augmented reality compositing technology is used to simulate the placement of the initially detected item at a position beyond the actual distance of the marker in the image detection information (AR compositing SDK: such as Apple ARKit's ARSCNView, Google ARCore) to form an image as the placement image.

[0140] S54: Collect user operation information.

[0141] Operation information refers to the user's feedback operation instructions on the system platform. Operation information is obtained by receiving the user's interactive instructions on the system platform regarding the placement of images, such as clicking the "Confirm Plan", "Adjust Plan", or "Reject Plan" button, and recording this interaction as operation information.

[0142] S55: Update the initial detected items based on the placement image selected based on the operational information.

[0143] If the user confirms a placement scheme corresponding to a certain image by judging the operation information, the size of the initial detection item corresponding to the placement scheme of the image will be filtered to update the initial detection item.

[0144] If the user refuses, the item will be removed from the candidate list so that the initial detection items can continue to be updated.

[0145] Other methods for updating the initial detection items include:

[0146] S60: Collects the user's delivery location.

[0147] The delivery location refers to the final geographical location used by the user to receive the item, which can be obtained through pre-input by the operator.

[0148] S61: Obtain the conveying dimension based on the conveying location and detection text.

[0149] Conveying dimensions refer to the dimensions that allow items to pass through the conveying location. Conveying dimensions are obtained by analyzing and detecting data parameters of the text at the conveying location (such as the diagonal length of an elevator, the corner width of a staircase, and the width of a staircase).

[0150] S62: Update the initial inspection items based on the conveyor dimensions.

[0151] The initial inspection items are updated by analyzing the conveyor dimensions.

[0152] Also includes:

[0153] S70: Collect initial transport type based on initial inspection of items.

[0154] The initial transport type refers to the type of transport of the initially inspected item. It can be retrieved from the system platform as the initial transport type. For example, whole transport versus split transport.

[0155] S71: Compare the initial transport type with the preset split type to obtain the split size.

[0156] The split type is the type of item that can be split and transported, as defined by the technicians.

[0157] The split size refers to the dimensions of each part of the initial inspection item after it has been split. By analyzing the consistency between the initial transportation type and the split type, if the initial transportation type and the split type are inconsistent, no adjustment will be made and the initial inspection size will still be output for subsequent use.

[0158] When the initial transport type is consistent with the split type, the dimensions of each part after splitting are retrieved from the initial detection dimensions as the split dimensions.

[0159] S72: Obtain the transport configuration based on the split dimensions and transport dimensions, or the initial inspection dimensions and transport dimensions.

[0160] The transportation form refers to the shape of an item in its split dimensions or initial inspection dimensions at the transportation location. The transportation form is determined by analyzing the placement of the split dimensions and transportation dimensions, or the initial inspection dimensions and transportation dimensions in three-dimensional space.

[0161] In this embodiment, if the placement shape in three-dimensional space is not satisfied, the initial detection item of that size and shape is discarded to update the initial detection item.

[0162] S73: Update the operation information and use the transportation mode selected in the updated operation information as the marked transportation mode.

[0163] The marked transportation mode refers to the transportation mode selected by the user. After the transportation mode is formed, the operation information is re-collected and the transportation mode selected by the operation information is used as the marked transportation mode.

[0164] In this embodiment, after the user selects the marked transportation form, other initially detected items whose similarity to the marked transportation form exceeds a preset benchmark similarity are retained. For example, similar forms where the hard material on the back of a sofa can come into contact with other surfaces during transportation need to be retained.

[0165] The baseline similarity is the minimum similarity set by technicians to retain the transport morphology of other initially inspected items, in order to reduce the need for operators to select from multiple options one by one.

[0166] S74: Update the initial inspection item by marking the transportation pattern.

[0167] Refer to S73 to retain the initial inspection items that are marked with the mode of transport and similar modes of transport.

[0168] Other methods for updating the initial detection items include:

[0169] S80: Collect the user's historical target items.

[0170] Historical target items refer to the target items that a user has historically selected or purchased. These historical target items can be retrieved from the system platform.

[0171] S81: Obtain duplicate items based on historical target items and initial detection items.

[0172] Duplicate items refer to items in the initial detection that are of the same type as historical target items. Items in the initial detection that are of the same type as historical target items are considered duplicate items.

[0173] S82: Obtain the baseline weight based on historical target items.

[0174] The baseline weight refers to the minimum weight value for repeatedly purchasing items of the same type as the historical target items. The baseline weight is matched from a preset weight lookup table using the historical target items.

[0175] The weight lookup table stores the baseline weights corresponding to different historical target items. The parameters in the weight lookup table are set in advance by those skilled in the art based on actual conditions and will not be elaborated here.

[0176] S83: Obtain duplicate weights based on duplicate items and historical target items.

[0177] The duplication weight is a numerical value used to quantify the degree of duplication between the initial associated item and the user's historical selections. The higher the weight, the greater the probability that the item is repeatedly selected by the user. The correction coefficient is obtained by retrieving the time difference between the duplicate item and the corresponding historical target item of the same type (different items have different correction coefficients; for example, for computers, the shorter the time of repeated computer queries, the smaller the weight, and the less likely it is to be repeatedly recommended). Then, the feature deviation parameters of the duplicate item and the corresponding historical target item of the same type are retrieved. The duplication weight is calculated by combining the correction coefficient, the feature deviation parameters, and the preset initial weight.

[0178] The initial weights are the base values ​​set by the technical staff for weight calculation.

[0179] S84: Compare the duplicate weights with the baseline weights to eliminate duplicate items corresponding to duplicate weights that do not exceed the baseline weights, and update the initial detection items.

[0180] By analyzing the cases where the duplicate weight exceeds the baseline weight, duplicate items corresponding to duplicate weights that do not exceed the baseline weight are removed to obtain new initial detection items.

[0181] Based on the same inventive concept, embodiments of the present invention provide a customer selection system based on three-dimensional multimodal interaction, comprising:

[0182] The acquisition module is used to acquire the detected text, historical detection parameters, and image detection information;

[0183] A memory for storing a program for a customer selection method based on three-dimensional multimodal interaction;

[0184] The processor is used to load and execute programs stored in memory.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0186] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A customer product selection method based on three-dimensional multimodal interaction, characterized in that, include: S10: Collect the detected text and historical detection parameters; S11: Obtain labeled text based on detected text; S12: Obtain the detection model based on historical detection parameters, and obtain the tagging key instructions based on the tagged text and the detection model; S13: Obtain the initial inspection item based on the marked key instructions; S14: Acquire image detection information in response to the detection signal; S15: Generate and send target items using image detection information and initial detected items, and update the detection model based on the target items.

2. The customer selection method based on three-dimensional multimodal interaction according to claim 1, characterized in that, Methods for generating the target item include: S20: Identify environmental objects from image detection information; S21: Determine the initial detection type based on the initial inspection item; S22: Identify items of the initial detection type from environmental items as environmental detection items; S23: Identify other features from environmentally inspected items; S24: Update the initial detected item using other features, and use the updated initial detected item as the target item.

3. The customer selection method based on three-dimensional multimodal interaction according to claim 2, characterized in that, Other methods for generating target items include: S30: Obtain the associated item type based on the initial detection type; S31: Obtain the initial usage environment by initially inspecting the items; S32: Obtain associated usage behavior based on the type of associated item and the initial usage environment; S33: Obtain associated usage characteristics based on associated usage behavior; S34: Use associated features to further assist in filtering the initial detection items, and use the remaining initial detection items after filtering as target items.

4. The customer selection method based on three-dimensional multimodal interaction according to claim 3, characterized in that, Also includes: S40: Update image detection information and detection text based on the initial usage environment; S41: Determine the scope of use of the item based on the updated image detection information and detection text; S42: Based on image detection information and preset movable features, obtain movable objects around the object's usage range; S43: Obtain the initial detection size based on the updated initial detection item; S44: Obtain the initial detection range by using the initial detection size and the detection text; S45: Compare the intersection of the initial detection range and the item usage range to update the initial detection item.

5. A customer selection method based on three-dimensional multimodal interaction according to claim 4, characterized in that, Methods for updating the initial detection items include: S50: Compare the intersection of the initial detection range and the item's usage range to determine if the range is exceeded; S51: Mark movable items located outside the range as movable items; S52: Retrieve the actual distance of the marker for the movable object from the image detection information; S53: Compare the moving distance of the marker with the actual distance of the marker to generate and send a placement image by combining the movable object of the marker with the image detection information; S54: Collect user operation information; S55: Update the initial detected items based on the placement image selected based on the operational information.

6. A customer selection method based on three-dimensional multimodal interaction according to claim 5, characterized in that, Other methods for updating the initial detection items include: S60: Collects the user's delivery location; S61: Obtain the conveying dimension based on the conveying location and detection text; S62: Update the initial inspection items based on the conveyor dimensions.

7. A customer product selection method based on three-dimensional multimodal interaction according to claim 6, characterized in that, Also includes: S70: Collect initial transport type based on initial inspection of items; S71: Compare the initial transport type with the preset split type to obtain the split dimensions; S72: Obtain the transport configuration based on the split dimensions and transport dimensions, or the initial inspection dimensions and transport dimensions; S73: Update the operation information and use the selected transportation mode in the updated operation information as the marked transportation mode; S74: Update the initial inspection item by marking the transportation pattern.

8. A customer selection method based on three-dimensional multimodal interaction according to claim 7, characterized in that, Other methods for updating the initial detection items include: S80: Collects the user's historical target items; S81: Obtain duplicate items based on historical target items and initial detected items; S82: Obtain baseline weights based on historical target items; S83: Obtain duplicate weights based on duplicate items and historical target items; S84: Compare the duplicate weights with the baseline weights to eliminate duplicate items corresponding to duplicate weights that do not exceed the baseline weights, and update the initial detection items.

9. A customer selection system based on three-dimensional multimodal interaction, characterized in that, include: The acquisition module is used to acquire the detected text, historical detection parameters, and image detection information; A memory for storing a program that implements a customer selection method based on three-dimensional multimodal interaction as described in any one of claims 1 to 8; The processor is used to load and execute programs stored in memory.

Images