An e-commerce platform barrier-free shopping method and system supporting multi-modal interaction

Abstract

This invention discloses an accessible shopping method and system for e-commerce platforms that supports multimodal interaction. The system includes supporting multiple types of input operations for users with disabilities; extracting and weighting multimodal commands to generate standardized operation commands that conform to system specifications; executing core shopping operations such as product browsing, searching, and ordering, adapting to accessible interaction processes; and outputting multiple types of accessible operation result feedback, such as voice navigation, Braille display, and tactile feedback. The system supports the acquisition and processing of multimodal interaction commands, including voice, gestures, Braille, and tactile feedback, and can automatically select the optimal interaction combination based on different disability types. The multimodal interaction method fully considers the physical characteristics and operating habits of users with disabilities, reducing the difficulty of shopping operations and enabling them to complete core shopping operations such as product browsing and ordering more conveniently and smoothly, greatly improving the shopping experience.

Description

Technical Field

[0001] This invention relates to the field of e-commerce platform shopping technology, and more specifically, to a method and system for barrier-free shopping on e-commerce platforms that supports multimodal interaction. Background Technology

[0002] In traditional e-commerce shopping scenarios, the interaction methods are mainly designed for able-bodied users, usually relying on visual and mouse / keyboard operations. However, due to physical limitations, users with disabilities face many difficulties in the process of shopping on traditional e-commerce platforms. For example, visually impaired users cannot obtain product information through vision and find it difficult to use visual-based interactive interfaces; users with physical disabilities may not be able to accurately operate the mouse and keyboard, making the shopping process complex and prone to errors.

[0003] With increasing societal emphasis on the rights of people with disabilities and the development of accessible environments, the demand for accessible shopping is becoming increasingly prominent. Although some e-commerce platforms have begun to offer accessible services, most only support single-modal interaction, such as providing only voice interaction or simple text prompts, which cannot meet the diverse operational needs of users with disabilities. At the same time, existing systems lack an effective mechanism for integrating and processing multimodal commands, resulting in insufficient accuracy and fluency of interaction. In addition, in terms of product recommendations, the special needs and usage habits of users with disabilities are not fully considered, and the accuracy of recommendations needs to be improved. Therefore, we propose an accessible shopping method and system for e-commerce platforms that supports multimodal interaction. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method and system for barrier-free shopping on an e-commerce platform that supports multimodal interaction, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method and system for barrier-free shopping on an e-commerce platform that supports multimodal interaction, comprising the following specific steps: Step 1: Collect multimodal interactive commands including voice, gestures, Braille, and tactile feedback to support multiple types of input operations for users with disabilities; Step 2: Extract features and perform weighted fusion on the multimodal interaction commands to generate standard commands; Step 3: Generate standard commands to execute core shopping operations such as browsing products, searching, and placing orders, adapting to an accessible interaction process; Step 4: Output multiple types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback; Step 5: Based on the user's purchased product categories and profile, generate related product recommendations; Step Six: Build and dynamically update user profiles, including spending levels and interest tags, to support accurate and personalized recommendations; Step 7: Collect shopping behavior and satisfaction data to evaluate the effectiveness of barrier-free shopping and optimize system performance; Step 8: Monitor the system's operational status in real time to ensure the stability of multimodal interaction and recommendation services.

[0006] Preferably, in step two, when performing feature extraction and weighted fusion of multimodal instructions, a weight-based fusion algorithm is used: Assign corresponding weights to the characteristics of instructions of different modalities, and fuse multimodal instructions into standardized operation instructions by weighted summation. The weights are set according to the reliability and accuracy factors of different instructions, and the sum of all weights is 1. The feature weights of voice commands are The feature weights of gesture commands are Braille instruction feature weights are The haptic feedback command feature weights are: ,and + + + =1, the weighted fusion calculation formula is: O = in For voice command feature values, For gesture command feature values, For Braille instructions, This represents the characteristic value of the haptic feedback command.

[0007] Preferably, in step five, the specific steps for generating related product recommendations are as follows: The system calls upon the user profile data that has been built and dynamically updated on the e-commerce platform, as well as the category data of all the products purchased by the user in history. The user profile data includes at least the user's consumption level and interest tags, and the product category data includes at least the primary category, the secondary category, and the category-related attributes. Statistical analysis is performed on the category data of users' historical purchases to extract the core purchase categories, high-frequency purchase categories, and secondary related categories. The correlation coefficient between each category is calculated, and the correlation coefficient is determined based on the frequency of simultaneous purchases by users, the time interval between purchases, and the similarity of category attributes. By combining the consumption level and interest tags in the user profile, the products corresponding to the core purchase category, high-frequency purchase category and secondary related category are initially screened, and products that deviate from the user's consumption level by more than a preset threshold or do not match the interest tags are removed, resulting in a set of candidate related products. Based on the category relevance coefficient, the matching degree between products and user interest tags, product reputation rating, and adaptability to barrier-free use scenarios, a weighted weight is set for each evaluation indicator, and products in the candidate related product set are prioritized to determine the recommended sequence of related products. Based on the user's commonly used multimodal interaction methods, the related product recommendation sequence is adapted to generate recommendation result data adapted to the corresponding interaction methods. The recommendation result data should at least include the core product information and accessibility operation guidance. The adapted related product recommendation results are synchronously transmitted to the e-commerce platform's multimodal feedback module. Combined with the accessibility operation result feedback method in step four, related product recommendation information is output to the user, thus completing the related product recommendation.

[0008] Preferably, the method for calculating the category correlation coefficient is as follows: Association coefficient = α × proportion of simultaneous purchase frequency + β × weight of purchase time interval + γ × similarity of category attributes; α, β, and γ are weighting coefficients, and α + β + γ = 1. The values ​​of α, β, and γ are dynamically adjusted based on user shopping behavior data.

[0009] Preferably, in step six, user profiles are constructed and dynamically updated; Based on users' purchase records, the total amount spent by users within a certain period of time, as well as the percentage of spending on different product categories, are calculated. By tracking the frequency of user purchases each week and month, and combining the amount and frequency of purchases, we can assess users' spending activity and spending power, and categorize users into high, medium, and low spending levels.

[0010] Preferably, the generation of interest tags includes: Based on users' browsing, searching, purchasing, favorites and add-to-cart records, we can analyze the degree of user attention to different product categories and assign an interest weight to each category. The higher the weight, the greater the user's interest in that category. Analyze the brand distribution of products purchased by users to identify brands that users frequently purchase. If a user has purchased clothing from a certain brand multiple times, an interest tag can be generated for them as a "clothing preference of the brand name". Interest tags are generated based on product attributes, which must include at least color, size, and material. By integrating interest tags generated from different dimensions and removing duplicate and less relevant tags, a comprehensive and accurate user interest tag system is formed.

[0011] Preferably, the system includes: The multimodal command acquisition module is used to acquire multimodal interactive commands such as voice, gestures, Braille, and tactile feedback. The instruction fusion and standardization module is used to extract features and weightedly fuse multimodal instructions to generate standardized operation instructions. The barrier-free shopping operation execution module is used to perform core shopping operations such as browsing products, searching, and placing orders; The accessibility result feedback module is used to output various types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback. The related product recommendation module is used to generate related product recommendations for tops and bottoms, and shoes and tights based on the user's purchased product categories and profile. The user profile building and updating module is used to build and dynamically update user profiles. The effectiveness evaluation and system optimization module is used to collect shopping behavior and satisfaction data, evaluate the effectiveness of barrier-free shopping, and optimize system performance. The system monitoring module is used to monitor the system's operating status in real time and ensure the stability of multimodal interaction and recommendation services.

[0012] The technical effects and advantages of this invention are as follows: In use, this system supports the acquisition and processing of multimodal interactive commands such as voice, gesture, Braille, and tactile feedback. It can automatically select the optimal combination of interactions based on different types of disabilities. For example, visually impaired users can use voice navigation, Braille confirmation, and tactile feedback to complete shopping operations, while users with physical disabilities can use gesture swipe filtering, voice search, and tactile click confirmation to shop. This multimodal interaction method fully considers the physical characteristics and operating habits of users with disabilities, reduces the difficulty of shopping operations, and enables them to complete core shopping operations such as browsing products, searching, and placing orders more conveniently and smoothly, greatly improving the shopping experience.

[0013] In use, this invention involves a system that constructs and dynamically updates user profiles. Combining user purchase categories with profile information, a collaborative filtering algorithm is used to calculate product relevance and generate recommended product combinations. For example, based on a user's historical purchase history of athletic shoes and the "sports enthusiast" profile tag, a "shoes-socks" combination recommendation is generated. Simultaneously, the recommendation list is sorted by user interest weights and displayed in various ways. Furthermore, the system collects shopping behavior and satisfaction data to continuously optimize recommendation strategies, such as adjusting the display position and text of recommendations based on A / B testing results. This precise, personalized recommendation better meets user needs, increases user attention to recommended products and purchase intention, thereby effectively improving product conversion rates. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] As attached Figures 1-2 The illustrated method and system for barrier-free shopping on an e-commerce platform that supports multimodal interaction includes the following specific steps: Step 1: Collect multimodal interactive commands including voice, gestures, Braille, and tactile feedback to support multiple types of input operations for users with disabilities; Step 2: Extract features and perform weighted fusion on the multimodal interaction commands to generate standard commands; Step 3: Generate standard commands to execute core shopping operations such as browsing products, searching, and placing orders, adapting to an accessible interaction process; Step 4: Output multiple types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback; Step 5: Based on the user's purchased product categories and profile, generate related product recommendations; Step Six: Build and dynamically update user profiles, including spending levels and interest tags, to support accurate and personalized recommendations; Step 7: Collect shopping behavior and satisfaction data to evaluate the effectiveness of barrier-free shopping and optimize system performance; Step 8: Monitor the system's operational status in real time to ensure the stability of multimodal interaction and recommendation services.

[0018] In step two, when performing feature extraction and weighted fusion of multimodal instructions, a weight-based fusion algorithm is used: Assign corresponding weights to the characteristics of instructions of different modalities, and fuse multimodal instructions into standardized operation instructions by weighted summation. The weights are set according to the reliability and accuracy factors of different instructions, and the sum of all weights is 1. The feature weights of voice commands are The feature weights of gesture commands are Braille instruction feature weights are The haptic feedback command feature weights are: ,and + + + =1, the weighted fusion calculation formula is: O = in For voice command feature values, For gesture command feature values, For Braille instructions, This represents the characteristic value of the haptic feedback command.

[0019] In step five, the specific steps for generating related product recommendations are as follows: The system calls upon the user profile data that has been built and dynamically updated by the e-commerce platform, as well as the category data of all the products purchased by the user in history. The user profile data includes at least the user's consumption level and interest tags, and the product category data includes at least the primary category, the secondary category, and the category association attributes. Statistical analysis is performed on the category data of users' historical purchases to extract the core purchase categories, high-frequency purchase categories, and secondary related categories. The correlation coefficient between each category is calculated, and the correlation coefficient is determined based on the frequency of simultaneous purchases by users, the time interval between purchases, and the similarity of category attributes. Combining the consumption level and interest tags in the user profile, the products corresponding to the core purchase category, high-frequency purchase category and secondary related category are initially screened, and products that deviate from the user's consumption level by more than a preset threshold or do not match the interest tags are removed, resulting in a candidate related product set; Based on the category relevance coefficient, the matching degree between products and user interest tags, product reputation rating, and adaptability to barrier-free use scenarios, a weighted weight is set for each evaluation indicator, and products in the candidate related product set are prioritized to determine the recommended sequence of related products. Based on the user's commonly used multimodal interaction methods, the associated product recommendation sequence is adapted to generate recommendation result data adapted to the corresponding interaction methods. The recommendation result data includes at least the core product information and accessibility operation guidance. The adapted related product recommendation results are synchronously transmitted to the multimodal feedback module of the e-commerce platform. Combined with the accessibility operation result feedback method in step four, related product recommendation information is output to the user to complete the related product recommendation.

[0020] The method for calculating the category correlation coefficient is as follows: Association coefficient = α × proportion of simultaneous purchase frequency + β × weight of purchase time interval + γ × similarity of category attributes; α, β, and γ are weighting coefficients, and α + β + γ = 1. The values ​​of α, β, and γ are dynamically adjusted based on user shopping behavior data.

[0021] In step six, user profiles are constructed and dynamically updated; Based on users' purchase records, the total amount spent by users within a certain period of time, as well as the percentage of spending on different product categories, are calculated. By tracking the frequency of user purchases each week and month, and combining the amount and frequency of purchases, we can assess users' spending activity and spending power, and categorize users into high, medium, and low spending levels.

[0022] The generation of interest tags includes: Based on users' browsing, searching, purchasing, favorites and add-to-cart records, we can analyze the degree of user attention to different product categories and assign an interest weight to each category. The higher the weight, the greater the user's interest in that category. Analyze the brand distribution of products purchased by users to identify brands that users frequently purchase. If a user has purchased clothing from a certain brand multiple times, an interest tag can be generated for them as a "clothing preference of the brand name". Interest tags are generated based on product attributes, which must include at least color, size, and material. By integrating interest tags generated from different dimensions and removing duplicate and less relevant tags, a comprehensive and accurate user interest tag system is formed.

[0023] The system includes: The multimodal command acquisition module is used to acquire multimodal interactive commands such as voice, gestures, Braille, and tactile feedback. The instruction fusion and standardization module is used to extract features and weightedly fuse multimodal instructions to generate standardized operation instructions. The barrier-free shopping operation execution module is used to perform core shopping operations such as browsing products, searching, and placing orders; The accessibility result feedback module is used to output various types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback. The related product recommendation module is used to generate related product recommendations for tops and bottoms, and shoes and tights based on the user's purchased product categories and profile. The user profile building and updating module is used to build and dynamically update user profiles. The effectiveness evaluation and system optimization module is used to collect shopping behavior and satisfaction data, evaluate the effectiveness of barrier-free shopping, and optimize system performance. The system monitoring module is used to monitor the system's operating status in real time and ensure the stability of multimodal interaction and recommendation services.

[0024] After the system is powered on, it automatically detects the connection status and functional integrity of devices such as microphones, cameras, Braille keyboards, and tactile sensors. If a microphone malfunction is detected, the system marks "voice input unavailable" in the log and prioritizes the use of gesture interaction mode. Load the system configuration file, including the product database address, database connection parameters, API interface address, initial weight allocation, and accessibility rules. If the voice command weight is 0.4 by default, extend the voice navigation response time to 3 seconds. Then start the log service to record the device status, the command processing process of successfully parsed gesture commands, and the user's browsing operation trajectory in the sports shoe category. Automatically select the optimal interaction combination based on the type of user device, such as mobile / smart wearable devices, and different types of disability; For visually impaired users: Voice navigation + Braille confirmation + tactile feedback, with vibration indicating successful operation.

[0025] For users with physical disabilities: Gesture swipe filtering + voice search + tactile click confirmation.

[0026] Data from each channel is linked by user ID to form a unified multimodal command stream. For example, user A's voice command "search for sports shoes" and gesture command "filter price range 200-500 yuan" are processed together. In scenario 1, product browsing and searching, user interaction and shopping operations require multimodal command input: Voice command: The user says "search for sneakers", and the system analyzes the keyword "sneakers" through voice recognition and extracts the voice feature value O1; Gesture commands: Users use gestures to swipe the screen to filter price ranges. The system uses computer vision to capture the coordinates of the gesture trajectory and extract gesture feature values, such as the swiping direction "from top to bottom". Braille instructions: The user inputs "men's clothing category" via the Braille keyboard, and the system decodes the Braille dot code to extract the Braille feature value corresponding to the code sequence of "men's clothing"; Haptic feedback: When a user presses and holds the screen to confirm the filter criteria, the system records the tactile pressure intensity and extracts tactile feature values. If the pressure value is >80%, confirmation is triggered. The system dynamically adjusts weights based on instruction reliability; for example, voice confidence 0.9 → w1 = 0.5, gesture accuracy 0.7 → w2 = 0.3. O = =0.5×0.9+0.3×0.7+0.1×0.8+0.1×0.6=0.78 Generate standardized operation instructions such as "Search for athletic shoes priced between 200-500 yuan under the men's clothing category" to drive the subsequent shopping process; The system automatically completes the following operations based on standardized instructions: When browsing products, the voice navigation leads to the "Men's Clothing → Sports Shoes" category, and the tactile vibration prompts "Current Category: Sports Shoes"; when searching and filtering, the user can use gestures to swipe through the price range of 200-500 yuan, and the system updates the filtering results in real time and announces "3 products have been filtered"; when confirming the order, the user enters the shipping address through the Braille keyboard, the system calls the order management system to generate the order, and prompts with the voice "Total price 299 yuan, confirm order?".

[0027] Related product recommendations User profile building involves collecting users' historical behavior data, such as whether they have purchased athletic shoes or browsed athletic socks, and using profile tags like "sports enthusiast" or "high-spending level" to dynamically update user profiles. Recommendation generation is also performed, such as generating "shoes-socks" combination recommendations based on an 80% co-occurrence probability of athletic shoes and athletic socks. The recommendation list is sorted by user interest weights and displayed via a pop-up window: "Enjoy 20% off when you buy athletic socks together, would you like to add them to your cart?". Accessibility feedback output: The system outputs operation results to the user through a multimodal channel. The voice announcement reads, "You have added the following to your shopping cart: Black sneakers, size 42. You can save 20 yuan by wearing them with sports socks." The Braille display shows the order summary as "Total price: 299 yuan, including 10 yuan shipping fee"; The tactile feedback is as follows: a short vibration (2 times) indicates a successful operation, and a long press and vibration (5 times) indicates an error (such as payment failure). Data collection records user browsing time, click-through rate (e.g., 60% click-through rate for recommended products), conversion rate (e.g., 20% order rate), and satisfaction data; if the gesture recognition error rate is higher than 10%, the system reduces the weight of gestures. Based on the A / B test results, if the conversion rate of the pop-up recommendation is 15% higher than that of the bottom navigation bar, adjust the recommendation display position and copy. Real-time monitoring of hardware status, such as alarms when microphone battery is <20%; software performance, such as triggering optimizations when command processing latency is >500ms; and interface stability, such as switching to a backup interface when payment API success rate is <95%. Notifications to operations and maintenance personnel via email / SMS to handle anomalies, such as "Camera offline, please check connection," and regular generation of system health reports, such as "Gesture recognition accuracy improved by 5% this week."

[0028] User A (visually impaired) logs into the system. The system automatically activates the voice + Braille + tactile interaction mode. User A says "Search for summer sports shoes". The system integrates the voice command (w1=0.5) and the Braille input "summer" (w3=0.3) to generate the standardized command "Search for summer men's sports shoes". The system provides voice navigation to the category and tactile vibration prompts "5 products have been selected".

[0029] User A selects size 42 using a Braille keyboard. The system places an order and announces via voice, "Total price: 299 yuan, estimated delivery tomorrow." The system recommends sports socks, and User A confirms via voice, "Add to cart." The system updates the order and vibrates to indicate "Operation successful." The system records User A's browsing behavior and will subsequently recommend more precise "sports equipment combinations" (such as sports shoes + sports socks + backpack). Users with disabilities can open the e-commerce platform's app or website that supports multimodal interaction and enter the registration page. The platform offers multiple accessible registration methods: visually impaired users can complete the information filling through voice prompts, hearing-impaired users can use text prompts and gesture guidance, and users with physical disabilities can choose a suitable input method based on their own circumstances, such as using head control devices in conjunction with the interface interaction to complete the registration. During the registration process, users need to fill in basic information, such as age, gender, and region. At the same time, the platform will ask users whether they are disabled and what their specific disability type is, in order to provide more accurate accessibility services and personalized recommendations in the future. Once users enter the e-commerce platform, they begin shopping. The platform's multimodal instruction acquisition module monitors various user inputs in real time. For example, visually impaired users can speak the product category they want to browse, such as "I want to look at sneakers"; hearing-impaired users can use gestures on the screen to indicate a search action and point to relevant product areas; users using Braille input devices can input search keywords by touching the Braille keyboard; and users with physical disabilities may use tactile feedback devices, such as pressure sensors, to trigger specific operation instructions, such as clicking on product images to view them. After receiving standardized operation instructions, the accessible shopping operation execution module performs core shopping operations such as browsing products, searching, and placing orders, and adapts to accessible interaction processes. For example, when a user expresses a voice command that they want to browse sneakers, the system automatically redirects to the sneaker product display page, while providing corresponding accessible browsing methods based on the user's disability type. Visually impaired users can listen to product information through voice navigation, such as "This sneaker is white, the brand is [brand name], and the price is [specific price]". Hearing-impaired users can learn about product details through text prompts and image displays. Users with physical disabilities can use haptic feedback devices, such as vibration prompts, to perceive certain features of the product, such as whether it is in stock.

[0030] If a user wants to purchase a product, they can place an order through multimodal interaction. The system will guide the user to complete the payment and other operations. The entire process fully considers the accessibility needs of users with disabilities. The accessibility feedback module outputs various types of accessibility feedback based on the user's operation results, including voice navigation, Braille display, and tactile feedback. For example, after a user successfully places an order, a visually impaired user will hear a voice prompt: "You have successfully placed an order for [product name], order number [specific number], estimated delivery time [time]". A hearing-impaired user will see a text prompt on the screen: "Order successful, order number [specific number], please pay attention to the logistics information". Users using Braille display devices can obtain relevant information about successful order placement by touching the Braille display screen. Users with physical disabilities will perceive that the order placement operation has been completed through tactile feedback devices, such as slight vibrations. The user profile building and updating module calculates a user's total spending over a specific period based on their purchase history, as well as the percentage of spending across different product categories. For example, it calculates a user's spending on clothing, food, and electronics over the past three months, determining the proportion of each category in their total spending. Simultaneously, it tracks the frequency of weekly and monthly purchases, combining spending amount and frequency to assess user activity and spending power, categorizing users into high, medium, and low spending levels. For instance, users with high spending and high purchase frequency are categorized as high-spending users.

[0031] Based on users' browsing, searching, purchasing, favorites and add-to-cart records, we can analyze the degree of user attention to different product categories and assign an interest weight to each category. The higher the weight, the greater the user's interest in that category. The related product recommendation module utilizes user profile data that has been built and dynamically updated by the e-commerce platform, as well as category data of all products purchased by the user in the past. The user profile data includes information such as user spending power and interest tags, while the product category data includes primary categories, secondary categories, and category-related attributes. Statistical analysis is performed on the category data of the user's historical purchases to extract the user's core purchase categories, frequently purchased categories, and secondary related categories. The correlation coefficient between product categories is calculated as follows: Correlation Coefficient = α × Shared Purchase Frequency + β × Purchase Time Interval Weight + γ × Category Attribute Similarity (where α, β, and γ are weighted coefficients, and α + β + γ = 1; the values ​​of α, β, and γ are dynamically adjusted based on user shopping behavior data). For example, if athletic shoes and sportswear are frequently purchased together by users, have a high shared purchase frequency, and a short purchase time interval, their category attribute similarity is high, then their correlation coefficient will be high. By combining user profiles with their spending levels and interest tags, a preliminary screening is performed on products corresponding to core purchase categories, high-frequency purchase categories, and secondary related categories. Products that deviate significantly from the user's spending level or do not match their interest tags are then removed, resulting in a candidate set of related products.

[0032] Finally, the following points should be noted: First, in the description of this invention, it should be noted that, unless otherwise specified and limited, the terms "installation", "connection", and "linkage" should be interpreted broadly, and can refer to mechanical connection or electrical connection, or internal connection between two components, or direct connection. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationship. When the absolute position of the object being described changes, the relative positional relationship may change. Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for barrier-free shopping on an e-commerce platform that supports multimodal interaction, characterized in that: The specific steps include the following: Step 1: Collect multimodal interactive commands including voice, gestures, Braille, and tactile feedback to support multiple types of input operations for users with disabilities; Step 2: Extract features and perform weighted fusion on the multimodal interaction commands to generate standard commands; Step 3: Generate standard commands to execute core shopping operations such as browsing products, searching, and placing orders, adapting to an accessible interaction process; Step 4: Output multiple types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback; Step 5: Based on the user's purchased product categories and profile, generate related product recommendations; Step Six: Build and dynamically update user profiles, including spending levels and interest tags, to support accurate and personalized recommendations; Step 7: Collect shopping behavior and satisfaction data to evaluate the effectiveness of barrier-free shopping and optimize system performance; Step 8: Monitor the system's operational status in real time to ensure the stability of multimodal interaction and recommendation services.

2. The method for barrier-free shopping on an e-commerce platform supporting multimodal interaction according to claim 1, characterized in that: In step two, when performing feature extraction and weighted fusion of multimodal instructions, a weight-based fusion algorithm is used: Assign corresponding weights to the characteristics of instructions of different modalities, and fuse multimodal instructions into standardized operation instructions by weighted summation. The weights are set according to the reliability and accuracy factors of different instructions, and the sum of all weights is 1. The feature weights of voice commands are The feature weights of gesture commands are Braille instruction feature weights are The haptic feedback command feature weights are: ,and + + + =1, the weighted fusion calculation formula is: O＝ in For voice command feature values, For gesture command feature values, For Braille instructions, This represents the characteristic value of the haptic feedback command.

3. The method for barrier-free shopping on an e-commerce platform supporting multimodal interaction according to claim 1, characterized in that: In step five, the specific steps for generating related product recommendations are as follows: The system calls upon the user profile data that has been built and dynamically updated by the e-commerce platform, as well as the category data of all the products purchased by the user in history. The user profile data includes at least the user's consumption level and interest tags, and the product category data includes at least the primary category, the secondary category, and the category association attributes. Statistical analysis is performed on the category data of users' historical purchases to extract the core purchase categories, high-frequency purchase categories, and secondary related categories. The correlation coefficient between each category is calculated, and the correlation coefficient is determined based on the frequency of simultaneous purchases by users, the time interval between purchases, and the similarity of category attributes. Combining the consumption level and interest tags in the user profile, the products corresponding to the core purchase category, high-frequency purchase category and secondary related category are initially screened, and products that deviate from the user's consumption level by more than a preset threshold or do not match the interest tags are removed, resulting in a candidate related product set; Based on the category relevance coefficient, the matching degree between products and user interest tags, product reputation rating, and adaptability to barrier-free use scenarios, a weighted weight is set for each evaluation indicator, and products in the candidate related product set are prioritized to determine the recommended sequence of related products. Based on the user's commonly used multimodal interaction methods, the associated product recommendation sequence is adapted to generate recommendation result data adapted to the corresponding interaction methods. The recommendation result data includes at least the core product information and accessibility operation guidance. The adapted related product recommendation results are synchronously transmitted to the multimodal feedback module of the e-commerce platform. Combined with the accessibility operation result feedback method in step four, related product recommendation information is output to the user to complete the related product recommendation.

4. The method for barrier-free shopping on an e-commerce platform supporting multimodal interaction according to claim 3, characterized in that: The method for calculating the category correlation coefficient is as follows: Association coefficient = α × proportion of simultaneous purchase frequency + β × weight of purchase time interval + γ × similarity of category attributes; α, β, and γ are weighting coefficients, and α + β + γ = 1. The values ​​of α, β, and γ are dynamically adjusted based on user shopping behavior data.

5. The method for barrier-free shopping on an e-commerce platform supporting multimodal interaction according to claim 1, characterized in that: Step six, which involves building and dynamically updating the user profile, includes: Based on users' purchase records, the total amount spent by users within a certain period of time, as well as the percentage of spending on different product categories, are calculated. By tracking the frequency of user purchases each week and month, and combining the amount and frequency of purchases, we can assess users' spending activity and spending power, and categorize users into high, medium, and low spending levels.

6. The method for barrier-free shopping on an e-commerce platform supporting multimodal interaction according to claim 3, characterized in that: The generation of interest tags includes: Based on users' browsing, searching, purchasing, favorites and add-to-cart records, we can analyze the degree of user attention to different product categories and assign an interest weight to each category. The higher the weight, the greater the user's interest in that category. Analyze the brand distribution of products purchased by users to identify brands that users frequently purchase. If a user has purchased clothing from a certain brand multiple times, an interest tag can be generated for them as a "clothing preference for the brand name". Interest tags are generated based on product attributes, which must include at least color, size, and material. By integrating interest tags generated from different dimensions and removing duplicate and less relevant tags, a comprehensive and accurate user interest tag system is formed.

7. A barrier-free shopping system for an e-commerce platform supporting multimodal interaction according to claim 1, used to implement the method of any one of claims 1 to 6, characterized in that, The system includes: The multimodal command acquisition module is used to acquire multimodal interactive commands such as voice, gestures, Braille, and tactile feedback. The instruction fusion and standardization module is used to extract features and weightedly fuse multimodal instructions to generate standardized operation instructions. The barrier-free shopping operation execution module is used to perform core shopping operations such as browsing products, searching, and placing orders; The accessibility result feedback module is used to output various types of accessibility operation result feedback, including voice navigation, Braille display, and tactile feedback. The related product recommendation module is used to generate related product recommendations for tops and bottoms, and shoes and tights based on the user's purchased product categories and profile. The user profile building and updating module is used to build and dynamically update user profiles. The effectiveness evaluation and system optimization module is used to collect shopping behavior and satisfaction data, evaluate the effectiveness of barrier-free shopping, and optimize system performance. The system monitoring module is used to monitor the system's operating status in real time and ensure the stability of multimodal interaction and recommendation services.

Images