1002 results about "Interactive technology" patented technology

The invention belongs to the technical field of digital modeling, and relates to a three-dimensional automatic modeling method and system based on an AI large model technology, and the method comprises the steps: obtaining multi-modal data inputted by a user, and converting the multi-modal data into feature vectors in a unified coding format; generating a conflict arbitration result based on a priority rule preset by the user or a default physical reasonability rule; according to a conflict arbitration result, generating an initial three-dimensional model through a large language model and a neural radiation field which are jointly trained; inputting the initial three-dimensional model into a physical simulation engine for physical verification, and outputting a correction model containing a structure defect mark; the correction instruction is analyzed into a model parameterization adjustment command; and based on a parameterized adjustment command, correcting an associated region in the model to perform incremental solution and re-rendering to generate a final three-dimensional model, and updating a verification result of the physical simulation engine. According to the method, the problem that a fuzzy semantic instruction is difficult to accurately convert into a model modification action due to the fact that a multi-modal interaction technology stays at a basic parameter adjustment level is solved.

The invention discloses a real-time interactive digital human system supporting high concurrency and an implementation method thereof, and relates to the technical field of digital human interaction.The system comprises a model instance pool module used for loading the weight of a deep learning model to a shared memory area through the memory mapping technology; the multi-thread scheduling module is used for scheduling user requests by adopting a lock-free queue and a dynamic priority algorithm; the asynchronous pipeline processing module is composed of decoupled micro-services, and the modules are connected in series through asynchronous message queues; the client SDK is used for dynamically switching a rendering mode according to terminal hardware performance and network conditions; the elastic capacity expansion and contraction module is used for monitoring resource loads in real time and automatically adjusting the number of service instances; and the audio and video synchronization calibration module is used for ensuring that the synchronization error of the audio and video frames is lower than a preset threshold value through a timestamp alignment algorithm. According to the scheme, core challenges in a high-concurrency digital human interaction scene can be systematically solved, and innovative support is provided for large-scale real-time application.

The invention relates to the technical field of multimedia interaction, in particular to an intelligent interaction control method and system for digital exhibition hall multimedia equipment, and the method comprises the following steps: S1, carrying out the arrangement of a sensor, and carrying out the intelligent sensing and data collection; s2, generating an intelligent interaction strategy; s3, equipment cooperative control and execution; and S4, performing real-time feedback and system optimization. According to the method, user behaviors and environment data are collected in real time through the multi-modal sensor, user requirements are accurately speculated in combination with a deep learning algorithm, the optimal interaction strategy and task priority are generated, individuation and interaction efficiency of user experience are remarkably improved, multi-device cooperation is optimized through distributed task scheduling, and user experience is improved. According to the method, display content and environment parameters are dynamically adjusted, self-optimization of the system is realized by utilizing user feedback, real-time performance, smoothness and reliability of interactive experience of the exhibition hall are ensured, rapid fault response and accurate maintenance demand prediction are realized through anomaly detection and equipment log analysis, and the stability and sustainability of operation of the exhibition hall system are greatly improved.

The invention relates to the technical field of AI interaction, and discloses an AI multi-modal emotion interaction memory terminal, which realizes microsecond-level synchronization of voice, facial expression and text data through a multi-thread acquisition engine, dynamically allocates each modal weight by adopting a multi-head cross attention mechanism, and adaptively adjusts modal importance based on a conversation context hidden state; when the cross-modal confidence difference exceeds a threshold value, a gating LSTM conflict resolution module is activated, and the multi-source data collaboration problem is solved; the emotional memory modeling constructs an emotional state transition topology based on a graph convolutional network, protects user privacy in combination with a differential privacy mechanism, and realizes associated event storage of millisecond backtracking of short-term memory and long-term memory. The technology integrates multi-modal dynamic perception, privacy security calculation and adaptive learning ability, significantly improves the real-time performance and personification degree of emotion interaction, and can be applied to the fields of intelligent customer service, emotion accompanying, health monitoring and the like.

The invention provides an interaction method and system applied to painting and calligraphy exhibition, and relates to the technical field of data interaction, and the method comprises the steps: building a user exhibition watching behavior data set through real-time collection based on an exhibit region; traversing the exhibit area to carry out multi-dimensional image calibration, and establishing a coordinate-label mapping network; traversing the coordinate-label mapping network set according to the user exhibition viewing behavior data set to activate an exhibit identification module, identifying a target exhibit number to perform linkage display of painting and calligraphy on a target user, generating display information, starting an exhibit interaction module to perform deep interaction on the target user in combination with the display information, and generating a personalized painting and calligraphy display guide path. The technical problem that in the prior art, due to the lack of real-time perception and accurate recognition of exhibition watching behaviors of users, personalized exhibit linkage and dynamic guide cannot be achieved is solved, and the technical effect that the personalized guide path is generated based on the interests of the users, so that the exhibition watching immersion and interactive experience are improved is achieved.

The invention provides an interactive integration method and system for an AI large model and a low-code platform, and relates to the technical field of interaction of the AI large model and the low-code platform, and the method comprises the steps: obtaining a task demand input by a user, analyzing the task demand into a structured task instruction, automatically matching an optimal AI model, and dynamically generating an adaptive API call parameter, according to the method, the AI large model and the low-code platform can be deeply integrated, the technical problems of integration complexity, interaction mode solidification, updating lag and the like are solved, and the method has the advantages that the method is simple and convenient to operate, and the method is suitable for popularization and application. According to the method, a user can call a complex model without AI professional knowledge, the low-code development efficiency is improved by more than 50%, and the security in a data processing process is enhanced by introducing a security verification mechanism, so that the integration efficiency of a low-code platform and an AI large model is greatly improved.

The invention discloses a voice interaction optimization method and system based on a multi-modal large model, and relates to the technical field of artificial intelligence and voice interaction, and the method comprises the steps: carrying out the voice recognition in response to the real-time voice of a user, and obtaining voice text information; according to the voice text information, combining the voice waveform of the real-time voice of the user as the input of a multi-modal recognition model, so as to judge whether the voice dialogue is interrupted and recognize the interruption intention, and obtaining a voice interruption result; obtaining a new intention of the user according to a voice interruption result, and dynamically adjusting a system response strategy to realize interaction optimization; the accuracy and comprehensiveness of interruption detection are improved through multi-modal fusion, so that the interruption intention is accurately recognized to respond to the user intention in real time, the system dialogue interaction efficiency and reliability are remarkably improved, and the defects that existing voice interruption detection is not high in detection accuracy, dynamic response of user interruption behaviors cannot be achieved, and user experience is poor are overcome. Therefore, the problems of low interaction efficiency and poor reliability of the voice dialogue system are solved.

The invention discloses a sound interaction intention recognition and intelligent decision-making method based on an AI large model, and relates to the technical field of intelligent voice interaction, and the method comprises the steps: collecting a voice signal through an acoustic sensor, carrying out the noise reduction processing and acoustic feature extraction, capturing a text instruction, and carrying out the semantic segmentation and text feature extraction, splicing the acoustic feature vector and the text feature vector to form a multi-modal data packet; retrieving a historical memory library based on the enhanced fusion feature vector to generate a memory context vector, identifying the category of a deliberate map through a two-stage intention reasoning model, analyzing operation parameters, and outputting a structured intention instruction; and performing parameter legality verification, equipment state verification and security risk assessment on the structured intention instruction, and packaging the structured intention instruction into an executable instruction set after correcting abnormal parameters. According to the method, through double screening of the frequency band energy ratio and the lexical item importance score, the acoustic-text feature scale difference is reduced.

The invention discloses a virtual-real fusion exhibition display interaction system and a multi-mode perception method, and belongs to the technical field of exhibition display interaction. The system collects audience eyeball fixation points, gesture actions and ambient light data through AR / VR equipment, analyzes coordinates of a region of interest through an eyeball fixation point attention mechanism, a gesture space-time encoder and a multi-modal fusion unit, triggers holographic projection explanation and virtual exhibition stand light and shadow dynamic adjustment (including illumination intensity, color, Gaussian blur and the like) based on a threshold value, and performs real-time display on the virtual exhibition stand. And multi-user collaborative interaction is realized through federal learning. According to the method, reinforcement learning is adopted to optimize an event-driven threshold value, and virtual and real visual splitting is eliminated in combination with ambient light adaptive mapping. The problems of low participation degree, insufficient single-mode interaction information and multi-user cooperation of traditional exhibition are solved, interest analysis accuracy is improved through multi-mode fusion, personalized experience is enhanced through dynamic interaction, the method is suitable for multiple scenes such as museums and science and technology museums, and exhibition intellectualization, immersion and group interaction efficiency are effectively improved.

The invention discloses a non-perpetual culture immersive interactive experience device and system based on a virtual digital human, and relates to the technical field of intangible cultural heritage intelligent interaction, and the device comprises a multi-mode sensing module which collects data such as user actions and voices through a depth camera and the like; the non-perpetual knowledge base module stores a non-perpetual knowledge graph and a case library; the digital human modeling engine fuses the inheritor characteristics and the user data to generate a virtual digital human; the immersion interaction engine constructs a cross-platform rendering environment to realize five-sense fusion experience; and the adaptive learning module analyzes the interaction sequence to optimize the digital human behavior, and all the modules cooperate to realize the intelligent interaction experience of non-genetic culture. Through the multi-modal perception module, the digital human modeling module and the like, the immersive interactive experience of the non-perpetual culture is realized, the user behavior can be accurately captured, the characteristic virtual digital human is generated, the five-sense fusion experience is provided, the content can be optimized based on the user interaction, the inheritance and propagation of the non-perpetual culture are promoted, and the participation degree and the sense of identity of the user are improved.

The invention discloses a real-time digital human generation system and method based on low-calculation-amount voice driving, and relates to the technical field of digital human interaction, and the system comprises an audio processing module which is configured to receive voice input in real time and extract an audio feature vector; the driving and rendering module is used for mapping the audio feature vectors into parameters representing mouth movement, generating a dynamic mouth image based on the preprocessed static face reference data and the parameters, and fusing the dynamic mouth image with the reference image; the synchronous control module is used for ensuring the synchronization of the audio features and the rendering video frames according to a timestamp alignment mechanism and a PID feedback control algorithm; and the dynamic scheduling module is used for monitoring a hardware resource load in real time and realizing dynamic allocation of computing resources through multi-thread parallel and task priority adjustment. According to the technical scheme provided by the invention, the breakthrough of low delay, high fidelity and low energy consumption of the digital human can be realized in mobile terminal, embedded and multi-platform scenes.

The invention provides a model-based interaction method and system, wearable equipment and a storage medium, and belongs to the technical field of intelligent interaction.The method comprises the steps that in response to a received interaction instruction, voice data, a gesture image, eye movement data and an environment image are obtained based on the interaction instruction; extracting user intention features based on the voice data, the gesture image and the eye movement data, and determining scene type features based on the environment image; determining an interaction theme based on the interaction instruction, obtaining user historical interaction information associated with the interaction theme from a context memory database, and generating a context feature vector based on the user historical interaction information; and inputting the user intention feature, the scene type feature and the context feature vector into an intention recognition model based on quantum enhancement to obtain a user intention, and generating interaction response data based on the user intention. According to the invention, the accuracy of user intention recognition can be improved, and the intelligence of interaction is improved.

The invention relates to the technical field of intelligent man-machine interaction, and discloses a customized virtual pet interactive simulation method, which comprises the following steps: S1, perceiving the emotional state of a user, S2, analyzing the behavior and emotion of the user, S3, establishing a virtual pet behavior model, and S4, establishing a virtual pet interaction model. The invention provides a customized virtual pet interactive simulation system, which comprises a sensing module, a decision-making module, an execution module and a feedback module. The method comprises the following steps: S1, selecting a virtual pet behavior, S2, selecting an optimal behavior, S5, carrying out deep reinforcement learning optimization, S6, executing the virtual pet behavior, and S7, feeding back and adjusting a behavior strategy. The invention also provides the customized virtual pet interactive simulation system, which comprises a sensing module, a decision-making module, an execution module and a feedback module. Through long-term emotional state tracking, personalized behavior pattern recognition and long-term reward correction of reinforcement learning, the virtual pet can continuously optimize a behavior strategy, dynamically adapt to long-term emotional changes of a user, and improve the personalized matching degree and long-term stability of interaction.

The embodiment of the invention provides a digital human interaction system and method based on multi-modal emotion recognition, and belongs to the technical field of digital human interaction. The system comprises a multi-modal sensing module used for collecting multi-modal data and preprocessing the multi-modal data to generate a standardized data stream; the cross-modal fusion and emotion recognition module is used for carrying out interactive modeling on the multi-modal features and outputting a current emotion label and emotion intensity; the reaction planning module is used for generating a composite reaction strategy; and the digital human rendering module is used for mapping the composite reaction strategy into control signals corresponding to the voice, the facial expression and the action respectively, and driving a digital human to execute corresponding voice output, facial expression change and limb action through the control signals so as to realize interaction. According to the method, multi-modal data are deeply fused through the cross-modal graph neural network and comparative learning, the weight is dynamically adjusted in combination with the modal confidence, and the emotion recognition accuracy and robustness are improved.

The invention relates to the technical field of digital interaction, in particular to a digital interaction enhancement system based on multi-modal voice, which comprises a multi-modal sensing module comprising a voice sensing unit, a visual sensing unit and an environment sensing unit and used for capturing multi-modal data input by a user; the multi-modal fusion module comprises a data synchronization unit, a feature extraction unit and a multi-modal fusion unit and is used for performing time synchronization, feature extraction and fusion processing on the multi-modal data; and the interactive understanding module comprises a natural language processing unit, a context understanding unit and an emotion analysis unit and is used for understanding the intention and the emotion state of the user. The deep learning technology is adopted to fuse voice, vision and environment data, and efficient and natural user interaction experience is achieved through the steps of feature extraction, feature fusion, intention recognition and the like. The system has excellent performance and a wide application prospect in a complex environment.

The invention relates to the technical field of interaction based on visual earphones, and discloses an interaction method and device based on a visual earphone, computer equipment and a medium, and the method comprises the steps: judging whether a user wearing the visual earphone sends an interaction instruction or not; analyzing content in the interaction instruction to determine a user demand; in response to the user demand, controlling a camera to scan current content to obtain a scanned picture; performing data processing on the scanned picture based on the user demand to obtain a processing result; and feeding back the processing result to the user. The method has the beneficial effects that the user demand is determined through the interaction instruction, and intelligent and personalized services can be provided according to the user demand. Through the camera integrated with the earphone, a user can obtain environment information in real time without depending on external equipment, and obtain instant feedback through the earphone, so that the practicability and interaction experience of the intelligent wearable equipment in a complex environment are greatly improved.

The invention belongs to the technical field of three-dimensional scene interaction, and particularly relates to a three-dimensional immersive content interaction method and system based on 3D Gaussian spattering. The method comprises the following steps: collecting a multi-view image and carrying out motion structure recovery to obtain a camera pose matrix; according to the camera pose matrix, performing three-dimensional reconstruction processing on the image data through a 3D Gaussian spattering algorithm to obtain a reconstruction model; performing behavior analysis on the historical viewport track of the user to obtain a visual angle prediction result; and according to the visual angle prediction result and the reconstruction result of the reconstruction model, video frame generation is carried out at a client through a real-time rendering engine, and a transmission result is obtained. According to the invention, the application of the 3DGS in the three-dimensional expression technology is realized, the rendering quality is improved, and the degree of participation and immersion of the user are improved.

The invention belongs to the technical field of new energy automobile teaching, and particularly relates to a VR interactive new energy automobile high-voltage electrical principle teaching method, system and equipment, and the method comprises the steps: obtaining the physical parameter data and dynamic circuit simulation data of a new energy automobile high-voltage electrical system; constructing a high-precision 3D model according to the physical parameter data, inputting dynamic circuit simulation data into the model, and generating interactive principle annotations and particle effect animations; based on a gesture recognition model and a new energy automobile VR interaction technology, circuit state changes in the particle effect animation are obtained, a new energy automobile high-voltage system principle and a signal flow are displayed, a practical operation simulation training scene is constructed, student operation data are recorded according to the practical operation simulation training scene, an AI evaluation model is trained, and an evaluation report is obtained. And adjusting interaction feedback parameters of the training scene, and meanwhile, generating a personalized learning report of the trainee. Therefore, the problems of safety risk, teaching lag, abstraction obstacle and the like in the prior art are solved.

The invention relates to the technical field of intelligent voice interaction, in particular to an intelligent Bluetooth voice remote control system based on AI semantic analysis. Comprising a voice acquisition unit, a Bluetooth communication unit, a voice recognition unit, an AI semantic analysis unit and a control execution unit, and the Bluetooth communication unit is used for establishing low-power-consumption Bluetooth connection with target equipment and supporting bidirectional data transmission. By combining advanced localized model library, edge calculation optimization, multi-modal data fusion and dynamic semantic map technologies, the recognition accuracy, the real-time response speed and the context understanding ability of the voice instruction are remarkably improved, and meanwhile, the data privacy is guaranteed, so that efficient, personalized, natural and smooth user interaction experience is realized.

The invention relates to the technical field of display terminal interaction, in particular to an intelligent display terminal multi-mode interaction method and system. Comprising a multi-module acquisition unit, and the multi-module acquisition unit is used for acquiring visual, voice and text data; the modal feature extraction unit is used for extracting key semantic features from vision, voice and texts, the differential fusion unit dynamically allocates fusion coefficients based on weight coefficients of all modals, and an optimal fusion logic is selected for different input modal combinations by adopting a differential fusion strategy; and the multi-module response unit is used for outputting the multi-mode content and outputting and displaying the multi-mode content through the terminal equipment. The current available modal combination is identified through the differential fusion strategy, and the optimal fusion sub-module is scheduled, so that different fusion logics are dynamically called during operation, different fusion sub-modules are dynamically selected according to modal availability and switched during operation, and the problems of resource waste and precision reduction caused by general fusion are avoided.

The invention discloses a virtual reality interaction method and system applied to classical famous picture display, and particularly relates to the technical field of virtual reality interaction. Multi-dimensional high-definition image acquisition and composition element three-dimensional modeling are performed on a painting, a cultural context label system is constructed in combination with historical data, a fixation point, an action posture and a staying behavior of a user are perceived in real time, accurate matching of a user behavior and a semantic label is realized, and multi-modal interaction feedback content consistent with the style of the painting is generated, so that the user experience is improved. And typical paths and misunderstanding areas are identified through clustering analysis of multi-user behavior data, and context tags and response logic are dynamically optimized, so that personalized recommendation and semantic guidance are realized, the understanding depth and immersion experience of the users on classical art are effectively improved, and the cultural expression ability and intelligent adaptability of the system are enhanced.

The invention discloses an AI digital human interactive response method based on a large language model, and relates to the technical field of digital human interaction, and the method comprises the steps: analyzing collected user voice data and visual data through a natural language processing method, generating a cross-modal feature vector, carrying out the cross-modal association analysis of the cross-modal feature vector, and carrying out the cross-modal association analysis of the cross-modal feature vector. Generating a semantic association topological graph; calculating a vertex coordinate and a joint activity threshold value of the semantic association topological graph through high-digital human correlation, inputting the vertex coordinate and the joint activity threshold value into a constructed coordinate index database to execute attention weight calibration, and outputting a multi-dimensional association graph; and performing information density analysis based on the multi-dimensional association map, generating an information density gradient vector field, and dividing a high-density core region and a low-density edge region, the high-density core region generating a semantic core coding tensor, and the low-density edge region generating an edge feature package. According to the method, the cross-modal fusion vector is converted into the cross-modal feature vector, so that the modeling of the cross-modal association relationship is realized.

The invention provides a context-based intelligent voice semantic understanding analysis method and system, and relates to the technical field of voice interaction, and the method comprises the steps: obtaining an input voice signal, extracting an acoustic feature, and decoding the acoustic feature to obtain a candidate text; constructing context semantic representation to carry out disambiguation processing; establishing a semantic dependency graph and carrying out multi-level correlation analysis; spreading context constraint information to perform multi-hop reasoning; and finally generating intention recognition and slot filling results and updating the session state. The semantic comprehension accuracy and the intelligent degree of the voice interaction system are improved by introducing the context information and the multi-level semantic dependency relationship.

The invention relates to a digital human interaction method and system based on multi-modal understanding, and belongs to the technical field of digital human intelligent interaction. The method comprises the following steps: acquiring multi-modal data, and obtaining multi-modal features through a feature extraction model according to the multi-modal data; obtaining enhanced modal features through a knowledge enhancement model according to the multi-modal features; according to the enhanced modal features, a modal fusion feature vector is obtained through a graph representation learning model; and the big language model learns modal fusion feature vectors through a multi-head attention mechanism to obtain natural language output. The digital human understanding of multi-modal data is realized, and the smoothness, naturalness and intellectualization in the digital human interaction process are improved.

The invention relates to the technical field of live broadcast interaction, in particular to an intelligent live broadcast interaction method and system based on an AI virtual human. The method comprises the following steps: identifying a real-time live broadcast interaction data stream, carrying out intelligent bullet screen filtering processing and multi-modal user interaction perception, and constructing a multi-modal user interaction perception map; key bullet screen extraction is carried out based on the multi-modal interaction perception map, and user demand prediction is carried out, so that user interaction demand features are generated; performing deep semantic space mapping on the user interaction demand features, and then performing reverse live broadcast semantic blank analysis to obtain demand blank covering information; user behavior dynamic analysis is carried out based on the multi-mode user interaction perception map, real-time interaction emotion global evolution fitting is carried out, and a real-time interaction emotion map is constructed. Through deep understanding and analysis of user demands, the emotion matching degree and the interaction content satisfaction degree of user interaction are improved.

The invention relates to the technical field of digital human interaction, and particularly discloses a multi-modal-based digital human interaction method and system, which is used for processing interactive voice information and interactive video stream information by adopting a text processing technology and a video analysis algorithm based on a deep learning model. And automatically judging whether the current interaction of the target user is finished or not according to the interaction response representation between the feature obtained by performing time sequence propagation reasoning on the local semantic association feature of each interaction action and the word granularity semantic association feature of each interaction voice text. Therefore, mutual influence and dependency relationship among different modal features can be caught more finely, so that information of different modals can be fused, dynamic change and deep association among the modals can be caught, more accurate feature fusion is realized, and the current interaction state of a user is judged more accurately.

The invention discloses an intelligent voice interaction method and device based on a voice large model, a terminal and a medium, and belongs to the technical field of voice recognition and interaction, and the method comprises the steps: recognizing the voiceprint information of a speaker through a voiceprint recognition model when a wake-up instruction is received; when the identity of the current pronunciator is the specified user registered through voiceprint, acquiring question information of the current pronunciator, calling a prompt word of a character event text of the specified user corresponding to the current pronunciator from a cloud end, and combining with a question proposed by the current pronunciator to obtain the question information of the current pronunciator; generating event text information related to a specified user character corresponding to the current pronunciator; and calling the speech synthesis model, synthesizing the generated event text information into speech according to the tone characteristics of the speaker, and outputting and playing the speech. According to the method, the user is supported to upload the character as the cue word, and the cue word is combined with the large model, so that the dialogue content generation based on the personalized information of the user is realized, and convenience is provided for the use of the user.

The invention relates to the technical field of artificial intelligence and multimedia interaction, and discloses an AI music rhythm-based dance movement real-time generation system, which comprises a music rhythm extraction module used for extracting rhythm time and intensity characteristics of input music and generating music rhythm distribution; the dance movement generation module generates initial dance movement distribution according to the extracted music rhythm distribution and is used for generating the initial dance movement distribution based on a pre-trained deep learning model; and the dynamic synchronization module is used for calculating a mapping relation between music rhythm distribution and dance movement distribution through a dynamic optimal transmission method. According to the technical scheme, dynamic optimal transmission and variational optimization are combined, the technical effect of high-precision alignment of actions and music rhythms is achieved by optimizing the synchronization relation between music rhythm distribution and dance action distribution, the problem of mismatching of actions and music in a complex music rhythm change scene is solved, and the accuracy of the movement and the music is improved. And the dancing action is more natural and smoother.

The invention provides an intelligent collaborative recommendation method and device oriented to multi-modal immersive content, and relates to the technical field of immersive interaction.The method comprises the steps that multi-modal data are collected, and the attention weight of interaction between the multi-modal data and other modalities is calculated for each modality; and obtaining a corresponding fusion feature through the calculated attention weight and the representation weight after self-modal internal attention processing, and determining a recommended action by using a predefined multi-agent according to the obtained fusion feature. According to the method and the device, the problem that the recommended immersive content cannot meet the actual demand of the user due to the fact that multiple modal data for understanding the user state in the virtual environment and the interaction among the modal data are not comprehensively considered in the prior art is solved.

The invention relates to the technical field of information interaction, in particular to a retail customer service response method and a retail customer service system. The response method comprises the following steps: based on retail data, constructing a retail large language model by using a Transform model architecture, and carrying out training and fine tuning, so that the retail large language model has a question and answer ability in the retail field; performing feature extraction based on the multi-modal data input by the user to generate question and answer text data; based on the question and answer text data, topK information with most matched semantics is retrieved from a preset vector knowledge base; and based on the text data and the topK information, calling the retail large language model to respond. According to the response method of the retail customer service, questions of the user can be deeply understood, intelligent response is carried out, and the shopping experience of the user is improved.