Question and answer method, system and device for cross-border trade and storage medium

Abstract

The invention discloses a cross-border trade question-answering method, system and device and a storage medium, and the cross-border trade question-answering method comprises the steps: determining a first feature based on text data; determining a second feature based on the image data; determining a third feature based on the voice data; determining a fourth feature based on the video data; based on the first feature, the second feature, the third feature and the fourth feature, performing feature fusion through a graph attention mechanism model to obtain a fused intention feature; inputting the fused intention features into a trained multi-label ViT-CNN hybrid model to obtain an intention classification result output by the trained multi-label ViT-CNN hybrid model; and on the basis of the intention classification result and the preset resource database, reply data of the target question is determined through a near-end strategy optimization algorithm, and the reply data is sent to the user, so that the question efficiency and accuracy of cross-border trade are improved.

Description

Technical Field

[0001] This application relates to the technical field of question-and-answer in cross-border trade, and in particular to a question-and-answer method, system, device and storage medium for cross-border trade. Background Technology

[0002] Driven by the deepening of economic globalization and the booming development of digital trade, the scale of cross-border trade continues to expand, and trade entities have increasingly stringent demands for the accuracy and efficiency of services. However, traditional cross-border trade service platforms are limited by inherent defects in their technical architecture and algorithm systems, failing to deeply explore the intrinsic relationships between different modalities of data. This results in low data processing efficiency and significant information loss, leading to low efficiency and accuracy in question answering. Summary of the Invention

[0003] This application aims to at least address the technical problems existing in the prior art. To this end, this application proposes a question-answering method, system, device, and storage medium for cross-border trade, which can realize deep intent parsing of multimodal inputs and improve the efficiency and accuracy of question answering in cross-border trade.

[0004] The first aspect of this application provides a question-and-answer method for cross-border trade, comprising the following steps: Acquire text, image, audio, and video data related to the target problem; Based on the text data, a first feature is determined; based on the image data, a second feature is determined; based on the voice data, a third feature is determined; and based on the video data, a fourth feature is determined. Based on the first feature, the second feature, the third feature, and the fourth feature, feature fusion is performed using a graph attention mechanism model to obtain the fused intent feature; The fused intent features are input into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; Based on the intent classification results and the preset resource database, the response data for the target question is determined by the near-end strategy optimization algorithm, and the response data is sent to the user.

[0005] The question-and-answer method for cross-border trade according to the embodiments of this application has at least the following beneficial effects: This method first acquires text, image, audio, and video data related to the target question. Based on the text data, a first feature is determined; based on the image data, a second feature is determined; based on the audio data, a third feature is determined; and based on the video data, a fourth feature is determined. Based on the first, second, third, and fourth features, a graph attention mechanism model is used to fuse these features, resulting in a fused intent feature. This multimodal data fusion provides more accurate data for subsequent steps. The fused intent feature is then input into a trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the model. This achieves deep intent parsing from multimodal inputs. Finally, based on the intent classification result and a pre-set resource database, a proximal policy optimization algorithm is used to determine the response data for the target question, which is then sent to the user, improving the efficiency and accuracy of cross-border trade inquiries.

[0006] According to some embodiments of this application, determining a first feature based on the text data; determining a second feature based on the image data; determining a third feature based on the voice data; and determining a fourth feature based on the video data includes: The text data is cleaned to obtain cleaned text data; The cleaned text data is input into the trained BERT-large model to obtain the first feature output by the trained BERT-large model; The image data is enhanced to obtain enhanced image data; The image-enhanced data is input into the trained ResNet-50 model to obtain the second feature output by the trained ResNet-50 model; The voice data is subjected to noise reduction processing to obtain noise-reduced data; The third feature is obtained by extracting features from the denoised data using a Mel frequency cepstral coefficient + bidirectional long short-term memory network model. Based on the video data, the fourth feature is determined by a network model based on an extended 3D convolutional network and a temporal self-attention mechanism network model.

[0007] According to some embodiments of this application, determining the fourth feature based on the video data using a dilated 3D convolutional network model and a temporal self-attention mechanism network model includes: Obtain the total duration of the video data; Based on the total duration of the video data, the initial sampling interval is determined using the following formula:

[0008] in, The initial sampling interval, The total duration of the video data; The video data is sampled according to the initial sampling interval to obtain sampled video data; Image features of the sampled video data are extracted using a third-generation lightweight convolutional neural network model, and cosine similarity between all adjacent frames is calculated based on the image features of each frame. The motion vector field between all adjacent frames in the sampled video data is calculated using the Falnebeck optical flow algorithm. The short-term temporal features of the sampled video data are extracted using the dilated 3D convolutional network model, and the temporal feature sequence of the short-term temporal features is obtained through a 16-frame sliding window. Calculate the temporal consistency score among all adjacent frames in the temporal feature sequence; Based on the cosine similarity, the motion vector field, and the temporal consistency score, video keyframes are selected from the sampled video data; The first video feature of the video keyframe is extracted by the dilated 3D convolutional network model; and the first video feature is input into the temporal self-attention mechanism network model to obtain the fourth feature output by the temporal self-attention mechanism network model.

[0009] According to some embodiments of this application, the step of performing feature fusion based on the first feature, the second feature, the third feature, and the fourth feature using a graph attention mechanism model to obtain fused intent features includes: Determine the first weight value of the first feature, the second weight value of the second feature, the third weight value of the third feature, and the fourth weight value of the fourth feature; Based on the first weight value, the second weight value, the third weight value, and the fourth weight value, the first feature, the second feature, the third feature, and the fourth feature are weighted and concatenated to obtain the fused intention feature.

[0010] According to some embodiments of this application, determining the first weight value of the first feature, the second weight value of the second feature, the third weight value of the third feature, and the fourth weight value of the fourth feature includes: Construct a historical training dataset, wherein the historical training dataset includes manually labeled historical intent labels for each historical training data, modality quality scores for each modality type of each historical training data, historical text data, historical image data, historical speech data and / or video data, wherein the modality types include text modality, image modality, speech modality and video modality, and the modality quality score is a constant value characterizing the data quality of each modality type of each historical training data; An initial graph attention mechanism network is constructed, and the initial graph attention mechanism network is trained using the historical training dataset to obtain a trained graph attention mechanism network and the first weight value, the second weight value, the third weight value, and the fourth weight value output by the trained graph attention mechanism network.

[0011] According to some embodiments of this application, the preset resource database includes an image recognition library, a policy knowledge base, video analysis tools, and a financial institution model. The step of determining the response data for the target question based on the intent classification results and the preset resource database using a near-end policy optimization algorithm includes: Obtain the resource load and historical matching success rate of the preset resource database; Construct a reward function, wherein the reward function includes resource matching accuracy, resource database response speed and modality fit, wherein the resource matching accuracy is the confidence value of historical log records, and the modality fit is obtained by matching modality type with a preset matching table; Using the intent classification result, resource load, and historical matching success rate as input data, and based on the reward function, the pre-defined resource database is invoked through a near-end policy optimization algorithm to obtain the response data for the target question.

[0012] According to some embodiments of this application, the response data includes text modal response data, voice modal response data, and video modal response data, and sending the response data to the user includes: Obtain the user type of the user, wherein the user type includes new users and old users; If the user type is a returning user, the text modal response data will be sent to the user. If the user type is a new user, the text modal response data, the voice modal response data, and the video modal response data are sent to the user.

[0013] A second aspect of this application provides a question-and-answer system for cross-border trade, the system comprising: The data acquisition module is used to acquire text data, image data, audio data, and video data related to the target problem. The feature determination module is used to determine a first feature based on the text data; a second feature based on the image data; a third feature based on the voice data; and a fourth feature based on the video data. The feature fusion module is used to perform feature fusion based on the first feature, the second feature, the third feature, and the fourth feature through a graph attention mechanism model to obtain the fused intent feature; The model output module is used to input the fused intent features into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; The response data determination module is used to determine the response data for the target question based on the intent classification results and a preset resource database using a near-end strategy optimization algorithm, and then send the response data to the user.

[0014] This system first acquires text, image, audio, and video data related to the target question. Based on the text data, it determines the first feature; based on the image data, it determines the second feature; based on the audio data, it determines the third feature; and based on the video data, it determines the fourth feature. Using a graph attention mechanism model, it fuses these features to obtain the fused intent features. This multimodal data fusion provides more accurate data for subsequent steps. The fused intent features are then input into a trained multi-label ViT-CNN hybrid model to obtain the intent classification results output by the model. This achieves deep intent parsing from multimodal inputs. Finally, based on the intent classification results and a pre-set resource database, a proximal policy optimization algorithm is used to determine the response data for the target question, which is then sent to the user, improving the efficiency and accuracy of cross-border trade inquiries.

[0015] A third aspect of this application provides a cross-border trade question-and-answer electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enables the at least one control processor to perform the aforementioned cross-border trade question-and-answer method.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the aforementioned cross-border trade question-and-answer method.

[0017] It should be noted that the beneficial effects of the second to fourth aspects of this application with respect to the prior art are the same as the beneficial effects of the question-and-answer system for cross-border trade with respect to the prior art described above, and will not be elaborated here.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating an embodiment of the question-and-answer method for cross-border trade provided in this application; Figure 2 This is a schematic diagram of the structure of an embodiment of the cross-border trade question-and-answer system provided in this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0020] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0021] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0022] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0023] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0024] Driven by the deepening of economic globalization and the booming development of digital trade, the scale of cross-border trade continues to expand, and trade entities have increasingly stringent demands for the accuracy and efficiency of services. However, traditional cross-border trade service platforms are limited by inherent defects in their technical architecture and algorithm systems, failing to deeply explore the intrinsic relationships between different modalities of data. This results in low data processing efficiency and significant information loss, leading to low efficiency and accuracy in question answering.

[0025] To address the aforementioned technical deficiencies, embodiments of this application provide a question-and-answer method, system, device, and storage medium for cross-border trade.

[0026] Please see Figure 1 This is a flowchart illustrating a question-and-answer method for cross-border trade provided in an embodiment of this application. The method is applied to electronic devices, such as servers. Figure 1 As shown, the question-and-answer method for cross-border trade includes: Step S101: Obtain text data, image data, audio data, and video data of the target problem; In step S101, the text data, image data, voice data, and video data of the target question can be the text data, image data, voice data, and video data of the target question uploaded by the user.

[0027] Step S102: Based on text data, determine the first feature; based on image data, determine the second feature; based on voice data, determine the third feature; based on video data, determine the fourth feature. Step S103: Based on the first feature, the second feature, the third feature, and the fourth feature, feature fusion is performed through the graph attention mechanism model to obtain the fused intent feature; The graph attention network (GAT) model described above is a deep learning model based on graph neural networks (GNNs) and attention mechanisms. Its core is to achieve differentiated modeling of the importance of nodes by assigning adaptive weights to the neighboring nodes of nodes in the graph.

[0028] Step S104: Input the fused intent features into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; The trained multi-label ViT-CNN hybrid model is a deep learning model that combines the global feature modeling capability of the Visual Transformer (ViT) with the local texture capture advantage of the Convolutional Neural Network (CNN), and is used for multi-label image classification tasks (i.e., a single image corresponds to multiple labels).

[0029] The above-mentioned intent classification results can be for document verification, tariff classification, policy inquiry, exhibition matchmaking, credit assessment, logistics tracking, customs declaration and / or compliance consultation.

[0030] Step S105: Based on the intent classification results and the preset resource database, determine the response data for the target question through the near-end strategy optimization algorithm, and send the response data to the user.

[0031] The aforementioned pre-defined resource database may include image recognition libraries, policy knowledge bases, video analytics tools, and financial institution models.

[0032] The aforementioned response data may include text modal response data, voice modal response data, and video modal response data.

[0033] This method first acquires text, image, audio, and video data related to the target question. Based on the text data, a first feature is determined; based on the image data, a second feature is determined; based on the audio data, a third feature is determined; and based on the video data, a fourth feature is determined. Based on the first, second, third, and fourth features, a graph attention mechanism model is used to fuse these features, resulting in a fused intent feature. This multimodal data fusion provides more accurate data for subsequent steps. The fused intent feature is then input into a trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the model. This achieves deep intent parsing from multimodal inputs. Finally, based on the intent classification result and a pre-set resource database, a proximal policy optimization algorithm is used to determine the response data for the target question, which is then sent to the user, improving the efficiency and accuracy of cross-border trade inquiries.

[0034] In some embodiments, step S102 may include steps S201 to S207: Step S201: Perform text cleaning on the text data to obtain cleaned text data; Step S202: Input the cleaned text data into the trained BERT-large model to obtain the first feature output by the trained BERT-large model; The trained BERT-large model described above is a large version of the BERT model.

[0035] Step S203: Perform image enhancement on the image data to obtain enhanced image data; Step S204: Input the image-enhanced data into the trained ResNet-50 model to obtain the second feature output by the trained ResNet-50 model; The trained ResNet-50 model is a classic deep convolutional neural network (50-layer residual structure).

[0036] Step S205: Perform noise reduction processing on the speech data to obtain noise-reduced data; Step S206: Extract features from the denoised data using the Mel frequency cepstral coefficients + bidirectional long short-term memory network model to obtain the third feature; In step S206, the above-mentioned feature extraction of the noise-reduced data using Mel frequency cepstral coefficients + bidirectional long short-term memory network model yields the third feature. This third feature can be obtained by extracting features of the noise-reduced data using Mel frequency cepstral coefficients (window length 25ms, frame shift 10ms, 40 Mel filter banks) to obtain acoustic features, and then extracting the temporal dependency features of the noise-reduced data using a 3-layer bidirectional long short-term memory network model (128 hidden units per layer). The acoustic features and temporal dependency features are then concatenated to obtain the third feature.

[0037] Step S207: Based on video data, determine the fourth feature by using a network model based on an extended 3D convolutional network and a temporal self-attention mechanism network model.

[0038] This application provides a more accurate data basis for subsequent intent classification by extracting feature data from four different modalities.

[0039] In some embodiments, step S207 may include steps S301 to S309: Step S301: Obtain the total duration of the video data; Step S302: Based on the total duration of the video data, determine the initial sampling interval using the following formula:

[0040] in, The initial sampling interval, The total duration of the video data; Step S303: Sample the video data according to the initial sampling interval to obtain the sampled video data; Step S304: Extract image features from the sampled video data using the third-generation lightweight convolutional neural network model (MobileNetV3), and calculate the cosine similarity between all adjacent frames based on the image features of each frame. Step S305: Calculate the motion vector field between all adjacent frames in the sampled video data using the Falnebeck optical flow algorithm; Step S306: Extract short-term temporal features of the sampled video data by using a dilated 3D CNN model, and obtain the temporal feature sequence of the short-term temporal features through a 16-frame sliding window; Step S307: Calculate the temporal consistency score between all adjacent frames in the temporal feature sequence; Step S308: Based on cosine similarity, motion vector field and temporal consistency score, select video keyframes from the sampled video data; The aforementioned video keyframes can be several video frames from the sampled video data.

[0041] In step S308, the above-mentioned selection of video keyframes from the sampled video data based on cosine similarity, motion vector field, and temporal consistency score can be achieved by statistically analyzing the proportion of regions in each video frame of the sampled video data whose motion field intensity is greater than a second preset threshold. All video frames with a cosine similarity less than a first preset threshold and a region proportion greater than a third preset threshold are selected from the sampled video data as the first video keyframe. All video frames with a temporal consistency score greater than a fourth preset threshold for three consecutive video frames are selected from the sampled video data as the second video keyframe. The first and second video keyframes are combined to obtain the aforementioned video keyframes. The first preset threshold is a value pre-set according to actual needs, which can be 0.75; the second preset threshold is a value pre-set according to actual needs, which can be 10 pixels per frame; the third preset threshold is a value pre-set according to actual needs, which can be 15%; and the fourth preset threshold is a value pre-set according to actual needs, which can be 15%.

[0042] Step S309: Extract the first video features (including local spatiotemporal features) of the video keyframes by dilating the three-dimensional convolutional network model; and input the first video features into the temporal self-attention mechanism network model to obtain the fourth feature output by the temporal self-attention mechanism network model.

[0043] In step S309, the fourth feature obtained by the temporal self-attention mechanism network model can be obtained by extracting global temporal features through the temporal self-attention mechanism network model, fusing the first video features and the global temporal features.

[0044] This application determines the fourth feature by using an extended 3D convolutional network model and a temporal self-attention mechanism network model, which can achieve complementary enhancement of "local spatiotemporal details and global temporal dependencies", thereby improving the expressiveness, robustness and interpretability of the fourth feature.

[0045] In some embodiments, step S103 may include steps S401 to S402: Step S401: Determine the first weight value of the first feature, the second weight value of the second feature, the third weight value of the third feature, and the fourth weight value of the fourth feature; The sum of the first, second, third, and fourth weights mentioned above is 1.

[0046] Step S402: Based on the first weight value, the second weight value, the third weight value, and the fourth weight value, the first feature, the second feature, the third feature, and the fourth feature are weighted and concatenated to obtain the fused intent feature.

[0047] In step S402, the first feature, second feature, third feature, and fourth feature are weighted and concatenated based on the first weight value, second weight value, third weight value, and fourth weight value to obtain the expression of the fused intent feature, which can be: ; in, For the fused intent features, As the first feature, As the first weight, As the second feature, As the second weight, As the third characteristic, As the third weight, The fourth characteristic, As the fourth weight, Feature splicing.

[0048] This application obtains a fused intent feature by weighting and concatenating the first, second, third, and fourth features based on a first weight value, a second weight value, a third weight value, and a fourth weight value. This achieves complementary advantages of multi-dimensional features and improves the accuracy, robustness, and generalization ability of intent recognition.

[0049] In some embodiments, step S401 may include steps S501 to S502: Step S501: Construct a historical training dataset, wherein the historical training dataset includes manually annotated historical intent labels for each historical training data, modality quality scores for each modality type of each historical training data, historical text data, historical image data, historical speech data and / or video data, and modality types include text modality, image modality, speech modality and video modality, and modality quality scores are constant values ​​that characterize the data quality of each modality type of each historical training data; Step S502: Construct an initial graph attention mechanism network. Train the initial graph attention mechanism network using the historical training dataset to obtain the trained graph attention mechanism network and the first, second, third, and fourth weight values ​​output by the trained graph attention mechanism network.

[0050] This application utilizes a graph attention mechanism network to learn the first, second, third, and fourth weight values. This retains the core advantage of the graph attention mechanism network in "adaptive modeling of neighbor importance" and achieves accurate weighting of multiple features / nodes through interpretable weight outputs, thereby improving the accuracy of the first, second, third, and fourth weight values.

[0051] In some embodiments, step S105 may include steps S601 to S603: Step S601: Obtain the resource load and historical matching success rate of the preset resource database; In step S601, the resource load for obtaining the preset resource database can be the resource load of the API interface for obtaining the preset resource database and the historical matching success rate. The historical matching success rate can be determined by the user satisfaction rate collected from historical user feedback.

[0052] Step S602: Construct a reward function, wherein the reward function includes resource matching accuracy, resource database response speed and modality fit, the resource matching accuracy is the confidence value of historical log records, and the modality fit is obtained by matching modality type with a preset matching table; The response speed of the aforementioned resource database can be determined by querying historical log records.

[0053] Step S603: Using intent classification results, resource load, and historical matching success rate as input data, and based on the reward function, the pre-defined resource database is invoked through the near-end policy optimization algorithm to obtain the response data for the target question.

[0054] In step S601, the above-mentioned reinforcement learning framework, which takes the intent classification result, resource load, and historical matching success rate as input data, and uses the reward function to call the preset resource database to obtain the response data of the target question, can be used as the reinforcement learning framework. The framework takes the intent classification result, resource load, and historical matching success rate as state input, and uses the response data of the target question obtained by calling the preset resource database as action output. The framework maximizes the reward function value through the Proximal Policy Optimization (PPO) algorithm to obtain the optimal response data, and uses the optimal response data as the response data of the target question.

[0055] This application uses intent classification results, resource load, and historical matching success rate as input data. Based on a reward function, it calls a preset resource database through a near-end policy optimization algorithm to obtain response data for the target question. This maximizes the matching accuracy between the response data and the user's intent, reduces invalid interactions, and improves the user service experience.

[0056] In some embodiments, step S105 may include steps S701 to S702: Step S701: Obtain the user type, where the user type includes new users and old users; Step S702: If the user type is an existing user, send the text modal response data to the user; Step S703: If the user type is a new user, send text modal response data, voice modal response data, and video modal response data to the user.

[0057] This application improves the user experience by sending different response data according to user type, providing efficient responses to existing users and detailed responses to new users.

[0058] Additionally, refer to Figure 2 One embodiment of this application provides a question-and-answer system for cross-border trade, including a data acquisition module 1100, a feature determination module 1200, a feature fusion module 1300, a model output module 1400, and a response data determination module 1500, wherein: The data acquisition module 1100 is used to acquire text data, image data, audio data, and video data of the target problem; The feature determination module 1200 is used to determine a first feature based on text data; a second feature based on image data; a third feature based on voice data; and a fourth feature based on video data. The feature fusion module 1300 is used to perform feature fusion based on the first feature, the second feature, the third feature, and the fourth feature through a graph attention mechanism model to obtain the fused intent feature; The model output module 1400 is used to input the fused intent features into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; The response data determination module 1500 is used to determine the response data for the target question based on the intent classification results and a preset resource database, using a near-end strategy optimization algorithm, and then send the response data to the user.

[0059] This system first acquires text, image, audio, and video data related to the target question. Based on the text data, it determines the first feature; based on the image data, it determines the second feature; based on the audio data, it determines the third feature; and based on the video data, it determines the fourth feature. Using a graph attention mechanism model, it fuses these features to obtain the fused intent features. This multimodal data fusion provides more accurate data for subsequent steps. The fused intent features are then input into a trained multi-label ViT-CNN hybrid model to obtain the intent classification results output by the model. This achieves deep intent parsing from multimodal inputs. Finally, based on the intent classification results and a pre-set resource database, a proximal policy optimization algorithm is used to determine the response data for the target question, which is then sent to the user, improving the efficiency and accuracy of cross-border trade inquiries.

[0060] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations. The acquisition, storage, use and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0061] It should be noted that the system embodiments described above are based on the same inventive concept as the method embodiments described above. Therefore, the relevant content of the method embodiments described above is also applicable to the system embodiments described above, and will not be repeated here.

[0062] Figure 3 A schematic diagram of the hardware structure for cross-border trade question answering provided in an embodiment of this application is shown.

[0063] Question answering devices used in cross-border trade may include a processor 301 and a memory 302 storing computer program instructions.

[0064] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0065] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0066] In some embodiments, memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0067] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the question-and-answer methods for cross-border trade in the above embodiments.

[0068] In one example, the question-and-answer device for cross-border trade may also include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0069] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0070] Bus 310 includes hardware, software, or both, that couples components of a cross-border communication device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0071] This cross-border trade question-and-answer device can execute the cross-border trade question-and-answer method in this application embodiment based on a 3D design model, thereby achieving a combination of... Figure 1 and Figure 2 The description of the question-and-answer methods and systems for cross-border trade.

[0072] Furthermore, in conjunction with the question-and-answer methods for cross-border trade in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the question-and-answer methods for cross-border trade in the above embodiments.

[0073] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0074] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0075] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0076] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0077] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A question-and-answer method for cross-border trade, characterized in that, The question-and-answer methods for cross-border trade include: Acquire text, image, audio, and video data related to the target problem; Based on the text data, a first feature is determined; based on the image data, a second feature is determined; based on the voice data, a third feature is determined; and based on the video data, a fourth feature is determined. Based on the first feature, the second feature, the third feature, and the fourth feature, feature fusion is performed using a graph attention mechanism model to obtain the fused intent feature; The fused intent features are input into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; Based on the intent classification results and the preset resource database, the response data for the target question is determined by the near-end strategy optimization algorithm, and the response data is sent to the user.

2. The question-and-answer method for cross-border trade according to claim 1, characterized in that, The first feature is determined based on the text data; Based on the image data, a second feature is determined; Based on the aforementioned voice data, a third feature is determined; Based on the video data, the fourth feature is determined, including: The text data is cleaned to obtain cleaned text data; The cleaned text data is input into the trained BERT-large model to obtain the first feature output by the trained BERT-large model; The image data is enhanced to obtain enhanced image data; The image-enhanced data is input into the trained ResNet-50 model to obtain the second feature output by the trained ResNet-50 model; The voice data is subjected to noise reduction processing to obtain noise-reduced data; The third feature is obtained by extracting features from the denoised data using a Mel frequency cepstral coefficient + bidirectional long short-term memory network model. Based on the video data, the fourth feature is determined by a network model based on an extended 3D convolutional network and a temporal self-attention mechanism network model.

3. The question-and-answer method for cross-border trade according to claim 2, characterized in that, The determination of the fourth feature based on the video data, using a network model based on a dilated 3D convolutional network and a temporal self-attention mechanism, includes: Obtain the total duration of the video data; Based on the total duration of the video data, the initial sampling interval is determined using the following formula: in, The initial sampling interval, The total duration of the video data; The video data is sampled according to the initial sampling interval to obtain sampled video data; Image features of the sampled video data are extracted using a third-generation lightweight convolutional neural network model, and cosine similarity between all adjacent frames is calculated based on the image features of each frame. The motion vector field between all adjacent frames in the sampled video data is calculated using the Falnebeck optical flow algorithm. The short-term temporal features of the sampled video data are extracted using the dilated 3D convolutional network model, and the temporal feature sequence of the short-term temporal features is obtained through a 16-frame sliding window. Calculate the temporal consistency score among all adjacent frames in the temporal feature sequence; Based on the cosine similarity, the motion vector field, and the temporal consistency score, video keyframes are selected from the sampled video data; The first video feature of the video keyframe is extracted by the dilated 3D convolutional network model; and the first video feature is input into the temporal self-attention mechanism network model to obtain the fourth feature output by the temporal self-attention mechanism network model.

4. The question-and-answer method for cross-border trade according to claim 3, characterized in that, The feature fusion based on the first feature, the second feature, the third feature, and the fourth feature, using a graph attention mechanism model, yields the fused intent feature, including: Determine the first weight value of the first feature, the second weight value of the second feature, the third weight value of the third feature, and the fourth weight value of the fourth feature; Based on the first weight value, the second weight value, the third weight value, and the fourth weight value, the first feature, the second feature, the third feature, and the fourth feature are weighted and concatenated to obtain the fused intention feature.

5. The question-and-answer method for cross-border trade according to claim 4, characterized in that, Determining the first weight value of the first feature, the second weight value of the second feature, the third weight value of the third feature, and the fourth weight value of the fourth feature includes: Construct a historical training dataset, wherein the historical training dataset includes manually labeled historical intent labels for each historical training data, modality quality scores for each modality type of each historical training data, historical text data, historical image data, historical speech data and / or video data, wherein the modality types include text modality, image modality, speech modality and video modality, and the modality quality score is a constant value characterizing the data quality of each modality type of each historical training data; An initial graph attention mechanism network is constructed, and the initial graph attention mechanism network is trained using the historical training dataset to obtain a trained graph attention mechanism network and the first weight value, the second weight value, the third weight value, and the fourth weight value output by the trained graph attention mechanism network.

6. The question-and-answer method for cross-border trade according to claim 4, characterized in that, The preset resource database includes an image recognition library, a policy knowledge base, video analysis tools, and financial institution models. The step of determining the response data for the target question based on the intent classification results and the preset resource database using a near-end policy optimization algorithm includes: Obtain the resource load and historical matching success rate of the preset resource database; Construct a reward function, wherein the reward function includes resource matching accuracy, resource database response speed and modality fit, wherein the resource matching accuracy is the confidence value of historical log records, and the modality fit is obtained by matching modality type with a preset matching table; Using the intent classification result, resource load, and historical matching success rate as input data, and based on the reward function, the pre-defined resource database is invoked through a near-end policy optimization algorithm to obtain the response data for the target question.

7. The question-and-answer method for cross-border trade according to claim 4, characterized in that, The response data includes text modal response data, voice modal response data, and video modal response data. Sending the response data to the user includes: Obtain the user type of the user, wherein the user type includes new users and old users; If the user type is a returning user, the text modal response data will be sent to the user. If the user type is a new user, the text modal response data, the voice modal response data, and the video modal response data are sent to the user.

8. A question-and-answer system for cross-border trade, characterized in that, The cross-border trade question and answer system includes: The data acquisition module is used to acquire text data, image data, audio data, and video data related to the target problem. The feature determination module is used to determine a first feature based on the text data; a second feature based on the image data; a third feature based on the voice data; and a fourth feature based on the video data. The feature fusion module is used to perform feature fusion based on the first feature, the second feature, the third feature, and the fourth feature through a graph attention mechanism model to obtain the fused intent feature; The model output module is used to input the fused intent features into the trained multi-label ViT-CNN hybrid model to obtain the intent classification result output by the trained multi-label ViT-CNN hybrid model; The response data determination module is used to determine the response data for the target question based on the intent classification results and a preset resource database using a near-end strategy optimization algorithm, and then send the response data to the user.

9. A question-and-answer device for cross-border trade, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform a cross-border trade question-and-answer method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a question-and-answer method for cross-border trade as described in any one of claims 1 to 7.

Images