Interactive information acquisition method, device, computer readable medium and electronic device
By performing multimodal sequence information processing on the interaction information of the target object, the correlation between the interaction type, time and amount sequence is extracted, and the problem of low accuracy of interaction information recognition in the prior art is solved, and more efficient interaction is achieved Information acquisition.
Patent Information
- Application Number
- CN202210014072.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-06
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-01-06
AI Technical Summary
The prior art is difficult to effectively identify the interaction sequence of the target object and its contextual correlation of the interactive operation object sequence, resulting in low accuracy of recognition of interaction information.
By obtaining the multimodal sequence information of the target object, including interaction type sequence, interaction time series and interaction amount sequence, multi-scale convolution and feature interaction processing are used to extract the correlation between sequence information to determine the interaction information.
It improves the accuracy of interactive information recognition and the universality of the model, can effectively learn the contextual correlation of interactive sequences and their operating object sequences, and enhances data processing efficiency.
Smart Images

Figure CN114549041B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to an interactive information acquisition method, an interactive information acquisition device, a computer-readable medium, and an electronic device. Background Art
[0002] With the development of computer technology, more and more interactive services can be realized through the Internet, such as online shopping, online payment, online transfer, etc. Although it can bring great convenience to people's lives, there are also corresponding disadvantages.
[0003] Currently, two methods are commonly used to control abnormal interactions. One is to monitor the interaction of the target object by monitoring the interaction amount and interaction frequency of the target object's account. However, this method has the problem of single judgment conditions and difficulty in effectively identifying abnormal interactions. The other is to model the interaction sequence of the target object, and use the established model to determine whether there are abnormalities in the subsequent interaction sequence of the target object. However, in the modeling process, the target object's interests implied behind the interaction are modeled, and the historical interests of the target object are understood and future interests are deduced, rather than modeling the contextual relevance of the interaction sequence and its interaction operation object sequence. Therefore, this method has the problem of missed judgments and false judgments. Summary of the invention
[0004] The purpose of this application is to provide an interactive information acquisition method, an interactive information acquisition device, a computer-readable medium and an electronic device, which can overcome the technical problems existing in the related art such as the inability to model the contextual relevance of the interaction sequence of the target object and its interactive operation object sequence, and the low accuracy of interactive information recognition.
[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0006] According to one aspect of an embodiment of the present application, a method for acquiring interaction information is provided, the method comprising: acquiring multimodal sequence information corresponding to an interaction with a target object; inputting the multimodal sequence information into an information processing model, extracting the correlation between each modal sequence information in the multimodal sequence information through the information processing model, and determining the interaction information corresponding to the interaction based on the correlation; wherein the information processing model is trained based on the correlation between each modal sequence samples in the multimodal sequence samples and the corresponding interaction prediction labels.
[0007] According to one aspect of an embodiment of the present application, there is provided an interactive information acquisition device, comprising: an information acquisition module, configured to acquire multimodal sequence information corresponding to an interaction with a target object; a model processing module, configured to input the multimodal sequence information into an information processing model, extract the correlation between each modal sequence information in the multimodal sequence information through the information processing model, and determine the interactive information corresponding to the interaction based on the correlation; wherein the information processing model is trained based on the correlation between each modal sequence samples in the multimodal sequence samples and the corresponding interaction prediction labels.
[0008] In some embodiments of the present application, the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; based on the above technical solution, the model processing module includes: a correlation acquisition unit, configured to perform multi-scale convolution and feature interaction processing on the interaction type sequence, the interaction time sequence, and the interaction amount sequence through the information processing model to obtain the correlation between the interaction type sequence, the interaction time sequence, and the interaction amount sequence; an interaction information acquisition unit, configured to process the interaction according to the correlation to output the interaction information.
[0009] In some embodiments of the present application, the information processing model includes multiple convolutional layers; based on the above technical solution, the correlation acquisition unit is configured to: set convolution operators of different lengths corresponding to each convolutional layer; use each convolution operator to convolve the interaction type sequence, the interaction time series and the interaction amount series respectively to obtain first convolution information corresponding to the interaction type sequence, second convolution information corresponding to the interaction time series and third convolution information corresponding to the interaction amount sequence; pool the first convolution information, the second convolution information and the third convolution information respectively, and splice the pooled information to obtain spliced information; perform feature interaction according to the spliced information to obtain the correlation between the interaction type sequence, the interaction time series and the interaction amount sequence.
[0010] In some embodiments of the present application, the pooling is average pooling; based on the above technical solution, feature interaction is performed according to the spliced information to obtain the correlation between the interaction type sequence, the interaction time series and the interaction amount sequence, including: obtaining single-dimensional abnormal information according to the feature components of each dimension in the spliced information and the first weight; taking the feature components of each dimension in the spliced information as target feature components, respectively, and obtaining multi-dimensional abnormal co-occurrence information according to the target feature components, the latent vectors corresponding to the target feature components, the non-target feature components in the spliced information, and the latent vectors corresponding to the non-target feature components; obtaining the correlation according to the single-dimensional abnormal information, the multi-dimensional abnormal co-occurrence information and preset bias information.
[0011] In some embodiments of the present application, the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; based on the above technical solution, the interaction information acquisition device also includes: a first vectorization module, configured to use a One-hot embedding method to perform embedding calculations on the information in the interaction type sequence.
[0012] In some embodiments of the present application, based on the above technical solution, the interactive information acquisition device further includes: a first amount processing module, configured to perform full connection processing on the interactive amount sequence through a biased Dense layer.
[0013] In some embodiments of the present application, based on the above technical solution, the interactive information acquisition device also includes: a sample acquisition module, configured to acquire multiple groups of the multimodal sequence samples; a sample processing module, configured to input each group of the multimodal sequence samples into the information processing model to be trained, extract the correlation between each modal sequence sample in the multimodal sequence samples through the information processing model to be trained, and determine the interactive prediction information corresponding to the multimodal sequence samples based on the correlation; a parameter adjustment module, configured to adjust the parameters of the information processing model to be trained according to the interactive prediction information to obtain the information processing model.
[0014] In some embodiments of the present application, based on the above technical solution, the sample processing module is configured to: perform multi-scale convolution operations and feature interaction operations on the samples of each modal sequence through the information processing model to be trained to obtain the correlation between the samples of each modal sequence.
[0015] In some embodiments of the present application, based on the above technical solution, the parameter adjustment module is configured to: determine the interaction prediction error of the information processing model to be trained according to the interaction prediction information and the interaction prediction label corresponding to the multimodal sequence sample; update the model parameters of the information processing model to be trained according to the interaction prediction error until the interaction prediction error reaches the minimum.
[0016] In some embodiments of the present application, the multimodal sequence samples include time series samples, interaction type sequence samples and amount series samples; based on the above technical solution, the interaction information acquisition device also includes: a second vectorization module, which is configured to use a one-hot embedding method to perform embedding calculations on the information in the interaction type sequence samples.
[0017] In some embodiments of the present application, based on the above technical solution, the interactive information acquisition device also includes: a second amount processing module, configured to perform full connection processing on the information in the amount sequence sample through a biased Dense layer.
[0018] In some embodiments of the present application, based on the above technical solution, the interactive information acquisition device is also configured to: before inputting each group of the multimodal sequence samples into the information processing model to be trained, shift the time in the time series samples to obtain time series enhanced samples; and / or, multiply or subtract the amount in the amount series samples to obtain amount series enhanced samples; construct a multimodal sequence enhanced sample according to the interactive type sequence samples, the time series enhanced samples and the amount series samples, or the interactive type sequence samples, the time series samples and the amount series enhanced samples, or the interactive type sequence samples, the time series enhanced samples and the amount series enhanced samples; input multiple groups of the multimodal sequence samples and multiple groups of the multimodal sequence enhanced samples into the information processing model to be trained for model training.
[0019] In some embodiments of the present application, based on the above technical solution, the interactive information acquisition device is further configured to: use a Focal Loss loss function to adjust the imbalance between positive samples and negative samples in multiple groups of multimodal sequence samples.
[0020] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for obtaining interactive information in the above technical solution is implemented.
[0021] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the interactive information acquisition method in the above technical solution by executing the executable instructions.
[0022] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the interactive information acquisition method in the above technical solution.
[0023] In the technical solution provided in the embodiment of the present application, multimodal sequence samples are used for aggregate modeling, and the multimodal sequence information corresponding to the interaction of the acquired target object is processed by a trained information processing model, and the correlation between each sequence information is extracted, and then the interaction information corresponding to the interaction is determined according to the extracted correlation. The present application can use non-standardized data for modeling, and the established model can learn the correlation between the interaction sequence corresponding to the target and the context of its operation object sequence, and then the interaction information corresponding to the input multimodal sequence information can be calculated according to the learned information, thereby improving the universality of the model, the efficiency of data processing and the accuracy of interaction information recognition.
[0024] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0026] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.
[0027] Figure 2 The flowchart of modeling using a single-path interaction sequence in the related art is schematically shown.
[0028] Figure 3 A flowchart of another modeling method in the related art is schematically shown.
[0029] Figure 4The flowchart schematically shows the steps of the method for obtaining interactive information in the present application.
[0030] Figure 5 The structural diagram of the information processing model in this application is schematically shown.
[0031] Figure 6 The flowchart of the method for obtaining the correlation between each modal sequence information in the multimodal sequence information in the present application is schematically shown.
[0032] Figure 7 The flowchart of converting the interaction type sequence into a vector in the present application is schematically shown.
[0033] Figure 8 The flowchart of the process of processing multimodal sequence information using the information processing model in the present application is schematically shown.
[0034] Fig. 9 The flowchart of the method for obtaining the information processing model in the present application is schematically shown.
[0035] Fig.10 The flowchart of adjusting the model parameters according to the interactive prediction information in the present application is schematically shown.
[0036] Fig.11 The flowchart of the method for data enhancement in the present application is schematically shown.
[0037] Fig.12 The flowchart of time shift and amount multiple is schematically shown.
[0038] Fig.13 The structural block diagram of the interactive information acquisition device in the present application is schematically shown.
[0039] Fig.14 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION
[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0041] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0044] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.
[0045] like Figure 1 As shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, and a smart car terminal. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.
[0046] According to the implementation requirements, the system architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, the server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to the terminal device 110, can also be applied to the server 130, or can be implemented by the terminal device 110 and the server 130 together, and the present application does not make any special restrictions on this.
[0047] For example, the target object can interact through the application installed on the terminal device 110, and the data generated during the interaction can be transmitted to the server 130 through the network 120. After the server 130 obtains the data generated during the transaction, it can call the trained information processing model to process it to output the interaction information corresponding to the interaction. The data generated during the transaction can specifically include: the interaction type, interaction time and interaction amount of each interaction. The interaction type can specifically be collection, transfer, red envelope, etc. The information processing model can be set in the server 130 or in other storage media. The data input into the information processing model is multimodal sequence information, which can specifically include interaction time sequence, interaction amount sequence and interaction type sequence. By processing the input interaction time sequence, interaction amount sequence and interaction type sequence, the correlation between each modal sequence information is obtained, and then the interaction information corresponding to the interaction can be determined according to the obtained correlation. The interaction information may specifically be whether the interaction is abnormal, and the interaction information may also be an abnormality level, for example: a high abnormality level, a medium abnormality level, and a low abnormality level. A high abnormality level means that the interaction is likely to be abnormal, a medium abnormality level means that the interaction may be abnormal, but there is no need to intervene too early, and only the subsequent interactions need to be monitored. A low abnormality level means that the interaction can be considered normal; of course, the interaction information may also be other information that can distinguish whether the interaction is abnormal, and the embodiments of the present application will not be repeated here. In the embodiments of the present application, the technical solution of the present application is described by taking the interaction information whether the interaction is abnormal as an example.
[0048] In some embodiments of the present application, the information processing model can also be configured in the terminal device 110. After the terminal device 110 collects the data generated by the interaction of the target object, it can directly call the information processing model and input the data into the information processing model to obtain the interaction information corresponding to the interaction.
[0049] In some embodiments of the present application, the information processing model installed on the terminal device 110 or the server 130 is a machine learning model that performs abnormal interaction judgment based on artificial intelligence technology.
[0050] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0051] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0052] Computer Vision Technology (CV) Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision that uses cameras and computers to replace human eyes to identify targets, and further performs graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, OCR, video processing, video semantic understanding, video content / image recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies.
[0053] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
[0054] In the related art of this application, there are methods for acquiring interaction information based on machine learning models, but the modeling methods of the machine learning models used are not the same, so the data processing methods and output results of the machine learning models when acquiring interaction information corresponding to the same interaction are also different. There are multiple methods for building a model based on the interaction sequence composed of the interactions of the target object.
[0055] One method is to first normalize multiple attributes in the target object's interaction; then stitch together the information of multiple attributes to form a representation of the target object's interaction, such as stitching together the interaction type, time, page, and amount; then combine multiple different interactions into a single-channel interaction sequence; finally, perform word2vec training on the single-channel interaction sequence and import it into the downstream classification task for modeling.
[0056] Figure 2 A schematic diagram of the process of modeling using a single-path interaction sequence is shown, such as Figure 2 As shown in the figure: multi-dimensional raw data is obtained, including time, interaction type, and amount, and then the multi-dimensional raw data is binned according to manual experience, such as binning at intervals of 10s, binning at intervals of 1w, and so on; after binning, the multi-dimensional data is spliced to form a single-channel interaction sequence, and the single-channel interaction sequence is trained with word2vec, and then imported into the downstream classification task for modeling.
[0057] exist Figure 2 In the process shown, the interaction sequences of the target objects with rich attributes are integrated into natural language sequences, and word2vec is used to model the sequences. Similarly, various methods in the natural language field are applicable, such as CNN (convolutional neural network), LSTM (long short-term memory network), Transformer, XGBoost, etc.
[0058] Another method is to embed multiple attributes separately, and then splice the embedding vectors of each attribute together according to the sequence position. That is, to splice the embedding vectors of each attribute that interacts at the same time point together to obtain a unified embedding vector for the interaction at that time point, thus forming the embedding vector of the interaction sequence of the target object.
[0059] Figure 3 A flow chart of another modeling method is shown, Figure 3 As shown in the figure, the time series is first binned and then embedded to form a time series vector. At the same time, the interaction type sequence and the amount sequence are respectively embedded to form an interaction type sequence vector and an amount sequence vector. Then, the time series vector, the interaction type sequence vector and the amount sequence vector are concatenated, and CNN, LSTM and Transformer are used to process the concatenated vectors to achieve modeling.
[0060] However, both of the above modeling methods first process the various attributes of the target object's interaction separately (the first modeling method is a feature engineering method that uses a standardized operation based on manual experience, and the latter modeling method directly uses an embedding method to model and encode), and then splice them into a single sequence and input them into a deep neural network for complex modeling. Relying on massive training data, the above methods are widely used in the field of recommendation systems and have achieved many results in improving recommendation effects (such as CTR, consumer market, per capita PV and other indicators). However, the above methods all model the interests of the target object hidden behind the interaction of the target object, gain insight into the historical interests of the object, and deduce future interests, rather than modeling the contextual relevance of the target object's interaction sequence and its interaction operation object sequence. In the control field, the context of the target object's interaction sequence is strongly correlated, so when the model established by the above method is used to obtain interaction information, problems such as poor prediction accuracy will occur.
[0061] The following describes in detail the technical solutions such as the interactive information acquisition method, interactive information acquisition device, computer-readable medium, and electronic device provided by the present application in conjunction with specific implementation methods.
[0062] Figure 4 The following is a schematic flow chart showing the steps of a method for acquiring interactive information in an embodiment of the present application. The method for acquiring interactive information can be executed by a terminal device or a server, or by both the terminal device and the server. Figure 4 As shown, the interactive information acquisition method in the embodiment of the present application may mainly include the following steps S410 to S420.
[0063] Step S410: Acquire multimodal sequence information corresponding to the interaction with the target object;
[0064] Step S420: inputting the multimodal sequence information into an information processing model, extracting the correlation between each modal sequence information in the multimodal sequence information through the information processing model, and determining the interaction information corresponding to the interaction according to the correlation;
[0065] The information processing model is trained based on the correlation between each modality sequence sample in the multimodal sequence sample and the corresponding interaction prediction labels.
[0066] In the interactive information acquisition method provided in the embodiment of the present application, multimodal sequence samples are used for aggregate modeling, and the multimodal sequence information corresponding to the interaction of the acquired target object is processed by a trained information processing model, and the correlation between each sequence information is extracted, and then the interactive information corresponding to the interaction is determined according to the extracted correlation. The present application can use non-standardized data for modeling, and the established model can learn the correlation between the interaction sequence of the target object and the context of its operation object sequence, and then the interactive information corresponding to the input multimodal sequence information can be calculated according to the learned information, thereby improving the universality of the information processing model, the efficiency of data processing and the accuracy of the interactive information.
[0067] The specific implementation of each method step of the interactive information acquisition method is described in detail below.
[0068] In step S410, multimodal sequence information corresponding to the interaction with the target object is obtained.
[0069] In one embodiment of the present application, in some scenarios, there is a strong contextual correlation between a series of interactions of the target object. These interactions can be directly adjacent or separated by a certain distance. For example, when dividing and selling stolen funds obtained by abnormal means, person A receives the amount of M10,000 yuan transferred to the account by the victim at time t1, and then transfers M20,000 yuan to the account of person B, and transfers M30,000 yuan to the account of person C. Then, the interaction of first receiving the transfer from the victim and subsequently dividing the stolen money between person B and person C has a strong contextual correlation. Therefore, when modeling and using the model to obtain the interaction information corresponding to the interaction, it is necessary to obtain the multimodal sequence information corresponding to the interaction. Among them, the definition of multimodality is as follows: the target object will generate a lot of interaction sequence data in the Internet business, and each interaction has different attributes. Different attributes can perform different operations. Some can only perform subtraction, some can be added or subtracted, and some are discrete variables that cannot be added or subtracted. These different types of attributes are multimodal attributes.
[0070] In one embodiment of the present application, the multimodal sequence information of the interaction includes three types of sequences, specifically: interaction type sequence, interaction time sequence and interaction amount sequence. Among them, the interaction type represents different operations of the target object, which can be specifically collection, transfer, purchase, red envelope, etc. It is a discrete variable and cannot be added or subtracted. It can only be judged whether it is equal or adjacent. If two interaction types are the same, they are equal. If the time of occurrence of two interaction types is adjacent, the two interaction types are adjacent; time represents the time of interaction in business. In this application, time is a unix timestamp, which is a natural number, and only subtraction calculation of time has business meaning, indicating the time interval between two interactions. The time series is a sequence composed of the time of occurrence of each interaction type; the amount identifies the amount of funds involved in the interaction of the target object in business. The amount is a natural number, and rich calculations can be performed, such as addition, subtraction, division and various derivative calculations.
[0071] In step S420, the multimodal sequence information is input into an information processing model, and the correlation between each modal sequence information in the multimodal sequence information is extracted through the information processing model, and the interaction information corresponding to the interaction is obtained based on the correlation; wherein the information processing model is trained based on the correlation between each modal sequence sample in the multimodal sequence sample and the corresponding interaction prediction label.
[0072] In one embodiment of the present application, after obtaining the multimodal sequence information, it can be input into the information processing model, and the correlation between the sequence information of each modality in the multimodal sequence information can be extracted by the information processing model, and the interaction information corresponding to the interaction can be determined based on the extracted correlation. When extracting the correlation between the sequence information in the multimodal sequence information through the information processing model, the information processing model can be used to perform multi-scale convolution on the interaction type sequence, the interaction time series, and the interaction amount series to obtain the corresponding convolution information, and then the convolution information can be subjected to feature interaction processing to obtain the correlation between the interaction type series, the interaction time series, and the interaction amount series; then the interaction can be processed based on the correlation to obtain the interaction information.
[0073] Figure 5 The structural diagram of the information processing model is shown in FIG. Figure 5 As shown, the information processing model includes an input layer 501, a convolution unit 502, a pooling layer 503, a splicing layer 504, a feature interaction layer 505, an interactive information prediction layer 506 and an output layer 507, wherein the convolution unit 502 includes multiple convolution layers, and each convolution layer has a convolution operator of different lengths.
[0074] Next, based on Figure 5The structure of the information processing model shown in the figure describes in detail the method of performing multi-scale convolution on the interaction type sequence, interaction time series and interaction amount series through the information processing model to obtain convolution information, and then performing feature interaction processing on the obtained convolution information to obtain the correlation between the interaction type sequence, interaction time series and interaction amount series.
[0075] Figure 6 A schematic diagram of a method for obtaining the correlation between each modal sequence information in multimodal sequence information is shown, such as Figure 6 As shown:
[0076] In step S601, convolution operators of different lengths are set corresponding to each convolution layer.
[0077] In one embodiment of the present application, a plurality of convolutional layers with convolutional operators of different lengths are set to perform multi-scale convolution on the input multimodal sequence information to extract convolution information of different sizes. By counting the interaction type sequences in the control scene, the length of the interaction subsequence is usually 2-5, so the length of the convolution operator can be set to 2-5 in the present application. Specifically, the convolution unit 502 can include convolutional layer 502-1, convolutional layer 502-2, convolutional layer 502-3 and convolutional layer 502-4, and the lengths of the convolution operators corresponding to them are 2, 3, 4, and 5, respectively. Of course, other numbers of convolutional layers can be set in the convolution unit, and the length of the convolution operator corresponding to each convolutional layer can be set to other values. The embodiment of the present application does not specifically limit this.
[0078] In step S602, convolution operators are used to convolve the interaction type sequence, the interaction time sequence and the interaction amount sequence respectively to obtain first convolution information corresponding to the interaction type sequence, second convolution information corresponding to the interaction time sequence and third convolution information corresponding to the interaction amount sequence.
[0079] In one embodiment of the present application, after the length of the convolution operator is set, multi-scale convolution can be performed on each sequence information through convolution operators of different lengths. Specifically, when the length of the convolution operator is 2, 3, 4, and 5, the interaction type sequence, interaction time sequence, and interaction amount sequence in the multimodal sequence information are respectively convolved by convolution operators of these four lengths to obtain corresponding convolution information. In the embodiment of the present application, the convolution information obtained by performing multi-scale convolution on the interaction type sequence is the first convolution information, the convolution information obtained by performing multi-scale convolution on the interaction time sequence is the second convolution information, and the convolution information obtained by performing multi-scale convolution on the interaction amount sequence is the third convolution information. In other words, the first convolution information, the second convolution information, and the third convolution information respectively include four different sizes of convolution information.
[0080] In one embodiment of the present application, since the interaction type is a discrete variable and coefficient multiplication calculation cannot be performed, before the interaction type sequence is input into the information processing model for convolution processing, it is necessary to perform embedding calculation on the interaction type sequence to convert it into a vector. In the embodiment of the present application, a one-hot embedding method can be used to embed the information in the interaction type sequence to convert the interaction type sequence into a vector in the form of a decimal, a natural number, etc. Figure 7 A schematic diagram of the process of converting the interaction type sequence into a vector is shown, Figure 7 As shown: The interaction type sequence includes interaction A, interaction B and interaction C. After embedding according to the mapping relationship between the interaction type and the one-hot encoding, interaction A is converted into vector a, interaction B is converted into vector b, and interaction C is converted into vector c.
[0081] In one embodiment of the present application, since the span of the amount value range in the amount sequence is relatively large, a Dense layer 508 can be set before the convolution layer of the information processing model to prevent the amount from being directly input into the model during the model training phase, causing obvious model loss jitter. Therefore, before the amount sequence is convoluted, the amount sequence can also be fully connected through the Dense layer 508 to prevent the output result from being biased due to the large span of the amount value range in the amount sequence. The Dense layer 508 performs biased full connection processing on the amount sequence samples, and the specific calculation formula is shown in formula (1):
[0082] y=w*x+b(1)
[0083] Among them, w is the weight, x is the amount in the amount sequence sample, b is the bias, and y is the amount sequence sample after full connection processing.
[0084] In step S603, the first convolution information, the second convolution information and the third convolution information are pooled respectively, and the pooled information is spliced to obtain spliced information.
[0085] In one embodiment of the present application, after obtaining the first convolution information, the second convolution information and the third convolution information, they can be pooled respectively through the pooling layer 503 to obtain the corresponding first pooling information, the second pooling information and the third pooling information, and then the obtained first pooling information, the second pooling information and the third pooling information can be spliced through the splicing layer 504 to obtain splicing information, which can be used to characterize the significant signals between the interaction type sequence, the interaction time series and the interaction amount series. Among them, the pooling layer 503 can specifically perform an average pooling operation to extract significant signals on the interaction type sequence, the interaction time series and the interaction amount series from the global.
[0086] In step S604, feature interaction is performed according to the splicing information to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence.
[0087] In one embodiment of the present application, in order to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence, it is also necessary to perform feature interaction on the spliced information through the feature interaction layer 505, and the signals on the interaction type sequence, the interaction time sequence and the interaction amount sequence can be mutually enhanced through feature interaction. Specifically, firstly, the single-dimensional abnormal information is obtained according to the feature components of each dimension in the spliced information and the first weight; then, the feature components of each dimension in the spliced information are respectively used as target feature components, and multi-dimensional abnormal co-occurrence information is obtained according to the target feature component, the latent vector corresponding to the target feature component, the non-target feature component in the spliced information, and the latent vector corresponding to the non-target feature component; finally, the correlation can be obtained according to the single-dimensional abnormal information, the multi-dimensional abnormal co-occurrence information and the preset bias information.
[0088] The specific calculation formula is shown in formula (2):
[0089]
[0090] Among them, w0 is the bias information, w i For x i The corresponding first weight, x i is the i-th dimension feature component in the splicing information, n is the feature dimension in the splicing information, x j is the j-th dimension feature component in the splicing information, v i For x i The corresponding latent vector, v j For x j The corresponding hidden vector.
[0091] From the analysis of formula (2), we can see that formula (2) consists of two parts, where the first part is the logistic regression (LR) model, which is used to describe single-dimensional anomalies, and the second part is the second-order polynomial model, which is actually the combination and multiplication of all feature values in the splicing information, used to describe the co-occurrence of multi-dimensional anomalies. Feature interaction can not only discover single-dimensional anomalies, but also make multi-dimensional anomalies co-occur, making the strong correlation of the interaction type sequence context more prominent.
[0092] According to formula (2), feature interaction can be realized, and the signals on the three sequences of interaction type sequence, interaction time series and interaction amount series can be mutually enhanced to obtain the correlation between the interaction type sequence, interaction time series and interaction amount series, and then the interaction information corresponding to the interaction can be obtained based on the correlation.
[0093] Figure 8 A schematic diagram of the process of processing multimodal sequence information using an information processing model is shown, such as Figure 8 As shown, in step S801, the interaction time series, interaction type series and interaction amount series are input into the information processing model through the input layer 501; in step S802, the interaction type information in the interaction type series is embedded to obtain the corresponding embedding vector; at the same time, the interaction amount in the interaction amount series is fully connected through the Dense layer 508 to obtain the updated interaction amount series; in step S803, the multi-modal sequence vector composed of the interaction time series, the embedding vector corresponding to the interaction type series and the updated interaction amount series is multi-scale convolutionally processed through the convolution unit 502 to obtain the first convolution information corresponding to the interaction time series, the second convolution information corresponding to the interaction type series and the third convolution information corresponding to the interaction amount series, wherein the first convolution information is the sum of the first and second convolution information. The first convolution information, the second convolution information and the third convolution information all contain four convolution vectors of different sizes; in step S804, the first convolution information, the second convolution information and the third convolution information are respectively average pooled through the pooling layer 503; in step S805, the first convolution information, the second convolution information and the third convolution information after average pooling are spliced through the splicing layer 504 to obtain splicing information; in step S806, the splicing information is subjected to feature interaction based on the factor decomposition machine through the feature interaction layer 505 to obtain the correlation between the interaction type sequence, the interaction time series and the interaction amount series; in step S807, the interaction is processed according to the correlation through the interaction information prediction layer 506 to obtain the corresponding interaction information; in step S808, the interaction information is output through the output layer 507.
[0094] In one embodiment of the present application, before applying the information processing model to predict the interaction information, a large number of training samples are required to train the information processing model to be trained to obtain an information processing model with stable performance.
[0095] Fig. 9 A schematic diagram of the method flow for obtaining an information processing model is shown. Fig. 9 As shown, the method flow specifically includes the following steps:
[0096] In step S901, multiple groups of multimodal sequence samples are obtained.
[0097] In one embodiment of the present application, in some special scenarios, it is necessary to study the strong correlation of the interaction context of the target object. For example, in a risk control scenario, the gang will divide the work and cooperate to perform abnormal operations to obtain funds. After receiving the funds transferred by the victim, they will divide the spoils. Then there will be a fixed interaction pattern in the interaction sequence. For example, within a certain period of time, the target account first receives the money and then transfers money to other accounts or sends red envelopes. Therefore, in the model training process, it is necessary to let the information processing model to be trained learn the interaction pattern from multiple groups of multimodal sequence samples, so that in the subsequent interaction information prediction, it can accurately identify the corresponding interaction pattern and mark the corresponding interaction as abnormal or highly abnormal level, etc. Therefore, when training the model, it is still necessary to train the information processing model to be trained according to the sequence information of the three dimensions of interaction type, interaction time and interaction amount. That is to say, similar to the composition of multimodal sequence information, multimodal sequence samples include interaction type sequence samples, time series samples and amount sequence samples.
[0098] In step S902, each group of multimodal sequence samples is input into the information processing model to be trained, the correlation between each modal sequence sample in the multimodal sequence samples is extracted through the information processing model to be trained, and the interaction prediction information corresponding to the multimodal sequence samples is obtained according to the correlation between each modal sequence sample.
[0099] In one embodiment of the present application, after obtaining multiple groups of multimodal sequence samples, they can be input into the information processing model to be trained, and the multimodal sequence samples can be subjected to multi-scale convolution, pooling, splicing, feature interaction and other operations through the information processing model to be trained to obtain the correlation between the sequence samples of each modality, and determine the interaction prediction information corresponding to the multimodal sequence samples based on the correlation.
[0100] and Figure 6 The model processing flow shown in is the same. When the multimodal sequence samples pass through the convolution unit, the convolution layer with convolution operators of different lengths in the convolution unit convolves the samples of each modal sequence in turn to generate the corresponding convolution information. Then, the generated convolution information is pooled through the pooling layer, and the information generated by the pooling is spliced through the splicing layer to obtain the splicing information corresponding to the multimodal sequence information. Next, the splicing information is transmitted to the feature interaction layer. Through the feature interaction layer, the signals of the three dimensions of time, amount, and interaction type in the splicing information can be mutually enhanced based on the factor decomposition machine to obtain the correlation between the three dimensional information.
[0101] After obtaining the correlation between the samples of each modal sequence, the interaction prediction information corresponding to the multimodal sequence sample can be determined according to the correlation and output through the output layer. It is worth noting that when the information processing model to be trained is initially trained, it has not learned any rules. Therefore, in order to determine the corresponding interaction prediction information according to the correlation, an initial rule can be set in advance so that the model can determine the interaction prediction information corresponding to the multimodal sequence information according to the correlation output by the feature interaction layer.
[0102] In step S903, the parameters of the information processing model to be trained are adjusted according to the interaction prediction information to obtain an information processing model.
[0103] In one embodiment of the present application, after obtaining the interactive prediction information output by the information processing model to be trained, the information processing model to be trained can be reversely adjusted according to the interactive prediction information, and the information processing model to be trained after the parameter adjustment is used for the next round of model training. A stable information processing model can be obtained by performing multiple iterative trainings of forward training-reverse parameter adjustment.
[0104] Fig.10 A schematic diagram of the process of adjusting model parameters according to interactive prediction information is shown, Fig.10 As shown, in step S1001, the interaction prediction error of the information processing model to be trained is determined according to the interaction prediction information and the interaction prediction labels corresponding to the multimodal sequence samples; in step S1002, the model parameters of the information processing model to be trained are updated according to the interaction prediction error until the interaction prediction error is minimized.
[0105] Among them, the interaction prediction label is the manually labeled interaction information corresponding to the input multimodal sequence sample. According to the interaction prediction information and interaction prediction label output by the model, the interaction prediction error of the information processing model to be trained can be determined. The interaction prediction error is the loss function of the information processing model to be trained. The smaller the loss function, the more stable the model. In the embodiment of the present application, the loss function can specifically be a cross entropy loss function, and of course it can also be other types of loss functions, which are not specifically limited in the present application.
[0106] In one embodiment of the present application, it is also possible to set the information processing model to be trained to be trained a preset number of times. After completing the preset number of training times, a stable information processing model can be obtained.
[0107] In one embodiment of the present application, in order to obtain a model with stable performance and high prediction accuracy, a large number of samples need to be used for training in the training stage. It is not enough to rely solely on manually collected and labeled samples, and manual collection and labeling are time-consuming and labor-intensive and the number of samples is limited. Therefore, in an embodiment of the present application, the sample size can be increased by data enhancement. In an embodiment of the present application, when predicting interactive information based on the interaction time series, interaction amount series, and interaction type series, the interaction time and interaction amount are not standardized. Therefore, during data enhancement, in order to ensure that the characteristics of the time series samples and the amount series samples are not lost, and at the same time to ensure that the information processing model to be trained can learn the correlation between the sequence samples, the time series samples and the amount series samples are still not standardized in the embodiment of the present application. At the same time, according to the data business characteristics, corresponding operations are performed on the time and / or amount to achieve data enhancement.
[0108] Fig.11 A schematic diagram of the data enhancement method flow chart is shown, such as Fig.11 As shown, in step S1101, the time in the time series sample is shifted to obtain the time series enhanced sample; and / or, in step S1102, the amount in the amount series sample is doubled or doubled to obtain the amount series enhanced sample; in step S1103, a multimodal sequence enhanced sample is constructed according to the interaction type sequence samples, the time series enhanced samples and the amount series samples, or the interaction type sequence samples, the time series enhanced samples and the amount series enhanced samples, or the interaction type sequence samples, the time series enhanced samples and the amount series enhanced samples; in step S1104, multiple groups of multimodal sequence samples and multiple groups of multimodal sequence enhanced samples are input into the information processing model to be trained for model training.
[0109] according to Fig.11 It can be seen that the time shift operation is used to enhance the time series samples, and the amount multiple operation is used to enhance the amount series samples. This is because when the target objects interact, the absolute value of the interaction time is not important, while the time interval is more important for model training. Therefore, the interaction time shift does not affect the judgment of whether the target object's interaction is abnormal. In addition, the absolute value of the amount has a very large range in the massive target objects, ranging from a few yuan to millions or even hundreds of millions, which has little impact on model training. On the contrary, the proportion of capital inflows and outflows has a greater impact on model training. Therefore, doubling or reducing the amount corresponding to the target object's interaction is equivalent to judging whether the target object's interaction is abnormal.
[0110] Fig.12 A flow chart showing time shift and amount multiples is shown, Fig.12As shown, the original input samples are: time series samples are: 1, 19, 40, amount series samples are: 3000, 2999, 4, interaction type sequence samples are: interaction A, interaction B, interaction C. Then the input samples generated after time shifting 100 are: time series samples are: 101, 119, 140, amount series samples are: 3000, 2999, 4, interaction type sequence samples are: interaction A, interaction B, interaction C. The input samples generated after reducing the amount by 10 times are: time series samples are: 1, 19, 40, amount series samples are: 300, 299.9, 0.4, interaction type sequence samples are: interaction A, interaction B, interaction C.
[0111] Multimodal sequence enhancement samples can be obtained based on the time series samples after time shift and / or the amount series samples after amount multiple processing, and then input samples can be constructed based on the original multiple groups of multimodal sequence samples and the enhanced multiple groups of multimodal sequence enhancement samples, and input them into the information processing model to be trained to train the model. Through the data enhancement method, the number of samples can be greatly increased, making the model training more sufficient. After multiple rounds of iterative training, the model is more stable and the accuracy of the prediction results is higher.
[0112] In one embodiment of the present application, since positive samples (interactions are abnormal) are very rare in control scenarios, especially when a control scenario just breaks out, the sample volume is even more scarce. Therefore, in order to improve the accuracy of the information processing model in predicting interaction information, after obtaining the interaction prediction information, it is also necessary to increase the processing of sample imbalance. The sample imbalance problem in the embodiment of the present application is mainly divided into two aspects. One is the problem of unbalanced sample ratios, and the other is the problem of unbalanced loss distribution between simple samples (easy case) and complex samples (hard case). In order to solve the above problems, the focal loss function is used in the embodiment of the present application to process the sample imbalance problem. The specific calculation formula is shown in formula (3):
[0113]
[0114] Among them, α and γ are both hyperparameters. The hyperparameter α is used to adjust the imbalance of sample proportions, and the hyperparameter γ is used to adjust the distribution of losses between simple samples and complex samples. A value > 1 will increase the contribution of complex samples to the loss of the training process; p is the probability that the predicted interaction is abnormal.
[0115] The information processing model training method in this application is based on multimodal sequence samples for aggregate modeling. In this method, non-standardized data such as time and amount are not standardized, but directly used for modeling, which reduces the amount of data processing, improves the universality and data processing efficiency of the information processing model, and reduces labor costs. In addition, in actual business scenarios, the accuracy of the output results of the information processing model trained by the information processing model training method in this application is higher. Compared with the existing solutions, the accuracy of the information processing model in this application for identifying abnormal interactions has increased by about 50%. According to the output results of the information processing model, managers can limit and restrict operations on relevant accounts in the first place, effectively preventing personnel who implement abnormal operations from continuing to interact abnormally. In addition, the abnormal information monitored can be fed back to relevant institutions in a timely manner, which greatly improves the intensity of investigation of abnormal interactions.
[0116] The interactive information acquisition method of the present application can be applied to various platforms involving fund transactions, such as online banking of various banks, various applications with wallet functions, such as WeChat, QQ, shopping platforms with wallet functions, etc. Next, the interactive information acquisition method in the present application is specifically described by taking the interaction of the target object through WeChat as an example. Among them, the target object is the user using WeChat, the interaction is the user conducting transactions through WeChat, and the interactive information corresponding to the interaction is the risk information of the user conducting transactions through WeChat.
[0117] Obtain the flow of funds corresponding to the WeChat account of object A and the type of transaction conducted by object A within a preset time period, and extract multimodal sequence information based on the obtained information, namely, interaction type sequence information, interaction time series information and interaction amount sequence information, where the interaction type can specifically be collection, transfer, sending red envelopes, etc. For example, within the time range of 0-5 minutes, object A receives a transfer at 1 minute, and the corresponding amount is 100,000 yuan. Then object A initiates transfers at 2 minutes, 3 minutes, and 4 minutes respectively, transferring 20,000 yuan, 10,000 yuan, and 20,000 yuan of funds to object B, object C, and object D respectively. Then, multimodal sequence information can be extracted based on this information, specifically: interaction type sequence information: collection, transfer, transfer, transfer; interaction time series information: 1 minute, 2 minutes, 3 minutes, 4 minutes; interaction amount sequence information: 100,000 yuan, 20,000 yuan, 20,000 yuan, and 20,000 yuan. Then, the information processing model can be called, and the obtained multimodal sequence information is input into the information processing model. The multimodal sequence information is processed by the information processing model through convolution, pooling, splicing, and feature interaction to obtain the correlation between the interaction type sequence information, the interaction time series information, and the interaction amount series information, and then the risk information corresponding to the interaction of object A is determined based on the correlation and the corresponding relationship between the correlation of the interaction sequence context learned by the information processing model and the risk information, and the risk information is output. For example, if during the model training process, the information processing model learns that the user account transfers funds to other accounts within 5 minutes after transferring funds, and the amount of funds transferred is greater than 60,000 and the total amount of funds transferred is greater than or equal to 50,000, then the interaction is identified as an abnormal interaction, and the interaction information output by the information processing model after processing the multimodal sequence information corresponding to object A is an abnormal interaction. When it is determined to be an abnormal interaction, the warning unit will be triggered to send a warning message to the risk control specialist so that the risk control specialist can limit or restrict the fund transfer in / out operation of the account of object A; or directly trigger the restriction unit to close the fund transfer in / out function of the WeChat account of object A.
[0118] It is worth noting that the interaction patterns of abnormal interactions learned by the information processing model are diverse. When identifying multimodal sequence information, misjudgment may occur, that is, normal interactions may be judged as abnormal interactions. In order to avoid misjudgment, after the interaction information is output, a specialist can perform manual judgment. If the conclusion of the manual judgment is the same as the interaction information output by the model, the target object's account will be limited or the transfer of funds in / out will be restricted. If the conclusion of the manual judgment is different from the interaction information output by the model, the conclusion of the manual judgment shall prevail.
[0119] It can be understood that in the specific implementation of the present application, the relevant data involving the target object's operations on account funds is required. When the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the target object's permission or consent, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0120] In the embodiment of the present application, multimodal sequence samples are used for aggregate modeling, and the multimodal sequence information corresponding to the interaction of the target object is processed by a trained information processing model, the correlation between each sequence information is extracted, and then the interaction information corresponding to the interaction is determined according to the extracted correlation. The present application can use non-standardized data for modeling, and the established model can learn the correlation between the interaction sequence of the target object and the context of its operation object sequence, and then the interaction information corresponding to the input multimodal sequence information can be obtained according to the learned information, thereby improving the universality of the information processing model, the efficiency of data processing and the accuracy of the interaction information.
[0121] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0122] The following introduces an apparatus embodiment of the present application, which can be used to execute the interactive information acquisition method in the above-mentioned embodiment of the present application. Fig.13 The structure block diagram of the interactive information acquisition device provided in the embodiment of the present application is schematically shown. Fig.13 As shown, the interactive information acquisition device 1300 includes: an information acquisition module 1310 and a model processing module 1320, specifically:
[0123] The information acquisition module 1310 is configured to acquire multimodal sequence information corresponding to the interaction with the target object; the model processing module 1320 is configured to input the multimodal sequence information into the information processing model, extract the correlation between each modal sequence information in the multimodal sequence information through the information processing model, and determine the interaction information corresponding to the interaction based on the correlation; wherein the information processing model is trained based on the correlation between each modal sequence sample in the multimodal sequence sample and the corresponding interaction prediction label.
[0124] In some embodiments of the present application, the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; based on the above embodiments, the model processing module 1320 includes: a correlation acquisition unit, configured to perform multi-scale convolution and feature interaction processing on the interaction type sequence, the interaction time sequence, and the interaction amount sequence through the information processing model to obtain the correlation between the interaction type sequence, the interaction time sequence, and the interaction amount sequence; an interaction information acquisition unit, configured to process the interaction according to the correlation to output the interaction information.
[0125] In some embodiments of the present application, the information processing model includes multiple convolutional layers; based on the above embodiments, the correlation acquisition unit is configured to: set convolution operators of different lengths corresponding to each of the convolutional layers; use each of the convolution operators to convolve the interaction type sequence, the interaction time series, and the interaction amount series, respectively, to obtain first convolution information corresponding to the interaction type sequence, second convolution information corresponding to the interaction time series, and third convolution information corresponding to the interaction amount sequence; pool the first convolution information, the second convolution information, and the third convolution information, respectively, and splice the pooled information to obtain spliced information; perform feature interaction according to the spliced information to obtain the correlation between the interaction type sequence, the interaction time series, and the interaction amount sequence.
[0126] In some embodiments of the present application, the pooling is average pooling; based on the above embodiments, feature interaction is performed according to the splicing information to obtain the correlation between the interaction type sequence, the interaction time series and the interaction amount sequence, including: obtaining single-dimensional abnormal information according to the feature components of each dimension in the splicing information and the first weight; using the feature components of each dimension in the splicing information as target feature components, respectively, and obtaining multi-dimensional abnormal co-occurrence information according to the target feature components, the latent vectors corresponding to the target feature components, the non-target feature components in the splicing information, and the latent vectors corresponding to the non-target feature components; obtaining the correlation according to the single-dimensional abnormal information, the multi-dimensional abnormal co-occurrence information and preset bias information.
[0127] In some embodiments of the present application, the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; based on the above embodiments, the interaction information acquisition device 1300 also includes: a first vectorization module, configured to use a One-hot embedding method to perform embedding calculations on the information in the interaction type sequence.
[0128] In some embodiments of the present application, based on the above embodiments, the interaction information acquisition device 1300 further includes: a first amount processing module, configured to perform full connection processing on the interaction amount sequence.
[0129] In some embodiments of the present application, based on the above embodiments, the interactive information acquisition device 1300 also includes: a sample acquisition module, configured to acquire multiple groups of the multimodal sequence samples; a sample processing module, configured to input each group of the multimodal sequence samples into the information processing model to be trained, extract the correlation between each modal sequence sample in the multimodal sequence samples through the information processing model to be trained, and determine the interactive prediction information corresponding to the multimodal sequence samples based on the correlation; a parameter adjustment module, configured to adjust the parameters of the information processing model to be trained based on the interactive prediction information to obtain the information processing model.
[0130] In some embodiments of the present application, based on the above embodiments, the sample processing module is configured to: perform multi-scale convolution and feature interaction processing on the samples of each modal sequence through the information processing model to be trained to obtain the correlation between the samples of each modal sequence.
[0131] In some embodiments of the present application, based on the above embodiments, the parameter adjustment module is configured to: determine the interaction prediction error of the information processing model to be trained according to the interaction prediction information and the interaction prediction label corresponding to the multimodal sequence sample; update the model parameters of the information processing model to be trained according to the interaction prediction error until the interaction prediction error reaches the minimum.
[0132] In some embodiments of the present application, the multimodal sequence samples include time series samples, interaction type sequence samples and amount series samples; based on the above embodiments, the interaction information acquisition device 1300 also includes: a second vectorization module, configured to use a one-hot embedding method to perform embedding calculation on the information in the interaction type sequence samples.
[0133] In some embodiments of the present application, based on the above embodiments, the interactive information acquisition device 1300 further includes: a second amount processing module, configured to perform full connection processing on the information in the amount sequence sample through a biased Dense layer.
[0134] In some embodiments of the present application, based on the above embodiments, the interactive information acquisition device 1300 is also configured to: before inputting each group of the multimodal sequence samples into the information processing model to be trained, shift the time in the time series samples to obtain time series enhanced samples; and / or, multiply or subtract the amount in the amount series samples to obtain amount series enhanced samples; construct a multimodal sequence enhanced sample according to the interactive type sequence samples, the time series enhanced samples and the amount series samples, or the interactive type sequence samples, the time series samples and the amount series enhanced samples, or the interactive type sequence samples, the time series enhanced samples and the amount series enhanced samples; input multiple groups of the multimodal sequence samples and multiple groups of the multimodal sequence enhanced samples into the information processing model to be trained for model training.
[0135] In some embodiments of the present application, based on the above embodiments, the interactive information acquisition device 1300 is further configured to: use a Focal Loss loss function to adjust the imbalance between positive samples and negative samples in the multiple groups of multimodal sequence samples.
[0136] The specific details of the interactive information acquisition device provided in each embodiment of the present application have been described in detail in the corresponding method embodiment and will not be repeated here.
[0137] Fig.14 The computer system structure block diagram of the electronic device used to implement the embodiment of the present application is schematically shown. The electronic device may be as follows: Figure 1 The terminal device 110 or the server 130 shown in FIG.
[0138] It should be noted that Fig.14 The computer system 1400 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0139] like Fig.14As shown, the computer system 1400 includes a central processing unit 1401 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1402 (ROM) or the program loaded from the storage part 1408 to the random access memory 1403 (RAM). Various programs and data required for system operation are also stored in the random access memory 1403. The central processing unit 1401, the read-only memory 1402 and the random access memory 1403 are connected to each other through a bus 1404. An input / output interface 1405 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1404.
[0140] In some embodiments, the following components are connected to the input / output interface 1405: an input section 1406 including a keyboard, a mouse, etc.; an output section 1407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a local area network card, a modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the input / output interface 1405 as needed. A removable medium 1411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1410 as needed, so that a computer program read therefrom is installed into the storage section 1408 as needed.
[0141] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1409, and / or installed from the removable medium 1411. When the computer program is executed by the central processor 1401, various functions defined in the system of the present application are executed.
[0142] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0143] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0144] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0145] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable an electronic device to execute the method according to the implementation methods of the present application.
[0146] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.
[0147] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for acquiring interactive information, characterized in that: include: Acquire multimodal sequence information corresponding to the interaction with the target object, wherein the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; Inputting the multimodal sequence information into an information processing model, extracting correlations between the modal sequence information in the multimodal sequence information through the information processing model, and determining interaction information corresponding to the interaction according to the correlations; The training process of the information processing model includes: Obtain multiple groups of multimodal sequence samples; Inputting each group of the multimodal sequence samples into the information processing model to be trained, extracting the correlation between each modality sequence sample in the multimodal sequence samples through the information processing model to be trained, and determining the interactive prediction information corresponding to the multimodal sequence samples according to the extracted correlation; The parameters of the information processing model to be trained are adjusted according to the interaction prediction information and the interaction prediction labels corresponding to the multimodal sequence samples to obtain the information processing model.
2. The method according to claim 1, characterized in that The extracting the correlation between each modal sequence information in the multimodal sequence information by the information processing model, and determining the interaction information corresponding to the interaction according to the correlation, includes: Performing multi-scale convolution and feature interaction processing on the interaction type sequence, the interaction time sequence and the interaction amount sequence through the information processing model to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence; The interaction is processed according to the correlation to output the interaction information.
3. The method according to claim 2, characterized in that The information processing model comprises a plurality of convolutional layers; The performing multi-scale convolution and feature interaction processing on the interaction type sequence, the interaction time sequence and the interaction amount sequence by the information processing model to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence includes: Convolution operators of different lengths are set corresponding to each of the convolutional layers; Using each of the convolution operators to convolve the interaction type sequence, the interaction time sequence, and the interaction amount sequence, respectively, to obtain first convolution information corresponding to the interaction type sequence, second convolution information corresponding to the interaction time sequence, and third convolution information corresponding to the interaction amount sequence; Pooling the first convolution information, the second convolution information, and the third convolution information respectively, and splicing the pooled information to obtain splicing information; Feature interaction is performed according to the splicing information to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence.
4. The method according to claim 3, characterized in that The pooling is average pooling; The performing feature interaction according to the splicing information to obtain the correlation between the interaction type sequence, the interaction time sequence and the interaction amount sequence includes: Acquire single-dimensional abnormal information according to the characteristic components of each dimension in the splicing information and the first weight; Taking the feature components of each dimension in the spliced information as target feature components respectively, and obtaining multi-dimensional abnormal co-occurrence information according to the target feature components, latent vectors corresponding to the target feature components, non-target feature components in the spliced information, and latent vectors corresponding to the non-target feature components; The correlation is obtained according to the single-dimensional anomaly information, the multi-dimensional anomaly co-occurrence information and preset bias information.
5. The method according to claim 1, characterized in that Before inputting the multimodal sequence information into the information processing model, the method further includes: A one-hot embedding method is used to embed the information in the interaction type sequence.
6. The method according to claim 3, characterized in that: Before using each of the convolution operators to convolve the interaction amount sequence respectively, the method further includes: The interaction amount sequence is fully connected through a biased Dense layer.
7. The method according to claim 1, characterized in that The extracting the correlation between each modality sequence sample in the multimodal sequence sample by using the information processing model to be trained includes: Multi-scale convolution and feature interaction processing are performed on the samples of each modal sequence through the information processing model to be trained to obtain the correlation between the samples of each modal sequence.
8. The method according to claim 1, characterized in that: The step of adjusting the parameters of the information processing model to be trained according to the interaction prediction information and the interaction prediction labels corresponding to the multimodal sequence samples to obtain the information processing model includes: Determining an interactive prediction error of the information processing model to be trained according to the interactive prediction information and the interactive prediction label corresponding to the multimodal sequence sample; The model parameters of the information processing model to be trained are updated according to the interactive prediction error until the interactive prediction error reaches a minimum.
9. The method according to claim 1, characterized in that: The multimodal sequence samples include time series samples, interaction type series samples and amount series samples; Before inputting each group of multimodal sequence samples into the information processing model to be trained, the method further includes: A one-hot embedding method is used to embed the information in the interaction type sequence sample.
10. The method according to claim 9, characterized in that Before extracting the correlation between the modal sequence samples in the multimodal sequence samples by using the information processing model to be trained, the method further includes: The information in the amount sequence sample is fully connected through a biased Dense layer.
11. The method according to claim 1, characterized in that: The multimodal sequence samples include time series samples, interaction type series samples and amount series samples; Before inputting each group of multimodal sequence samples into the information processing model to be trained, the method further includes: Shifting the time in the time series samples to obtain time series enhanced samples; and / or The amounts in the amount sequence samples are multiplied or reduced to obtain an enhanced amount sequence sample; Constructing a multimodal sequence enhancement sample according to the interaction type sequence sample, the time series enhancement sample and the amount series sample, or the interaction type sequence sample, the time series sample and the amount series enhancement sample, or the interaction type sequence sample, the time series enhancement sample and the amount series enhancement sample; Inputting a plurality of groups of the multimodal sequence samples and a plurality of groups of the multimodal sequence enhancement samples into the information processing model to be trained for model training.
12. The method according to claim 1, characterized in that The method further comprises: The FocalLoss loss function is used to adjust the imbalance between positive samples and negative samples in multiple groups of multimodal sequence samples.
13. An interactive information acquisition device, characterized in that: include: An information acquisition module, configured to acquire multimodal sequence information corresponding to the interaction with the target object, wherein the multimodal sequence information includes an interaction type sequence, an interaction time sequence, and an interaction amount sequence corresponding to the interaction; a model processing module, configured to input the multimodal sequence information into an information processing model, extract the correlation between each modal sequence information in the multimodal sequence information through the information processing model, and determine the interaction information corresponding to the interaction according to the correlation; The training process of the information processing model includes: Obtain multiple groups of multimodal sequence samples; Inputting each group of the multimodal sequence samples into the information processing model to be trained, extracting the correlation between each modality sequence sample in the multimodal sequence samples through the information processing model to be trained, and determining the interactive prediction information corresponding to the multimodal sequence samples according to the extracted correlation; The parameters of the information processing model to be trained are adjusted according to the interaction prediction information and the interaction prediction labels corresponding to the multimodal sequence samples to obtain the information processing model.
14. A computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for obtaining interactive information according to any one of claims 1 to 12 is implemented.
15. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the interactive information acquisition method described in any one of claims 1 to 12 by executing the executable instructions.
16. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; the processor of the electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the electronic device executes the interactive information acquisition method described in any one of claims 1 to 12.
