A method, device and equipment for generating customer portraits

Abstract

The embodiments of this specification provide a method, device and equipment for generating a customer portrait, which can be used in the field of big data technology. The method includes obtaining information about the target customer in multiple modes; converting the information in each mode to obtain feature information corresponding to each mode; determining the multimodal data relationship network of the target customer based on the feature information corresponding to each mode in multiple modes; fusing the information in multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer; generating a portrait of the target customer based on the fused feature information and the label information of the target customer. The embodiments of this specification can be used to automatically fill in the missing dimensional information in the customer portrait, so that the generated customer portrait is more accurate and comprehensive.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method, device and equipment for generating customer portraits. Background Art

[0002] With the development of digital technology, building an enterprise-level customer portrait with full information integration, full customer coverage, full relationship disclosure, and full process support will have a very important impact on helping banks provide customer-centric and precise financial services.

[0003] In the prior art, when constructing a customer profile, some dimensions of the customer profile may be incomplete, which requires manual verification and supplementation. However, when the number of customers is large, manual verification and supplementation is not only time-consuming and laborious, but also prone to errors, resulting in a low accuracy of the generated customer profile.

[0004] Therefore, the industry is in urgent need of a technical solution that can solve the above technical problems. Summary of the invention

[0005] The embodiments of this specification provide a method, device and equipment for generating a customer portrait, which can automatically fill in the missing dimensional information in the customer portrait, so that the generated customer portrait is more accurate and comprehensive.

[0006] The method, device and equipment for generating customer portraits provided in this specification are implemented in the following ways.

[0007] A method for generating a customer portrait comprises: obtaining information of a target customer in multiple modes; converting the information in each mode to obtain feature information corresponding to each mode; determining a multimodal data relationship network of the target customer based on the feature information corresponding to each mode in the multiple modes; fusing the information in the multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer; and generating a portrait of the target customer based on the fused feature information and the label information of the target customer.

[0008] A device for generating a customer portrait, comprising: an acquisition module for acquiring information of a target customer in multiple modes; a conversion module for converting the information in each mode to obtain feature information corresponding to each mode; a determination module for determining a multimodal data relationship network of the target customer based on the feature information corresponding to each mode in the multiple modes; a fusion module for fusing the information in the multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer; and a generation module for generating a portrait of the target customer based on the fused feature information and the label information of the target customer.

[0009] A device for generating a customer portrait includes at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of any method embodiment in the embodiments of this specification are implemented.

[0010] A computer-readable storage medium stores computer instructions, which, when executed, implement the steps of any method embodiment in the embodiments of this specification.

[0011] This specification provides a method, device and equipment for generating customer portraits. In some embodiments, information of target customers in multiple modes can be obtained, and the information in each mode can be converted to obtain feature information corresponding to each mode. It is also possible to determine the multimodal data relationship network of the target customer based on the feature information corresponding to each mode under multiple modes, and fuse the information in multiple modes based on the multimodal data relationship network to obtain the fused feature information of the target customer. Further, a portrait of the target customer is generated based on the fused feature information and the label information of the target customer. The embodiments of this specification can not only systematically integrate and fuse data of different modes of customers, reduce manual workload, and reduce the difficulty of manually integrating and verifying data from different sources, but also predict customer portrait labels through the inherent correlation between data when some data cannot be obtained, and automatically fill in customer portrait labels, thereby making the generated customer portrait more accurate and comprehensive. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of this specification, constitute a part of this specification, and do not constitute a limitation of this specification. In the drawings:

[0013] Figure 1 It is a flowchart of an embodiment of a method for generating a customer portrait provided in this specification;

[0014] Figure 2 This is a schematic diagram of the module structure of an embodiment of a device for generating a customer portrait provided in this specification;

[0015] Figure 3 This is a hardware structure block diagram of an embodiment of a server for generating a customer portrait provided in this specification. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments in this specification, not all of the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the embodiments of this specification.

[0017] The following is an example of a specific application scenario to illustrate the implementation plan of this specification. Specifically, Figure 1 It is a flowchart of an embodiment of a method for generating a customer portrait provided in this specification. Although this specification provides the method operation steps or device structure shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial combination based on routine or no creative labor.

[0018] An implementation scheme provided in this specification can be applied to a client, a server, etc. The client can include a terminal device, such as a smart phone, a tablet computer, etc. The server can include a single computer device, or a server cluster composed of multiple servers, or a server structure of a distributed system, etc.

[0019] It should be noted that the following embodiment description does not limit the technical solutions in other application scenarios that can be extended based on this specification. Figure 1 As shown, in an embodiment of a method for generating a customer portrait provided in this specification, the method may include the following steps.

[0020] S0: Obtain information about target customers in multiple modes.

[0021] Among them, the target customer can be any customer for whom a customer portrait needs to be generated.

[0022] In some embodiments, the information in multiple modes may mainly include text, video, audio, image, etc. Of course, the above is only an exemplary description, and the information in multiple modes is not limited to the above examples. Technicians in the relevant field may make other changes under the inspiration of the technical essence of this application, but as long as the functions and effects achieved are the same or similar to those of this application, they should be covered within the protection scope of this application.

[0023] In some embodiments, customer information in multiple modes can be collected through API (application programming interface) docking with external systems, can be collected through front-end business systems, can also be collected through crawling and other methods, and this specification does not limit this.

[0024] In some implementation scenarios, after collecting and acquiring information about customers in multiple modes, the collected information can be preprocessed to remove redundant information, so that the customer profile subsequently generated based on the information is more accurate. The preprocessing may include cleaning, screening, etc.

[0025] In some implementation scenarios, the information in each mode may include one or more pieces of information. For example, the information of the target customer in multiple modes may include 50 texts, 30 videos, 20 audios, and 100 images.

[0026] S2: Convert the information under each mode to obtain the feature information corresponding to each mode.

[0027] In the embodiments of this specification, after obtaining information of target customers in multiple modes, the information in each mode can be converted to obtain feature information corresponding to each mode. Since the information in each mode can include one or more pieces of information, the feature information corresponding to each mode can include one or more pieces of information.

[0028] In some embodiments, the conversion of information under each mode to obtain characteristic information corresponding to each mode may include: obtaining first information under the first mode; converting the first information to obtain an initial numerical vector corresponding to the first information; determining the position vector of the first information based on the initial numerical vector; determining the input vector corresponding to the first information based on the initial numerical vector and the position vector; converting the input vector to a preset length based on a preset conversion model to obtain a characteristic vector of the first information; obtaining characteristic information corresponding to the first mode based on the characteristic vector of each piece of information under the first mode. The first mode may be any one of a plurality of modes. The first information may be any piece of information under the master of ceremonies mode.

[0029] In some implementation scenarios, after obtaining the first information in the first mode, the first information can be converted into a numerical form, thereby obtaining an initial numerical vector corresponding to the first information. In this specification, the method of converting the first information into a numerical form is not limited.

[0030] Since the data streams generated by text, video, audio, etc. are in order, for example, each frame of a video has a front-to-back order, in order to make the subsequent customer portraits generated based on different modal information more accurate, in some implementation scenarios, after the first information is converted into an initial numerical vector, the position vector of the first information can be determined according to the initial numerical vector. Each element in the position vector can represent the position of each element in the initial numerical vector.

[0031] In some implementation scenarios, determining the position vector of the first information based on the initial numerical vector may include: assigning an identifier to each element in the initial numerical vector in order; obtaining the identifier of the target element; when the identifier of the target element is a first identifier, determining the position information of the target element according to a first preset method; when the identifier of the target element is a second identifier, determining the position information of the target element according to a second preset method; arranging the position information of each element in the initial numerical vector according to the identifier of each element to obtain the position vector of the first information. The identifier assigned to each element in the initial numerical vector may be a numerical sequence that increases from 1, so that the identifier of each element may represent the dimension of the element in the initial numerical vector. The target element may be any element in the initial numerical vector. The first identifier may represent that the element dimension is an odd length, and the second identifier may represent that the element dimension is an even length.

[0032] In some implementation scenarios, when the identifier of the target element is the first identifier, the location information of the target element may be determined according to the following method:

[0033] PE (k,2j) = sin(k / 10000 2j / L )

[0034] Among them, PE (k,2j) represents the position information of the target element, k represents the identifier of the target element, 2j represents the even position number, and L represents the length of the feature vector.

[0035] In some implementation scenarios, when the identifier of the target element is the second identifier, the location information of the target element may be determined according to the following method:

[0036] PE (k,2j+1) =cos(k / 10000 2j / L )

[0037] Among them, PE (k,2j+1) represents the position information of the target element, k represents the identifier of the target element, 2j+1 represents the odd position number, and L represents the length of the feature vector.

[0038] In some implementation scenarios, after the position information of the target element is determined, the position information of each element in the initial numerical vector may be arranged according to the identifier of each element to obtain the position vector of the first information.

[0039] In some implementation scenarios, after obtaining the position vector of the first information, the input vector corresponding to the first information can be determined according to the initial numerical vector and the position vector. The input vector can be further converted into a feature vector of a preset length using a preset conversion model. The feature vector of the preset length can be understood as feature information corresponding to the first information. The preset length can be set according to the actual scenario, for example, it can be 10, 15, etc., and this specification does not limit this.

[0040] In some implementation scenarios, the preset conversion model may be an FFN model. The FFN model is used to convert a vector into a vector of fixed length.

[0041] Similarly, each piece of information under the first mode can be converted into a corresponding feature vector in the above manner, thereby obtaining feature information corresponding to the first mode (also referred to as unimodal feature information). Similarly, feature information corresponding to each mode can be obtained in the above manner. Among them, the feature information corresponding to each mode can include one or more feature vectors, and the length of each feature vector is the same. The length of the feature vectors in the feature information of different modes is the same.

[0042] In some implementation scenarios, after obtaining the feature information corresponding to each modality, all feature vectors included in the feature information may be sorted according to the generation time of each feature vector. In some implementation scenarios, after obtaining the feature information corresponding to each modality, all feature vectors included in the feature information may also be sorted according to the acquisition time of the acquisition information corresponding to each feature vector. The result of sorting by acquisition time is the same as the result of sorting by generation time.

[0043] The above implementation is described below using a specific implementation scenario. The target customer's multiple modal information includes four categories: text (F), video (O), audio (A), and image (I). The FFN model is used to convert the information in each modality into a feature vector of length L.

[0044] Specifically, in this embodiment, each category of information can be converted into a corresponding initial numerical vector, where the text initial numerical vector is denoted as V input_f ,input_f=1,…,F′,video initial numerical vector V input_o , input_o=1,…,O′, the initial audio value vector is denoted as V input_a , input_a=1,…,A′, the initial numerical vector of the image is denoted as V input_i, input_i=1,…,I′, F′ represents the number of text initial numerical vectors, O′ represents the number of video initial numerical vectors, A′ represents the number of audio initial numerical vectors, and I′ represents the number of image initial numerical vectors. Among them, F′, O′, A′, and I′ can be the same or different. The length of the text initial numerical vector, the length of the video initial numerical vector, the length of the audio initial numerical vector, and the length of the image initial numerical vector can be the same or different.

[0045] Furthermore, the corresponding position vector can be determined based on the initial numerical vector. input =[x1,…,x K ], we can determine the corresponding position element according to the corresponding dimension of each element and obtain the corresponding position vector P = [p1,…,p K ]. The element with even dimension length is sin, and the element with odd dimension length is cos. For example, for element x2, its subscript is the corresponding identifier, that is, k=2 is an even number, that is, 2j=2, at this time, PE can be used (k,2j) = sin(k / 10000 2j / L ) Calculate the position element p2 corresponding to x2. For element x5, its subscript is the corresponding identifier, that is, k=5 is an odd number, that is, 2j+1=5, at this time, PE (k,2j+1) =cos(k / 10000 2j / L ) Calculate the position element p5 corresponding to x5.

[0046] Furthermore, the input vector can be obtained according to the initial vector and the corresponding position vector. input +P=[x1+p1,…,x k +p k ]Get the input vector.

[0047] Furthermore, the FFN model can be used to convert the input vector into a feature vector of length L. The feature vector corresponding to the text class can be recorded as V f , f=1,…,F, the feature vector corresponding to the video class can be recorded as V o , o=1,…,O, the feature vector corresponding to the audio class can be recorded as V a , a=1,…,A, the feature vector corresponding to the image class can be recorded as V i, i=1,…,I, F represents the number of text feature vectors, O represents the number of video feature vectors, A represents the number of audio feature vectors, and I represents the number of image feature vectors. Among them, F, O, A, and I can be the same or different. Since multiple input vectors can be merged into one feature vector, F and F′, O and O′, A and A′, and I and I′ can be the same or different. Among them, the length of the text feature vector, the length of the video feature vector, the length of the audio feature vector, and the length of the image feature vector are all L.

[0048] In this embodiment, after the feature vector corresponding to each category is obtained, all feature vectors may be sorted according to the generation time of each feature vector.

[0049] In the embodiments of the present specification, by converting the information in each mode, the information of the target user in different modes can be converted into feature vectors of the same length, thereby improving the efficiency of subsequent processing.

[0050] S4: Determine a multimodal data relationship network of the target customer based on feature information corresponding to each modality under multiple modalities.

[0051] In the embodiments of this specification, after the feature information corresponding to each modality is obtained, the multimodal data relationship network of the target customer can be determined according to the feature information corresponding to each modality under multiple modalities.

[0052] In some embodiments, determining the multimodal data relationship network of the target customer based on the feature information corresponding to each modality under multiple modalities may include: taking the feature vector of each information under each modality as a point to obtain a point set associated with the target customer; determining the direction information of the edge based on the generation time of each point; and connecting the points in the point set based on the direction information to obtain the multimodal data relationship network of the target customer.

[0053] In some implementation scenarios, after obtaining the feature information corresponding to each modality, points and edges can be defined based on the feature vectors of each piece of information under each modality, thereby obtaining a multimodal data relationship network of the target customers.

[0054] For example, in some implementation scenarios, after obtaining the text feature vector V f , video class feature vector V o , audio feature vector V a , image class feature vector V i After that, different V f , V o , V a , V i Defined as points, thus obtaining the point set associated with the target customer.

[0055] In some implementation scenarios, the predefined edge categories can be expressed as Among them, F, O, A, and I represent the points corresponding to the text feature vector, the points corresponding to the video feature vector, the points corresponding to the audio feature vector, and the points corresponding to the image feature vector, respectively.

[0056] In some implementation scenarios, after the edge category is defined, the direction information of the edge can be determined according to the generation time of each point. The direction information indicates the direction of the edge. In this embodiment, the direction of the edge can be from a point with an earlier generation time to a point with a later generation time.

[0057] In some implementation scenarios, after defining the edge categories, a weight may be assigned to each category of the edge.

[0058] In some implementation scenarios, after determining the points, edges, and edge directions, the point-edge relationships can be connected based on this information to obtain a multimodal data relationship network for each customer.

[0059] S6: Based on the multimodal data relationship network, information under multiple modes is integrated to obtain integrated feature information of the target customer.

[0060] In the embodiments of this specification, after obtaining the multimodal data relationship network of the target customer, information in multiple modes can be fused based on the multimodal data relationship network to obtain fused feature information of the target customer.

[0061] In some embodiments, fusing information under multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer may include: obtaining a set of associated points corresponding to the target point in the multimodal data relationship network; wherein the generation time of each point in the associated point set satisfies a preset relationship with the generation time of the target point; determining the multimodal feature vector of the target point based on the feature vector corresponding to the target point and the feature vector of each associated point in the associated point set; and determining the fused feature information of the target customer based on the multimodal feature vector of each point in the multimodal data relationship network.

[0062] The target point may be any point in the multimodal data relationship network. The associated point set corresponding to the target point may include one or more points associated with the target point.

[0063] In some implementation scenarios, the preset relationship between the generation time of each point in the associated point set and the generation time of the target point can be understood as taking the generation time of the target point as the starting point, and the generation time of each point in the associated point set is within the range from the starting point to the preset time point. For example, if the generation time of the target point is January 3, and the preset time point is January 6, then the text feature vectors, video feature vectors, audio feature vectors, and image feature vectors generated between January 3 and January 6 are all points associated with the target point, and the set of these points is the associated point set corresponding to the target point.

[0064] In some implementation scenarios, before determining the multimodal feature vector of the target point based on the feature vector corresponding to the target point and the feature vector of each associated point in the associated point set, the method may include: determining the associated edge set corresponding to the target point based on the associated point set; determining the weight of each edge in the associated edge set based on the weight pre-assigned to each category of edges; and normalizing the weight of each edge in the associated edge set to obtain the normalized weight of each edge in the associated edge set.

[0065] In some implementation scenarios, the target point can be recorded as dot d , and its corresponding set of associated points can be recorded as N d ={dot d,1 ,…,dot d,N}, dot d,1 Indicates the first associated point, dot d,N In this way, the line connecting the target point and each point in the associated point set is taken as an edge, and the set of all edges can be understood as the associated edge set corresponding to the target point.

[0066] In some implementation scenarios, after determining the associated point set corresponding to the target point, the weight of each edge in the associated edge set can be determined according to the weight pre-assigned to each category of edge. The weight of each edge in the associated edge set can be further normalized according to the following formula:

[0067]

[0068] Among them, β d,n Represents the target point and N d The normalized weight of the edge corresponding to the nth associated point in , Indicates the target point dot d With N d The weight of the {φ,π} type relationship between the nth associated point and the corresponding edge, φ∈Φ, π indicates the direction of the edge, Target point dot d With N dThe weight of the {φ,π} type relationship between the edges corresponding to the n′th associated point in .

[0069] In some implementation scenarios, after obtaining the normalized weight of each edge in the associated edge set, the multimodal feature vector of the target point can be determined in the following manner:

[0070]

[0071] Among them, MV d,v Represents the multimodal feature vector of the target point, V d,v Represents the feature vector corresponding to the target point, n represents the sequence number, N d represents the associated edge set corresponding to the target point, β d,n Represents the target point and N d The normalized weight of the edge corresponding to the nth associated point in V d,n,v Represents the feature vector of the nth associated point in the associated point set, mv d,v,1 Represents the first element in the multimodal feature vector, mv d,v,L represents the Lth element in the multimodal feature vector, L represents the vector length, v∈{f,o,a,i}, f, o, a, i represent the points corresponding to the text feature vector, the video feature vector, the audio feature vector, and the image feature vector, respectively.

[0072] Similarly, the multimodal feature vector of each point in the multimodal data relationship network can be obtained in a similar manner as described above, and then the multimodal feature vectors corresponding to all points in the multimodal data relationship network are fused to obtain the fused feature information of the target customer.

[0073] In some implementation scenarios, the fusion feature information of the target customer can be determined by the following methods:

[0074]

[0075] Among them, G g Represents the fusion feature information of the target customer, MV d,v Represents the multimodal feature vector of a point. The conunt function is used to count the number of multimodal feature vectors in the multimodal data relationship network. ∑MV d,v Represents the sum of the multimodal feature vectors of all points in the multimodal data relationship network.

[0076] S8: Generate a portrait of the target customer based on the fused feature information and the label information of the target customer.

[0077] In the embodiments of the present specification, after obtaining the fused feature information of the target customer, a portrait of the target customer can be generated based on the fused feature information and the label information of the target customer.

[0078] In some embodiments, generating a portrait of the target customer based on the fused feature information and the label information of the target customer may include: obtaining the label information of the target customer; wherein the label information includes a first label and a second label, the first label has a label value, and the second label has no label value; obtaining a label vector corresponding to the first label based on a dictionary value corresponding to the first label and a label value corresponding to the first label; determining a label vector corresponding to the second label based on the label vector corresponding to the first label and the fused feature information; and merging the label vector corresponding to the first label with the label vector corresponding to the second label to generate a portrait of the target customer.

[0079] In some implementation scenarios, the label information may be pre-set label indicators that describe the customer portrait, such as gender, age, shopping preferences, education level, asset level, investment preferences, etc.

[0080] Because some of the pre-set label indicators for a customer may have label values ​​and some may not, there will be incomplete information on some dimensions in the customer portrait. In order to make the generated customer portrait more complete and more accurate, in some implementation scenarios, OneHotVector is used to convert label indicators with label values ​​into label vectors. For example, taking gender as an example, gender generally has three dictionary values: male, female, and unknown. The length of OneHotVector is three. Assuming that the target customer is female, the label vector converted from the gender label using OneHotVector is [0,1,0]. Among them, the dictionary value corresponding to each label indicator can be determined according to the actual business.

[0081] In some implementation scenarios, after determining the label vector corresponding to the label indicator with a label value (hereinafter referred to as "known label"), the label vector corresponding to the label indicator without a label value can be determined based on the label vector and the fused feature information. In the following, the label vector corresponding to the label indicator with a label value is referred to as "known label", and the label vector corresponding to the label indicator without a label value is referred to as "missing label".

[0082] In some implementation scenarios, based on the random forest model, the model can be pre-trained using the label information of customers whose label indicators all have label values ​​and the corresponding fusion feature information to obtain the corresponding label prediction model.

[0083] For example, if the number of customer label indicators is three, the corresponding label vectors can be recorded as: Then the fusion feature information G of each customer can be obtained according to the above method g , then G gMerge with two of the label vectors respectively, train the random forest model, obtain model 1, model 2, and model 3, and then use model 1, model 2, and model 3 to predict the remaining label vector respectively. Among them, the input of model 1 is G g With V A,g 、V B,g The merged vector is denoted as Accordingly, Model 1 can be used to predict V C,g ; The input of model 2 is G g With V A,g 、V C,g The merged vector is denoted as Accordingly, model 2 can be used to predict V B,g ; The input of model 3 is G g With V B,g 、V C,g The merged vector is denoted as Accordingly, model 3 can be used to predict V A,g . It can represent dimensional expansion, for example, G g is [1,2,3], V A,g is [3,4,5], V B,g is [6,7], then [1,2,3,3,4,5,6,7]. It should be noted that in some implementation scenarios, when training the label prediction model, n-1 label vectors and fused feature information are usually combined for training, and then the remaining 1 label vector is predicted. Wherein, n represents the total number of label vectors. Of course, the above is only an exemplary description, and the embodiments of this specification are not limited to the above examples. For example, the more important k vectors among the n label vectors can be combined with the fused feature information to train the model, and then the more important m label vectors among the remaining label vectors are predicted (wherein n represents the total number of label vectors, and the sum of k and m is less than or equal to n). Under the inspiration of the technical essence of this application, technical personnel in the relevant field may also make other changes, but as long as the functions and effects achieved are the same or similar to those of this application, they should be included in the scope of protection of this application. In addition, the number of models trained can be determined according to the actual scenario. For example, there are a total of 3 label vectors, but the trained models can be 3, 2 or 1, and this specification does not limit this.

[0084] In some implementation scenarios, after obtaining the label prediction model, the missing labels of the target customers can be predicted based on the label prediction model by using the known labels of the target customers and the fused feature information.

[0085] For example, the three label vectors of the target customers are V A,g , V B,g , V C,g , assuming VA,g is a known label, V B,g 、V C,g If the label is missing, the fusion feature information G of the target customer can be g With V A,g Merge (where the merge result can be recorded as ) Input model 1 to obtain V C,g,0 , the fusion feature information G of the target customer g With V A,g Merge (where the merge result can be recorded as ) Input model 2 to obtain V B,g，0 Among them, V C,g,0 、V B,g，0 They represent the results obtained from the initial prediction.

[0086] Furthermore, we can set V A,g，0 =V A,g , repeat Q rounds, each round uses the V obtained in the previous round B,g,q 、V C,g,q , respectively into model 1, model 2, and model 3, that is, in model 1, input Prediction V C,g,q , in the model input Prediction V B,g,q , input in model 3 Prediction V A,g,q , where q represents the number of cycles.

[0087] When |V A,g,q -V A,g When |≤ε, the prediction is completed, and the customer missing label can be obtained, that is, V B,g =V B,g,Q , V C,g =V C,g,Q Among them, ε can be set according to the actual scenario, and this specification does not limit this.

[0088] In some implementation scenarios, after obtaining the missing tag of the customer, the missing tag V B,g , V C,g With known label V A,g Merge to get a complete customer label system, namely V A,g 、V B,g , V C,g The value of can represent the profile of the target customer.

[0089] The embodiments of this specification can be applied to the label prediction and completion of customer portraits.

[0090] The embodiments of this specification can not only reduce manual workload and the difficulty of manually integrating and verifying data from different sources by systematically integrating and fusing customer data of different modalities, but also predict customer portrait labels through the inherent correlation between data when some data cannot be obtained, and automatically fill in customer portrait labels, thereby making the generated customer portrait more accurate and comprehensive.

[0091] In the embodiments of this specification, by automatically integrating the collected data in different modalities such as voice, text, image, video, etc., the missing dimensions in the customer portrait can be automatically filled and a complete customer portrait can be generated, thereby solving the problem that some information may not be able to be collected directly from customers and is difficult to complete manually.

[0092] In the embodiments of this specification, by fusing and complementing the customer's multi-modal data, the quality of the generated customer portrait can be improved, providing support for banks to provide personalized financial services to each individual.

[0093] Of course, the above is only an exemplary description, and the embodiments of this specification are not limited to the above examples. Technical personnel in the relevant field may make other changes based on the technical essence of this application. However, as long as the functions and effects achieved are the same or similar to those of this application, they should be included in the scope of protection of this application.

[0094] From the above description, it can be seen that the embodiments of the present application can obtain information about target customers in multiple modes, convert the information in each mode, and obtain feature information corresponding to each mode. It is also possible to determine the multimodal data relationship network of the target customer based on the feature information corresponding to each mode under multiple modes, fuse the information in multiple modes based on the multimodal data relationship network, and obtain the fused feature information of the target customer. Further, a portrait of the target customer is generated based on the fused feature information and the label information of the target customer. The embodiments of this specification can not only systematically integrate and fuse data of different modes of customers, reduce manual workload, and reduce the difficulty of manually integrating and verifying data from different sources, but also predict customer portrait labels through the inherent correlation between data when some data cannot be obtained, and automatically fill in customer portrait labels, thereby making the generated customer portrait more accurate and comprehensive.

[0095] In this specification, each embodiment of the above method is described in a progressive manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. For relevant parts, refer to the partial description of the method embodiment.

[0096] Based on the above-mentioned method for generating a customer portrait, one or more embodiments of this specification also provide a device for generating a customer portrait. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiment of this specification and a device combined with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware of predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0097] Specifically, Figure 2 is a schematic diagram of a module structure of an embodiment of a device for generating a customer portrait provided in this specification, such as Figure 2 As shown, a device for generating a customer portrait provided in this specification may include: an acquisition module 120, a conversion module 122, a determination module 124, a fusion module 126, and a generation module 128.

[0098] The acquisition module 120 can be used to acquire information of target customers in multiple modes;

[0099] The conversion module 122 may be used to convert information in each mode to obtain feature information corresponding to each mode;

[0100] The determination module 124 may be used to determine the multimodal data relationship network of the target customer according to the feature information corresponding to each modality under the multiple modalities;

[0101] A fusion module 126 may be used to fuse information in multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer;

[0102] The generation module 128 can be used to generate a portrait of the target customer based on the fused feature information and the label information of the target customer.

[0103] It should be noted that the above-mentioned device may also include other implementation modes according to the description of the method embodiment. The specific implementation mode may refer to the description of the relevant method embodiment, which will not be described one by one here.

[0104] This specification also provides an embodiment of a device for generating a customer portrait, including a processor and a memory for storing processor executable instructions, wherein the instructions, when executed by the processor, implement the following steps: obtaining information of a target customer in multiple modalities; converting the information in each modality to obtain feature information corresponding to each modality; determining a multimodal data relationship network of the target customer based on the feature information corresponding to each modality in the multiple modalities; fusing the information in multiple modalities based on the multimodal data relationship network to obtain fused feature information of the target customer; and generating a portrait of the target customer based on the fused feature information and the label information of the target customer.

[0105] It should be noted that the above-mentioned device may also include other implementations according to the description of the method or device embodiment. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0106] The method embodiments provided in this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 3 This is a hardware structure block diagram of an embodiment of a server for generating a customer portrait provided in this specification. The server may be the device for generating a customer portrait or the equipment for generating a customer portrait in the above embodiment. Figure 3 As shown, the server 10 may include one or more (only one is shown in the figure) processors 100 (the processor 100 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 200 for storing data, and a transmission module 300 for communication functions. A person skilled in the art can understand that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components shown in the figure may also include other processing hardware, such as a database or multi-level cache, GPU, or other hardware with Figure 3 Different configurations are shown.

[0107] The memory 200 can be used to store software programs and modules of application software, such as program instructions / modules corresponding to the method for generating customer portraits in the embodiment of this specification. The processor 100 executes various functional applications and data processing by running the software programs and modules stored in the memory 200. The memory 200 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 200 may further include a memory remotely arranged relative to the processor 100, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0108] The transmission module 300 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission module 300 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 300 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0109] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0110] The methods or devices described in the above embodiments provided in this specification can implement business logic through computer programs and record them on storage media, and the storage media can be read and executed by computers to achieve the effects of the solutions described in the embodiments of this specification. The storage medium may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical methods. The storage medium may include: devices that use electrical energy to store information, such as various types of memories, such as RAM, ROM, etc.; devices that use magnetic energy to store information, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and U disks; devices that use optical methods to store information, such as CDs or DVDs. Of course, there are other types of readable storage media, such as quantum memories, graphene memories, and so on.

[0111] The above-mentioned method or device embodiment for generating customer portraits provided in this specification can be implemented by a processor in a computer executing corresponding program instructions, such as using the C++ language of the Windows operating system to implement it on a PC, on a Linux system, or other methods such as using Android or iOS system programming languages ​​to implement it on a smart terminal, as well as based on the processing logic of a quantum computer.

[0112] It should be noted that the devices, equipment, and systems described in the specification may also include other implementation methods according to the description of the relevant method embodiments. The specific implementation methods can refer to the description of the corresponding method embodiments, which will not be described one by one here.

[0113] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0114] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of some modules can be implemented in the same or more software and / or hardware, and the modules that implement the same function can also be implemented by a combination of multiple sub-modules or sub-units.

[0115] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and systems according to the embodiments of the present invention. It should be understood that it can be implemented by computer program instructions, and these computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the specified function. These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, and the instruction device is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] Those skilled in the art will appreciate that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may be provided in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware.

[0117] The above is only an example of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the scope of the claims.

Claims

1. A method for generating a customer portrait, characterized in that: include: Obtain information about target customers in multiple modes; The information in the multiple modalities includes text, video, audio and image; Transform the information under each mode to obtain the feature information corresponding to each mode; Determine a multimodal data relationship network of the target customer according to feature information corresponding to each modality under multiple modalities; Based on the multimodal data relationship network, information under multiple modes is integrated to obtain integrated feature information of the target customer; Generate a profile of the target customer based on the fused feature information and the label information of the target customer; Wherein, determining the multimodal data relationship network of the target customer according to the feature information corresponding to each mode under multiple modes includes: The text feature vector V f , video class feature vector V o , audio feature vector V a , image class feature vector V i Defined as a point; The edge category is represented as Among them, F, O, A, and I represent the points corresponding to the text feature vector, the points corresponding to the video feature vector, the points corresponding to the audio feature vector, and the points corresponding to the image feature vector, respectively; After defining the edge category, the direction information of the edge is determined according to the generation time of each point, where the direction information indicates the direction of the edge; After determining the points, edges, and edge directions, connect the point-edge relationships to obtain the multimodal data relationship network of each customer.

2. The method according to claim 1, characterized in that Each mode includes at least one piece of information; The information under each mode is converted to obtain feature information corresponding to each mode, including: Obtaining first information in a first mode; Converting the first information to obtain an initial numerical vector corresponding to the first information; Determining a position vector of the first information according to the initial numerical vector; Determine an input vector corresponding to the first information according to the initial numerical vector and the position vector; Converting the input vector to a preset length based on a preset conversion model to obtain a feature vector of the first information; Based on the feature vector of each piece of information in the first modality, feature information corresponding to the first modality is obtained.

3. The method according to claim 2, characterized in that The step of determining the position vector of the first information according to the initial numerical vector includes: Assigning an identifier to each element in the initial numerical vector in order; Get the target element's ID; When the identifier of the target element is the first identifier, determining the position information of the target element according to a first preset method; When the identifier of the target element is the second identifier, determining the position information of the target element according to a second preset method; The position information of each element in the initial numerical vector is arranged according to the identifier of each element to obtain the position vector of the first information.

4. The method according to claim 1, characterized in that The fusing information in multiple modes based on the multimodal data relationship network to obtain fused feature information of the target customer includes: Acquire a set of associated points corresponding to the target point in the multimodal data relationship network; wherein the generation time of each point in the associated point set satisfies a preset relationship with the generation time of the target point; Determine a multimodal feature vector of the target point according to the feature vector corresponding to the target point and the feature vector of each associated point in the associated point set; Based on the multimodal feature vector of each point in the multimodal data relationship network, the fused feature information of the target customer is determined.

5. The method according to claim 4, characterized in that Before determining the multimodal feature vector of the target point according to the feature vector corresponding to the target point and the feature vector of each associated point in the associated point set, the method includes: Determine, according to the associated point set, an associated edge set corresponding to the target point; Determine the weight of each edge in the associated edge set according to the weight pre-assigned to each category of edges; The weight of each edge in the associated edge set is normalized to obtain the normalized weight of each edge in the associated edge set.

6. The method according to claim 5, characterized in that The multimodal feature vector of the target point is determined as follows: Among them, MV d,v Represents the multimodal feature vector of the target point, V d,v Represents the feature vector corresponding to the target point, n represents the sequence number, N d represents the associated edge set corresponding to the target point, β d,n Represents the target point and N d The normalized weight of the edge corresponding to the nth associated point in V d,n,v Represents the feature vector of the nth associated point in the associated point set, mv d,v,1 Represents the first element in the multimodal feature vector, mv d,v,L Represents the Lth element in the multimodal feature vector, where L represents the vector length.

7. The method according to claim 1, characterized in that The generating a portrait of the target customer according to the fused feature information and the label information of the target customer includes: Acquire tag information of the target customer; wherein the tag information includes a first tag and a second tag, the first tag has a tag value, and the second tag has no tag value; Obtaining a label vector corresponding to the first label according to the dictionary value corresponding to the first label and the label value corresponding to the first label; Determine a label vector corresponding to the second label based on the label vector corresponding to the first label and the fused feature information; The label vector corresponding to the first label is combined with the label vector corresponding to the second label to generate a profile of the target customer.

8. A device for generating a customer portrait, characterized in that: include: The acquisition module is used to obtain information about target customers in multiple modes; The information in the multiple modalities includes text, video, audio and image; A conversion module is used to convert the information under each mode to obtain the feature information corresponding to each mode; A determination module, used to determine the multimodal data relationship network of the target customer according to the feature information corresponding to each mode under multiple modes; A fusion module, used to fuse information under multiple modes based on the multimodal data relationship network to obtain fusion feature information of the target customer; A generating module, used for generating a portrait of the target customer according to the fused feature information and the label information of the target customer; Wherein, determining the multimodal data relationship network of the target customer according to the feature information corresponding to each mode under multiple modes includes: The text feature vector V f , video class feature vector V o , audio feature vector V a , image class feature vector V i Defined as a point; The edge category is represented as Among them, F, O, A, and I represent the points corresponding to the text feature vector, the points corresponding to the video feature vector, the points corresponding to the audio feature vector, and the points corresponding to the image feature vector, respectively; After defining the edge category, the direction information of the edge is determined according to the generation time of each point, where the direction information indicates the direction of the edge; After determining the points, edges, and edge directions, connect the point-edge relationships to obtain the multimodal data relationship network of each customer.

9. A device for generating a customer portrait, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the instructions.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the instructions are executed, the steps of the method described in any one of claims 1 to 7 are implemented.

Images