Claim settlement processing method and device, claim settlement model training method and device and storage medium

Abstract

The embodiment of the invention provides a claim settlement processing method, a claim settlement model training method, equipment and a storage medium. For a claim settlement request, image claim settlement credentials and text claim settlement credentials corresponding to the claim settlement request can be obtained, and image features of the image claim settlement credentials and text features of the text claim settlement credentials are extracted by using the claim settlement auditing model. And the claim settlement auditing model can execute the classification task by using the association degree of the image features and the text features to obtain a claim settlement auditing result. Therefore, on one hand, the claim settlement auditing model extracts multi-modal features from multiple dimensions such as vision and text, and executes a classification task based on the association degree of the multi-modal features, so that whether the claim settlement request of the user corresponds to a reasonable claim settlement basis can be accurately identified, and an auditing result can be accurately output; on the other hand, on the premise that the multi-modal features are extracted, the claim audit model can quickly infer the claim audit result in an end-to-end mode according to the association degree of the multi-modal features, the inference process is simplified, and the claim audit efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a claims processing, model training method, device and storage medium. Background Art

[0002] With the popularization of takeout and community group purchase, users can buy takeout meals, fruits and vegetables online, and have them delivered to their door offline by couriers. As a special consumer product, food may be damaged, bumped, or spoiled during the consumption process due to stale raw ingredients, short shelf life, unreasonable food packaging, or improper transportation, which may lead to consumer complaints and claims.

[0003] In the traditional claims process, users need to submit various credentials, such as image credentials and text report descriptions. Claims adjusters need to obtain user claims from the text report descriptions and check whether the submitted image credentials can effectively prove the rationality of their claims. However, this claims processing method relies on manual participation, so the detection efficiency is low. Therefore, a new solution needs to be proposed. Summary of the invention

[0004] Multiple aspects of the present application provide a claims processing, model training method, device and storage medium to reduce the reliance of claims review operations on manual labor and improve the efficiency of claims review.

[0005] An embodiment of the present application provides a claims processing method, including: responding to a claim request, obtaining a multimodal claims credential corresponding to the claim request; inputting the multimodal claims credential into a claims review model to extract multimodal features corresponding to the multimodal claims credential; performing a classification task according to the correlation of the multimodal features to obtain a claims review result; the correlation is used to describe the degree of semantic matching of features of different modes in the multimodal features; wherein the claims review model is obtained based on joint training of multimodal samples.

[0006] Optionally, the multimodal claim voucher includes: a text claim voucher and at least one image claim voucher; the multimodal claim voucher is input into a claim review model to extract multimodal features corresponding to the multimodal claim voucher, including: in the claim review model, using a text encoder to feature encode the text claim voucher to obtain a text feature vector; and, using an image encoder to feature encode the at least one image claim voucher to obtain at least one image feature vector; wherein the text encoder and the image encoding are jointly trained based on text-image pairs using a contrastive learning training method.

[0007] Optionally, a classification task is performed according to the correlation of the multimodal features, including: utilizing a matching sub-model in the claim review model to perform feature matching on the text feature vector and the at least one image feature vector to obtain at least one matching score; and outputting a claim review result according to the at least one matching score and a set score condition.

[0008] Optionally, feature matching is performed on the text feature vector and the at least one image feature vector to obtain at least one matching score, including: determining at least one feature vector group; including the text feature vector and any image feature vector in any feature vector group; and respectively calculating the similarity of the feature vectors in the at least one feature vector group to obtain the at least one matching score.

[0009] Optionally, based on the at least one matching score and the set score conditions, a claim review result is output, including: selecting a target matching score that meets the set score conditions from the at least one matching score; if the target matching score is greater than a preset score threshold, outputting a claim approval review result; if the target matching score is less than or equal to the preset score threshold, outputting a claim rejection review result.

[0010] Optionally, before using the claim review model to extract the text features of the text claim voucher and the image features of the at least one image claim voucher, it also includes: obtaining multiple training data groups; any training data group includes: a text claim voucher sample, an image claim voucher sample and a true value of the claim result of any historical claim request; determining a pre-trained neural network model; the neural network model is pre-trained using a text-image pair pre-training method; inputting the multiple training data groups into the neural network model respectively to obtain the claim review results of each of the multiple training data groups; the neural network model is used to extract the text features of the text claim voucher sample and the image features of the image claim voucher sample in any training data group, and perform a classification task based on the correlation between the text features and the image features to obtain the claim review results of the training data group; obtaining the difference between the claim review results of each of the multiple training data groups and the true value of the claim result; with the goal of converging the difference, fine-tuning the parameters of the neural network model, and when the difference converges to a specified range, outputting the neural network model as the claim review model.

[0011] An embodiment of the present application also provides a claims processing method, including: responding to a claim request for a food order, obtaining a text claim credential and at least one image claim credential corresponding to the claim request; inputting the text claim credential and at least one image claim credential into a claims review model to extract text features of the text claim credential and image features of the at least one image claim credential; performing a classification task based on the correlation between the text features and the image features to obtain a claims review result; the correlation is used to describe the degree of semantic matching between the text features and the image features; wherein the claims review model is obtained based on joint training of text-image sample pairs.

[0012] The embodiment of the present application also provides a model training method, including: obtaining multiple training data groups; any training data group includes: a text claim voucher sample, an image claim voucher sample and a true value of a claim result of any historical claim request; determining a pre-trained neural network model; the neural network model is pre-trained using a text-image pair pre-training method; the multiple training data groups are respectively input into the neural network model to obtain the claim review results of each of the multiple training data groups; the neural network model is used to extract the text features of the text claim voucher sample and the image features of the image claim voucher sample in any training data group, and perform a classification task based on the correlation between the text features and the image features to obtain the claim review results of the training data group; obtaining the difference between the claim review results of each of the multiple training data groups and the true value of the claim result; with the goal of converging the difference, fine-tuning the parameters of the neural network model, and when the difference converges to a specified range, outputting the neural network model as the claim review model.

[0013] Optionally, determining a pre-trained neural network model includes: obtaining a plurality of sample pairs based on a plurality of text-image pairs, the plurality of sample pairs including positive sample pairs and negative sample pairs; inputting the plurality of sample pairs into the neural network model; using an image encoder and a text encoder in the neural network model to respectively encode the text and the image in the plurality of sample pairs to obtain text features and image features of the plurality of sample pairs; determining the correlation between the text features and the image features of the plurality of sample pairs; and updating the parameters of the image encoder and the text encoder with the goal of reducing the correlation between the image features and the text features of the positive sample pairs and increasing the correlation between the image features and the text features of the negative sample pairs to pre-train the neural network model.

[0014] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.

[0015] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed by a processor.

[0016] In an embodiment of the present application, for a claim request, an image claim voucher and a text claim voucher corresponding to the claim request can be obtained, and the claim review model can be used to extract the image features of the image claim voucher and the text features of the text claim voucher. The claim review model can use the correlation between the image features and the text features to perform classification tasks and obtain the claim review results. In this embodiment, on the one hand, the claim review model extracts multimodal features from multiple dimensions such as vision and text, and performs classification tasks based on the correlation of multimodal features, which can accurately identify whether the user's claim request corresponds to a reasonable claim basis and thus accurately output the review results; on the other hand, under the premise of extracting multimodal features, the claim review model can quickly infer the claim review results "end-to-end" according to the correlation of multimodal features, which simplifies the reasoning process and improves the efficiency of claim review. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A flowchart of a claims processing method provided for an exemplary embodiment of the present application;

[0019] Figure 2 A schematic diagram of the structure and processing flow of a claims review model provided for an exemplary embodiment of the present application;

[0020] Figure 3 A flowchart of a model training method provided for an exemplary embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure and training process of a claims review model provided for an exemplary embodiment of the present application;

[0022] Figure 5 A flowchart of a claims processing method provided for an exemplary embodiment of the present application;

[0023] Figure 6A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0025] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0026] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0027] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0028] In order to solve the technical problem that the claims processing method in the prior art relies on manual operation and thus has low detection efficiency, a solution is provided in some embodiments of the present application. The technical solutions provided in each embodiment of the present application are described in detail below in conjunction with the accompanying drawings.

[0029] Figure 1 is a flowchart of a claim processing method provided by an exemplary embodiment of the present application, which may include: Figure 1 Steps shown:

[0030] Step 101: respond to a claim request for a food order, and obtain a text claim voucher and at least one image claim voucher corresponding to the claim request.

[0031] Step 102: input the text claim voucher and at least one image claim voucher into a claim review model to extract text features of the text claim voucher and image features of the at least one image claim voucher.

[0032] Step 103, performing a classification task according to the correlation between the text features and the image features to obtain a claim review result; the correlation is used to describe the degree of semantic matching between the text features and the image features; the claim review model is obtained by joint training of text-image sample pairs.

[0033] The execution subject of the embodiment of the present application can be any electronic device, on which a claims processing device can be run, and the claims processing device can be implemented as an independently running application or as a plug-in. Among them, the electronic device can be implemented as a terminal device on the user side, such as a mobile phone, a tablet computer, etc., and the electronic device can also be implemented as a server, such as a conventional server, a cloud server or a virtualized data center, etc., which is not limited in this embodiment. Among them, when deployed on the user side, the claims processing device can run on the ARM (Advanced RISC Machines, RISC (Reduced Instruction Set Computer) microprocessor), CPU (Central Processing Unit, Central Processing Unit), GPU (Graphics Processing Unit, Graphics Processing Unit), FPGA (Field Programmable Gate Array, Field Programmable Gate Array), DSP (Digital Signal Processing, Digital Signal Processing) and other computing units on the electronic device, which is not limited in this embodiment.

[0034] Among them, the claim processing device can be applied to the claim review service in the after-sales scenario of food orders, and the after-sales scenario of the food order includes: after-sales scenario of catering takeaway, after-sales scenario of fresh takeaway, etc. The after-sales scenario of the above-mentioned food orders is a high-incidence scenario for claims. Based on the claim processing device, claims review can be efficiently performed to determine whether to make a claim for the food order. The following will be explained in detail.

[0035] The claim processing device can obtain the claim request for the food order. Optionally, if the claim processing device runs on the terminal device side, the claim processing device can detect the claim request for the food order based on the user's operation on the terminal device. For example, the claim processing device can provide an interface on which the food order and the claim application control corresponding to the food order are displayed. The user can initiate a claim request for any food order by triggering the claim application control corresponding to the food order. Alternatively, optionally, if the claim processing device runs on the terminal device side, the claim processing device can obtain the claim request for the food order based on the call operation of other applications on the terminal device (such as a takeaway application or a fresh food ordering application).

[0036] Optionally, if the claim processing device runs on the server side, the claim processing device can communicate and interact with the terminal device to obtain the claim request for the food order sent by the terminal device. In this embodiment, the claim processing device running on the server can provide a shared claim review service to multiple terminal devices to reduce the occupation of terminal resources.

[0037] Among them, the claim processing device can respond to the claim request of the food order and obtain the claim voucher corresponding to the claim request. Among them, the claim voucher is used to describe the user's claim demand and the basis for initiating the claim. Among them, the claim voucher can be a multimodal claim voucher to obtain the user's claim demand and claim description information from multiple dimensions. Among them, multimodality is used to represent data forms in different forms, such as text, image, audio, video and other modalities. In this embodiment, the multimodal claim voucher may at least include: a claim voucher in text mode and a claim voucher in image mode, that is, a text claim voucher and an image claim voucher. Of course, in other embodiments, the multimodal claim voucher may also include: a claim voucher in audio mode (such as the vibration sound produced by slapping a watermelon) or a claim voucher in other modes.

[0038] The text claim voucher may include a claim case description entered by the user through voice or keyboard, and the claim case description may generally include at least one of the key fields of food name, food price, merchant name, after-sales claim, and purchase record. For example, the text claim voucher may be "I bought a cup of fruit tea at A beverage store, but there was a bug in it."

[0039] Among them, the image claim voucher may include the claim basis input by the user, which may be at least one of the product pictures taken by the user of the purchased product, the screenshots of the negotiation records with the merchant, and the screenshots of the order. Among them, the product pictures may include one or more of the details of product damage, details of foreign objects, and details of deterioration. In the after-sales scenario of food orders, product images can more realistically reflect the problems with the food, and thus can be used as the basis for applying for claims.

[0040] After the claim processing device obtains the above-mentioned multimodal claim vouchers, it can use the claim review model to perform automatic claim review. Among them, the input of the claim review model is the multimodal claim voucher, and the output is the claim review result. The claim review result can be inferred "end-to-end" at one time, which is efficient and fast. The claim model is obtained by joint training based on text-image sample pairs, so that the claim review model can combine visual and text clues to more accurately infer the review results. Among them, the specific training process of the claim review model will be introduced in subsequent embodiments and will not be repeated here.

[0041] The claim review model can extract features from multimodal claim vouchers and determine whether to make a claim for a food order based on the extracted features. For text claim vouchers, the claim review model can extract text features; for any image claim voucher, the claim review model can extract image features. The text feature is an abstract expression of the semantic information contained in the text claim voucher, and the image feature is an abstract expression of the image semantics contained in the image claim voucher. Image semantics refers to the meaning of the information expressed by the image. For example, when an image contains food claim targets such as insects, foreign objects, and mold spots, the semantics of the image may include information used to describe the location and type of the above claim targets. After obtaining the text features and image features, the claim review model can perform a classification task based on the correlation between the text features and the image features to obtain the claim review result. The claim review result includes: passing the claim or rejecting the claim.

[0042] Among them, the correlation between text features and image features is used to characterize the degree of semantic matching between text features and image features, so as to detect whether the text claim voucher and the image claim voucher are consistent. Among them, the closer the semantics expressed by the text claim voucher is to the semantics expressed by the image claim voucher, the higher the correlation between the text features and the image features. For example, when the text claim voucher contains descriptive information related to food quality such as "bugs", "blackening" and "mold spots", if the image claim voucher contains "bug images", "blackened food images" and "mold spot images", the semantics expressed by the text claim voucher is highly consistent with the semantics expressed by the image claim voucher, and the corresponding text features and image features have a high correlation. On the contrary, if the image claim voucher does not contain "bug images", "blackened food images" and "mold spot images", the semantics expressed by the text claim voucher is less consistent with the semantics expressed by the image claim voucher, and the corresponding text features and image features have a low correlation. When the correlation between text features and image features is high, it can be considered that the claim request has a reasonable basis for claim settlement; conversely, when the correlation between text features and image features is low, it can be considered that the claim request lacks a reasonable basis for claim settlement.

[0043] Among them, the correlation between text features and image features can be determined according to the similarity between text features and image features, or can be determined according to the difference between text features and image features, or can be determined according to the fusion features of text features and image features, and this is not limited in this embodiment. After obtaining the text features and image features, the claims review model can obtain the correlation between the text features and the image features, and analyze the degree of match between the claim case description expressed by the text features and the claim basis expressed by the image features based on the correlation between the text features and the image features. A binary classification task is performed based on the degree of match, and a claim review result of claim approval or claim rejection can be obtained. In this embodiment, there is no need to perform independent classification tasks based on text features and image features respectively, which can simplify the model structure, simplify the model processing flow, and improve reasoning efficiency.

[0044] In this embodiment, for the claim request of the food order, the image claim voucher and the text claim voucher corresponding to the claim request can be obtained, and the image features of the image claim voucher and the text features of the text claim voucher can be extracted using the claim review model. The claim review model can use the correlation between the image features and the text features to perform the classification task and obtain the claim review result. In this embodiment, on the one hand, the claim review model extracts multimodal features from multiple dimensions such as vision and text, and performs the classification task based on the correlation of the multimodal features, which can accurately identify whether the user's claim request corresponds to a reasonable claim basis and thus accurately output the review result; on the other hand, under the premise of extracting the multimodal features, the claim review model can quickly infer the claim review result "end-to-end" according to the correlation of the multimodal features, which simplifies the reasoning process and improves the efficiency of the claim review.

[0045] In some exemplary embodiments, the claim review model may include multiple encoders to perform feature extraction on multi-modal input data respectively. When the input data includes claim basis in text mode and image mode, the claim review model includes at least a text encoder and an image encoder.

[0046] Figure 2 The structure of the claims review model is shown in the figure. Figure 2 As shown, the claims review model includes a parallel image encoder and text encoder, a matching sub-model connected to the image encoder and the text encoder, and an output network connected to the matching sub-model. Among them, the image encoder and the text encoder can be implemented based on a deep learning network model. For example, the image encoder can be implemented as a convolutional neural network model (Convolutional Neural Networks, referred to as CNN), a visual self-attention model (vision transformer, referred to as ViT). The text encoder can be implemented as a Transformer (a natural language processing model based on data transformation) or other NLP (Natural Language Processing) models, such as LSTM (Long Short-Term Memory, long short-term memory network) model, RNN (Recurrent Neural Network, recurrent neural network), etc., which are not limited in this embodiment.

[0047] Optionally, after the claim processing device obtains the text claim voucher and at least one image claim voucher corresponding to the claim request, the text claim voucher and at least one image claim voucher may be input into the claim review model. In the claim review model, the text claim voucher may be feature encoded using a text encoder to obtain a text feature vector. In the claim model, the at least one image claim voucher may be feature encoded using an image encoder to obtain at least one image feature vector.

[0048] The text encoder and the image encoder are jointly trained using a contrastive learning training method. During the joint training process, the image encoder and the text encoder can encode the image and text respectively to map the image and text to a shared embedding space, and make the matching image and text closer in the embedding space, while the mismatching image and text are farther away in the embedding space. Thus, the image encoder and the text encoder can learn feature expression capabilities on the premise of understanding the semantic association between the image and the text.

[0049] Based on the contrastive learning training method, the text encoder and the image encoder can make the features extracted from the text and pictures with higher matching degrees closer, and make the features extracted from the text and pictures with lower matching degrees farther. Furthermore, in this embodiment, after the image claim voucher is input into the image encoder, the image encoder can perform feature extraction operations based on the pre-learned feature expression capabilities to obtain image features that can more accurately represent semantic information. Similarly, after the text claim voucher is input into the text encoder, the text encoder can perform feature extraction operations based on the pre-learned feature expression capabilities to obtain text features that can more accurately represent semantic information. The more accurate image features and text features can provide reliable data support for the subsequent reasoning process, thereby improving the accuracy of the reasoning results.

[0050] Optionally, when the claim review model performs a classification task based on the correlation between the text feature and the image feature, the matching sub-model in the claim review model can be used to perform feature matching on the text feature vector and the at least one image feature vector. A matching score can be calculated by performing feature matching between the text feature vector and any image feature vector. The feature matching operation between the text feature vector and the at least one image feature vector can obtain at least one matching score. The claim review model can output the claim review result based on the at least one matching score.

[0051] In some optional embodiments, the matching scores of text feature vectors and image feature vectors can be calculated by similarity. In the matching sub-model of the claim model, at least one feature vector group can be determined; wherein, any feature vector group contains a text feature vector and any image feature vector. That is, any feature vector group is composed of feature vectors of different modalities. The similarity of the feature vectors in the at least one feature vector group is calculated respectively, and at least one matching score can be obtained. Among them, the similarity of the feature vectors in any feature vector group can be used to express the degree of matching between the user's claim case description and the claim evidence. The higher the matching score, the higher the degree of matching between the user's claim case description and the claim evidence, and the higher the rationality of the user's claim request. Furthermore, based on the matching score, the rationality of the claim request can be accurately judged to improve the accuracy of the claim review results.

[0052] After determining the at least one matching score, the output network in the claim review model may output the claim review result according to the at least one matching score. In some optional embodiments, the output network may select a target matching score that satisfies a set score condition from the at least one matching score. Optionally, the set score condition may include: the matching score is the maximum value of the at least one matching score, or the matching score is greater than the average value of the at least one matching score, or the matching score is greater than the median of the at least one matching score, and this embodiment includes but is not limited to this. If the target matching score is greater than the preset score threshold, it can be considered that the user's claim claim is more reasonable, and the output network of the claim review model can output a claim approval review result; conversely, if the target matching score is less than or equal to the preset score threshold, it can be considered that the user's claim claim is less reasonable, and the output network of the claim review model can output a claim rejection review result.

[0053] In this implementation, by screening the at least one matching score, the screened target matching score can more accurately express the matching relationship between the image feature and the text feature, thereby facilitating a more accurate analysis of the rationality of the claim basis based on the matching relationship. By outputting the claim review result by judging whether the target matching score is greater than the score threshold, the review conclusion can be quickly given, thereby improving the efficiency of the claim review.

[0054] The following will continue to combine Figure 2 , a specific application scenario is used as an example for exemplary description.

[0055] In a specific scenario, a user purchases a cup of fruit tea at a beverage station A on a food delivery platform through a terminal device, but there is foreign matter in the fruit tea. The user then initiates a claim operation through the claim service provided by the food delivery platform. The terminal device can respond to the user's claim operation, initiate a claim request to the server, and submit the claim credentials entered by the user to the server. Figure 2 As shown in FIG, the text claim voucher entered by the user is a description of the claim case, that is, I bought a cup of fruit tea at A beverage store, but there was a bug in it. Figure 2 As shown, the image claim voucher input by the user can be N pictures containing bugs obtained by photographing the details of the beverage, where N is a positive number N≥1. The server can perform automatic claim review using the claim review model.

[0056] like Figure 2 As shown, the server can input N image claim vouchers and 1 text claim voucher into the claim review model. In the claim review model, N image claim vouchers are input into the image encoder to obtain N d-dimensional visual feature vectors, where d is a positive integer, and d can be 256, 128 or other data values, which are not limited in this embodiment. Figure 2 As shown, the text claim voucher is input into the text encoder to obtain a d-dimensional text feature vector. N d-dimensional visual feature vectors and 1 d-dimensional text feature vector are input into the matching sub-model. In the matching sub-model, the text feature vector can be matched with the N image feature vectors respectively to obtain N matching scores. When matching, the matching sub-model can calculate the feature similarity between the text feature vector and any image feature vector. When calculating the feature similarity, the matching sub-model can normalize the text feature vector and the image feature vector respectively, and perform inner product calculation on the normalized feature vector to obtain a matching score. Usually, the matching score is a value in [0,1].

[0057] After obtaining N matching scores based on the above calculation method, the matching sub-model can select a higher score from the N matching scores as the matching score of the claim application. Figure 2 As shown, the claim score is input into the output network of the claim review model. The output network can determine whether the claim score is greater than a preset score threshold. If it is greater than the preset score threshold, the review conclusion that the claim is passed can be output. Conversely, if the claim score is less than or equal to the preset score threshold, the output network can output the review conclusion that the claim is rejected. After the server obtains the review conclusion output by the claim review model, it can send the review conclusion to the terminal device for display. If the review conclusion is that the claim is passed, the server can further send the review conclusion to the claim payment platform to make a claim payment to the user, which will not be repeated.

[0058] Based on this implementation method, in scenarios such as food delivery and fresh food delivery where claims are more frequent and the basis for claims is relatively clear, the claim review model can be used to automatically review food orders for claims. This can greatly reduce reliance on manual review, reduce claims review costs, and effectively improve review efficiency.

[0059] In addition to the claims processing method described in the aforementioned embodiments, the embodiments of the present application also provide a model training method, which is used to train a converged claims review model to improve the accuracy of the claims review model.

[0060] Figure 3 is a flow chart of a model training method provided by an exemplary embodiment of the present application, such as Figure 3 As shown, the method includes:

[0061] Step 301, obtain multiple training data sets; any training data set includes: a text claim voucher sample, an image claim voucher sample and a true value of a claim result of any historical claim request.

[0062] Step 302: determine a pre-trained neural network model; the neural network model is pre-trained using a text-image pair pre-training method.

[0063] Step 303: Input the multiple training data groups into the neural network model respectively to obtain the claims review results of the multiple training data groups respectively.

[0064] Step 304: Obtain the difference between the claim review results of each of the multiple training data groups and the true value of the claim results.

[0065] Step 305: fine-tune the parameters of the neural network model with the goal of converging the difference, and when the difference converges to a specified range, output the neural network model as the claim review model.

[0066] The model training method provided in the embodiments of the present application is used to train the claims review model used in the aforementioned embodiments of the present application. The claims review model is used to review claim requests in a variety of scenarios to give a conclusion on whether to make a claim. The various scenarios include, but are not limited to: food safety claims review scenarios, takeaway order timeout claims scenarios, vehicle insurance claims scenarios, medical insurance claims scenarios, or other possible insurance claims scenarios. During the training process, multiple training data groups can be input into the claims review model so that the claims review model learns the connection between the input claim credentials and the claims review results, thereby learning the ability to automatically review claim requests.

[0067] Among them, any set of training data includes: a text claim voucher sample, an image claim voucher sample and a true value of the claim result of any historical claim request. Among them, the true value of the claim result can be manually annotated during the claim review. The true value of the claim result is usually represented by 0 or 1. If the true value of the claim review is 0, it corresponds to a claim rejection; if the true value of the claim review is 1, it corresponds to a claim approval.

[0068] After obtaining multiple training data sets, the pre-trained neural network model can be trained to fine-tune the pre-trained neural network model to obtain a claims review model suitable for a specific scenario. The neural network model can be pre-trained using a text-image pair pre-training method, and the implementation method of this pre-training operation will be introduced later.

[0069] In some optional embodiments, the training method for fine-tuning the pre-trained neural network model may include a stock training stage, that is, a stage of offline training based on stock claims data. In the stock training stage, stock claim application data of historical claims requests that have been manually reviewed can be obtained, and multiple training data groups can be generated based on the claim application data. Any training data group contains image claim credentials, claim text credentials, and true value labels provided by the user when initiating a claim request for a historical claim request. Among them, the true value label refers to the claim review result obtained by manual review, and the conclusion of whether the claim is paid obtained by manual review has high reliability and accuracy. After multiple training data groups are input into the claim review model, the claim review model can perform forward reasoning based on the image claim credentials and claim text credentials in each training data group, and the claim review result output by the claim review model for each training data group can be obtained.

[0070] Optionally, if the number of training data groups generated by the existing case report data is greater than a certain threshold, the claim audit model can be gradient back-propagated according to the difference between the claim audit result and the true value label of each training data group to optimize the parameters of the model. In this embodiment, when the number of training data groups is large, the sample capacity is large, and the claim audit model can better learn the claim audit knowledge, thereby improving the ability to accurately conduct claim audits.

[0071] Optionally, if the number of training data groups generated by the existing case report data is less than or equal to a certain threshold, the gradient back propagation operation of the model may not be performed temporarily, and the incremental training phase may be entered, and sample accumulation may continue in the incremental training phase to increase the sample capacity. In this case, the claim review model may be run online, and manual review and automatic review based on the claim review model may be performed synchronously on the subsequent claim requests. In the process of the above synchronous execution, incremental case report data may be continuously acquired, and incremental training data groups may be generated based on the incremental case report data. When the number of accumulated training data groups is greater than a certain threshold, the gradient back propagation of the claim review model may be performed based on the difference between the claim review result and the true value label of each training data group to optimize the parameters of the model. In this embodiment, the automatic review task processed by the claim review model is a binary classification task, that is, the output result is a claim approval or a claim rejection. Therefore, a binary cross entropy calculation may be used to calculate the difference between the claim review result and the true value label of each training data group.

[0072] The above online training operation can be executed in a loop until the difference between the claims review result of the training data group and the true value label converges to a specified range. Manual review and gradient back propagation optimization can be stopped, and the claims review model can be used to independently perform claims review.

[0073] Among them, the encoder of the claim review model may include a text encoder and an image encoder, which are used to extract features from text claim voucher samples and image claim voucher samples respectively.

[0074] In some optional embodiments, a neural network model including a text encoder and an image encoder may be constructed, and the neural network model may be pre-trained using a text-image pair pre-training method to obtain a text encoder and an image encoder with excellent feature expression capabilities. An exemplary description will be given below.

[0075] Optionally, multiple sample pairs required for pre-training can be obtained based on multiple text-image pairs, and the multiple sample pairs may include positive sample pairs and negative sample pairs. Among them, any text-image pair consists of a paragraph (or a sentence) of text and an image. In the positive sample pair, the text and the image are matched with each other. For example, in the food safety claims scenario, the positive sample pair may include: fly-fly picture (positive sample pair P1), caterpillar-caterpillar picture (positive sample pair P2), mold-mold picture (positive sample pair P3), rot-rot picture (positive sample pair P4). The text and image in the positive sample pair are scattered and randomly arranged and combined to generate a negative sample pair. For example, the following negative sample pairs can be generated based on the above positive sample pairs: fly-mold picture (negative sample pair N1), caterpillar-fly picture (negative sample pair N2), mold-rot picture (negative sample pair N3), rot-caterpillar picture (negative sample pair N4).

[0076] During pre-training, the above-mentioned multiple sample pairs can be input into the neural network model, and the image encoder and text encoder in the neural network model can be used to encode the text and image in the multiple sample pairs respectively, so as to obtain the text features and image features of the multiple sample pairs. For example, the text features and image features of the positive sample pairs P1, P2, P3, and P4 can be obtained respectively, as well as the text features and image features of the negative sample pairs N1, N2, N3, and N4 can be obtained respectively. Afterwards, the correlation between the text features and image features of the multiple sample pairs can be determined, that is, the correlation between the text features and image features of the positive sample pairs and the correlation between the text features and image features of the negative sample pairs can be determined respectively. For example, the correlation between the text features and image features of the positive sample pair P1 and the correlation between the text features and image features of the negative sample pair N1 can be determined.

[0077] In order to improve the recognition ability of the neural network model for texts and images with the same or similar semantics and the ability to distinguish texts and images with different semantics, a contrast loss function can be constructed, and the neural network model can be optimized according to the contrast loss function. In the contrast loss function, the higher the correlation between the text features and image features of the positive sample pair, the smaller the loss value; conversely, the lower the correlation between the text features and image features of the positive sample pair, the larger the loss value. The higher the correlation between the text features and image features of the negative sample pair, the larger the loss value; the lower the correlation between the text features and image features of the negative sample pair, the smaller the loss value. Based on the contrast loss function, the parameters of the image encoder and the text encoder can be updated to pre-train the neural network model with the goal of reducing the correlation between the image features and text features of the positive sample pair and increasing the correlation between the image features and text features of the negative sample pair. Based on the pre-training operation, the neural network model can learn the semantic correlation between images and texts, so that the feature expressions of images and texts with the same or similar semantics are closer.

[0078] In other optional embodiments, the text encoder and the image encoder can be a text feature extraction sub-model and an image feature extraction sub-model in a pre-trained large model. Among them, the pre-trained large model refers to an artificial intelligence model with large-scale parameters and complex structure. The pre-trained large model adopts a multimodal artificial intelligence algorithm of large-scale self-supervised learning, integrating language and visual modal understanding, and has powerful capabilities in visual and text understanding generation tasks. For example, in some embodiments, M6 (a multimodal large model) can be obtained as a pre-trained large model. For example, in other embodiments, the CLIP (Contrastive Language-Image Pretraining, a pre-training method based on text-image pairs) model can be obtained as a pre-trained large model, which is not limited in this embodiment.

[0079] In this implementation, the text feature and image feature extraction operations are performed based on the pre-trained large model. On the one hand, the accuracy of the extracted text features and image features can be improved based on the powerful task processing capabilities of the pre-trained large model. On the other hand, the adaptability of the claims review model to different claims scenarios can be improved based on the strong generalization capabilities of the pre-trained large model.

[0080] The following will take the implementation of a pre-trained large model as a CLIP model as an example for exemplary explanation.

[0081] The CLIP model may include a text encoder and an image encoder trained by a pre-training method based on a text-image pair. Among them, the image encoder trained by the pre-training method based on the text-image pair can be described as a CLIP image encoder; the text encoder based on the pre-training method based on the text-image pair can be described as a CLIP text encoder. The CLIP image encoder and the CLIP text encoder can be obtained by joint training using image samples and text samples based on a pre-training method of contrastive learning. In the process of joint training, the image and text can be mapped to a shared embedding space by encoding the image and text, and the distance between the matching image and text in the embedding space is made closer, while the distance between the non-matching image and text in the embedding space is farther, so that the model can understand the semantic association between the image and the text.

[0082] In the process of visual encoding, the CLIP image encoder can extract features from images through CNN or ViT models, and map the extracted image features to the embedding space. In the process of text encoding, the CLIP text encoder uses the Transformer model to extract features from text, and also maps the extracted features to the embedding space. The CLIP image encoder and the CLIP encoder share parameters to ensure that images and texts have consistent embedding spaces.

[0083] During the joint training process, a contrast loss function can be used to measure the difference between matched and unmatched image-text pairs. For matched image-text pairs, the CLIP model encourages them to be close in the embedding space; for unmatched image-text pairs, CLIP encourages them to be far apart in the embedding space. In turn, the model is able to learn the semantic association between images and texts. By performing multiple joint trainings on large-scale image and text datasets, a CLIP model with strong generalization ability can be obtained, and the CLIP image encoder and CLIP text encoder in the CLIP model have good performance in visual and language tasks, respectively. After determining the image encoder and text encoder based on this implementation, the matching sub-model and the output network can be connected in sequence at the output ends of the image encoder and the text encoder to obtain the initialization model of the claims review model (i.e., the pre-trained neural network model), such as Figure 4 shown.

[0084] In the process of stock training and incremental training recorded in the aforementioned embodiments, the parameters of the pre-trained neural network model can be fine-tuned according to the difference between the claim review results of the training data group and the true value label to make it more suitable for the claim scenario of food orders. Among them, the model parameters of the pre-trained neural network model have strong generalization ability. When fine-tuning, a smaller learning rate can be used to make the model converge quickly on the data set in the food field, so that the pre-trained neural network model can maintain the strong feature expression ability in vision and text, and learn the claim review ability of food orders.

[0085] It is worth noting that, in some exemplary embodiments, before the image claim voucher is input into the image encoder, a data augmentation operation may be performed on the image claim voucher to increase the generalization ability of the network model, such as Figure 4 As shown. The data enhancement operation may include but is not limited to at least one of the following: color adjustment, brightness adjustment, saturation adjustment, adding image noise, scaling the image resolution, randomly cropping the image, randomly blocking the image, cutting out the local area of ​​the image, adding mosaic to the image, etc. Based on the above image enhancement operations, the amount of sample data can be enriched and the coverage of sample data can be increased, thereby greatly increasing the generalization ability of the claims review model.

[0086] In this implementation, by training the claims audit model offline and further optimizing the claims audit model online, the task understanding ability of the claims audit model can be improved, thereby obtaining more accurate audit results. Based on the trained claims audit model, on the one hand, fully automated claims operations can be realized, greatly saving the labor costs of auditors; on the other hand, a one-time end-to-end audit operation can be performed online, with a high audit response speed, thus improving audit efficiency.

[0087] In addition to the claims processing method provided in the aforementioned embodiments, the embodiments of the present application also provide a claims processing method applicable to a variety of scenarios, which will be exemplified below.

[0088] Figure 5 is a flowchart of a claims processing method provided by another exemplary embodiment of the present application, such as Figure 5 As shown, the method includes:

[0089] Step 501: respond to a claim request and obtain a multimodal claim voucher corresponding to the claim request.

[0090] Step 502: input the multimodal claim voucher into a claim review model to extract multimodal features of the multimodal claim voucher.

[0091] Step 503, performing a classification task according to the correlation of the multimodal features to obtain a claim review result; the correlation is used to describe the degree of semantic matching of features of different modes in the multimodal features; the claim review model is obtained by joint training of multimodal samples.

[0092] The claim processing method provided in this embodiment can be executed by a claim processing device, which can run on the terminal side or the server side. The claim processing device can be applied to food safety claims review scenarios, takeaway order timeout claims scenarios, vehicle insurance claims scenarios, medical insurance claims scenarios, or other possible insurance claims scenarios. In the above-mentioned claims scenarios, the claim processing device can receive a claim request and obtain a multimodal claim credential corresponding to the claim request. The multimodal claim credential includes: image credentials and text credentials. For example, in the vehicle insurance claims scenario, the claim processing device can obtain text-modal claim description information and at least one vehicle damage picture. For another example, in the medical insurance claims scenario, the claim processing device can obtain text-modal claim description information, prescription pictures, medical record pictures, and invoice pictures.

[0093] After obtaining the multimodal claim voucher, the claim processing device can use the claim review model to extract multimodal features from the multimodal claim voucher, and perform classification tasks according to the correlation of the multimodal features to obtain the claim review result. Among them, the claim review model is trained using training data in the claim scenario. Optionally, during the training stage, multiple training data groups can be obtained; any training data group includes: multimodal claim voucher samples and claim result true values. The multiple training data groups are respectively input into the claim review model to obtain the claim review results of each of the multiple training data groups. The difference between the claim review results of each of the multiple training data groups and the claim result true value is obtained, and the parameters of the claim review model are fine-tuned with the goal of converging the difference.

[0094] Among them, the claims review models corresponding to different application scenarios may be the same or different. In some optional embodiments, during the training phase, sample data under a variety of scenarios may be used to train the claims review model, so that the claims review model can learn the claims review knowledge of different scenarios, and adaptively output the claims results accurately according to the multimodal claims credentials of the claim requests in different scenarios. In other optional embodiments, during the training phase, multiple claims review models may be determined, and the parameters of the multiple claims review models may be initialized. Afterwards, sample data under different scenarios may be used to train the claims review models respectively, so as to obtain claims review models corresponding to different functional scenarios, which is not limited in this embodiment.

[0095] Optionally, the claim review model may include encoders corresponding to multiple modalities, which can be used to extract features from the multi-modal claims basis to obtain multi-modal features. For example, the claim model may include an encoder of the first modality, which is used to extract features from the claims basis of the first modality to obtain features of the first modality. The claim model may include an encoder of the second modality, which is used to extract features from the claims basis of the second modality to obtain features of the second modality. In the claim review model, the encoders corresponding to the multiple modalities can feature encode the multi-modal claim voucher according to the modal correspondence to obtain the feature vectors of the multi-modal claim vouchers. In some optional embodiments, the multi-modal claim voucher corresponding to the claim request includes: a text claim voucher and at least one image claim voucher. Accordingly, the claim model can feature encode the text claim voucher using a text encoder to obtain a text feature vector; and feature encode the at least one image claim voucher using an image encoder to obtain at least one image feature vector. Encoders corresponding to multiple modalities can be trained using contrastive learning methods so that encoders corresponding to multiple modalities can be aligned in terms of feature expression capabilities. For example, a text encoder and an image encoder can be jointly trained using contrastive learning training methods based on text-image pairs.

[0096] After the claim review model extracts multimodal features from the multimodal claim voucher, it can perform classification tasks based on the correlation of the multimodal features to obtain the claim review results. Among them, the correlation of the multimodal features is used to describe the degree of semantic matching of the multimodal features. The correlation of the multimodal features can be determined based on the similarity of the multimodal features, or can be determined based on the difference of the multimodal features, or can be determined based on the fusion features of the multimodal features, which is not limited in this embodiment.

[0097] Accordingly, when the claim review model performs the classification task according to the multimodal feature, the matching sub-model in the claim review model is used to perform feature matching on the feature vectors of different modes in the multimodal feature to obtain at least one matching score. Wherein, when there are multiple feature vectors under each mode, feature matching can be performed on the feature vectors of any different modes to obtain one or more matching scores. According to the at least one matching score, the claim review result can be input. In some optional embodiments, when the multimodal claim voucher corresponding to the claim request includes a text claim voucher and at least one image claim voucher, the matching sub-model in the claim review model can be used to perform feature matching on the text feature vector of the text claim voucher and at least one image feature vector corresponding to the at least one image claim voucher to obtain at least one matching score. According to the at least one matching score and the set score condition, the claim review result can be output.

[0098] Optionally, the matching sub-model in the claim review model can determine at least one feature vector group when performing feature matching on feature vectors of different modes in the multimodal feature. Among them, multiple feature vectors in any feature vector group correspond to claim vouchers of different modes. The matching sub-model can calculate the similarity of the feature vectors in the at least one feature vector group respectively to obtain at least one matching score. For example, assume that the multimodal feature includes M text feature vectors and N image feature vectors, and M and N are positive integers. The M text feature vectors and the N image feature vectors can be divided into M*N feature vector groups, and any feature vector group contains a text feature vector and an image feature vector. The matching sub-model can calculate the similarity of the feature vectors in the M*N feature vector groups to obtain M*N matching scores. For example, when performing feature matching on a text feature vector and at least one image feature vector, at least one feature vector group can be determined, and any feature vector group contains the text feature vector and any image feature vector. The similarity of the feature vectors in the at least one feature vector group is calculated respectively to obtain at least one matching score.

[0099] After obtaining the at least one matching score, the claim review model may output the claim review result according to the at least one matching score. Optionally, it includes: the claim review model may select a target matching score that meets the set score condition from the at least one matching score. Among them, the set score condition may include: the matching score is the maximum value of the at least one matching score, or the matching score is greater than the average value of the at least one matching score, or the matching score is greater than the median of the at least one matching score. This embodiment includes but is not limited to this. If the selected target matching score is greater than the preset score threshold, the review result of claim approval is output; if the selected target matching score is less than or equal to the preset score threshold, the review result of claim rejection is output.

[0100] In this embodiment, in response to a claim request, a multimodal claim voucher corresponding to the claim request can be obtained, and the claim review model can be used to obtain the feature vectors of each of the multimodal claim vouchers as multimodal features. Among them, the multimodal features can characterize the claim clues corresponding to the claim request from multiple aspects, which is conducive to obtaining more accurate claim review results. After obtaining the multimodal features, the claim review model can be used to perform classification tasks according to the correlation of the multimodal features to obtain the claim review results. In this embodiment, on the one hand, the claim review results can be accurately output in combination with the multi-dimensional information expressed by the multimodal features. On the other hand, the claim review model can be used to quickly infer the claim review results "end-to-end", thereby reducing the reliance of the claim review operation on manual labor and improving the efficiency of the claim review.

[0101] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 301 to 104 can be device A; for another example, the execution subject of steps 301 and 302 can be device A, and the execution subject of step 303 can be device B; and so on.

[0102] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0103] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0104] Figure 6 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 6 As shown, the electronic device includes: a memory 601 , a processor 602 and a communication component 603 .

[0105] The memory 601 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.

[0106] Processor 602 is coupled to memory 601 and is used to execute a computer program in memory 601, so as to: respond to a claim request and obtain a multimodal claim voucher corresponding to the claim request; input the multimodal claim voucher into a claim review model to extract multimodal features corresponding to the multimodal claim voucher; perform a classification task according to the correlation of the multimodal features to obtain a claim review result; the correlation is used to describe the degree of semantic matching of features of different modes in the multimodal features; wherein the claim review model is obtained based on joint training of multimodal samples.

[0107] Optionally, the multimodal claim voucher includes: a text claim voucher and at least one image claim voucher; when the processor 602 inputs the multimodal claim voucher into the claim review model to extract the multimodal features corresponding to the multimodal claim voucher, it is specifically used to: in the claim review model, use a text encoder to feature encode the text claim voucher to obtain a text feature vector; and use an image encoder to feature encode the at least one image claim voucher to obtain at least one image feature vector; wherein the text encoder and the image encoding are jointly trained according to the text-image pair using a contrastive learning training method.

[0108] Optionally, when the processor 602 performs a classification task based on the correlation of the multimodal features, it is specifically used to: utilize the matching sub-model in the claim review model to perform feature matching on the text feature vector and the at least one image feature vector to obtain at least one matching score; and output the claim review result based on the at least one matching score and the set score conditions.

[0109] Optionally, when the processor 602 performs feature matching on the text feature vector and the at least one image feature vector to obtain at least one matching score, it is specifically used to: determine at least one feature vector group; any one feature vector group includes the text feature vector and any one image feature vector; and respectively calculate the similarity of the feature vectors in the at least one feature vector group to obtain the at least one matching score.

[0110] Optionally, when the processor 602 outputs the claim review result based on the at least one matching score and the set score condition, it is specifically used to: select a target matching score that meets the set score condition from the at least one matching score; if the target matching score is greater than a preset score threshold, output a review result of claim approval; if the target matching score is less than or equal to the preset score threshold, output a review result of claim rejection.

[0111] Optionally, Figure 6 The electronic device shown can also be used to execute a claim processing method for food orders, wherein the processor 602 is specifically used to: respond to a food order claim request, obtain a text claim credential and at least one image claim credential corresponding to the claim request; input the text claim credential and at least one image claim credential into a claim review model to extract text features of the text claim credential and image features of the at least one image claim credential; perform a classification task based on the correlation between the text features and the image features to obtain a claim review result; the correlation is used to describe the degree of semantic matching between the text features and the image features; wherein the claim review model is obtained based on joint training of text-image sample pairs.

[0112] Optionally, Figure 6The electronic device shown can also be used to execute a model training method, wherein the processor 602 is specifically used to: obtain multiple training data groups; any training data group includes: a text claim voucher sample, an image claim voucher sample and a true value of a claim result of any historical claim request; determine a pre-trained neural network model; the neural network model is pre-trained using a text-image pair pre-training method; the multiple training data groups are respectively input into the neural network model to obtain the claim review results of each of the multiple training data groups; the neural network model is used to extract the text features of the text claim voucher sample and the image features of the image claim voucher sample in any training data group, and perform a classification task based on the correlation between the text features and the image features to obtain the claim review results of the training data group; obtain the difference between the claim review results of each of the multiple training data groups and the true value of the claim result; with the goal of converging the difference, fine-tune the parameters of the neural network model, and when the difference converges to a specified range, output the neural network model as the claim review model.

[0113] Optionally, determining a pre-trained neural network model includes: obtaining a plurality of sample pairs based on a plurality of text-image pairs, the plurality of sample pairs including positive sample pairs and negative sample pairs; inputting the plurality of sample pairs into the neural network model; using an image encoder and a text encoder in the neural network model to respectively encode the text and the image in the plurality of sample pairs to obtain text features and image features of the plurality of sample pairs; determining the correlation between the text features and the image features of the plurality of sample pairs; and updating the parameters of the image encoder and the text encoder with the goal of reducing the correlation between the image features and the text features of the positive sample pairs and increasing the correlation between the image features and the text features of the negative sample pairs to pre-train the neural network model.

[0114] In this embodiment, for a claim request, the image claim voucher and the text claim voucher corresponding to the claim request can be obtained, and the claim review model can be used to extract the image features of the image claim voucher and the text features of the text claim voucher. The claim review model can use the correlation between the image features and the text features to perform classification tasks and obtain the claim review results. In this embodiment, on the one hand, the claim review model extracts multimodal features from multiple dimensions such as vision and text, and performs classification tasks based on the correlation of multimodal features. It can accurately identify whether the user's claim request corresponds to a reasonable claim basis and thus accurately output the review results; on the other hand, under the premise of extracting multimodal features, the claim review model can quickly infer the claim review results "end-to-end" according to the correlation of multimodal features, which simplifies the reasoning process and improves the efficiency of claim review.

[0115] Figure 6The illustrated electronic device can also be used to execute the following claims processing method, wherein the processor 602 is used to: receive a claim request; obtain a multimodal claim voucher corresponding to the claim request; use a claim review model to extract multimodal features from the multimodal claim voucher, and perform a classification task based on the correlation of the multimodal features to obtain a claims review result; the claim review model is obtained by joint training based on multimodal samples.

[0116] Further, if Figure 6 As shown, the electronic device also includes: a power supply component 604 and other components. Figure 6 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 6 Components shown.

[0117] Among them, the memory 601 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0118] The communication component 603 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as Wi-Fi (wireless network communication technology), 2G (such as Global System for Mobile Communications (GSM)), 3G (such as Wideband Code Division Multiple Access (WCDMA), 4G (such as Long Term Evolution (LTE)), 4G+ (such as upgraded Long Term Evolution (LTE-Advanced, LTE-A)) or 5G (Fifth Generation Mobile Communication Technology), or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0119] The power supply component 604 is used to provide power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power to the device where the power supply component is located.

[0120] In this embodiment, in response to a claim request, a multimodal claim voucher corresponding to the claim request can be obtained, and the claim review model can be used to obtain the feature vectors of each of the multimodal claim vouchers as multimodal features. Among them, the multimodal features can characterize the claim clues corresponding to the claim request from multiple aspects, which is conducive to obtaining more accurate claim review results. After obtaining the multimodal features, the claim review model can be used to perform classification tasks according to the correlation of the multimodal features to obtain the claim review results. In this implementation, on the one hand, the claim review results can be accurately output in combination with the multi-dimensional information expressed by the multimodal features. On the other hand, the claim review model can be used to quickly infer the claim review results "end-to-end", thereby reducing the reliance of the claim review operation on manual labor and improving the efficiency of the claim review.

[0121] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0122] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM (Compact Disc Read-Only Memory, read-only optical disc read-only memory), optical storage, etc.) containing computer-usable program codes.

[0123] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0127] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0128] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Parallel Random Access Machine (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0130] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for processing claims, characterized in that: include: Respond to a claim request and obtain a multimodal claim voucher corresponding to the claim request; Inputting the multimodal claim voucher into a claim review model to extract multimodal features corresponding to the multimodal claim voucher; Performing a classification task according to the correlation of the multimodal features to obtain a claim review result; the correlation is used to describe the degree of semantic matching between features of different modes in the multimodal features; Among them, the claim review model is obtained by joint training based on multimodal samples.

2. The method according to claim 1, characterized in that The multimodal claim voucher includes: a text claim voucher and at least one image claim voucher; The multimodal claim voucher is input into the claim review model to extract multimodal features corresponding to the multimodal claim voucher, including: In the claim review model, a text encoder is used to perform feature encoding on the text claim voucher to obtain a text feature vector; and Using an image encoder to perform feature encoding on the at least one image claim voucher to obtain at least one image feature vector; The text encoder and the image encoder are jointly trained based on text-image pairs using a contrastive learning training method.

3. The method according to claim 2, characterized in that Performing a classification task according to the correlation of the multimodal features includes: Using the matching sub-model in the claim review model, feature matching is performed on the text feature vector and the at least one image feature vector to obtain at least one matching score; The claim review result is output based on the at least one matching score and the set score conditions.

4. The method according to claim 3, characterized in that Performing feature matching on the text feature vector and the at least one image feature vector to obtain at least one matching score includes: Determine at least one feature vector group; any one of the feature vector groups contains the text feature vector and any one of the image feature vectors; The similarities of the feature vectors in the at least one feature vector group are respectively calculated to obtain the at least one matching score.

5. The method according to claim 3, characterized in that: Outputting a claim review result according to the at least one matching score and the set score condition, including: Selecting a target matching score that satisfies a set score condition from the at least one matching score; If the target matching score is greater than the preset score threshold, the review result of claim approval is output; If the target matching score is less than or equal to the preset score threshold, the review result of claim rejection is output.

6. A method for handling claims, characterized in that: include: In response to a food order claim request, obtaining a text claim voucher and at least one image claim voucher corresponding to the claim request; Inputting the text claim voucher and at least one image claim voucher into a claim review model to extract text features of the text claim voucher and image features of the at least one image claim voucher; Performing a classification task according to the correlation between the text features and the image features to obtain a claim review result; the correlation is used to describe the degree of semantic matching between the text features and the image features; The claim review model is obtained by joint training based on text-image sample pairs.

7. A model training method, characterized in that: include: Obtain multiple training data sets; Any training data set includes: a text claim voucher sample, an image claim voucher sample, and a true value of a claim result of any historical claim request; Determining a pre-trained neural network model; the neural network model is pre-trained using a text-image pair pre-training method; The plurality of training data groups are respectively input into the neural network model to obtain the claims review results of the plurality of training data groups; the neural network model is used to extract the text features of the text claim voucher samples and the image features of the image claim voucher samples in any training data group, and perform a classification task according to the correlation between the text features and the image features to obtain the claims review results of the training data group; Obtaining the difference between the claim review results of each of the plurality of training data groups and the true value of the claim result; With the goal of converging the difference, the parameters of the neural network model are fine-tuned, and when the difference converges to a specified range, the neural network model is output as the claim review model.

8. The method according to any one of claim 7, characterized in that: Determine the pre-trained neural network model, including: According to the plurality of text-image pairs, a plurality of sample pairs are obtained, wherein the plurality of sample pairs include positive sample pairs and negative sample pairs; Inputting the plurality of sample pairs into the neural network model; Encode the texts and images in the plurality of sample pairs respectively using the image encoder and the text encoder in the neural network model to obtain text features and image features of the plurality of sample pairs; Determining the correlation between the text features and the image features of the plurality of sample pairs; With the goal of reducing the correlation between the image features and the text features of the positive sample pairs and increasing the correlation between the image features and the text features of the negative sample pairs, the parameters of the image encoder and the text encoder are updated to pre-train the neural network model.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement the claims processing method described in any one of claims 1 to 8.

Images