Financial service processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a financial service processing method and device, a storage medium and electronic equipment. Relates to the technical field of satellite remote sensing, and the method comprises the steps: obtaining a target remote sensing image and a target cue word of a to-be-handled loan service of a target object, the target cue word being used for indicating a target model to process the target remote sensing image; determining the target model based on a task type corresponding to the target remote sensing image; processing the target remote sensing image and the target cue word through the target model to obtain a processing result; and based on the processing result, determining the risk of handling the loan service by the target object. Through the method and the device, the problem of relatively low risk assessment efficiency caused by performing risk assessment on the remote sensing image corresponding to the financial service based on a manual mode in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of satellite remote sensing technology, and in particular to a method and device for processing financial services, a storage medium and an electronic device. Background Art

[0002] In the financial industry, remote sensing technology is deeply integrated with financial services. Satellite remote sensing has the characteristics of large observation coverage, short revisit cycle and high accuracy. It is applied to credit management, post-loan monitoring of engineering construction, inclusive finance agricultural credit issuance assessment, insurance loss assessment and claims settlement and other business areas, which can comprehensively improve the service level of the financial industry in credit, insurance and other fields.

[0003] Remote sensing technology is increasingly used in banking business scenarios, involving change detection in the field of industrial credit, identifying change information of target objects in the images before and after the infrastructure project; target detection in the field of industrial credit, identifying the location and category information of specified targets in remote sensing images to obtain their quantity and direction information, etc. However, the existing technology often conducts risk assessment of remote sensing images corresponding to financial business based on manual methods, which has the problem of low assessment efficiency.

[0004] Currently, no effective solution has been proposed to address the problem that risk assessment of remote sensing images corresponding to financial business is performed manually in related technologies, resulting in relatively low risk assessment efficiency. Summary of the invention

[0005] The main purpose of this application is to provide a method and device for processing financial business, a storage medium and an electronic device to solve the problem in the related art that risk assessment of remote sensing images corresponding to financial business is performed manually, resulting in relatively low risk assessment efficiency.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for processing financial business is provided. The method comprises: obtaining a target remote sensing image and a target prompt word of a loan business to be processed by a target object, wherein the target prompt word is used to instruct a target model to process the target remote sensing image; determining the target model based on the task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection, and semantic segmentation; processing the target remote sensing image and the target prompt word through the target model to obtain a processing result; and determining the risk of the target object in processing the loan business based on the processing result.

[0007] Furthermore, the basic model is obtained by adopting the following steps: obtaining sample remote sensing images and sample prompt words; constructing a training sample set based on the sample remote sensing images and the sample prompt words; and training the initial model through the training sample set to obtain the basic model.

[0008] Furthermore, the initial model is trained by the training sample set to obtain the basic model, including: encoding the sample remote sensing image by the image encoder in the initial model to obtain a visual feature sequence; encoding the sample prompt words by the text encoder in the initial model to obtain a text feature sequence; decoding processing is performed by the image decoder in the initial model based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image; decoding processing is performed by the text decoder in the initial model based on the visual feature sequence and the text feature sequence to obtain a reconstructed prompt word; the initial model is trained based on the reconstructed remote sensing image and the reconstructed prompt word to obtain the basic model.

[0009] Furthermore, the image decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image, including: performing feature fusion on the visual feature sequence through the masked self-attention layer in the image decoder to obtain a first feature sequence; performing feature fusion on the first feature sequence and the text feature sequence through the cross-attention layer in the image decoder to obtain a second feature sequence; and performing feature transformation on the second feature sequence through the multilayer perceptron in the image decoder to obtain the reconstructed remote sensing image.

[0010] Furthermore, performing feature transformation on the second feature sequence through the multilayer perceptron in the image decoder to obtain the reconstructed remote sensing image includes: performing feature transformation on the second feature sequence through the multilayer perceptron to obtain multiple reconstructed image blocks; and splicing the multiple reconstructed image blocks according to a preset order to obtain the reconstructed remote sensing image.

[0011] Furthermore, the initial model is trained based on the reconstructed remote sensing image and the reconstructed prompt words to obtain the basic model, including: calculating based on the reconstructed remote sensing image and the sample remote sensing image to obtain a first loss function; calculating based on the reconstructed prompt words and the sample prompt words to obtain a second loss function; calculating based on the visual feature sequence and the text feature sequence to obtain a third loss function; training the initial model based on the first loss function, the second loss function and the third loss function to obtain the basic model.

[0012] Furthermore, the third loss function is obtained by calculating based on the visual feature sequence and the text feature sequence, including: calculating the vector length of the visual feature sequence to obtain a first vector length; calculating the vector length of the text feature sequence to obtain a second vector length; performing multiplication calculation on the visual feature sequence and the text feature sequence to obtain a vector inner product; and obtaining the third loss function based on the first vector length, the second vector length and the vector inner product.

[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a device for processing financial business is provided. The device includes: an acquisition unit, used to acquire a target remote sensing image and a target prompt word of a loan business to be processed by a target object, wherein the target prompt word is used to instruct a target model to process the target remote sensing image; a first determination unit, used to determine the target model based on the task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection and semantic segmentation; a processing unit, used to process the target remote sensing image and the target prompt word through the target model to obtain a processing result; a second determination unit, used to determine the risk of the target object processing the loan business based on the processing result.

[0014] Furthermore, the basic model is obtained by using the following devices: an acquisition unit, used to acquire sample remote sensing images and sample prompt words; a construction unit, used to construct a training sample set based on the sample remote sensing images and the sample prompt words; a training unit, used to train the initial model through the training sample set to obtain the basic model.

[0015] Furthermore, the training unit includes: a first encoding subunit, used to encode the sample remote sensing image through the image encoder in the initial model to obtain a visual feature sequence; a second encoding subunit, used to encode the sample prompt word through the text encoder in the initial model to obtain a text feature sequence; a first decoding subunit, used to perform decoding processing based on the visual feature sequence and the text feature sequence through the image decoder in the initial model to obtain a reconstructed remote sensing image; a second decoding subunit, used to perform decoding processing based on the visual feature sequence and the text feature sequence through the text decoder in the initial model to obtain a reconstructed prompt word; a training subunit, used to train the initial model based on the reconstructed remote sensing image and the reconstructed prompt word to obtain the basic model.

[0016] Furthermore, the first decoding subunit includes: a first fusion module, used to perform feature fusion on the visual feature sequence through the masked self-attention layer in the image decoder to obtain a first feature sequence; a second fusion module, used to perform feature fusion on the first feature sequence and the text feature sequence through the cross-attention layer in the image decoder to obtain a second feature sequence; a transformation module, used to perform feature transformation on the second feature sequence through the multi-layer perceptron in the image decoder to obtain the reconstructed remote sensing image.

[0017] Furthermore, the transformation module includes: a transformation submodule, used to perform feature transformation on the second feature sequence through the multilayer perceptron to obtain multiple reconstructed image blocks; and a splicing submodule, used to splice the multiple reconstructed image blocks according to a preset order to obtain the reconstructed remote sensing image.

[0018] Furthermore, the training subunit includes: a first calculation module, used to perform calculations based on the reconstructed remote sensing image and the sample remote sensing image to obtain a first loss function; a second calculation module, used to perform calculations based on the reconstructed prompt words and the sample prompt words to obtain a second loss function; a third calculation module, used to perform calculations based on the visual feature sequence and the text feature sequence to obtain a third loss function; a training module, used to train the initial model based on the first loss function, the second loss function and the third loss function to obtain the basic model.

[0019] Furthermore, the third calculation module includes: a first calculation submodule, used to calculate the vector length of the visual feature sequence to obtain a first vector length; a second calculation submodule, used to calculate the vector length of the text feature sequence to obtain a second vector length; a third calculation submodule, used to perform product calculation on the visual feature sequence and the text feature sequence to obtain a vector inner product; a determination submodule, used to obtain the third loss function based on the first vector length, the second vector length and the vector inner product.

[0020] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any one of the above-mentioned method embodiments.

[0021] In order to achieve the above-mentioned purpose, according to another aspect of the present application, there is further provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned method embodiments.

[0022] In the embodiment of the present application, the following steps are adopted: acquiring the target remote sensing image and target prompt word of the target object's pending loan business, wherein the target prompt word is used to indicate that the target model processes the target remote sensing image; determining the target model based on the task type corresponding to the target remote sensing image, wherein the target model is composed of the output layer corresponding to the basic model and the task type, and the task type is one of the following: object recognition, target detection and semantic segmentation; processing the target remote sensing image and the target prompt word through the target model to obtain the processing result; determining the risk of the target object handling the loan business based on the processing result, solving the technical problem that the risk assessment of the remote sensing image corresponding to the financial business is performed manually in the related technology, resulting in a relatively low risk assessment efficiency. In this scheme, the target remote sensing image of the target object's pending loan business is collected and the target prompt word is obtained, and then the target model is determined according to the task type corresponding to the target remote sensing image, and the target remote sensing image and the target prompt word are processed by the target model, and finally the risk of the target object handling the loan business is determined according to the processing result, avoiding the analysis of the remote sensing image corresponding to the financial business based on the manual method, improving the processing efficiency of the remote sensing image, and thus achieving the effect of improving the risk assessment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0024] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a processing method for financial services provided in an embodiment of the present application;

[0025] Figure 2 is a flowchart of a method for processing financial services provided in an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of training of a basic model provided according to an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a financial service processing device provided according to an embodiment of the present application;

[0028] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0032] Example 1

[0033] According to an embodiment of the present application, a method embodiment for processing financial business is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1The hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for processing financial services is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0035] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the processing method of financial services in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the processing method of the above-mentioned financial services. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] The display may be a touch screen type liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0039] Under the above operating environment, this application provides Figure 2 The processing method of financial business shown. Figure 2 1 is a flowchart of a method for processing financial services according to Embodiment 1 of the present application. The method includes:

[0040] Step S201, obtaining a target remote sensing image and a target prompt word of a target object's pending loan business, wherein the target prompt word is used to instruct a target model to process the target remote sensing image.

[0041] Optionally, remote sensing technology is increasingly used in the field of industrial / agricultural credit of financial institutions. Therefore, when a user needs to apply for industrial / agricultural credit, it is necessary to collect remote sensing images related to the user's domain loan business to be processed. For example, in industrial credit scenarios, hydropower station construction monitoring requires the collection of hydropower station remote sensing images, industrial park construction monitoring requires the collection of industrial park remote sensing images, and so on. For example, in agricultural credit scenarios, agricultural greenhouse construction monitoring tasks require the collection of agricultural greenhouse remote sensing images, and offshore fish cage construction monitoring requires the collection of fish cage remote sensing images, and so on. Then obtain the target prompt word (i.e. prompt). It should be noted that the target prompt word is used to instruct the target model to process the target remote sensing image, for example. Instruct the target model to classify and parse each pixel in the target remote sensing image, and finally give the classification information of the target object corresponding to different positions.

[0042] Step S202, based on the task type corresponding to the target remote sensing image, determine the target model, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection and semantic segmentation.

[0043] Optionally, according to the task type corresponding to the target remote sensing image, for example, object recognition, target detection, and semantic segmentation, the output layer corresponding to the task type is determined, and then the above-mentioned target model is determined according to the output layer and the basic model. For example, the output layer may include a detection head, a classification head, or a segmentation head network.

[0044] Step S203, processing the target remote sensing image and the target prompt word by using the target model to obtain a processing result.

[0045] Optionally, after obtaining the target model, the target remote sensing image and target prompt words are input into the target model, and the target remote sensing image and target prompt words are processed by the target model to obtain corresponding processing results. For example, if it is a land object recognition task, the corresponding processing result is what land objects are included in the target remote sensing image.

[0046] Step S204: based on the processing result, determine the risk of the target object applying for a loan.

[0047] Optionally, the risk of the target object applying for a loan is determined based on the processing result. For example, the processing result may be the area of ​​the target object's crops used for agricultural loans, and the risk of the target object applying for a loan is determined based on the area information.

[0048] To summarize, the target remote sensing image of the target object's loan business to be processed and the target prompt word are collected, and then the target model is determined according to the task type corresponding to the target remote sensing image. The target remote sensing image and the target prompt word are processed through the target model, and finally the risk of the target object's loan business is determined according to the processing result. This avoids the manual analysis of the remote sensing images corresponding to the financial business, improves the processing efficiency of the remote sensing images, and thus achieves the effect of improving the efficiency of risk assessment.

[0049] Optionally, in the financial business processing method provided in the embodiment of the present application, the following steps are used to obtain the basic model: obtain sample remote sensing images and sample prompt words; construct a training sample set based on the sample remote sensing images and sample prompt words; train the initial model through the training sample set to obtain the basic model.

[0050] In an optional embodiment, the following steps are used to train a basic model: obtain sample remote sensing images and sample prompt words, and train the initial model based on the sample remote sensing images and the sample prompt words to obtain a basic model. It should be noted that the basic model includes a visual-language encoder and a visual-language decoder. The basic model can more accurately capture the relevant features of the sample remote sensing images and the sample prompt words, which helps to improve the accuracy of the subsequent remote sensing image processing.

[0051] Optionally, in the financial business processing method provided in the embodiment of the present application, the initial model is trained by a training sample set to obtain a basic model, including: encoding a sample remote sensing image by an image encoder in the initial model to obtain a visual feature sequence; encoding a sample prompt word by a text encoder in the initial model to obtain a text feature sequence; decoding processing is performed by an image decoder in the initial model based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image; decoding processing is performed by a text decoder in the initial model based on the visual feature sequence and the text feature sequence to obtain a reconstructed prompt word; and the initial model is trained based on the reconstructed remote sensing image and the reconstructed prompt word to obtain a basic model.

[0052] In an optional embodiment, training the initial model using the training sample set includes the following steps: using the image encoder in the initial model to encode the sample remote sensing image to obtain a visual feature sequence, for example, the image encoder can be a 12-layer ViT-B / 16 structure (B refers to the Base model size, which specifies the width and depth of the model, that is, the number of Transformer layers (encoder blocks) and the number of hidden units in each layer. 16 refers to dividing the image into patches of 16x16 pixels). The visual feature sequence can be

[0053] The sample prompt words are encoded by the text encoder in the initial model to obtain a text feature sequence. For example, the text encoder adopts a 12-layer BERT-Base model structure. After the input natural language text prompt words are processed by the text encoder, the text semantic feature sequence is extracted as follows:

[0054] After obtaining the above-mentioned visual feature sequence and text feature sequence, the image decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image. For example, the image decoder is a structure of two shallow transformer decoders. After the image decoder fuses the visual feature sequence and the text feature sequence, the remote sensing image is reconstructed according to the fused feature output.

[0055] The text decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain the reconstructed prompt word. For example, the text decoder is two shallow transformer decoder structures. After the text encoder fuses the visual feature sequence and the text feature sequence, it reconstructs the prompt word according to the fused feature output.

[0056] Finally, the initial model is trained according to the reconstructed remote sensing image and the reconstructed prompt words to obtain the basic model.

[0057] By introducing rich semantic information of natural language features on the basis of visual features, a stable and efficient supervision signal is provided to guide the model to learn feature representations with good generalization, thereby improving the image processing performance of subsequent models.

[0058] Optionally, in the financial business processing method provided in the embodiment of the present application, the image decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image, including: performing feature fusion of the visual feature sequence through the masked self-attention layer in the image decoder to obtain a first feature sequence; performing feature fusion of the first feature sequence and the text feature sequence through the cross-attention layer in the image decoder to obtain a second feature sequence; performing feature transformation of the second feature sequence through the multilayer perceptron in the image decoder to obtain a reconstructed remote sensing image.

[0059] In an optional embodiment, the decoding process is performed by the following steps: the image decoder includes a masked self-attention layer, a cross-attention layer, and a multi-layer perceptron. The visual feature sequence is further fused through the masked self-attention layer, and then the first feature sequence and the text feature sequence are multimodally fused through the cross-attention layer to obtain a second feature sequence. Finally, the multi-layer perceptron performs feature transformation on the second feature sequence to obtain a reconstructed remote sensing image.

[0060] In an optional embodiment, the following steps can be used to obtain the reconstructed prompt word: receiving a text feature sequence, further fusing the text features through a masked self-attention layer, and then performing multimodal feature fusion on the fused text features and the visual feature sequence through a cross-attention layer, and finally completing the feature transformation through a multi-layer perceptron to generate an autoregressive text sequence to obtain the final reconstructed prompt word.

[0061] The masked self-attention layer and the cross-attention layer enable the initial model to learn multimodal features more accurately and improve the performance of the model.

[0062] Optionally, in the financial business processing method provided in the embodiment of the present application, the second feature sequence is subjected to feature transformation through a multilayer perceptron in an image decoder to obtain a reconstructed remote sensing image, including: the second feature sequence is subjected to feature transformation through a multilayer perceptron to obtain a plurality of reconstructed image blocks; and the plurality of reconstructed image blocks are spliced ​​according to a preset order to obtain a reconstructed remote sensing image.

[0063] In an optional embodiment, the following steps are used to obtain the above-mentioned reconstructed remote sensing image: First, the multi-layer perceptron performs feature transformation on the second feature sequence to obtain multiple reconstructed image blocks, that is, the second feature sequence is converted into 16*16 image blocks (patch), and then spliced ​​in the sequence generation order to finally obtain the reconstructed image. The remote sensing image can be effectively reconstructed by performing feature transformation and image block splicing processing through MLP.

[0064] Optionally, in the financial business processing method provided in the embodiment of the present application, the initial model is trained based on the reconstructed remote sensing image and the reconstructed prompt words to obtain the basic model, including: calculating based on the reconstructed remote sensing image and the sample remote sensing image to obtain the first loss function; calculating based on the reconstructed prompt words and the sample prompt words to obtain the second loss function; calculating based on the visual feature sequence and the text feature sequence to obtain the third loss function; training the initial model based on the first loss function, the second loss function and the third loss function to obtain the basic model.

[0065] In an optional embodiment, the initial model is trained according to the reconstructed remote sensing image and the reconstructed prompt word to obtain the basic model, including the following steps: a first loss function is calculated according to the reconstructed remote sensing image and the sample remote sensing image. For example, the first loss function is:

[0066]

[0067] Among them, y i and i ' are the pixel values ​​at position i of the original input image and the reconstructed image, respectively, and n is the total number of pixels in the image.

[0068] The second loss function is calculated based on the reconstruction prompt word and the sample prompt word. For example, the second loss function is:

[0069]

[0070] Among them, u j To reconstruct the words with heavy hints, P(u j |u j-K ,...,u j-1 ; θ) means that given the model parameters θ and the previous u j-K ,...,u j-1 Under the condition of words, the uth j The conditional probability of a word. K is the size of the context sliding window (the default is 2048, which can be adjusted according to the scenario).

[0071] The third loss function is calculated based on the visual feature sequence and the text feature sequence. The third loss function can be used to effectively integrate the visual-linguistic multimodal features.

[0072] Finally, the initial model is trained according to the first loss function, the second loss function and the third loss function to obtain the basic model.

[0073] Through image loss, text loss, and visual-language multimodal feature fusion loss, the initial model can be effectively iteratively trained to improve the training effect of the initial model.

[0074] Optionally, in the financial business processing method provided in the embodiment of the present application, calculation is performed based on the visual feature sequence and the text feature sequence to obtain the third loss function, including: calculating the vector length of the visual feature sequence to obtain a first vector length; calculating the vector length of the text feature sequence to obtain a second vector length; performing multiplication calculation on the visual feature sequence and the text feature sequence to obtain a vector inner product; obtaining the third loss function based on the first vector length, the second vector length and the vector inner product.

[0075] In an optional embodiment, the third loss function can be obtained by the following steps: the vector length of the visual feature sequence is calculated to obtain the first vector length and the vector length of the text feature sequence is calculated to obtain the second vector length, then, the visual feature sequence and the text feature sequence are multiplied to obtain the vector inner product, and finally the third loss function is obtained based on the first vector length, the second vector length and the vector inner product.

[0076] For example, the third loss function is

[0077] Among them, the temperature coefficient τ is a hyperparameter (which can be set according to different business requirements), and the denominator is the vector inner product of the visual features and text features of each image-text pair, are the lengths of the visual and text feature vectors corresponding to each image-text pair; is the vector inner product of the visual features and text features of the current input image-text pair, are the lengths of the corresponding input visual and text feature vectors, respectively.

[0078] In an optional embodiment, the following may be used: Figure 3The schematic diagram shown realizes the training of the basic model, which specifically includes: the input image is extracted through the image encoder to extract the visual feature sequence, the input text prompt word is extracted through the text encoder to extract the text semantic feature sequence, and the visual text contrast learning loss is calculated; the visual feature sequence is input into the image decoder, and is fused with the text feature sequence through the masked self-attention layer and the cross-attention feature, and then the feature transformation is completed through the multi-layer perceptron, and the image autoregressive generation sequence is output, and then the vector is sequentially converted into 16*16 image blocks (patch) and sequentially spliced ​​to reconstruct the image; according to the original image and the reconstructed image, the image autoregressive reconstruction generation loss is calculated; the text feature sequence is input into the text decoder, and is fused with the visual feature sequence through the masked self-attention layer and the cross-attention feature, and then the feature transformation is completed through the multi-layer perceptron to generate an autoregressive text sequence, and then the vector is sequentially de-embedded back into the vocabulary and spliced ​​to generate a text sentence, and the autoregressive text sequence generation loss is calculated; the gradient is back-propagated to update the weight to obtain the final basic model.

[0079] The processing method of financial business provided by the embodiment of the present application, by acquiring the target remote sensing image and target prompt word of the loan business to be processed by the target object, wherein the target prompt word is used to indicate that the target model processes the target remote sensing image; based on the task type corresponding to the target remote sensing image, the target model is determined, wherein the target model is composed of the output layer corresponding to the basic model and the task type, and the task type is one of the following: object recognition, target detection and semantic segmentation; the target remote sensing image and the target prompt word are processed by the target model to obtain the processing result; based on the processing result, the risk of the target object handling the loan business is determined, which solves the technical problem that the risk assessment of the remote sensing image corresponding to the financial business is performed manually in the related technology, resulting in a relatively low risk assessment efficiency. In this scheme, the target remote sensing image of the loan business to be processed by the target object is collected and the target prompt word is obtained, and then the target model is determined according to the task type corresponding to the target remote sensing image, and the target remote sensing image and the target prompt word are processed by the target model, and finally the risk of the target object handling the loan business is determined according to the processing result, which avoids the analysis of the remote sensing image corresponding to the financial business based on the manual method, improves the processing efficiency of the remote sensing image, and thus achieves the effect of improving the risk assessment efficiency.

[0080] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0081] Example 2

[0082] The embodiment of the present application also provides a financial service processing device. It should be noted that the financial service processing device of the embodiment of the present application can be used to execute the financial service processing method provided in the embodiment of the present application. The financial service processing device provided in the embodiment of the present application is introduced below.

[0083] According to an embodiment of the present application, a device for implementing the above-mentioned financial service processing method is also provided. Figure 4 As shown, the device includes: an acquisition unit 401, a first determination unit 402, a processing unit 403 and a second determination unit 404.

[0084] An acquisition unit 401 is used to acquire a target remote sensing image and a target prompt word of a loan business to be processed by a target object, wherein the target prompt word is used to instruct a target model to process the target remote sensing image;

[0085] A first determining unit 402 is used to determine a target model based on a task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection, and semantic segmentation;

[0086] The processing unit 403 is used to process the target remote sensing image and the target prompt word through the target model to obtain a processing result;

[0087] The second determining unit 404 is used to determine the risk of the target object applying for a loan business based on the processing result.

[0088] The financial business processing device provided in the embodiment of the present application obtains the target remote sensing image and target prompt word of the loan business to be processed by the target object through the acquisition unit 401, wherein the target prompt word is used to instruct the target model to process the target remote sensing image; the first determination unit 402 determines the target model based on the task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection and semantic segmentation; the processing unit 403 processes the target remote sensing image and the target prompt word through the target model to obtain a processing result; the second determination unit 404 determines the risk of the target object processing the loan business based on the processing result, thereby solving the technical problem that the risk assessment of the remote sensing images corresponding to the financial business is performed manually in the related technology, resulting in relatively low risk assessment efficiency. In this scheme, the target remote sensing image of the target object's pending loan business and the target prompt word are collected, and then the target model is determined according to the task type corresponding to the target remote sensing image. The target remote sensing image and the target prompt word are processed through the target model, and finally the risk of the target object's loan business is determined according to the processing results. This avoids manual analysis of the remote sensing images corresponding to the financial business, improves the processing efficiency of the remote sensing images, and thus achieves the effect of improving the efficiency of risk assessment.

[0089] Optionally, in the financial business processing device provided in the embodiment of the present application, the following devices are used to obtain the basic model: an acquisition unit, used to acquire sample remote sensing images and sample prompt words; a construction unit, used to construct a training sample set based on the sample remote sensing images and sample prompt words; a training unit, used to train the initial model through the training sample set to obtain the basic model.

[0090] Optionally, in the financial business processing device provided in the embodiment of the present application, the training unit includes: a first encoding subunit, used to encode the sample remote sensing image through the image encoder in the initial model to obtain a visual feature sequence; a second encoding subunit, used to encode the sample prompt words through the text encoder in the initial model to obtain a text feature sequence; a first decoding subunit, used to perform decoding processing based on the visual feature sequence and the text feature sequence through the image decoder in the initial model to obtain a reconstructed remote sensing image; a second decoding subunit, used to perform decoding processing based on the visual feature sequence and the text feature sequence through the text decoder in the initial model to obtain a reconstructed prompt word; a training subunit, used to train the initial model based on the reconstructed remote sensing image and the reconstructed prompt word to obtain a basic model.

[0091] Optionally, in the financial business processing device provided in the embodiment of the present application, the first decoding subunit includes: a first fusion module, used to perform feature fusion on the visual feature sequence through the masked self-attention layer in the image decoder to obtain a first feature sequence; a second fusion module, used to perform feature fusion on the first feature sequence and the text feature sequence through the cross-attention layer in the image decoder to obtain a second feature sequence; a transformation module, used to perform feature transformation on the second feature sequence through a multi-layer perceptron in the image decoder to obtain a reconstructed remote sensing image.

[0092] Optionally, in the financial business processing device provided in the embodiment of the present application, the transformation module includes: a transformation submodule, used to perform feature transformation on the second feature sequence through a multilayer perceptron to obtain multiple reconstructed image blocks; and a splicing submodule, used to perform splicing processing on the multiple reconstructed image blocks according to a preset order to obtain a reconstructed remote sensing image.

[0093] Optionally, in the financial business processing device provided in the embodiment of the present application, the training subunit includes: a first calculation module, used to calculate based on the reconstructed remote sensing image and the sample remote sensing image to obtain a first loss function; a second calculation module, used to calculate based on the reconstructed prompt words and the sample prompt words to obtain a second loss function; a third calculation module, used to calculate based on the visual feature sequence and the text feature sequence to obtain a third loss function; a training module, used to train the initial model based on the first loss function, the second loss function and the third loss function to obtain a basic model.

[0094] Optionally, in the financial business processing device provided in the embodiment of the present application, the third calculation module includes: a first calculation sub-module, used to calculate the vector length of the visual feature sequence to obtain a first vector length; a second calculation sub-module, used to calculate the vector length of the text feature sequence to obtain a second vector length; a third calculation sub-module, used to perform product calculation on the visual feature sequence and the text feature sequence to obtain a vector inner product; a determination sub-module, used to obtain a third loss function based on the first vector length, the second vector length and the vector inner product.

[0095] It should be noted that the acquisition unit 401, the first determination unit 402, the processing unit 403 and the second determination unit 404 correspond to steps S201 to S204 in the first embodiment, and the four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the first embodiment. It should be noted that the modules or units may be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the modules may also be part of the device and run in the computer terminal 10 provided in the first embodiment.

[0096] Example 3

[0097] An embodiment of the present application may provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (only one is shown) processor 502, memory 504, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0098] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0099] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtaining a target remote sensing image and a target prompt word of a loan business to be processed by the target object, wherein the target prompt word is used to instruct the target model to process the target remote sensing image; determining a target model based on a task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection, and semantic segmentation; processing the target remote sensing image and the target prompt word through the target model to obtain a processing result; and determining the risk of the target object processing the loan business based on the processing result.

[0100] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: the basic model is obtained by adopting the following steps: obtaining sample remote sensing images and sample prompt words; constructing a training sample set based on the sample remote sensing images and sample prompt words; training the initial model through the training sample set to obtain the basic model.

[0101] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: training the initial model through the training sample set to obtain the basic model, including: encoding the sample remote sensing image through the image encoder in the initial model to obtain a visual feature sequence; encoding the sample prompt words through the text encoder in the initial model to obtain a text feature sequence; decoding processing based on the visual feature sequence and the text feature sequence through the image decoder in the initial model to obtain a reconstructed remote sensing image; decoding processing based on the visual feature sequence and the text feature sequence through the text decoder in the initial model to obtain a reconstructed prompt word; training the initial model based on the reconstructed remote sensing image and the reconstructed prompt word to obtain the basic model.

[0102] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: decoding processing is performed based on the visual feature sequence and the text feature sequence by the image decoder in the initial model to obtain a reconstructed remote sensing image, including: feature fusion of the visual feature sequence is performed through the masked self-attention layer in the image decoder to obtain a first feature sequence; feature fusion of the first feature sequence and the text feature sequence is performed through the cross-attention layer in the image decoder to obtain a second feature sequence; feature transformation of the second feature sequence is performed through the multi-layer perceptron in the image decoder to obtain a reconstructed remote sensing image.

[0103] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: performing feature transformation on the second feature sequence through the multi-layer perceptron in the image decoder to obtain a reconstructed remote sensing image, including: performing feature transformation on the second feature sequence through the multi-layer perceptron to obtain multiple reconstructed image blocks; splicing the multiple reconstructed image blocks according to a preset order to obtain a reconstructed remote sensing image.

[0104] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: training the initial model based on the reconstructed remote sensing image and the reconstructed prompt word to obtain the basic model, including: calculating based on the reconstructed remote sensing image and the sample remote sensing image to obtain the first loss function; calculating based on the reconstructed prompt word and the sample prompt word to obtain the second loss function; calculating based on the visual feature sequence and the text feature sequence to obtain the third loss function; training the initial model based on the first loss function, the second loss function and the third loss function to obtain the basic model.

[0105] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: calculate based on the visual feature sequence and the text feature sequence to obtain the third loss function, including: calculate the vector length of the visual feature sequence to obtain the first vector length; calculate the vector length of the text feature sequence to obtain the second vector length; multiply the visual feature sequence and the text feature sequence to obtain the vector inner product; obtain the third loss function based on the first vector length, the second vector length and the vector inner product.

[0106] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0107] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0108] Example 4

[0109] The embodiment of the present application also provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the financial service processing method provided in the first embodiment.

[0110] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0111] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program steps of the processing method of financial business.

[0112] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0118] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing financial services, characterized in that: include: Acquire a target remote sensing image and a target prompt word of a loan business to be processed by a target object, wherein the target prompt word is used to instruct a target model to process the target remote sensing image; Determine the target model based on the task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection, and semantic segmentation; Processing the target remote sensing image and the target prompt word by using the target model to obtain a processing result; Based on the processing result, the risk of the target object applying for the loan business is determined.

2. The method according to claim 1, characterized in that The basic model is obtained by following the steps below: Obtain sample remote sensing images and sample prompt words; Constructing a training sample set according to the sample remote sensing image and the sample prompt words; The initial model is trained using the training sample set to obtain the basic model.

3. The method according to claim 2, characterized in that The initial model is trained by the training sample set to obtain the basic model including: Encoding the sample remote sensing image by the image encoder in the initial model to obtain a visual feature sequence; Encoding the sample prompt words through the text encoder in the initial model to obtain a text feature sequence; The image decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image; The text decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed prompt word; The initial model is trained according to the reconstructed remote sensing image and the reconstruction prompt words to obtain the basic model.

4. The method according to claim 3, characterized in that The image decoder in the initial model performs decoding processing based on the visual feature sequence and the text feature sequence to obtain a reconstructed remote sensing image, including: Performing feature fusion on the visual feature sequence through a masked self-attention layer in the image decoder to obtain a first feature sequence; Performing feature fusion on the first feature sequence and the text feature sequence through a cross attention layer in the image decoder to obtain a second feature sequence; The second feature sequence is subjected to feature transformation by a multi-layer perceptron in the image decoder to obtain the reconstructed remote sensing image.

5. The method according to claim 4, characterized in that Performing feature transformation on the second feature sequence by a multilayer perceptron in the image decoder to obtain the reconstructed remote sensing image includes: Performing feature transformation on the second feature sequence by the multi-layer perceptron to obtain a plurality of reconstructed image blocks; The plurality of reconstructed image blocks are spliced ​​together according to a preset order to obtain the reconstructed remote sensing image.

6. The method according to claim 3, characterized in that The initial model is trained according to the reconstructed remote sensing image and the reconstructed prompt words to obtain the basic model including: Performing calculations based on the reconstructed remote sensing image and the sample remote sensing image to obtain a first loss function; Calculating according to the reconstruction prompt word and the sample prompt word to obtain a second loss function; Calculating according to the visual feature sequence and the text feature sequence to obtain a third loss function; The initial model is trained according to the first loss function, the second loss function and the third loss function to obtain the basic model.

7. The method according to claim 6, characterized in that The third loss function is obtained by calculating according to the visual feature sequence and the text feature sequence: Calculating the vector length of the visual feature sequence to obtain a first vector length; Calculating the vector length of the text feature sequence to obtain a second vector length; Performing product calculation on the visual feature sequence and the text feature sequence to obtain a vector inner product; The third loss function is obtained according to the first vector length, the second vector length and the vector inner product.

8. A financial service processing device, characterized in that: include: An acquisition unit, used to acquire a target remote sensing image and a target prompt word of a loan business to be processed by a target object, wherein the target prompt word is used to instruct a target model to process the target remote sensing image; A first determining unit is used to determine the target model based on a task type corresponding to the target remote sensing image, wherein the target model is composed of a basic model and an output layer corresponding to the task type, and the task type is one of the following: object recognition, target detection, and semantic segmentation; A processing unit, used for processing the target remote sensing image and the target prompt word through the target model to obtain a processing result; The second determining unit is used to determine the risk of the target object handling the loan business based on the processing result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the financial business processing method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program, when running, executes the method for processing financial business described in any one of claims 1 to 7.

Images