Collateral loan risk monitoring method and device and computer program product

Abstract

The invention provides a collateral loan risk monitoring method and device and a computer program product, and the method comprises the steps: obtaining the multi-source data of a collateral, and the multi-source data comprises the collection images collected by a plurality of image collection devices at the same time; pixel offset adjustment is carried out on the acquired images of the multi-source data to obtain a plurality of aligned feature images, and the aligned feature images are in one-to-one correspondence with the acquired images; fusing the plurality of aligned feature images by using a gated feature pyramid fusion network to obtain fused features; and performing risk analysis according to the fusion features to obtain the risk index of the collateral, and sending out early warning according to the risk index, thereby solving the problem that the collateral risk monitoring in the prior art depends on manual risk judgment.

Description

Technical Field

[0001] This application relates to the field of risk assessment technology, and more specifically, to a method, apparatus, computer-readable storage medium, and computer program product for monitoring risks in collateralized loans. Background Technology

[0002] As collateral for financial loans becomes more diversified, financial risks also become more complex. The valuation of collateral for loans such as real estate development loans, livestock breeding loans, and winery tank loans is rapidly evolving from the old paradigm of "one-time manual on-site inspection, static filing of paper ownership certificates, and delayed quarterly assessment reports" to a new paradigm of "all-weather dynamic perception, continuous data-driven, and intelligent risk early warning." However, manual photography and measurement, paper ownership certificates, and infrequent quarterly assessment reports not only fail to promptly identify the value depreciation of collateral due to natural disasters, market fluctuations, or changes in use during its existence, but also fail to provide effective feedback within hours to days after sudden events such as typhoons and fires. Faced with the high concurrency, rapid transmission, and unpredictable new risks brought about by frequent extreme weather events and urban renewal, financial institutions can only passively wait for losses to be exposed before initiating post-loan disposal. This lagging information chain can no longer meet the requirements of supervision and management for early identification, early intervention, and early resolution of risks.

[0003] Meanwhile, banks, asset management companies, and insurance companies have attempted to incorporate structured data such as macroeconomic indicators, regional transaction characteristics, and borrower profiles into their internal risk control systems. Through machine learning, they have achieved preliminary quantitative calculations of valuation fluctuations by updating default probabilities and loss expectations on a T+1 daily basis. However, a significant gap still exists between the physical and digital worlds. Satellite remote sensing can cover millions of square kilometers but is limited by meter-level resolution; drones can provide centimeter-level details but are constrained by airspace and weather; fixed cameras can transmit on-site images in minutes but have a fixed perspective and a single spectrum; and infrared thermal imaging can detect hidden leaks but is difficult to accurately correspond to property units. The huge differences in spatiotemporal benchmarks, spatial resolution, spectral bands, and update frequencies of these multi-source heterogeneous data mean that existing systems can only integrate them in a coarse-grained manner through offline stitching and manual comparison. This results in a large number of "visible damages" such as roof cracks, hidden additions, and heat leaks being missed. Furthermore, the risk scores given by deep models are difficult to justify in supervisory and management spot checks due to the lack of traceable and auditable interpretation paths. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, computer-readable storage medium, and computer program product for monitoring the risk of collateralized loans, so as to at least solve the problem that the existing collateralized risk monitoring relies on manual risk assessment.

[0005] To achieve the above objectives, according to one aspect of this application, a method for monitoring the risk of collateralized loans is provided, comprising: acquiring multi-source data of the collateral, the multi-source data including images acquired by multiple image acquisition devices at the same time; adjusting the pixel offset of the acquired images of the multi-source data to obtain multiple aligned feature images, the aligned feature images corresponding one-to-one with the acquired images; fusing the multiple aligned feature images using a gated feature pyramid fusion network to obtain fused features; performing risk analysis based on the fused features to obtain a risk index of the collateral, and issuing an early warning based on the risk index.

[0006] Optionally, the acquired images from the multi-source data are subjected to pixel offset adjustment to obtain multiple aligned feature images, including: calculating the sub-pixel offset of each pixel in each target image based on a reference image using a cascaded offset field, wherein the reference image is the acquired image with the highest resolution in the multi-source data, and the target image is the acquired image in the multi-source data other than the reference image; re-evaluating each pixel of the target image according to the corresponding sub-pixel offset to obtain an adjusted target image; and combining all the adjusted target images and the reference image into multiple aligned feature images.

[0007] Optionally, re-evaluating each pixel of the target image according to the corresponding sub-pixel offset to obtain an adjusted target image includes: re-evaluating each pixel of the target image according to the corresponding sub-pixel offset to obtain an initial adjusted target image; stitching the initial adjusted target image and the reference image to obtain a stitched image; generating a confidence map corresponding to the stitched image based on the stitched image, wherein the value of each pixel in the confidence map is a quantized score of the alignment reliability of the corresponding pixel in the stitched image; and adjusting the parameters of the cascaded offset field when the proportion of pixels in the confidence map that are less than the confidence threshold is greater than or equal to a predetermined proportion, until the proportion of pixels in the adjusted confidence map that are less than the confidence threshold is less than the predetermined proportion, and determining the initial adjusted target image corresponding to the current adjustment as the adjusted target image.

[0008] Optionally, a gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features, including: converting multiple aligned feature images into feature tensors to obtain continuous risk embedding vectors; constructing a spatial pyramid pooling layer to extract multi-scale features from the continuous risk embedding vectors to obtain multiple extracted features, wherein the extracted features correspond one-to-one with the scale; generating fusion weights corresponding to the extracted features using a gated weight generation function; fusing multiple extracted features using the fusion weights to obtain aggregated features; generating a global risk embedding vector and a pixel-level heatmap based on the aggregated features, and generating multiple modal-level contribution weights based on the multiple aligned feature images, wherein the global risk embedding vector is a feature vector describing risk, the pixel-level heatmap is a visualization image with a single pixel as the smallest unit and temperature mapped by color gradient, and the modal-level contribution weights are used to characterize the degree of influence of different modalities of the multi-source data on risk.

[0009] Optionally, risk analysis is performed based on the fusion features to obtain a risk index of the collateral, and an early warning is issued based on the risk index, including: decoding the global risk embedding vector into multiple risk indices through an orthogonal decomposition matrix, wherein the orthogonal decomposition matrix is ​​obtained by training with sample data, the sample data including historical multi-source data and corresponding historical risk indices; and issuing an early warning information for the risk type corresponding to the risk index when at least one of the risk indices is greater than a predetermined threshold, wherein the risk type includes at least fire and external damage.

[0010] Optionally, after performing risk analysis based on the fusion features to obtain the risk index of the collateral and issuing an early warning based on the risk index, the method further includes: in the case of issuing the early warning information for the fire, analyzing the temperature change trend based on the pixel-level heat map; in the case of issuing the early warning information for the external force damage, determining the largest modal-level contribution weight as the main contributing mode, and analyzing the cause of the external force damage based on the acquired image corresponding to the main contributing mode.

[0011] Optionally, obtaining multi-source data of the collateral includes: obtaining original multi-source data of the collateral; converting the acquisition time of the original multi-source data to the same time on the same time axis; and converting the images of the original multi-source data to images of the same resolution to obtain the multi-source data.

[0012] According to another aspect of this application, a collateral loan risk monitoring device is provided, comprising: an acquisition unit for acquiring multi-source data of the collateral, the multi-source data including images acquired by multiple image acquisition devices at the same time; an adjustment unit for adjusting the pixel offset of the acquired images of the multi-source data to obtain multiple aligned feature images, the aligned feature images corresponding one-to-one with the acquired images; a fusion unit for fusing the multiple aligned feature images using a gated feature pyramid fusion network to obtain fused features; and an analysis unit for performing risk analysis based on the fused features to obtain a risk index of the collateral, and issuing an early warning based on the risk index.

[0013] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0014] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the methods described.

[0015] Applying the technical solution of this application, the above-mentioned collateral loan risk monitoring method has limitations in the images acquired by a single image acquisition device, which may lead to missed risk assessments. For example, an inappropriate shooting angle may result in the failure to capture clear cracks. Furthermore, the differences in angle and resolution of images acquired by different image acquisition devices mean that existing systems can only integrate images in a coarse-grained manner through offline stitching and manual comparison. This method, by adjusting the pixel offset of the acquired images from multiple sources, avoids misalignment in the stitching of images from multiple sources, thereby facilitating the fusion of multiple aligned feature images to obtain fused features. Risk analysis is then performed based on the fused features, and warnings are issued. This ensures the accuracy of risk warnings while eliminating the need for manual risk assessment, thus solving the problem of existing collateral risk monitoring relying on manual risk assessment. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a collateral loan risk monitoring method according to an embodiment of this application is shown;

[0018] Figure 2 A flowchart illustrating a risk monitoring method for collateralized loans according to an embodiment of this application is shown.

[0019] Figure 3 A schematic diagram of a remote collateral loan risk monitoring system based on multi-source data, according to an embodiment of this application, is shown.

[0020] Figure 4 The diagram illustrates a heterogeneous data access and spatiotemporal indexing module and its inputs and outputs according to an embodiment of this application.

[0021] Figure 5 A schematic diagram of a deformable pyramid alignment module subunit provided according to an embodiment of this application is shown;

[0022] Figure 6 A schematic diagram of a three-layer decoding structure for a risk identification module according to an embodiment of this application is shown;

[0023] Figure 7 A structural block diagram of a collateral loan risk monitoring device provided according to an embodiment of this application is shown.

[0024] The above figures include the following reference numerals:

[0025] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] Multi-source heterogeneous data: Data sets with different sources, structures, and formats. In this invention, it specifically refers to four types of spatiotemporally asynchronous data: satellite remote sensing, UAVs, video monitoring, and infrared thermal imaging. With future technological advancements, other available data sources can be dynamically integrated.

[0031] Bilinear interpolation: a position resampling method based on linear weighting of adjacent pixels.

[0032] Deformable Pyramid: A multi-level deformation modeling architecture that achieves geometric correction by progressively refining the offset field.

[0033] Cascaded offset field: A multi-level linked spatial deformation vector field that describes the pixel position correction amount.

[0034] Gated Feature Pyramid Network (GFPN): The Gated Feature Pyramid Network is a model design that introduces a gating mechanism on the basis of the classic feature pyramid. The Feature Pyramid Network (FPN) is a basic structure in target detection and segmentation tasks. It improves the model's ability to recognize targets of different sizes by fusing multi-scale features. Gating units (such as GRU and gated convolution) are often used for feature selection and dynamic fusion.

[0035] As described in the background section, existing collateral risk monitoring relies on manual risk assessment. To address this issue, embodiments of this application provide a collateral loan risk monitoring method, apparatus, computer-readable storage medium, and computer program product.

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0037] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a collateral loan risk monitoring method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the collateral loan risk monitoring method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one instance, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0039] This embodiment provides a method for monitoring the risk of collateralized loans that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 2 This is a flowchart of a collateral loan risk monitoring method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0041] Step S201: Obtain multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0042] Specifically, the collateral consists of multi-source data composed of images collected by satellite remote sensing, video surveillance, drones, and infrared detection. The collateral can be base liquor, etc.

[0043] Step S202: The pixel offset of the acquired images of the above multi-source data is adjusted to obtain multiple aligned feature images, and the above aligned feature images correspond one-to-one with the above acquired images.

[0044] Specifically, such as Figure 3 As shown, the variable pyramid alignment module adjusts the pixel offset of the acquired images from the above multi-source data to align images of different resolutions and perspectives to the same high-resolution coordinate system, thereby obtaining multiple aligned feature images.

[0045] Step S203: A gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features;

[0046] Specifically, such as Figure 3 As shown, the cross-modal spatiotemporal fusion module uses a gated feature pyramid fusion network to receive the upstream "aligned but multi-source separated" data, extract features, and output a unified and interpretable risk representation downstream to obtain fused features.

[0047] Step S204: Perform risk analysis based on the above-mentioned fusion characteristics to obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0048] Specifically, such as Figure 3 As shown, the risk identification module outputs a binary interpretation of risk representation, describing risk factors in the data in the form of identification, and providing a visible risk identification capability.

[0049] In this embodiment, the image acquired by a single image acquisition device in the above-mentioned collateral loan risk monitoring method has limitations, which may lead to missed risk assessments. For example, an inappropriate shooting angle may result in the failure to capture a clear image of a crack. Furthermore, the differences in angle and resolution of images acquired by different image acquisition devices mean that existing systems can only integrate images in a coarse-grained manner through offline stitching and manual comparison. This method adjusts the pixel offset of the acquired images from multiple sources to avoid misalignment of the stitched images, thereby facilitating the fusion of multiple aligned feature images to obtain fused features. Risk analysis is then performed based on the fused features, and warnings are issued. This ensures the accuracy of risk warnings while eliminating the need for manual risk assessment, thus solving the problem of existing collateral risk monitoring relying on manual risk assessment.

[0050] To achieve the above-mentioned normalization processing of multi-source data, in one optional implementation, step S201 includes:

[0051] Step S2011: Obtain the original multi-source data of the collateral;

[0052] Step S2012: Convert the acquisition time of the above-mentioned original multi-source data to the same time on the same time axis, and convert the images of the above-mentioned original multi-source data into images of the same resolution to obtain the above-mentioned multi-source data.

[0053] In the above embodiments, such as Figure 3 As shown, before the deformable pyramid alignment module performs alignment processing, the heterogeneous data access and spatiotemporal indexing module is mainly used to access different types of raw risk monitoring data. It processes different flows of structured and unstructured data simultaneously in an offline and real-time parallel manner to adapt to the heterogeneity of multi-source data, while also normalizing the spatiotemporal data. Specifically, the heterogeneous data access and spatiotemporal indexing module, acting as the "data bus" and "spatiotemporal registry center" of the entire collateral risk monitoring system, is mainly responsible for the following tasks: 1. Protocol parsing, metadata extraction, and quality verification of multi-source heterogeneous data; 2. Registering all observation units with unique keys based on a unified spatiotemporal benchmark; 3. Generating indexes and data passports for subsequent algorithm layers to ensure spatiotemporal consistency during cross-module, cross-language, and cross-process calls.

[0054] Specifically, such as Figure 4As shown, the time normalization unit: Based on the fundamental conditions of inconsistent time bases of heterogeneous sensors, sampling clock drift, and cross-domain errors, this module adopts a "single time axis, drift-free, interpolable" time coordinate system. The time normalization unit provides repeatable and auditable time alignment capabilities. It includes the following five processors: a reference anchor: using the first successfully parsed sensor recording time T0 as the global zero time; an absolute timescale converter: uniformly converting multiple source fields such as UTC, GPS weeks, NTP timestamps, and camera RTC time into Unix nanosecond integers; and a relative timescale calculator: for any recording time T... i Generate normalized time offset Δt i = (T i -T0) / 1ns, drift compensator: optionally, using a PTP or GNSS 1PPS pulse to adjust Δt i Sub-millisecond drift correction; Extended interpolator: based on Δt i The sequence uses linear or cubic spline interpolation to generate virtual timestamps for missing sampling points, with an error of less than or equal to 1 ms. The input and output of the time normalization unit can be summarized as shown in Tables 1 and 2. Spatial normalization unit: Under the conditions of spatial misalignment and scale misalignment caused by differences in field of view, geometric imaging, projection system, and ground sampling distance between satellite, UAV, remote sensing, and video sensors, this module provides a spatial normalization unit with "single coordinate system, single resolution, and single elevation benchmark," enabling different pixels and units to perform cross-membrane calculations within the same geometric framework. The system includes the following five processors: a) Coordinate reference unit: Binds multi-source images to a unified spatial reference through a single parametric reprojection and least-squares adjustment, ensuring residuals <1cm; b) Resolution tuner: Using 0.5m as the sole reference, pools and shrinks coarse pixels such as satellite images and convolves and enlarges fine pixels such as drone images, aligning resolution at the physical scale; c) Tile divider: Divides the image into standard 256×256 tiles, performs zero-padding at the edges, and records the effective range, ensuring reading efficiency and cross-platform index consistency; d) Integrity checker: Calculates SHA256 for the tile pixel matrix and performs hash verification with the data passport; if they do not match, resampling is performed; e) Spatial indexer: Creates a composite index for the database {spatial dimension, temporal dimension}, exposing a RESTful query interface.

[0055] Table 1

[0056]

[0057] Table 2

[0058]

[0059] To achieve automatic alignment, in one optional implementation, step S202 above includes:

[0060] Step S2021: Calculate the sub-pixel offset of each pixel in each target image based on the reference image using the cascaded offset field. The reference image is the acquired image with the highest resolution among the multi-source data, and the target image is the acquired image other than the reference image among the multi-source data.

[0061] Step S2022: Re-value each pixel of the target image according to the corresponding sub-pixel offset to obtain the adjusted target image;

[0062] Step S2023: Combine all the adjusted target images and the reference images into multiple alignment feature images.

[0063] In the above embodiments, the spatial normalization unit can unify the input image to a fixed ground sampling distance, such as... Figure 5 As shown, the deformable pyramid alignment module needs to further provide the ability to automatically align under "multi-scale, cross-module, and sub-pixel" conditions. It includes the following sequentially coupled sub-units: A multi-scale pyramid construction unit constructs an L-layer feature pyramid for a single-source image, halving the resolution layer by layer. The i-th layer features are obtained through convolution-pooling operations. The specific process is as follows: a 3×3 GroupNorm-ReLU6 convolution is performed on the single-source image to extract features from channels 32-128, followed by halving the resolution layer by layer through 2×2 average pooling, forming a multi-level feature pyramid. Convolution and pooling are sequentially coupled to ensure that texture and edge information are not lost during downsampling, and the weights and activations are uniformly quantized to INT8 and cached for subsequent direct reuse. A cascaded offset field calculation unit performs offset calculations in a "coarse-to-fine" manner, using the highest-resolution drone as a reference for alignment by default. At the top layer, a 3×3 convolution is used to estimate the coarse offset ΔP of the image. i ; will ΔP i After 2× nearest neighbor upsampling ΔP i As the "initial guess" for the next layer, it is obtained by adding the feature difference of the next layer. Repeat the upsampling, fine-tuning, and summing process until the finest layer ΔP is obtained. i At this point, each pixel has an independent sub-pixel offset. The deformable convolutional execution unit takes a given feature map F to be aligned and a pixel-wise offset ΔP, and re-samples each point of F according to the sub-pixel position specified by ΔP, outputting a "aligned" feature map. That is, perform the following for each output position (i,j): Where K is a 3×3 conventional convolutional sampling network; (k x , k j ) is a fixed offset of a 3×3 grid; F(x) is the subpixel value read using bilinear interpolation; w kThese are the convolution weights; M[i,j] is the confidence weight, which defaults to 1 if it does not exist.

[0064] To ensure alignment reliability, in one optional implementation, step S2022 includes:

[0065] Step S20221: Re-value each pixel of the target image according to the corresponding sub-pixel offset to obtain the initially adjusted target image;

[0066] Step S20222: The initially adjusted target image and the reference image are stitched together to obtain a stitched image;

[0067] Step S20223: Generate a confidence map corresponding to the stitched image based on the stitched image. The value of each pixel in the confidence map is a quantized score of the alignment reliability of the corresponding pixel in the stitched image.

[0068] Step S20224: If the proportion of pixels smaller than the confidence threshold in the confidence image is greater than or equal to a predetermined proportion, adjust the parameters of the cascaded offset field until the proportion of pixels smaller than the confidence threshold in the adjusted confidence image is less than the predetermined proportion, and determine the initial adjusted target image corresponding to the current adjustment as the adjusted target image.

[0069] In the above implementation, the confidence map generation unit concatenates the aligned feature map and the reference map along the channel dimension to form a 64-channel difference tensor. Then, a 3×3 convolution is used to capture misaligned texture differences within the complementary 3×3 neighborhood, generating a 32-channel intermediate feature. A 1×1 convolution is then used to compress the 32 channels into a single channel and pass it through a sigmoid function, outputting a 0-1 confidence map of the same size as the original image. The value of each pixel represents the quantized score of the alignment reliability at that position, which can be directly used for downstream fusion and interpretable display. The error closed-loop control unit compares the alignment difference and geometric reprojection residual between the aligned features and the reference features in real time. If any indicator exceeds the limit, a local or global recalculation is triggered, iteratively optimizing the offset field until the error converges. A pass flag and an updated offset field and confidence map are output to ensure that the final alignment accuracy meets sub-pixel level requirements.

[0070] In order to integrate the features of multi-source data, in one optional implementation, step S203 above includes:

[0071] Step S2031: Convert the multiple aligned feature images mentioned above into feature tensors to obtain continuous risk embedding vectors;

[0072] Step S2032: Construct a spatial pyramid pooling layer to extract multi-scale features of the above continuous risk embedding vectors, and obtain multiple extracted features, which correspond one-to-one with the scale.

[0073] Step S2033: Generate the fusion weights corresponding to the extracted features using a gated weight generation function based on the extracted features.

[0074] Step S2034: The above-mentioned fusion weights are used to fuse multiple extracted features to obtain aggregated features;

[0075] Step S2035: Generate a global risk embedding vector and a pixel-level heatmap based on the above-mentioned aggregated features, and generate multiple modal-level contribution weights based on the above-mentioned aligned feature images. The above-mentioned global risk embedding vector is a feature vector describing the risk, the above-mentioned pixel-level heatmap is a visualization image that uses a color gradient to map temperature with a single pixel as the smallest unit, and the above-mentioned modal-level contribution weights are used to characterize the degree of influence of different modalities of the above-mentioned multi-source data on the risk.

[0076] In the above implementation, under a unified spatiotemporal coordinate system, four types of heterogeneous, asynchronous, and heterogeneous feature tensors—satellite remote sensing (0.3-3m), UAV visible light (1-5cm), fixed camera video (1080p 30fps), and infrared thermal imaging (640×512 25fps)—are encoded into a 512-dimensional continuous risk embedding vector. Simultaneously, pixel-level cross-modal attention weights and modal-level contributions are output to support sub-second-level risk binary classification. For this purpose, this module employs a GatedFeature Pyramid Network (GFPN) fusion engine. Let the input feature tensor be... H=W=256 is the spatial dimension, T is the number of time frames, and C is the spatial dimension. k The number of channels is denoted as , and the confidence plot is denoted as . The offset field is denoted as The core processing flow is as follows: Construct a Spatial Pyramid Pooling Layer (SPP) to extract multi-scale features: Where l corresponds to three scales: l=1: 256×256 (original scale), l=2: 128×128 (1 / 2 downsampling), l=3: 64×64 (1 / 4 downsampling). A gated weight generation function is defined to generate the fused weights: Cross-modal feature fusion for feature filtering: Hierarchical feature aggregation: This involves upsampling and fusing multi-scale features to generate output features. Where Up(x) is the bilinear interpolation upsampling. The output generation mechanism is defined as follows: global risk embedding vector. Where GeM(x) is generalized mean pooling, with an output dimension of 512. Pixel-level heatmap. The output size is 256×256×T×4, with the channel dimension corresponding to four modal contributions. Modal-level contribution weights: .

[0077] To identify risks, in an optional implementation, step S204 above includes:

[0078] Step S2041: The global risk embedding vector is decoded into multiple risk indices through an orthogonal decomposition matrix. The orthogonal decomposition matrix is ​​obtained by training with sample data, which includes historical multi-source data and corresponding historical risk indices.

[0079] Step S2042: If at least one of the aforementioned risk indices is greater than a predetermined threshold, issue a warning message for the risk type corresponding to the aforementioned risk index, wherein the aforementioned risk type includes at least fire and external damage.

[0080] In the above embodiments, such as Figure 6 As shown, the risk identification module is used to describe the risk of collateral by taking the output of the previous module (global risk vector, heatmap, contribution weight). It improves risk readability by converting the characteristic expression of risk factors into an identifier form. The 512-dimensional risk vector is decoded into n-class risk indices through orthogonal decomposition matrix. The nth type of risk index (0~1), W dec The dataset and training are labeled by experts to meet the requirements. (Risk category distinguishability).

[0081] To provide accurate early warning, in one optional implementation, after performing risk analysis based on the aforementioned fusion features to obtain a risk index for the collateral, and issuing an early warning based on the aforementioned risk index, the method further includes:

[0082] Step S301: When the above-mentioned fire warning information is issued, analyze the temperature change trend based on the above-mentioned pixel-level heat map;

[0083] Step S302: When the above-mentioned warning information of external force damage is issued, the modal level contribution weight with the largest value is determined as the main contributing mode, and the cause of the external force damage is analyzed based on the above-mentioned acquired image corresponding to the main contributing mode.

[0084] In the above implementation method, the specific sources of risk are traced through different risk indices to help relevant personnel to handle emergencies and reduce the loss of value of the collateral, as shown in Table 3.

[0085] Table 3

[0086]

[0087] This application also provides a collateral loan risk monitoring device. It should be noted that this collateral loan risk monitoring device can be used to execute the collateral loan risk monitoring method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0088] The following describes the collateral loan risk monitoring device provided in the embodiments of this application.

[0089] Figure 7 This is a schematic diagram of a collateral loan risk monitoring device according to an embodiment of this application. Figure 7 As shown, the device includes:

[0090] Acquisition unit 10 is used to acquire multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0091] The adjustment unit 20 is used to adjust the pixel offset of the acquired images of the multi-source data to obtain multiple aligned feature images, and the aligned feature images correspond one-to-one with the acquired images.

[0092] Fusion unit 30 is used to fuse multiple aligned feature images using a gated feature pyramid fusion network to obtain fused features;

[0093] Analysis unit 40 is used to perform risk analysis based on the above-mentioned fusion characteristics, obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0094] In the aforementioned collateral loan risk monitoring device, the images acquired by a single image acquisition device have limitations, which may lead to missed risk assessments. For example, an inappropriate shooting angle may result in the failure to capture clear images of cracks. Furthermore, the differences in angle and resolution between images acquired by different image acquisition devices mean that existing systems can only integrate images in a coarse-grained manner through offline stitching and manual comparison. This method adjusts the pixel offset of the acquired images from multiple sources to avoid misalignment during image stitching. This facilitates the fusion of multiple aligned feature images to obtain fused features, which are then used for risk analysis and to issue warnings. This ensures the accuracy of risk warnings without requiring manual risk assessment, thus solving the problem of existing collateral risk monitoring technologies relying on manual risk assessment.

[0095] To achieve the above-mentioned normalization processing of multi-source data, in one optional embodiment, the acquisition unit includes:

[0096] The acquisition module is used to acquire the original multi-source data of the collateral;

[0097] The first conversion module is used to convert the acquisition time of the above-mentioned original multi-source data into the time of the same time axis, and to convert the images of the above-mentioned original multi-source data into images of the same resolution, so as to obtain the above-mentioned multi-source data.

[0098] In the above embodiments, such as Figure 3 As shown, before the deformable pyramid alignment module performs alignment processing, the heterogeneous data access and spatiotemporal indexing module is mainly used to access different types of raw risk monitoring data. It processes different flows of structured and unstructured data simultaneously in an offline and real-time parallel manner to adapt to the heterogeneity of multi-source data, while also normalizing the spatiotemporal data. Specifically, the heterogeneous data access and spatiotemporal indexing module, acting as the "data bus" and "spatiotemporal registry center" of the entire collateral risk monitoring system, is mainly responsible for the following tasks: 1. Protocol parsing, metadata extraction, and quality verification of multi-source heterogeneous data; 2. Registering all observation units with unique keys based on a unified spatiotemporal benchmark; 3. Generating indexes and data passports for subsequent algorithm layers to ensure spatiotemporal consistency during cross-module, cross-language, and cross-process calls.

[0099] Specifically, such as Figure 4 As shown, the time normalization unit: Based on the fundamental conditions of inconsistent time bases of heterogeneous sensors, sampling clock drift, and cross-domain errors, this module adopts a "single time axis, drift-free, interpolable" time coordinate system. The time normalization unit provides repeatable and auditable time alignment capabilities. It includes the following five processors: a reference anchor: using the first successfully parsed sensor recording time T0 as the global zero time; an absolute timescale converter: uniformly converting multiple source fields such as UTC, GPS weeks, NTP timestamps, and camera RTC time into Unix nanosecond integers; and a relative timescale calculator: for any recording time T... i Generate normalized time offset Δt i = (T i -T0) / 1ns, drift compensator: optionally, using a PTP or GNSS 1PPS pulse to adjust Δt i Sub-millisecond drift correction; Extended interpolator: based on Δt iThe sequence uses linear or cubic spline interpolation to generate virtual timestamps for missing sampling points, with an error of less than or equal to 1 ms. The input and output of the time normalization unit can be summarized as shown in Tables 1 and 2. Spatial normalization unit: Under the conditions of spatial misalignment and scale misalignment caused by differences in field of view, geometric imaging, projection system, and ground sampling distance between satellite, UAV, remote sensing, and video sensors, this module provides a spatial normalization unit with "single coordinate system, single resolution, and single elevation benchmark," enabling different pixels and units to perform cross-membrane calculations within the same geometric framework. The system includes the following five processors: a) Coordinate reference unit: Binds multi-source images to a unified spatial reference through a single parametric reprojection and least-squares adjustment, ensuring residuals <1cm; b) Resolution tuner: Using 0.5m as the sole reference, pools and shrinks coarse pixels such as satellite images and convolves and enlarges fine pixels such as drone images, aligning resolution at the physical scale; c) Tile divider: Divides the image into standard 256×256 tiles, performs zero-padding at the edges, and records the effective range, ensuring reading efficiency and cross-platform index consistency; d) Integrity checker: Calculates SHA256 for the tile pixel matrix and performs hash verification with the data passport; if they do not match, resampling is performed; e) Spatial indexer: Creates a composite index for the database {spatial dimension, temporal dimension}, exposing a RESTful query interface.

[0100] To achieve automatic alignment, in one optional implementation, the adjustment unit includes:

[0101] The calculation module is used to calculate the sub-pixel offset of each pixel of each target image based on the reference image through the cascaded offset field. The reference image is the acquired image with the highest resolution among the multi-source data, and the target image is the acquired image other than the reference image among the multi-source data.

[0102] The adjustment module is used to re-value each pixel of the target image according to the corresponding sub-pixel offset to obtain the adjusted target image.

[0103] The combination module is used to combine all the above-mentioned adjusted target images and the above-mentioned reference images into multiple above-mentioned aligned feature images.

[0104] In the above embodiments, the spatial normalization unit can unify the input image to a fixed ground sampling distance, such as... Figure 5As shown, the deformable pyramid alignment module needs to further provide the ability to automatically align under "multi-scale, cross-module, and sub-pixel" conditions. It includes the following sequentially coupled sub-units: A multi-scale pyramid construction unit constructs an L-layer feature pyramid for a single-source image, halving the resolution layer by layer. The i-th layer features are obtained through convolution-pooling operations. The specific process is as follows: a 3×3 GroupNorm-ReLU6 convolution is performed on the single-source image to extract features from channels 32-128, followed by halving the resolution layer by layer through 2×2 average pooling, forming a multi-level feature pyramid. Convolution and pooling are sequentially coupled to ensure that texture and edge information are not lost during downsampling, and the weights and activations are uniformly quantized to INT8 and cached for subsequent direct reuse. A cascaded offset field calculation unit performs offset calculations in a "coarse-to-fine" manner, using the highest-resolution drone as a reference for alignment by default. At the top layer, a 3×3 convolution is used to estimate the coarse offset ΔP of the image. i ; will ΔP i After 2× nearest neighbor upsampling ΔP i As the "initial guess" for the next layer, it is obtained by adding the feature difference of the next layer. Repeat the upsampling, fine-tuning, and summing process until the finest layer ΔP is obtained. i At this point, each pixel has an independent sub-pixel offset. The deformable convolutional execution unit takes a given feature map F to be aligned and a pixel-wise offset ΔP, and re-samples each point of F according to the sub-pixel position specified by ΔP, outputting a "aligned" feature map. That is, perform the following for each output position (i,j): Where K is a 3×3 conventional convolutional sampling network; (k x , k j ) is a fixed offset of a 3×3 grid; F(x) is the subpixel value read using bilinear interpolation; w k These are the convolution weights; M[i,j] is the confidence weight, which defaults to 1 if it does not exist.

[0105] To ensure alignment reliability, in one optional implementation, the adjustment module includes:

[0106] The value acquisition submodule is used to re-acquire the values ​​of each pixel in the target image according to the corresponding sub-pixel offset to obtain the initially adjusted target image.

[0107] The stitching submodule is used to stitch the initially adjusted target image and the reference image to obtain a stitched image.

[0108] The generation submodule is used to generate a confidence map corresponding to the stitched image based on the stitched image. The value of each pixel in the confidence map is a quantized score of the alignment reliability of the corresponding pixel in the stitched image.

[0109] The adjustment submodule is used to adjust the parameters of the cascaded offset field when the proportion of pixels with a value less than the confidence threshold in the confidence image is greater than or equal to a predetermined proportion, until the proportion of pixels with a value less than the confidence threshold in the adjusted confidence image is less than the predetermined proportion, and the initial adjusted target image corresponding to the current adjustment is determined as the adjusted target image.

[0110] In the above implementation, the confidence map generation unit concatenates the aligned feature map and the reference map along the channel dimension to form a 64-channel difference tensor. Then, a 3×3 convolution is used to capture misaligned texture differences within the complementary 3×3 neighborhood, generating a 32-channel intermediate feature. A 1×1 convolution is then used to compress the 32 channels into a single channel and pass it through a sigmoid function, outputting a 0-1 confidence map of the same size as the original image. The value of each pixel represents the quantized score of the alignment reliability at that position, which can be directly used for downstream fusion and interpretable display. The error closed-loop control unit compares the alignment difference and geometric reprojection residual between the aligned features and the reference features in real time. If any indicator exceeds the limit, a local or global recalculation is triggered, iteratively optimizing the offset field until the error converges. A pass flag and an updated offset field and confidence map are output to ensure that the final alignment accuracy meets sub-pixel level requirements.

[0111] To fuse features from multiple sources, in one optional implementation, the fusion unit includes:

[0112] The second conversion module is used to convert multiple aligned feature images into feature tensors to obtain continuous risk embedding vectors.

[0113] The construction module is used to construct a spatial pyramid pooling layer to extract multi-scale features of the continuous risk embedding vectors mentioned above, resulting in multiple extracted features, which correspond one-to-one with the scale.

[0114] The first generation module is used to generate the fusion weights corresponding to the extracted features using a gated weight generation function based on the extracted features.

[0115] The fusion module is used to fuse multiple extracted features using the aforementioned fusion weights to obtain aggregated features;

[0116] The second generation module is used to generate a global risk embedding vector and a pixel-level heatmap based on the above-mentioned aggregated features, and to generate multiple modality-level contribution weights based on multiple above-mentioned aligned feature images. The above-mentioned global risk embedding vector is a feature vector describing the risk, the above-mentioned pixel-level heatmap is a visualization image with a single pixel as the smallest unit and temperature mapped by color gradient, and the above-mentioned modality-level contribution weights are used to characterize the degree of influence of different modalities of the above-mentioned multi-source data on the risk.

[0117] In the above implementation, under a unified spatiotemporal coordinate system, four types of heterogeneous, asynchronous, and heterogeneous feature tensors—satellite remote sensing (0.3-3m), UAV visible light (1-5cm), fixed camera video (1080p 30fps), and infrared thermal imaging (640×512 25fps)—are encoded into a 512-dimensional continuous risk embedding vector. Simultaneously, pixel-level cross-modal attention weights and modal-level contributions are output to support sub-second-level risk binary classification. For this purpose, this module employs a GatedFeature Pyramid Network (GFPN) fusion engine. Let the input feature tensor be... H=W=256 is the spatial dimension, T is the number of time frames, and C is the spatial dimension. k The number of channels is denoted as , and the confidence plot is denoted as . The offset field is denoted as The core processing flow is as follows: Construct a Spatial Pyramid Pooling Layer (SPP) to extract multi-scale features: Where l corresponds to three scales: l=1: 256×256 (original scale), l=2: 128×128 (1 / 2 downsampling), l=3: 64×64 (1 / 4 downsampling). A gated weight generation function is defined to generate the fused weights: Cross-modal feature fusion for feature filtering: Hierarchical feature aggregation: This involves upsampling and fusing multi-scale features to generate output features. Where Up(x) is the bilinear interpolation upsampling. The output generation mechanism is defined as follows: global risk embedding vector. Where GeM(x) is generalized mean pooling, with an output dimension of 512. Pixel-level heatmap. The output size is 256×256×T×4, with the channel dimension corresponding to four modal contributions. Modal-level contribution weights: .

[0118] To identify risks, in one optional implementation, the analysis unit includes:

[0119] The decoding module is used to decode the global risk embedding vector into multiple risk indices through an orthogonal decomposition matrix. The orthogonal decomposition matrix is ​​obtained by training with sample data, which includes historical multi-source data and corresponding historical risk indices.

[0120] The sending module is used to issue a warning message for the risk type corresponding to the risk index when at least one of the aforementioned risk indices is greater than a predetermined threshold. The aforementioned risk types include at least fire and external damage.

[0121] In the above embodiments, such as Figure 6As shown, the risk identification module is used to describe the risk of collateral by taking the output of the previous module (global risk vector, heatmap, contribution weight). It improves risk readability by converting the characteristic expression of risk factors into an identifier form. The 512-dimensional risk vector is decoded into n-class risk indices through orthogonal decomposition matrix. The nth type of risk index (0~1), W dec The dataset and training are labeled by experts to meet the requirements. (Risk category distinguishability).

[0122] For accurate early warning, in one optional implementation, the above-mentioned device further includes:

[0123] The first determining unit is used to analyze the temperature change trend based on the pixel-level heat map after performing risk analysis based on the above-mentioned fusion features to obtain the risk index of the above-mentioned collateral and issuing an early warning based on the above-mentioned risk index, and after issuing the above-mentioned early warning information of the above-mentioned fire.

[0124] The second determining unit is used to determine the mode level contribution weight with the largest value as the main contributing mode when the above-mentioned warning information of external force damage is issued, and to analyze the cause of the external force damage based on the above-mentioned acquired image corresponding to the main contributing mode.

[0125] In the above implementation method, the specific sources of risk are traced through different risk indices to help relevant personnel to handle emergencies and reduce the loss of value of the collateral, as shown in Table 3.

[0126] The aforementioned collateral loan risk monitoring device includes a processor and a memory. The acquisition unit, adjustment unit, fusion unit, and analysis unit are all stored as program units in the memory, and the processor executes these program units to achieve their respective functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0127] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of existing collateral risk monitoring relying on manual risk assessment.

[0128] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0129] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the collateral loan risk monitoring method.

[0130] Specifically, methods for monitoring risks in collateralized loans include:

[0131] Step S201: Obtain multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0132] Step S202: The pixel offset of the acquired images of the above multi-source data is adjusted to obtain multiple aligned feature images, and the above aligned feature images correspond one-to-one with the above acquired images.

[0133] Step S203: A gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features;

[0134] Step S204: Perform risk analysis based on the above-mentioned fusion characteristics to obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0135] This invention provides a processor for running a program, wherein the program executes the collateral loan risk monitoring method.

[0136] Specifically, methods for monitoring risks in collateralized loans include:

[0137] Step S201: Obtain multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0138] Step S202: The pixel offset of the acquired images of the above multi-source data is adjusted to obtain multiple aligned feature images, and the above aligned feature images correspond one-to-one with the above acquired images.

[0139] Step S203: A gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features;

[0140] Step S204: Perform risk analysis based on the above-mentioned fusion characteristics to obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0141] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0142] Step S201: Obtain multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0143] Step S202: The pixel offset of the acquired images of the above multi-source data is adjusted to obtain multiple aligned feature images, and the above aligned feature images correspond one-to-one with the above acquired images.

[0144] Step S203: A gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features;

[0145] Step S204: Perform risk analysis based on the above-mentioned fusion characteristics to obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0146] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0147] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0148] Step S201: Obtain multi-source data of the collateral, including images acquired by multiple image acquisition devices at the same time.

[0149] Step S202: The pixel offset of the acquired images of the above multi-source data is adjusted to obtain multiple aligned feature images, and the above aligned feature images correspond one-to-one with the above acquired images.

[0150] Step S203: A gated feature pyramid fusion network is used to fuse multiple aligned feature images to obtain fused features;

[0151] Step S204: Perform risk analysis based on the above-mentioned fusion characteristics to obtain the risk index of the above-mentioned collateral, and issue an early warning based on the above-mentioned risk index.

[0152] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0159] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0162] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0163] 1) In the collateral loan risk monitoring method of this application, the images acquired by a single image acquisition device have limitations and may lead to missed risk assessments. For example, an inappropriate shooting angle may result in the failure to capture clear cracks. Furthermore, the differences in angle and resolution of images acquired by different image acquisition devices mean that existing systems can only integrate images in a coarse-grained manner through offline stitching and manual comparison. This method adjusts the pixel offset of the acquired images from multiple sources to avoid misalignment of the acquired images from multiple sources. This facilitates the fusion of multiple aligned feature images to obtain fused features. Risk analysis is then performed based on the fused features, and warnings are issued. This ensures the accuracy of risk warnings while eliminating the need for manual risk assessment, thus solving the problem of existing collateral risk monitoring relying on manual risk assessment.

[0164] 2) In the collateral loan risk monitoring device of this application, the images acquired by a single image acquisition device have limitations, which may lead to missed risk assessments. For example, an inappropriate shooting angle may result in the failure to capture clear images of cracks. Furthermore, the differences in angle and resolution of images acquired by different image acquisition devices mean that existing systems can only integrate images in a coarse-grained manner through offline stitching and manual comparison. This method adjusts the pixel offset of the acquired images from multiple sources to avoid misalignment of the acquired images from multiple sources. This facilitates the fusion of multiple aligned feature images to obtain fused features. Risk analysis is then performed based on the fused features, and warnings are issued. This ensures the accuracy of risk warnings while eliminating the need for manual risk assessment, thus solving the problem of existing collateral risk monitoring relying on manual risk assessment.

[0165] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of monitoring risk of a loan against a collateral, characterized by, The method comprises the following steps: acquiring multi-source data of a mortgage, the multi-source data comprising acquisition images collected by multiple image collection devices at the same time; adjusting the acquisition images of the multi-source data by pixel offset to obtain multiple aligned feature images, the aligned feature images corresponding one-to-one to the acquisition images; fusing the multiple aligned feature images by using a gated feature pyramid fusion network to obtain fused features; performing risk analysis according to the fused features to obtain a risk index of the mortgage, and issuing a warning according to the risk index.

2. The method of claim 1, wherein, adjusting the acquisition images of the multi-source data by pixel offset to obtain multiple aligned feature images, comprising: calculating sub-pixel offsets of each pixel of each target image based on a reference image through a cascaded offset field, the reference image being the acquisition image with the largest resolution in the multi-source data, and the target image being the acquisition image other than the reference image in the multi-source data; revaluing each pixel of the target image according to the corresponding sub-pixel offset to obtain an adjusted target image; composing all the adjusted target images and the reference image into multiple aligned feature images.

3. The method of claim 2, wherein, revaluing each pixel of the target image according to the corresponding sub-pixel offset to obtain an adjusted target image, comprising: revaluing each pixel of the target image according to the corresponding sub-pixel offset to obtain an initial adjusted target image; splicing the initial adjusted target image and the reference image to obtain a spliced image; generating a confidence map corresponding to the spliced image according to the spliced image, each pixel of the confidence map being a quantified score of alignment reliability of a corresponding pixel of the spliced image; in a case where a proportion of pixel points less than a confidence threshold in the confidence map is greater than or equal to a predetermined proportion, adjusting parameters of the cascaded offset field until the proportion of pixel points less than the confidence threshold in the adjusted confidence map is less than the predetermined proportion, and determining the initial adjusted target image corresponding to the current adjustment as the adjusted target image.

4. The method of claim 1, wherein, fusing the multiple aligned feature images by using a gated feature pyramid fusion network to obtain fused features, comprising: converting the multiple aligned feature images into feature tensors to obtain continuous risk embedding vectors; constructing a spatial pyramid pooling layer to extract multi-scale features of the continuous risk embedding vectors to obtain multiple extracted features, the extracted features corresponding one-to-one to scales; generating fusion weights corresponding to the extracted features by using a gated weight generation function according to the extracted features; fusing the multiple extracted features by using the fusion weights to obtain aggregated features; generating a global risk embedding vector and a pixel-level heat map according to the aggregated features, and generating multiple modality-level contribution weights according to the multiple aligned feature images, the global risk embedding vector being a feature vector describing risk, the pixel-level heat map being a visualized image mapping temperature by color gradient with a single pixel as the minimum unit, and the modality-level contribution weights being used to represent the influence degree of different modalities of the multi-source data on risk.

5. The method of claim 4, wherein, According to the fusion feature, risk analysis is performed to obtain a risk index of the mortgage, and a warning is issued according to the risk index, including: The global risk embedding vector is decoded into a plurality of risk indexes through an orthogonal decomposition matrix, the orthogonal decomposition matrix being obtained by training sample data, and the sample data including historical multi-source data and corresponding historical risk indexes; In a case where at least one of the risk indexes is greater than a predetermined threshold, warning information of a risk type corresponding to the risk index is issued, and the risk type at least includes fire and external force damage.

6. The method of claim 5, wherein, After the risk analysis according to the fusion feature, the risk index of the mortgage is obtained, and the warning is issued according to the risk index, the method further includes: In a case where the warning information of the fire is issued, a temperature change trend is analyzed according to the pixel-level heat map; In a case where the warning information of the external force damage is issued, the maximum modal-level contribution weight is determined as a main contribution mode, and a cause of the external force damage is analyzed according to the acquisition image corresponding to the main contribution mode.

7. The method of claim 1, wherein, Obtaining multi-source data of a mortgage, including: Obtaining original multi-source data of a mortgage; Converting the acquisition time of the original multi-source data to a time point on a same time axis, converting the image of the original multi-source data to an image with a same resolution, and obtaining the multi-source data.

8. A collateral loan risk monitoring apparatus characterized by comprising: Including: An acquisition unit is configured to acquire multi-source data of a mortgage, the multi-source data including acquisition images acquired by a plurality of image acquisition devices at a same time point; An adjustment unit is configured to perform pixel offset adjustment on the acquisition images of the multi-source data to obtain a plurality of aligned feature images, the aligned feature images corresponding to the acquisition images one by one; A fusion unit is configured to fuse a plurality of the aligned feature images by using a gated feature pyramid fusion network to obtain fusion features; An analysis unit is configured to perform risk analysis according to the fusion features to obtain a risk index of the mortgage, and issue a warning according to the risk index.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program controls the device where the computer-readable storage medium is located to execute the method of any one of claims 1 to 7 when the program is running.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images