Multi-modal data federated learning-based farmer credit assessment method and system

Abstract

The invention discloses a peasant household credit assessment method and system based on multi-modal data federated learning, and relates to the technical field of computer intelligence, and the method comprises the steps: constructing a dual-channel representation mode of land assets based on GIS space coding and property and age limit information, and generating multi-dimensional feature input of peasant household production capacity and asset conditions in combination with multi-source data; a three-level federated learning structure is constructed, data is safely exchanged through TLS communication encryption and equipment identity authentication, and differential privacy noise is injected; a Top-K gradient selection algorithm is adopted to carry out parameter sparse transmission, a sparse rate is dynamically adjusted in combination with a round increasing strategy, a grouping sparse convolution and parameter freezing mechanism is introduced, and a federal deep learning model is compressed and an adaptive edge is deployed through 8-bit distillation quantification; according to the method, multi-modal feature construction, layered federated training and a privacy enhancement mechanism are fused, and fine modeling, lightweight deployment and data security collaboration of peasant household credit assessment are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer intelligence technology, and particularly to a method and system for farmer credit assessment based on multi-modal data federated learning. Background Art

[0002] In the rural financial service system, farmer credit assessment, as the basic work of credit decision-making and risk control, depends on a comprehensive understanding of farmers' asset status, production capacity, and behavioral characteristics. However, the existing technologies mainly adopt centralized modeling methods, relying on various financial transaction data, such as traditional financial behavior data like bank loans and repayment records, for modeling and analysis. This approach faces several fundamental problems in practical applications: In the rural scenario, the data distribution is highly fragmented. Land data is usually held by the land and resources department, crop planting information is scattered among agricultural cooperatives, and agricultural product sales information exists in the supply and marketing system or e-commerce platforms. As a result, it is difficult to complete unified data aggregation at the same central node, seriously affecting the integrity and timeliness of credit modeling.

[0003] Currently, existing credit models have a problem of singularity in the feature dimension, often ignoring unstructured or weakly structured asset data such as real estate status, harvest expectations, agricultural machinery equipment, and family labor force structure. Such data often contains important clues about farmers' actual repayment ability. Some studies have tried to introduce the federated learning framework to achieve distributed data modeling. However, mainstream federated learning models generally have problems such as large model size, frequent parameter updates, and large communication volume during the training process, and cannot adapt to the operating environment with weak computing power and limited bandwidth of edge computing nodes such as rural credit cooperatives and township platforms, resulting in high costs and poor effectiveness for federated training deployment. Currently, the data types, standards, and formats mastered by various institutions are not unified. Some asset data, such as land ownership certificates and crop yield estimates, have non-standard expressions in the form of images or texts, further increasing the technical difficulty of cross-institutional data collaborative modeling. The privacy protection means in the existing system are relatively weak, and it is difficult to meet the privacy computing compliance requirements in cross-subject collaboration of financial data. Especially when dealing with sensitive information related to identity and assets, it faces risks such as raw data leakage and model reverse inference. Therefore, establishing a farmer credit intelligent assessment framework that can integrate multi-modal data sources, balance training performance and privacy protection, and adapt to rural network and terminal capabilities has become the key direction of current technological development. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the technical problem to be solved by the present invention is: the existing rural household credit assessment method has problems such as insufficient modeling coverage, single feature dimension, redundant federated learning structure, and lack of privacy protection mechanism leading to data leakage risks. How to construct a federated modeling system that supports multi-modal feature fusion, can be deployed in heterogeneous edge environments, and has differential privacy capabilities, and efficiently realizes the technical problem of credit assessment on the premise of ensuring the leakage of sensitive information of rural households.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a rural household credit assessment method based on multi-modal data federated learning, including constructing a two-channel representation method of land assets based on GIS spatial coding and property right years information, combining multi-source data to generate multi-dimensional feature inputs of rural household production capacity and asset status; constructing a three-level federated learning structure, securely exchanging data through TLS communication encryption and device identity authentication, and injecting differential privacy noise; using the Top-K gradient selection algorithm for parameter sparse transmission, dynamically adjusting the sparsity rate in combination with the round increment strategy, introducing grouped sparse convolution and parameter freezing mechanism, and compressing and deploying the federated deep learning model adapted to the edge through 8-bit distillation quantization; the two-channel representation method includes spatial-time limit two-channel land modeling, encoding land assets using two independent dimensions of GIS spatial location and property right years; the three-level federated learning structure includes constructing a three-level federated topology structure composed of terminal devices, edge nodes, and a central coordinator, communicating with each other through a secure communication protocol, and constructing a hierarchical heterogeneous federated network architecture; device identity authentication includes realizing terminal identity authentication and binding of upstream and downstream nodes through the device unique ID and timestamp token; injecting differential privacy noise includes adding Laplace noise to the uploaded parameters during the aggregation process to uniformly control the perturbation intensity at the server side; parameter sparse transmission includes dynamically adjusting the Top-K gradient upload ratio based on the increase of training rounds, and flexibly adjusting gradient transmission in combination with communication status and local loss changes; compressing the federated deep learning model through 8-bit distillation quantization includes using the joint strategy of 8-bit fixed-point quantization and knowledge distillation to complete compression and accuracy transfer before the deployment of the global federated deep learning model.

[0007] As a preferred solution of the rural household credit assessment method based on multi-modal data federated learning described in the present invention, wherein: the construction of the two-channel representation method of land assets includes collecting the GPS coordinates of rural households' land, converting the coordinates into UTM encoding using the unified Mercator projection, constructing a land spatial adjacency matrix to represent the plot topology structure; based on rural land ownership data, extracting the registration years field, and extracting key time factors through text recognition and format standardization methods to construct a spatio-temporal integrated land asset feature input combined with the spatial location encoding.

[0008] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the multi-dimensional feature input for generating the production capacity and asset status of farmers includes extracting continuous features from NDVI data through a sliding window, constructing a meteorological factor matrix to model and extract crop output features, generating housing depreciation features based on building material durability parameters and years, and splicing all features under a unified dimension.

[0009] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the constructing a meteorological factor matrix to model and extract crop output features includes constructing a data table of crop types and historical climate fluctuation factors, archiving by region based on agricultural technology promotion, synchronizing timestamps, filling missing values by interpolation, participating in the combined modeling of output prediction factors, and outputting an estimated value vector.

[0010] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the constructing a three-level federated learning structure includes setting a central coordinator node to manage the global federated deep learning model and the aggregation and distribution process, setting regional edge nodes to receive the federated deep learning models uploaded by terminal devices, performing privacy processing, locally collecting and encrypting farmer data based on terminal devices to participate in federated training, and completing mapping and identity binding between each layer through a unique device ID.

[0011] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the communication encryption and device identity authentication through TLS includes establishing a secure communication channel of the TLS 1.3 protocol between nodes at all levels, performing identity token authentication on participating devices based on keys and timestamps, allocating a temporary session channel after successful authentication, encrypting and transmitting the parameter exchange of the federated deep learning model in each round through the temporary session channel, and rejecting unregistered and authentication-failed nodes from joining the training round.

[0012] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the injecting differential privacy noise includes applying Laplace distribution random perturbation to the parameter update of the federated deep learning model and injecting all the noise into the server side.

[0013] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the parameter sparse transmission using the Top-K gradient selection algorithm includes sorting the local gradient values by absolute value on the client device, selecting the top K gradient values and uploading them to the upper layer node, setting the sparse transmission value, adaptively adjusting the sparsity rate, and updating the selection strategy.

[0014] As a preferred solution of the farmer credit assessment method based on multi-modal data federated learning according to the present invention, wherein: the introduction of grouped sparse convolution and parameter freezing mechanism includes dividing multiple groups in the convolutional layer of the federated deep learning model based on the temporary session channel dimension, training independent parameter sets, freezing the update channels of the parameter layer with stable convergence after the number of rounds reaches a set threshold, using the knowledge distillation method to map the floating-point model to an equivalent 8-bit fixed-point network, and deploying low-power edge devices.

[0015] Another object of the present invention is to provide a farmer credit assessment system based on multi-modal data federated learning, which can solve the problem of insufficient feature expression caused by the lack of modeling support for the spatial attributes of farmers' real estate and the property right time factor in the current credit assessment technology by introducing a land asset quantification scheme based on the joint modeling of GIS coding and property right years.

[0016] As a preferred solution of the farmer credit assessment system based on multi-modal data federated learning according to the present invention, wherein: it includes a multi-modal feature construction module, a federated collaborative training module, and a model compression and transmission control module; the multi-modal feature construction module is responsible for collecting and processing heterogeneous raw data in multiple dimensions of land, housing, and crops, and generating a farmer credit feature vector for training by using unified spatial coding; the federated collaborative training module is used to construct a three-level federated architecture, support device registration, synchronize tasks and local training, and complete distributed model updates based on encrypted communication and identity authentication; the model compression and transmission control module is used to perform structured compression and training control on the model parameters through Top-K gradient screening, using sparse convolution, parameter freezing, and 8-bit distillation quantization strategies.

[0017] A computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the farmer credit assessment method based on multi-modal data federated learning.

[0018] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, it implements the steps of the farmer credit assessment method based on multi-modal data federated learning.

[0019] Advantages of the present invention: The method for evaluating farmers' credit based on multi-modal data federated learning provided by the present invention improves the feature modeling ability of spatial location and property rights duration through a dual-channel representation method of land assets, and solves the problem of insufficient utilization of static asset information by traditional models; the constructed three-level federated learning architecture effectively realizes joint training under the premise of data privacy, and improves the intensity of data privacy protection; the introduced Top-K gradient sparse transmission, grouped sparse convolution, and 8-bit quantization mechanism significantly reduce the computing and communication load during the operation of the model on edge devices; the present invention has achieved better effects in multi-dimensional fusion of feature expression, privacy and security guarantee of federated training, and resource adaptation for edge deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings according to these drawings without creative efforts.

[0021] Figure 1 It is the overall flowchart of a method for evaluating farmers' credit based on multi-modal data federated learning provided by the first embodiment of the present invention.

[0022] Figure 2 It is a diagram showing the difference in parameter distribution between a traditional fully connected network and the sparse convolution network adopted by the present invention in a comparison of a method for evaluating farmers' credit based on multi-modal data federated learning provided by the second embodiment of the present invention.

[0023] Figure 3 It is the overall schematic diagram of a system for evaluating farmers' credit based on multi-modal data federated learning provided by the third embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0025] Embodiment 1, referring to Figure 1 , which is an embodiment of the present invention, provides a method for evaluating farmers' credit based on multi-modal data federated learning, including: S1: Based on GIS spatial coding and property right duration information, construct a dual-channel representation method for land assets, and combine multi-source data to generate multi-dimensional feature inputs of farmers' production capacity and asset status.

[0026] Further, constructing the dual-channel representation method for land assets includes collecting the GPS coordinates of farmers' land, converting the coordinates into UTM coding using the Universal Transverse Mercator projection, constructing a land spatial adjacency matrix to represent the plot topology; based on rural land ownership data, extracting the registration duration field, extracting key time factors through character recognition and format standardization methods, and constructing a spatio-temporal integrated land asset feature input combined with the spatial location coding.

[0027] Furthermore, the multi-source data includes remote sensing images, NDVI vegetation index, building material types, pest and disease damage, and meteorological records.

[0028] It should be noted that generating the multi-dimensional feature inputs of farmers' production capacity and asset status includes extracting continuous features from NDVI data (time series data) through a sliding window, constructing a meteorological factor matrix to model and extract crop output features, generating house depreciation features based on building material durability parameters and years, and splicing all features under a unified dimension.

[0029] It should also be noted that constructing a meteorological factor matrix to model and extract crop output features includes constructing a data table of crop types and historical climate fluctuation factors, archiving by region based on agricultural technology promotion, synchronizing timestamps, using interpolation to fill in missing values, participating in the combined modeling of output prediction factors, and outputting an estimated value vector.

[0030] S2: Construct a three-level federated learning structure, securely exchange data through TLS communication encryption and device identity authentication, and inject differential privacy noise.

[0031] Further, constructing the three-level federated learning structure includes setting a central coordinator node to manage the global federated deep learning model and the aggregation and distribution process, setting regional edge nodes to receive the federated deep learning models uploaded by terminal devices, performing privacy processing, locally collecting and encrypting farmers' data based on terminal devices to participate in federated training, and completing mapping and identity binding between each layer through a unique device ID.

[0032] Furthermore, the three - level federated learning structure consists of a central coordinator, regional edge nodes, and terminal devices. The central coordinator layer is deployed on the provincial rural commercial bank server, responsible for aggregating and distributing global model parameters, and ensuring the coordination and synchronization of the overall training. The regional node layer is deployed on the edge computing server of the county - level rural credit union, responsible for the initial aggregation and privacy processing of multi - terminal data within the region, reducing the load on the central coordinator. The terminal device layer is configured on the village - level agricultural assistance service terminals, such as portable tablets or edge devices, which directly collect multi - modal raw data on the farmer side and perform local encryption processing, and participate in the federated training process.

[0033] It should be noted that through TLS communication encryption and device identity authentication, a secure communication channel of the TLS 1.3 protocol is established between nodes at all levels. Identity token authentication based on keys and timestamps is performed on participating devices. After successful authentication, a temporary session channel is allocated. Each round of federated deep learning model parameter exchange is encrypted and transmitted through the temporary session channel, and unregistered and authentication - failed nodes are refused to join the training round.

[0034] It should also be noted that injecting differential privacy noise includes applying Laplace - distributed random perturbations to the updates of the federated deep learning model parameters and injecting all the noise into the server - side.

[0035] It should also be noted that the Laplace - distributed random noise is expressed as: ; where represents the updated value of the global model parameters generated in the current round of federated aggregation, represents the updated value of the model weight parameters of the participating clients, represents the model weight parameters of the participating clients, represents the index value of the number of participating client models, represents the number of clients participating in the current round of federated training, represents the Laplace - distributed random noise added based on the differential privacy mechanism, represents the privacy budget parameter in the differential privacy mechanism, and the smaller the value, the stronger the privacy protection, represents the global sensitivity.

[0036] S3: Adopt the Top - K gradient selection algorithm for parameter - sparse transmission, dynamically adjust the sparsity rate in combination with the round - increment strategy, introduce grouped sparse convolution and parameter freezing mechanisms, and compress the federated deep learning model and deploy it for edge adaptation through 8 - bit distillation quantization.

[0037] Further, the use of the Top-K gradient selection algorithm for parameter sparse transmission includes that the client device sorts the local gradient values by absolute value, selects the top K gradient values and uploads them to the upper-layer node, sets the sparse transmission value, adaptively adjusts the sparsity rate, and updates the selection strategy.

[0038] It should be noted that the gradient update amount judgment formula is expressed as: ; Wherein, is the loss function of the local model of the current terminal device, is the partial derivative symbol, represents the number of client devices participating in the current round of federated training, For the th participating client in the th model weight parameter, represents the model weight parameters of the participating clients, is the gradient transmission threshold for the current round, which is dynamically issued by the central coordinator or the regional node, used to control the amount of uploaded data, and judge which gradients contribute to the global optimization of the model.

[0039] It should also be noted that the introduction of grouped sparse convolution and parameter freezing mechanism includes that based on the temporary session channel dimension, the convolutional layer of the federated deep learning model is divided into multiple groups, and the independent parameter sets are trained. The parameter layer with stable convergence freezes the update channel after the number of rounds reaches the set threshold. The knowledge distillation method is used to map the floating-point model into an equivalent 8-bit fixed-point network and deploy low-power edge devices.

[0040] It should also be noted that during the federated training process, the client device only uploads the gradient parameters with higher contribution in the current round of training. The contribution is selected according to the sorting of the gradient magnitude, and the sparsity rate is dynamically adjusted to adapt to the bandwidth limitation under different communication conditions, thus significantly reducing the communication overhead; the convolutional layer in the model is divided into ≥4 groups of channels according to the channel dimension, and each group of convolutional kernels is independently trained to reduce redundant calculations and improve the model inference efficiency, which is suitable for resource-constrained terminals to execute; as the number of rounds increases, the parameter layer with stable convergence in the model is gradually frozen, and the computational cost is reduced with a linearly increasing freezing ratio, and the stability and convergence speed of the model are improved.

[0041] Example 2, referring to Figure 2 , which is an embodiment of the present invention, provides a method for evaluating the credit of farmers based on multi-modal data federated learning. In order to verify the beneficial effects of the present invention, scientific demonstrations are carried out through economic benefit calculations and simulation experiments.

[0042] This embodiment aims to verify the adaptability, data privacy protection ability, and credit scoring performance of the proposed "Farmer Credit Assessment Method Based on Multi-modal Data Federated Learning" in resource-constrained edge scenarios.

[0043] First, the experiment was conducted on edge devices with the ARM Cortex-A76 architecture, configured with 8GB of memory, and enabled TensorRT inference optimization to approximate real deployment conditions; the communication environment simulated the access situation of rural agricultural assistance service points, with the local area network transmission stable within 5ms, featuring typical low latency and low bandwidth characteristics. The software environment used Python 3.8 and Pytorch 2.0, and the federated learning framework was based on FATE 1.9.

[0044] Second, the data selected were real multi-modal data samples from 15,000 farmers, covering agricultural insurance records, agricultural supplies consumption flows, remote sensing image NDVI indices, farmland sensor acquisition values, and financial loan records. Each modal data was normalized through a unified standard, and the sliding window segmentation method was used in the time series dimension to construct dynamic features. Missing values were repaired using the multiple imputation algorithm to form a complete and consistent input vector with time series characteristics.

[0045] Then, the control group adopted a centralized XGBoost model, a traditional training paradigm, with an input dimension of 58, simulating the conventional process of centralized training and centralized deployment in a data set. The experimental group adopted the federated deep learning model proposed in the present invention, which integrated multi-modal input, distributed modeling, and differential privacy protection mechanisms, and the input dimension was extended to 213. Both groups of models were trained with the same data volume and number of training rounds to maintain the objectivity of the experiment.

[0046] The evaluation dimensions included the model parameter order of magnitude, the number of feature dimensions, the data leakage risk (quantified by the reverse inference ability), the inference latency, the scoring accuracy (using AUC as an indicator), the edge memory occupancy, and the communication load. The index statistics were carried out through the average value of multiple rounds of tests.

[0047] Table 1 Comparison data table of farmer credit assessment experiments

[0048] As can be seen from Table 1, the present invention is superior to the traditional centralized model in multiple key indicators. First, the number of model parameters decreased from 12.7M to 3.2M, and the compression ratio exceeded 75%, which is closely related to the introduction of grouped sparse convolution and Top-K gradient screening algorithms in the federated structure of the present invention. Through selective sparse gradient updates during the parameter upload stage, both computation and communication were optimized, adapting to the real environment of insufficient bandwidth and hardware limitations of rural agricultural assistance terminals.

[0049] In terms of feature dimensions, the model of the present invention expands the input dimension to 213 through a multi-modal data fusion mechanism, covering land GIS information, NDVI dynamic index, historical credit behavior, housing depreciation, crop output prediction factors, etc., providing richer discriminant basis for the model. In contrast, the traditional model is limited by the structure and input type, with a dimension of only 58 and insufficient expressive ability.

[0050] The data privacy risk score is significantly reduced from 9.1 to 2.3, thanks to the introduction of the differential privacy mechanism. Adding Laplace perturbations uniformly during the parameter aggregation stage makes it difficult for attackers to reverse-engineer the original data at the edge device through gradients, structurally ensuring the security of farmers' data and avoiding the high-risk problem of "one leak, global leak" in traditional centralized modeling.

[0051] The AUC of the model is improved from 0.742 to 0.816, indicating that the structure of the present invention also has an essential improvement in actual evaluation accuracy, reflecting the synergistic effect of multi-dimensional structure expressiveness and joint modeling strategy. In addition, the inference latency on the edge device is shortened from 380ms to 90ms, indicating that after model compression and quantization distillation, the present invention is suitable for real-time response in low-power environments, further enhancing the deployment feasibility.

[0052] Generally speaking, as Figure 2 shows the comparison of the parameter distributions of the fully connected network and the sparse convolutional network adopted in the present invention. The blue area represents the distribution of parameter values in the fully connected network, and the red area represents the distribution of the sparse convolutional network parameters after adopting the grouped sparse strategy in the present invention. The overlapping part of the two is purple, indicating a similar distribution. Figure 2 The red distribution is more concentrated around 0 compared to the blue distribution, indicating that the sparse convolutional strategy introduced in the present invention effectively compresses unnecessary parameters, making most weights approach zero, which improves the deployment efficiency of the model in resource-constrained terminals; the tail of the red distribution is less, indicating that excessive redundant parameters are effectively suppressed, which is beneficial to improving the stability and interpretability of the model; the model structure with a more compact parameter distribution in the present invention, combined with the Top-K gradient sparse upload and 8-bit quantization mechanism introduced in the present invention, significantly reduces the memory occupancy and computational latency when running on edge devices.

[0053] Example 3, referring to Figure 3 is an embodiment of the present invention, providing a farmer credit assessment system based on multi-modal data federated learning, including a multi-modal feature construction module 100, a federated collaborative training module 200, and a model compression and transmission control module 300.

[0054] Among them, X1: The multi-modal feature construction module 100 includes an asset space-time series fusion coding sub-module 101 and a crop dynamic production capacity and risk modeling sub-module 102.

[0055] It should be noted that the asset spatio-temporal fusion coding sub-module 101 is used to uniformly integrate the land GPS coordinates and property rights duration, and adopts UTM projection and OCR parsing technology to form the plot topology; the crop dynamic production capacity and risk modeling sub-module 102 is used to construct a crop health status and climate fluctuation response matrix based on the NDVI time series and the meteorological records of agricultural technology stations, and generate an output prediction feature vector through timestamp synchronization and interpolation processing, which characterizes the agricultural operation ability of farmers.

[0056] It should also be noted that the high-dimensional farmer feature vector output by the multi-modal feature construction module 100 serves as the input data basis for the federated collaborative training module 200, and is locally trained by the terminal device to generate local model updates.

[0057] X2: The federated collaborative training module 200 includes a secure communication and device authentication management sub-module 201 and a privacy aggregation and client dynamic screening sub-module 202.

[0058] It should be noted that the secure communication and device authentication management sub-module 201 is used to construct a three-level federated structure, establish a secure channel and complete device identity binding through the TLS 1.3 protocol and the timestamp token mechanism, ensuring node trust and communication encryption during the federated training process; the privacy aggregation and client dynamic screening sub-module 202 is used to perform differential privacy noise injection during the aggregation stage, apply Laplace perturbation to the uploaded parameters to protect privacy, and screen terminal devices based on local data freshness and network status, and dynamically allocate training tasks.

[0059] It should also be noted that the federated collaborative training module 200 uploads and aggregates the client parameters during the training stage, and distributes them to the model compression and transmission control module 300 for compression and deployment after the parameters are updated.

[0060] X3: The model compression and transmission control module 300 includes a Top-K gradient sparse upload sub-module 301, a grouped sparse convolution training sub-module 302, and an 8-bit quantization distillation deployment sub-module 303.

[0061] It should be noted that the Top-K gradient sparse upload sub-module 301 is used to only select the top K gradients with the largest absolute values for upload after local training at the terminal, and the sparsity rate is dynamically adjusted according to the training rounds and bandwidth status to minimize the communication load; the grouped sparse convolution training sub-module 302 is used to divide the convolution layer into multiple groups for independent training, reduce parameter redundancy, reduce the computational overhead, and adapt to low-performance edge devices; the 8-bit quantization distillation deployment sub-module 303 is used to distill the floating-point model into a fixed-point quantization version, significantly reducing the memory occupancy and inference latency while maintaining the performance, and supporting the immediate deployment of ARM-A76 level devices.

[0062] It should also be noted that after the model compression and transmission control module 300 deploys the compressed model to the terminal device, it enters the next round of training with the new data generated by the multimodal feature construction module 100, forming a closed loop.

[0063] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0064] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0065] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because, for example, the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or, if necessary, other suitable processing, and then stored in a computer memory.

[0066] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following techniques well known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for evaluating farmers' credit based on multi-modal data federated learning, characterized in that Including: Based on GIS spatial coding and property right duration information, construct a dual-channel representation method for land assets, combine multi-source data, and generate multi-dimensional feature inputs of farmers' production capacity and asset status; Construct a three-level federated learning structure, securely exchange data through TLS communication encryption and device identity authentication, and inject differential privacy noise; Adopt the Top-K gradient selection algorithm for parameter sparse transmission, dynamically adjust the sparsity rate in combination with the round increment strategy, introduce grouped sparse convolution and parameter freezing mechanism, and compress and deploy the federated deep learning model adapted to the edge through 8-bit distillation quantization; The dual-channel representation method includes spatial-time dual-channel land modeling, which encodes land assets using two independent dimensions of GIS spatial location and property right duration; The three-level federated learning structure includes constructing a three-level federated topology composed of terminal devices, edge nodes, and a central coordinator, communicating through a secure communication protocol, and constructing a hierarchical heterogeneous federated network architecture; Device identity authentication includes implementing terminal identity authentication and binding of upstream and downstream nodes through the device unique ID and timestamp token; Injecting differential privacy noise includes adding Laplace noise to the uploaded parameters during the aggregation process and uniformly controlling the perturbation intensity on the server side; Parameter sparse transmission includes dynamically adjusting the Top-K gradient upload ratio based on the increase in training rounds, and flexibly adjusting gradient transmission in combination with the communication status and local loss changes; Compressing the federated deep learning model through 8-bit distillation quantization includes completing compression and accuracy transfer before deploying the global federated deep learning model using the combined strategy of 8-bit fixed-point quantization and knowledge distillation.

2. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 1, wherein: The construction of the dual-channel representation method for land assets includes: Collect the GPS coordinates of farmers' land, convert the coordinates to UTM encoding using the unified Mercator projection, and construct a land spatial adjacency matrix to represent the plot topology; Based on rural land ownership data, extract the registration duration field, extract key time factors through text recognition and format standardization methods, and construct a spatio-temporal integrated land asset feature input combined with spatial location coding.

3. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 1 or 2, characterized in that: The generation of multi-dimensional feature inputs of farmers' production capacity and asset status includes: Extract continuous features from NDVI data through a sliding window, construct a meteorological factor matrix to model and extract crop output features, generate housing depreciation features based on building material durability parameters and years, and splice all features in a unified dimension.

4. The method for evaluating farmers' credit based on multi-modal data federated learning according to claim 3, wherein: The construction of the meteorological factor matrix to model and extract crop output features includes: Construct a data table of crop types and historical climate fluctuation factors, file it regionally based on agricultural technology promotion, synchronize timestamps, use interpolation to fill in missing values, participate in the combined modeling of output prediction factors, and output an estimated value vector.

5. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 1, characterized in that: The construction of the three-level federated learning structure includes: Set up a central coordinator node to manage the global federated deep learning model and the aggregation and distribution process, set up regional edge nodes to receive the federated deep learning models uploaded by terminal devices, perform privacy processing, locally collect and encrypt farmers' data based on terminal devices to participate in federated training, and complete mapping and identity binding between each layer through the unique device ID.

6. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 5, wherein: The TLS communication encryption and device identity authentication include: Establish a secure communication channel of the TLS 1.3 protocol between nodes at all levels, perform identity token authentication on participating devices based on keys and timestamps. After successful authentication, allocate a temporary session channel, and each round of federated deep learning model parameter exchange is encrypted and transmitted through the temporary session channel. Reject unregistered and nodes with failed authentication from joining the training round.

7. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 6, wherein: The injection of differential privacy noise includes Applying Laplace distribution random perturbation to the update of federated deep learning model parameters and injecting all the noise into the server side.

8. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 1, wherein: The adoption of the Top-K gradient selection algorithm for parameter sparse transmission includes The client device sorts the local gradient values by absolute value, selects the top K gradient values and uploads them to the upper-level node, sets the sparse transmission value, adaptively adjusts the sparsity rate, and updates the selection strategy.

9. The method for evaluating the credit of farmers based on multi-modal data federated learning according to claim 8, wherein: The introduction of grouped sparse convolution and parameter freezing mechanism includes Based on the dimension of the temporary session channel, the convolutional layer of the federated deep learning model is divided into multiple groups for independent parameter set training. For the parameter layer with stable convergence, after the number of rounds reaches the set threshold, the update channel is frozen. The knowledge distillation method is used to map the floating-point model to an equivalent 8-bit fixed-point network and deploy low-power edge devices.

10. A farmer credit assessment system based on multi-modal data federated learning, characterized in that: It includes a multi-modal feature construction module (100), a federated collaborative training module (200), and a model compression and transmission control module (300); The multi-modal feature construction module (100) is responsible for collecting and processing heterogeneous raw data in multiple dimensions of land, houses, and crops, and generating farmer credit feature vectors for training using unified spatial coding; The federated collaborative training module (200) is used to build a three-level federated architecture, support device registration, synchronize tasks and local training, and complete distributed model updates based on encrypted communication and identity authentication; The model compression and transmission control module (300) is used to perform structured compression and training control on model parameters through Top-K gradient screening, adopting sparse convolution, parameter freezing, and 8-bit distillation quantization strategies.

11. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the farmer credit assessment method based on multi-modal data federated learning according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the farmer credit assessment method based on multi-modal data federated learning according to any one of claims 1 to 9.

Images