Data acquisition and management system for accurate help in big data environment

Abstract

The invention relates to the technical field of information management, in particular to a data acquisition and management system for accurate help in a big data environment, and the system comprises a survey information analysis module, a data anomaly analysis module, a home geographic positioning module, a resource demand analysis module, an economic trend analysis module and a data management optimization module. According to the method, key information of low-income reasons and family structures is extracted through semantic analysis, the quality and accuracy of data are optimized, seasonal and tendency analysis of time information is combined, input errors and fraudulent behaviors are recognized, the reliability of the data is improved, and through analysis of geographic positions and social economic attributes of low-income families, the accuracy of the data is improved. The efficiency of classifying and identifying the low-income families by the government and the organization is improved, the economic condition of the low-income families is dynamically evaluated, the resource allocation priority is adjusted in time, the reasonability of resource allocation is optimized, the pertinence and effectiveness of assistance work are improved, and accurate putting and maximum utilization of resources are ensured.

Description

Technical Field

[0001] The present invention relates to the field of information management technology, and in particular to a data collection and management system for precise assistance in a big data environment. Background Art

[0002] The field of information management technology focuses on the collection, storage, processing, protection, transmission and retrieval of information. It uses computer science and information science to optimize the management of information flow and data resources. It aims to improve the organization's information utilization efficiency, ensure data security and privacy, and support the organization's strategic decision-making and daily operations through information systems. It is applied to database management systems, content management systems and knowledge management systems to help enterprises and organizations effectively process batches of data and information, ensure the integration, security and efficient use of data, and maximize the value of information.

[0003] Among them, the data collection and management system used for precise assistance focuses on providing support for assistance decision-making through efficient data collection and precise data management. It is used to collect relevant data on low-income areas and low-income groups, including economic conditions, educational resources, and medical conditions. By managing and analyzing target data, it adjusts and optimizes the allocation and optimization strategies of assistance resources, improves the pertinence and effectiveness of assistance projects, and helps governments and related organizations implement assistance plans more effectively, ensuring that resources are delivered accurately to related populations.

[0004] Traditional data collection and management systems lack sufficient flexibility and adaptability when processing batch data, and perform poorly in analyzing the causes of low income and real-time data processing. They are unable to handle complex data analysis needs, making it difficult to adjust assistance strategies in a timely manner when responding to sudden economic events or seasonal changes, affecting the timeliness and effectiveness of assistance measures. They rely on static data and less-changing indicators in assessing resource allocation priorities, lack a sensitive response to changes in the conditions of low-income families, and are unable to accurately reflect the immediate needs of low-income families, resulting in waste of resources and failure of assistance resources to reach the groups most in need. The lack of effective geographic information analysis limits the precise identification and targeted intervention of low-income areas, reducing the geographic targeting and overall efficiency of assistance projects. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a data collection and management system for precise assistance in a big data environment.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a data collection and management system for precise assistance in a big data environment comprises:

[0007] The survey information analysis module analyzes the low-income reasons and family member structure information in the text through semantic analysis based on the social survey text information, and generates low-income information processing results;

[0008] The data anomaly analysis module performs seasonality and trend analysis on the data according to the low-income information processing result and time information, identifies abnormal data, and generates abnormal data identification results;

[0009] The family geographic positioning module identifies the geographic location data of multiple low-income families based on the abnormal data identification result, and records the characteristic information of the multiple low-income families to generate low-income family characteristic information;

[0010] The resource demand analysis module classifies a plurality of low-income families according to the characteristic information of the low-income families, evaluates the resource demands and resource allocation priorities of the plurality of low-income families, and generates a classification result of help information;

[0011] The economic trend analysis module uses the classification results of the help information to analyze the changing trends of the economic conditions of multiple low-income families, calculates the resource allocation priorities that need to be adjusted for the target families, and generates dynamic trend evaluation information;

[0012] The data management optimization module is based on the dynamic trend evaluation information, analyzes the query and access frequency of various data, adjusts the index structure to optimize the query efficiency, and generates results to help data management.

[0013] As a further solution of the present invention, the low-income information processing results include classification labels for low-income families, statistical frequency information on reasons for low income, and family member composition data; the abnormal data identification results include seasonal abnormality marking results, trend abnormality indicator information, and data integrity verification results; the low-income family characteristic information includes geographical agglomeration area division results, economic activity types, and social participation assessment information; the help information classification results include resource demand rating results, low-income level classification data, and resource allocation priority ranking lists; the dynamic trend assessment information includes resource adjustment information, family economic status change trend analysis data, and long-term low-income risk prediction information; the help data management results include index optimization records, data access time information, and data access frequency information.

[0014] As a further solution of the present invention, the survey information analysis module includes:

[0015] The text extraction submodule identifies the text content in the target information based on the social survey text information, records the reasons for low income and family member structure information, and generates text content recognition records;

[0016] The semantic analysis submodule identifies and records the semantic roles and key information in the text based on the text content recognition record, analyzes the roles and economic status of multiple family members, and generates a semantic information analysis record;

[0017] The data standardization submodule formats and standardizes the data based on the semantic information analysis records, optimizes the consistency of the data structure, and generates low-income information processing results.

[0018] As a further solution of the present invention, the data anomaly analysis module includes:

[0019] The time information identification submodule identifies time tag information based on the low-income information processing result, analyzes the time change trend of the data, performs trend evaluation on the time series, and obtains time trend analysis data;

[0020] The data fluctuation analysis submodule analyzes the data based on the time trend, performs seasonal decomposition on the data, identifies data fluctuations in periodic patterns and predicts data trends to obtain seasonal analysis results;

[0021] The data deviation identification submodule combines the seasonal analysis results and identifies abnormal data points that deviate from the normal state, including data entry errors and fraudulent behavior, by comparing the deviations between data fluctuations and standard fluctuation models, to obtain abnormal data identification results.

[0022] As a further solution of the present invention, the family geolocation module includes:

[0023] The location information analysis submodule uses a K-means clustering algorithm based on the abnormal data identification result to identify and analyze the geographical location information of multiple low-income families, evaluate the geographical distribution characteristics of multiple low-income families, and generate a geographical distribution analysis result;

[0024] The socio-economic attribute analysis submodule collects and analyzes the income status, family member information and labor capacity data of multiple low-income families based on the geographic distribution analysis results, and evaluates the impact of multiple target attributes on the family's economic status to generate an economic status evaluation result;

[0025] The income characteristic identification submodule analyzes and identifies key economic and social characteristics of low-income families based on the economic status assessment results, including income level, educational background, and health status, and generates low-income family characteristic information.

[0026] As a further solution of the present invention, the K-means clustering algorithm is based on the formula:

[0027]

[0028] Calculate the root mean square distance from each cluster center to its members to generate geographic distribution analysis results, where C i represents the concentration index of the ith cluster, which is used to measure the concentration of the geographical locations of all households in the cluster. The smaller the value, the closer the geographical locations of the households in the cluster are. i represents the index of the cluster, that is, the ith cluster is currently being calculated. j represents the index of the household in the ith cluster, that is, the jth household in the ith cluster is being examined. x ij represents the longitude value of the jth family in the i-th cluster, μ i represents the average longitude of all households in the ith cluster, y ij represents the latitude value of the jth family in the i-th cluster, ν i represents the average latitude of all households in the ith cluster, and n represents the total number of households in the ith cluster.

[0029] As a further solution of the present invention, the resource demand analysis module includes:

[0030] The family information grouping submodule uses the low-income family characteristic information to group the low-income family information according to the family's income status, geographical location and labor capacity, records the characteristic information of multiple groups, and generates a family classification result;

[0031] The help demand assessment submodule assesses the resource demand type information of families in multiple categories based on the family classification results, including educational support, health services, and employment assistance, and generates resource demand analysis data;

[0032] The priority analysis submodule adjusts the resource allocation priorities of multiple households and household groups according to the resource demand analysis data, optimizes the efficiency and fairness of resource allocation, and generates help information classification results.

[0033] As a further solution of the present invention, the economic trend analysis module includes:

[0034] The time series analysis submodule uses an autoregressive moving average model based on the classification results of the help information to collect economic data of multiple low-income families, identifies the cyclical characteristics and change trends of economic activities through time series analysis, and generates economic change trend information;

[0035] The trend prediction submodule evaluates and predicts the economic status changes of multiple low-income families based on the economic change trend information, including income fluctuations and economic growth rates, and generates economic trend prediction results;

[0036] The priority calculation submodule calculates the resource demand priorities that need to be adjusted for multiple low-income families based on the economic trend forecast results, including education, health and employment assistance needs, and generates dynamic trend assessment information.

[0037] As a further embodiment of the present invention, the autoregressive moving average model is based on the formula

[0038]

[0039] Calculate the time series of the household economic data and generate economic change trend information, where t represents the current time point, which is used to mark a specific observation point in the time series. In the summation formula of the autoregressive part, i is the index, which means looking at i time units from the current time point t to the past, which is used to access historical data points. In the summation formula of the moving average part, j is the index, which means looking at j time units from the current time point t to the past, which is used to access historical error values. t-i Represents the observation value i time units before the current time point t, ∈ t-j represents the prediction error j time units before the current time point t, X t represents the observed economic indicator value at time t, reflecting the economic status of the family at this point in time, c represents the constant term in the model, adjusting the baseline level to match the economic situation of the individual family, ∈ t represents the error term, which indicates the deviation between the current actual value and the model predicted value. p represents the order of the autoregressive term, which means that we need to look back at the data of the past p periods to predict the current value. i is the autoregressive coefficient, which measures the influence of the economic data of the i-th period on the current economic status. q represents the order of the moving average term, which determines the impact of the errors of the past q periods on the current forecast. θ j It is the moving average coefficient, which is used to adjust the contribution of past errors to the current economic indicator forecast.

[0040] As a further solution of the present invention, the data management optimization module includes:

[0041] The access frequency calculation submodule identifies the frequencies of various data queries and accesses based on the dynamic trend evaluation information, analyzes the data access patterns and usage frequencies, and generates data access analysis results;

[0042] The index structure adjustment submodule adjusts the database index structure based on the data access analysis results, including adding and modifying data indexes to optimize query efficiency and generating data index adjustment records;

[0043] The performance evaluation submodule analyzes and records the data query and storage efficiency after the index structure adjustment based on the data index adjustment record, optimizes the operation efficiency of the data management system, and generates results to help data management.

[0044] Compared with the prior art, the advantages and positive effects of the present invention are:

[0045] In the present invention, key information on the causes of low income and family structure is extracted through semantic analysis to optimize the quality and accuracy of data. Combined with the seasonal and trend analysis of time information, input errors and fraudulent behavior are identified to improve the reliability of data. By analyzing the geographical location and socioeconomic attributes of low-income families, the efficiency of government and organization in classifying and identifying low-income families is improved, the economic status of low-income families is dynamically assessed, and resource allocation priorities are adjusted in a timely manner to optimize the rationality of resource allocation, improve the pertinence and effectiveness of assistance work, and ensure accurate allocation of resources and maximum utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a system flow chart of the present invention;

[0047] Figure 2 It is a schematic diagram of the system framework of the present invention;

[0048] Figure 3 This is a flow chart of the survey information analysis module of the present invention;

[0049] Figure 4 It is a flow chart of the data anomaly analysis module of the present invention;

[0050] Figure 5 This is a flow chart of the family geolocation module of the present invention;

[0051] Figure 6 It is a flow chart of the resource demand analysis module of the present invention;

[0052] Figure 7 It is a flow chart of the economic trend analysis module of the present invention;

[0053] Figure 8 This is a flow chart of the data management optimization module of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0056] See also Figure 1 to Figure 2 , a data collection and management system for precise assistance in a big data environment includes:

[0057] The survey information analysis module analyzes the low-income reasons and family member structure information in the text through semantic analysis based on the social survey text information, and generates low-income information processing results;

[0058] The data anomaly analysis module performs seasonal and trend analysis on the data based on the low-income information processing results and time information, identifies abnormal data, and generates abnormal data identification results;

[0059] The family geolocation module identifies the geographic location data of multiple low-income families based on the abnormal data identification results, and records the characteristic information of multiple low-income families to generate low-income family characteristic information;

[0060] The resource demand analysis module classifies multiple low-income families according to their characteristic information, evaluates their resource demands and resource allocation priorities, and generates classification results for help information;

[0061] The economic trend analysis module uses the help information classification results to analyze the changing trends of the economic conditions of multiple low-income families, calculate the resource allocation priorities that need to be adjusted for the target families, and generate dynamic trend evaluation information;

[0062] The data management optimization module is based on dynamic trend evaluation information, analyzes the query and access frequency of various data, adjusts the index structure to optimize query efficiency, and generates results to help data management.

[0063] The results of low-income information processing include classification labels for low-income families, statistical frequency information on reasons for low income, and family member composition data. The results of abnormal data identification include seasonal abnormality marking results, trend abnormality indicator information, and data integrity verification results. The characteristic information of low-income families includes geographical agglomeration area division results, economic activity types, and social participation assessment information. The results of helpful information classification include resource demand rating results, low-income level classification data, and resource allocation priority ranking lists. Dynamic trend assessment information includes resource adjustment information, family economic status change trend analysis data, and long-term low-income risk prediction information. The results of helpful data management include index optimization records, data access time information, and data access frequency information.

[0064] See also Figure 2 and Figure 3 , the survey information analysis module includes:

[0065] The text extraction submodule identifies the text content in the target information based on the social survey text information, records the reasons for low income and family member structure information, and generates text content recognition records;

[0066] In the text extraction submodule, the text content recognition process based on social survey text information is carried out according to the formula

[0067]

[0068] Calculate the text feature weight score, where F represents the feature weight score of the text content, which is the result obtained by weighting multiple feature items, w i represents the weight of the i-th feature, x i represents the value of the i-th feature item, n represents the total number of feature items,

[0069] Detailed explanation of the formula and the process of formula calculation and derivation:

[0070] Assuming that the weight of the reason for low income is 0.6, the weight of the number of family members is 0.3, and the weight of the complexity of family structure is 0.1, if in a certain survey report, the frequency of low income reasons is 10 times, the number of family members is 5, and the family structure complexity score is 7, the calculation process is as follows:

[0071] F=(0.6×10)+(0.3×5)+(0.1×7)=6+1.5+0.7=8.2

[0072] The result was a score of 8.2, reflecting the prominence of the reasons for low income in the text and the complexity of the information about family members.

[0073] The semantic analysis submodule identifies and records the semantic roles and key information in the text based on the text content recognition record, analyzes the roles and economic status of multiple family members, and generates a semantic information analysis record;

[0074] In the semantic analysis submodule, semantic roles and key information in the text are identified and recorded based on text content recognition records. The text is analyzed through deep learning natural language processing technology to identify key semantic roles, including beneficiaries, supporters, economic status and other key words. Each identified role and keyword is scored and classified. The classification algorithm in the machine learning model is used to evaluate and classify the influence and relevance of multiple semantic roles. Socioeconomic data is extracted from the text for low-income analysis and formulation of resource allocation strategies to ensure the high quality and relevance of information processing.

[0075] The data standardization submodule formats and standardizes the data based on semantic information analysis records, optimizes data structure consistency, and generates low-income information processing results;

[0076] In the data standardization submodule, after obtaining the semantic information analysis records, the data is formatted and standardized to ensure that the data follows a unified format and standard for subsequent data processing and analysis, including data cleaning, removing noise and irrelevant information in the data, correcting spelling errors, unifying quantification standards, converting the number of members in data from differentiated sources from text descriptions into numerical information, standardizing data formats, and uniformly formatting dates and times to improve data consistency and availability, enhance the accuracy and efficiency of data analysis, ensure that the analysis is based on accurate and consistent data, generate low-income information processing results, and provide solid data support for decision-making.

[0077] See also Figure 2 and Figure 4 , the data anomaly analysis module includes:

[0078] The time information identification submodule identifies the time tag information based on the low-income information processing results, analyzes the time change trend of the data, performs trend evaluation on the time series, and obtains the time trend analysis data;

[0079] In the time information identification submodule, based on the low-income information processing results, through the identification and analysis of time tag information, data mining technology is used, including the use of regular expressions to extract date and time information from text, and the extracted time data is formatted to ensure the consistency and accuracy of the time data. The time series analysis method is used to analyze the time change trend of the data. By considering the autoregressive nature and seasonal fluctuations of the time data, the trend of data evolution over time is analyzed to provide support for the formulation of targeted assistance strategies. The obtained time trend analysis data provides an empirical basis for subsequent decision-making.

[0080] The data fluctuation analysis submodule analyzes data based on time trends, decomposes data seasonally, identifies data fluctuations in periodic patterns, and predicts data trends to obtain seasonal analysis results;

[0081] In the data fluctuation analysis submodule, complex seasonal decomposition technology is used to analyze the time trend analysis data. Through seasonal decomposition and periodic pattern recognition in time series analysis, the seasonal decomposition function in the statistical software package is used to identify and strip off the seasonal components in the time series data, including using the X-12-ARIMA method to process monthly or quarterly economic data, identify periodic patterns in the data, including wage payment or agricultural production cycles, and data fluctuations caused by seasonal changes, predict data trends, and provide a basis for formulating seasonal adjustment policies. The seasonal analysis results are used to analyze the inherent laws of data fluctuations and provide a scientific basis for resource allocation and policy adjustments.

[0082] The data deviation identification submodule combines the seasonal analysis results and identifies abnormal data points that deviate from the norm, including data entry errors and fraudulent behaviors, by comparing the deviations between data fluctuations and the standard fluctuation model, and obtains abnormal data identification results;

[0083] In the data deviation identification submodule, abnormal data points are identified based on the seasonal analysis results, according to the formula

[0084]

[0085] Calculate the standard deviation of the data and identify abnormal data points that deviate from the normal state. i Represents the value of the ith data point, μ represents the mean of the data set, N represents the total number of data points, and σ represents the standard deviation of the data. It identifies abnormal data points in the data set that deviate from the mean by more than twice the standard deviation, identifies data entry errors or potential fraud, and provides a basis for data cleaning and quality control. The abnormal data identification results obtained can ensure the accuracy and reliability of data analysis.

[0086] Detailed explanation of the formula and the process of formula calculation and derivation:

[0087] Assume that the data points collected during the observation period are: 100, 102, 98, 95, 101,

[0088] The data point mean μ is calculated as:

[0089]

[0090] The standard deviation σ is calculated as:

[0091]

[0092] The resulting standard deviation value is 2.48, indicating that most data points are concentrated within the range of plus or minus 2.48 of the mean value. Data points outside the target range are considered abnormal, and abnormal data are filtered to ensure the accuracy of data processing.

[0093] See also Figure 2 and Figure 5 , the home geolocation module includes:

[0094] Based on the abnormal data identification results, the location information analysis submodule uses the K-means clustering algorithm to identify and analyze the geographical location information of multiple low-income families, evaluate the geographical distribution characteristics of multiple low-income families, and generate geographical distribution analysis results;

[0095] K-means clustering algorithm, according to the formula:

[0096]

[0097] Calculate the root mean square distance from each cluster center to its members to generate geographic distribution analysis results, where C i represents the concentration index of the ith cluster, which is used to measure the concentration of the geographical locations of all households in the cluster. The smaller the value, the closer the geographical locations of the households in the cluster are. i represents the index of the cluster, that is, the ith cluster is currently being calculated. j represents the index of the household in the ith cluster, that is, the jth household in the ith cluster is being examined. x ij represents the longitude value of the jth family in the i-th cluster, μ i represents the average longitude of all households in the ith cluster, y ij represents the latitude value of the jth family in the i-th cluster, ν i represents the average latitude of all households in the ith cluster, and n represents the total number of households in the ith cluster.

[0098] By calculating the square root of the average Euclidean distance from each cluster center to its cluster members, the geographic agglomeration characteristics of different low-income households are evaluated. The low-income households are clustered according to their geographical locations through the K-means clustering algorithm, and the geographic distribution patterns and potential regional low-income concentration areas are identified, providing precise geographic information support for resource allocation and policy making.

[0099] formula:

[0100]

[0101] Detailed explanation of the formula and the process of formula calculation and derivation:

[0102] The formula used to calculate the geographic concentration of each low-income household cluster includes the square root of the sum of the squared average distances in both longitude and latitude.

[0103] The parameters in the formula are as follows:

[0104] x ij and ij are the geographic coordinates of a specific household, i.e. longitude and latitude,

[0105] μ i and ν i is the average longitude and latitude of all households in the ith cluster,

[0106] n is the total number of households in the ith cluster.

[0107] Suppose there are three households in a cluster, and their geographical locations are as follows:

[0108] Home 1, longitude 100.0, latitude 20.0,

[0109] Home 2, longitude 102.0, latitude 22.0,

[0110] Home 3, longitude 98.0, latitude 18.0,

[0111] Substitute the parameters into the formula for calculation:

[0112]

[0113] Compute the mean longitude and mean latitude:

[0114]

[0115] Calculate the sum of the squared distances of each household from the mean location:

[0116] The square of the distance from home 1 to the mean location (100.0-100.0) 2 +(20.0-20.0) 2 =0,

[0117] The square of the distance from home 2 to the mean location (102.0-100.0) 2 +(22.0-20.0) 2 =4+4=8,

[0118] The square of the distance from home 3 to the mean location (98.0-100.0) 2 +(18.0-20.0) 2 =4+4=8,

[0119] The total distance square sum is 0+8+8=16,

[0120] Calculate the aggregation degree C i :

[0121]

[0122] The results show that when calculating the average Euclidean distance from each household in the cluster to the average geographical location, the value obtained is 3.27, indicating that the households in the cluster have a certain degree of geographical agglomeration. The smaller the value, the closer the households are geographically, and the larger the value, the higher the geographical dispersion of the households. It is possible to identify the geographically concentrated areas that need to be focused on and provide decision support for precise assistance.

[0123] The socio-economic attribute analysis submodule collects and analyzes the income status, family member information, and labor capacity data of multiple low-income families based on the results of the geographic distribution analysis, and evaluates the impact of multiple target attributes on the family's economic status to generate economic status assessment results;

[0124] In the socio-economic attribute analysis submodule, the income status, family member information and labor capacity data of multiple low-income families are collected and analyzed through the results of geographical distribution analysis. Linear regression and variance analysis are used to identify and evaluate the impact of various socio-economic attributes on the family's economic status. The socio-economic information of the family is obtained through data collection tools, and invalid or erroneous data records are eliminated using data cleaning technology. Statistical analysis methods are used to perform pattern recognition and association analysis on clean data sets, including calculating the correlation between income level and education level, understanding the impact of various factors on the family's economic status, and providing a basis for formulating effective assistance measures. The generated economic status assessment results are crucial for understanding the low-income phenomenon and formulating intervention strategies.

[0125] The income characteristics identification submodule analyzes and identifies the key economic and social characteristics of low-income families based on the economic status assessment results, including income level, educational background, and health status, and generates low-income family characteristic information;

[0126] In the income characteristic identification submodule, the key economic and social characteristics of low-income families are analyzed by utilizing the results of economic status assessment, including multi-dimensional information on income level, educational background and health status. Data mining and machine learning techniques are used to collect education and health records of family members through database queries and API calls. Normalization and encoding conversion are used to process target information. Machine learning algorithms are applied to identify key variables that affect family economic status, including the overall influence of family education level, revealing the socioeconomic motivations behind differentiated income levels, identifying the shortcomings of low-income families in education and health, and providing customized assistance strategies and intervention points for policymakers. The generated low-income family characteristic information provides a scientific basis for helping policy making.

[0127] See also Figure 2 and Figure 6 , the resource demand analysis module includes:

[0128] The family information grouping submodule uses the characteristic information of low-income families to group low-income family information according to the family's income status, geographical location and labor capacity, records the characteristic information of multiple groups, and generates family classification results;

[0129] In the family information grouping submodule, low-income family characteristic information is used to group low-income family information according to the family's income status, geographical location and labor capacity, including the classification and analysis of multi-dimensional data, and the use of decision tree algorithms to analyze and classify family data. The occupational information, health records and educational background of family members are extracted from the database, and families are divided according to economic status according to preset classification rules. For example, families with incomes below the regional median are classified as low-income families, including the impact of total family income, number of family members and geographical location on employment opportunities. Low-income families are subdivided according to needs to improve the pertinence and efficiency of resource allocation. The generated family classification results are used to help policy making and resource allocation.

[0130] The help demand assessment submodule evaluates the resource demand type information of families in multiple categories based on the family classification results, including educational support, health services, and employment assistance, and generates resource demand analysis data;

[0131] In the Help Needs Assessment submodule, resource needs assessment is performed based on the household classification results, according to the formula

[0132]

[0133] Calculate the total resource demand of each family group, where R represents the total resource demand, w i represents the weight of the i-th resource, v iRepresents the demand for the i-th type of resource, and n represents the total number of resource types. Through this method, the specific resource needs of different family groups in terms of education support, health services, employment assistance, etc. can be quantified, providing a quantitative basis for the precise allocation and optimization of resources. Detailed explanation of the formula and the derivation process of the formula calculation: Assuming that for a family group, the weight of education support is 0.5, the weight of health services is 0.3, and the weight of employment assistance is 0.2, and the demands are 10, 5, and 3 respectively. The total resource demand is calculated as follows:

[0134] R = 0.5 × 10 + 0.3 × 5 + 0.2 × 3

[0135] R=5+1.5+0.6

[0136] R=7.1

[0137] The results show that during the assessment process, the total resource demand of the family group is 7.1 units. The value reflects the priority and total demand of the group in terms of different resources, which is used to adjust priorities and rationally allocate resources according to the availability of resources.

[0138] The priority analysis submodule analyzes data based on resource requirements, adjusts resource allocation priorities for multiple households and household groups, optimizes the efficiency and fairness of resource allocation, and generates help information classification results;

[0139] In the priority analysis submodule, resource demand analysis data is used to adjust the resource allocation priorities among different families and family groups, including using linear programming and optimization algorithms to balance resource supply and demand, determining the intensity of each family group's demand for different resources based on resource demand analysis data, considering the total amount and availability of resources, and using linear programming models to optimize resource allocation plans to achieve fair distribution of multiple key resources such as education, medical care, and employment. Sensitivity analysis is used to evaluate the impact of changes in resource quantity on allocation priorities to ensure that when resources are tight, the groups most in need receive priority support. The generated help information classification results provide policymakers with scientific resource allocation strategies to maximize fairness.

[0140] See also Figure 2 and Figure 7 , the economic trend analysis module includes:

[0141] The time series analysis submodule uses the autoregressive moving average model based on the help information classification results to collect economic data of multiple low-income families, identify the cyclical characteristics and change trends of economic activities through time series analysis, and generate economic change trend information;

[0142] The autoregressive moving average model, according to the formula

[0143]

[0144] Calculate the time series of the household economic data and generate economic change trend information, where t represents the current time point, which is used to mark a specific observation point in the time series. In the summation formula of the autoregressive part, i is the index, which means looking at i time units from the current time point t to the past, which is used to access historical data points. In the summation formula of the moving average part, j is the index, which means looking at j time units from the current time point t to the past, which is used to access historical error values. t-i Represents the observation value i time units before the current time point t, ∈ t-j represents the prediction error j time units before the current time point t, X t represents the observed economic indicator value at time t, reflecting the economic status of the family at this point in time, c represents the constant term in the model, adjusting the baseline level to match the economic situation of the individual family, ∈ t represents the error term, which indicates the deviation between the current actual value and the model predicted value. p represents the order of the autoregressive term, which means that we need to look back at the data of the past p periods to predict the current value. i is the autoregressive coefficient, which measures the influence of the economic data of the i-th period on the current economic status. q represents the order of the moving average term, which determines the impact of the errors of the past q periods on the current forecast. θ j It is the moving average coefficient, which is used to adjust the contribution of past errors to the current economic indicator forecast.

[0145] By analyzing the dynamic changes in the economic status of multiple low-income families, real-time data support is provided for precise assistance strategies.

[0146] formula:

[0147]

[0148] Detailed explanation of the formula and the process of formula calculation and derivation:

[0149] Evaluate the changing trend of the economic status of target low-income families through the prediction of time series data.

[0150] Parameter setting and data collection:

[0151] c is the constant term of the model, reflecting the baseline economic level, and is assumed to be 1.2.

[0152] ∈ t is the prediction error at the current time point t, is a simulated value, used for example calculations, and is assumed to be 0.3.

[0153] p is the order of the autoregressive term, which is assumed to be 1, indicating that we only consider one period of historical data.

[0154] φ 1is the autoregression coefficient, assumed to be 0.8, which indicates the impact of the economic indicators of the previous period on the current forecast.

[0155] q is the order of the moving average term, which is assumed to be 1, indicating that the error impact of one period is considered.

[0156] θ 1 is the moving average coefficient, assuming it is -0.5, which is used to adjust the deviation of the current forecast caused by the error of the previous period.

[0157] i assumes 1, j assumes 1, which means considering the data and errors of period t-1.

[0158] X t-1 represents the observed value of the previous period. Assuming that the economic indicator at time t-1 is 500,

[0159] ∈ t-1 Represents the prediction error at time point t-1. Assume that the prediction error at time t-1 is 0.3.

[0160] Substitute the parameters into the formula for calculation:

[0161]

[0162] X t =1.2+0.3+0.8×500-(-0.5×0.3)

[0163] X t =1.2+0.3+400+0.15

[0164] X t =401.65

[0165] The results show that after considering the impact of baseline, historical economic data and previous errors, the predicted economic indicator value at the current time point t is 401.65, which means that the family's economic situation at time t is expected to improve compared with the previous period. The data is used to analyze the family's resource needs and allocation priorities, effectively guide resource allocation decisions, and ensure the accuracy and effectiveness of assistance measures.

[0166] The trend forecasting submodule evaluates and forecasts the economic status changes of multiple low-income families based on the economic change trend information, including income fluctuations and economic growth rate, and generates economic trend forecast results;

[0167] In the trend forecasting submodule, by analyzing the economic change trend information, the economic status of low-income families is predicted in the future, and the time series analysis technology is used to evaluate the income fluctuations and economic growth rate. The economic data of the past few years are collected, including family income, employment rate and regional economic development status. The autoregressive moving average model is used to analyze the data. The model parameters are automatically adjusted according to the historical data trend to match the changes in the economic cycle. The model analysis predicts the income changes and economic growth trends of families in the next few months or years, which helps policymakers and social service agencies understand the economic challenges faced by low-income families and provides a scientific basis for the formulation of related assistance policies. The generated economic trend forecast results are used for resource allocation and policy optimization.

[0168] The priority calculation submodule calculates the resource demand priorities that need to be adjusted for multiple low-income families based on the economic trend forecast results, including education, health and employment assistance needs, and generates dynamic trend assessment information;

[0169] In the priority calculation submodule, the resource demand priorities of low-income families in different support areas are calculated based on the economic trend forecast results. Through multivariate decision analysis, the data points in the economic trend forecast results are classified according to education, health, and employment assistance needs, and the urgency and resource sensitivity of families in each category are evaluated. The weighted sorting method is used to prioritize the needs of multiple families. The weight setting is adjusted according to the historical assistance effects and family feedback to ensure that resource allocation meets urgent needs and adapts to the dynamic changes in economic trends. It also ensures that education, health and employment assistance resources are reasonably allocated to improve the effectiveness and fairness of assistance. The generated dynamic trend assessment information provides decision makers with a basis for adjusting resource allocation to cope with changes in economic conditions.

[0170] See also Figure 2 and Figure 8 , the data management optimization module includes:

[0171] The access frequency calculation submodule identifies the frequencies of various data queries and accesses based on dynamic trend evaluation information, analyzes data access patterns and usage frequencies, and generates data access analysis results;

[0172] In the access frequency calculation submodule, data query and access frequency are identified and analyzed based on dynamic trend evaluation information. All users' query requests and access records are extracted from the system log. Time series analysis tools are used to identify data access patterns and change trends. The peaks and troughs of access frequency are determined by calculating the moving average and standard deviation of the daily access times. The spectrum analysis method is used to analyze periodic changes and identify access patterns, including the differences in access between weekdays and weekends. Data access behaviors are grouped through clustering algorithms to identify access characteristics of different user groups. The generated data access analysis results show the frequency and pattern of data access, revealing the temporal dynamics of user behavior and providing a basis for adjusting data management strategies.

[0173] The index structure adjustment submodule adjusts the database index structure based on the data access analysis results, including adding and modifying data indexes to optimize query efficiency and generating data index adjustment records;

[0174] In the index structure adjustment submodule, adjust the database index structure according to the data access analysis results, according to the formula

[0175]

[0176] Calculate the access efficiency comparison before and after index adjustment. In the formula, D represents the total efficiency improvement after index adjustment, and f i Represents the access frequency of the i-th query, and n represents the total number of query types. Considering the query frequency of each type, its contribution to efficiency improvement after optimization is obtained through inverse conversion. The index adjustment of each query is aimed at reducing access time and improving response speed, while maintaining overall performance optimization, ensuring reasonable allocation of resources and maximizing utilization efficiency.

[0177] Detailed explanation of the formula and the process of formula calculation and derivation:

[0178] Assuming that there are three main query types in the database system, with access frequencies of 100 times / day, 50 times / day, and 25 times / day respectively, the efficiency improvement is calculated as follows:

[0179]

[0180] D=0.01+0.02+0.04

[0181] D = 0.07

[0182] The results show that after adjusting the index structure, the overall efficiency of the system is improved by 0.07 units, which means that index optimization improves the overall query efficiency by reducing the processing time of high-frequency queries. The numerical results reflect the degree of improvement in the system efficiency after optimization, providing a quantitative basis for performance evaluation and system optimization.

[0183] The performance evaluation submodule adjusts the data index records, analyzes and records the data query and storage efficiency after the index structure is adjusted, optimizes the operating efficiency of the data management system, and generates results to help data management;

[0184] In the performance evaluation submodule, based on the data index adjustment records, the data query and storage efficiency after the index structure adjustment is analyzed and recorded, including the collection of system operation logs and performance indicators before and after the adjustment. Data analysis tools are used to compare the system response time and processing capacity at different time points, including the performance changes of query operations after index optimization. The actual effect of the index structure adjustment is evaluated by calculating the average query response time and transaction processing rate before and after the adjustment. The system's load balancing and resource utilization are considered. Through the target performance data, data management results are generated to help reflect the performance improvement brought about by index optimization and provide empirical support for data management strategies.

[0185] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A data collection and management system for precise assistance in a big data environment, characterized by: The system comprises: The survey information analysis module analyzes the low-income reasons and family member structure information in the text through semantic analysis based on the social survey text information, and generates low-income information processing results; The data anomaly analysis module performs seasonality and trend analysis on the data according to the low-income information processing result and time information, identifies abnormal data, and generates abnormal data identification results; The family geographic positioning module identifies the geographic location data of multiple low-income families based on the abnormal data identification result, and records the characteristic information of the multiple low-income families to generate low-income family characteristic information; The resource demand analysis module classifies a plurality of low-income families according to the characteristic information of the low-income families, evaluates the resource demands and resource allocation priorities of the plurality of low-income families, and generates a classification result of help information; The economic trend analysis module uses the classification results of the help information to analyze the changing trends of the economic conditions of multiple low-income families, calculates the resource allocation priorities that need to be adjusted for the target families, and generates dynamic trend evaluation information; The data management optimization module is based on the dynamic trend evaluation information, analyzes the query and access frequency of various data, adjusts the index structure to optimize the query efficiency, and generates results to help data management.

2. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The low-income information processing results include classification labels for low-income families, statistical frequency information on reasons for low income, and family member composition data; the abnormal data identification results include seasonal abnormality marking results, trend abnormality indicator information, and data integrity verification results; the low-income family characteristic information includes geographical agglomeration area division results, economic activity types, and social participation assessment information; the help information classification results include resource demand rating results, low-income level classification data, and resource allocation priority ranking lists; the dynamic trend assessment information includes resource adjustment information, family economic status change trend analysis data, and long-term low-income risk prediction information; the help data management results include index optimization records, data access time information, and data access frequency information.

3. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The survey information analysis module includes: The text extraction submodule identifies the text content in the target information based on the social survey text information, records the reasons for low income and family member structure information, and generates text content recognition records; The semantic analysis submodule identifies and records the semantic roles and key information in the text based on the text content recognition record, analyzes the roles and economic status of multiple family members, and generates a semantic information analysis record; The data standardization submodule formats and standardizes the data based on the semantic information analysis records, optimizes the consistency of the data structure, and generates low-income information processing results.

4. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The data anomaly analysis module includes: The time information identification submodule identifies time tag information based on the low-income information processing result, analyzes the time change trend of the data, performs trend evaluation on the time series, and obtains time trend analysis data; The data fluctuation analysis submodule analyzes the data based on the time trend, performs seasonal decomposition on the data, identifies data fluctuations in periodic patterns and predicts data trends to obtain seasonal analysis results; The data deviation identification submodule combines the seasonal analysis results and identifies abnormal data points that deviate from the normal state, including data entry errors and fraudulent behavior, by comparing the deviations between data fluctuations and standard fluctuation models, to obtain abnormal data identification results.

5. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The family geolocation module includes: The location information analysis submodule uses a K-means clustering algorithm based on the abnormal data identification result to identify and analyze the geographical location information of multiple low-income families, evaluate the geographical distribution characteristics of multiple low-income families, and generate a geographical distribution analysis result; The socio-economic attribute analysis submodule collects and analyzes the income status, family member information and labor capacity data of multiple low-income families based on the geographic distribution analysis results, and evaluates the impact of multiple target attributes on the family's economic status to generate an economic status evaluation result; The income characteristic identification submodule analyzes and identifies key economic and social characteristics of low-income families based on the economic status assessment results, including income level, educational background, and health status, and generates low-income family characteristic information.

6. The data collection and management system for precise assistance in a big data environment according to claim 5, characterized in that: The K-means clustering algorithm is based on the formula: Calculate the root mean square distance from each cluster center to its members to generate geographic distribution analysis results, where C i represents the concentration index of the ith cluster, which is used to measure the concentration of the geographical locations of all households in the cluster. The smaller the value, the closer the geographical locations of the households in the cluster are. i represents the index of the cluster, that is, the ith cluster is currently being calculated. j represents the index of the household in the ith cluster, that is, the jth household in the ith cluster is being examined. x ij represents the longitude value of the jth family in the i-th cluster, μ i represents the average longitude of all households in the ith cluster, y ij represents the latitude value of the jth family in the i-th cluster, ν i represents the average latitude of all households in the ith cluster, and n represents the total number of households in the ith cluster.

7. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The resource demand analysis module includes: The family information grouping submodule uses the low-income family characteristic information to group the low-income family information according to the family's income status, geographical location and labor capacity, records the characteristic information of multiple groups, and generates a family classification result; The help demand assessment submodule assesses the resource demand type information of families in multiple categories based on the family classification results, including educational support, health services, and employment assistance, and generates resource demand analysis data; The priority analysis submodule adjusts the resource allocation priorities of multiple households and household groups according to the resource demand analysis data, optimizes the efficiency and fairness of resource allocation, and generates help information classification results.

8. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The economic trend analysis module includes: The time series analysis submodule uses an autoregressive moving average model based on the classification results of the help information to collect economic data of multiple low-income families, identifies the cyclical characteristics and change trends of economic activities through time series analysis, and generates economic change trend information; The trend prediction submodule evaluates and predicts the economic status changes of multiple low-income families based on the economic change trend information, including income fluctuations and economic growth rates, and generates economic trend prediction results; The priority calculation submodule calculates the resource demand priorities that need to be adjusted for multiple low-income families based on the economic trend forecast results, including education, health and employment assistance needs, and generates dynamic trend assessment information.

9. The data collection and management system for precise assistance in a big data environment according to claim 8, characterized in that: The autoregressive moving average model, according to the formula Calculate the time series of the household economic data and generate economic change trend information, where t represents the current time point, which is used to mark a specific observation point in the time series. In the summation formula of the autoregressive part, i is the index, which means looking at i time units from the current time point t to the past, which is used to access historical data points. In the summation formula of the moving average part, j is the index, which means looking at j time units from the current time point t to the past, which is used to access historical error values. t-i Represents the observation value i time units before the current time point t, ∈ t-j represents the prediction error j time units before the current time point t, X t represents the observed economic indicator value at time t, reflecting the economic status of the family at this point in time, c represents the constant term in the model, adjusting the baseline level to match the economic situation of the individual family, ∈ t represents the error term, which indicates the deviation between the current actual value and the model predicted value. p represents the order of the autoregressive term, which means that we need to look back at the data of the past p periods to predict the current value. i is the autoregressive coefficient, which measures the influence of the economic data of the i-th period on the current economic status. q represents the order of the moving average term, which determines the impact of the errors of the past q periods on the current forecast. θ j It is the moving average coefficient, which is used to adjust the contribution of past errors to the current economic indicator forecast.

10. The data collection and management system for precise assistance in a big data environment according to claim 1, characterized in that: The data management optimization module includes: The access frequency calculation submodule identifies the frequencies of various data queries and accesses based on the dynamic trend evaluation information, analyzes the data access patterns and usage frequencies, and generates data access analysis results; The index structure adjustment submodule adjusts the database index structure based on the data access analysis results, including adding and modifying data indexes to optimize query efficiency and generating data index adjustment records; The performance evaluation submodule analyzes and records the data query and storage efficiency after the index structure adjustment based on the data index adjustment record, optimizes the operation efficiency of the data management system, and generates results to help data management.

Images