Adaptive incremental update method and system based on digital rural large language model

Abstract

The present invention proposes an adaptive incremental update method and system based on a digital village large language model, which belongs to the field of computers. The method includes: S1, setting a counter and a variable flag in a digital village system, and initializing the counter and the variable flag; S2, using the digital village system to collect new data of the system in real time; S3, converting the new data of the system into text information, and determining the type of the new data of the system according to the text information; S4, updating the counter according to the type of the new data of the system, and calculating the total number of text information, the total amount of information and the active index of the new data of the system according to the text information; S5, updating the variable flag according to the total number of text information, the total amount of information and the active index of the new data of the system, judging whether the new data of the system meets the update conditions according to the variable flag, and if so, updating the digital village large language model. The present application simultaneously considers the three dimensions of the total number of text information, the total amount of information and the active index, realizes a comprehensive evaluation of the new data of the system, effectively saves computing resources, and can adapt to different scenarios and needs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an adaptive incremental update method and system based on a digital village large language model. Background Art

[0002] Digital villages are an important direction for the development of modern rural areas, and they use information technology to improve rural governance capabilities and service levels. Among them, large language models, as the core technology of intelligent services, are widely used in rural management and services at different levels, such as villages, townships, and counties. These models can process and generate text information related to the local area, providing farmers and managers with accurate information services and decision-making support.

[0003] With the deepening of digital village construction, each region has accumulated a large amount of localized data, and the existing model update method mainly updates the model at fixed time intervals. However, the update method at fixed time intervals may also update when the amount of information is insufficient, resulting in a waste of computing resources; or it may fail to update in time when the amount of information surges, affecting model performance, and unable to dynamically adjust the update strategy according to actual conditions, and unable to adapt to fluctuations in the amount of information.

[0004] Therefore, finding a model updating method that can update a large language model in a timely and efficient manner while saving computing resources to improve the updating efficiency is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention proposes an adaptive incremental update method and system based on the digital village large language model, which can simultaneously consider the three dimensions of the total number of text messages, the total amount of information and the activity index, thereby realizing a comprehensive evaluation of the new data added to the system. By setting multiple conditions and thresholds, computing resources are effectively saved. At the same time, each threshold can be adjusted according to actual conditions, so that the system can adapt to different scenarios and needs.

[0006] The technical solution of the present invention is achieved in this way:

[0007] In a first aspect, the present invention provides an adaptive incremental update method based on a digital village large language model, comprising the following steps:

[0008] S1. Setting counters and variable flags in the digital village system and initializing the counters and variable flags;

[0009] S2. Use the digital village system to collect new data in real time;

[0010] S3, converting the system newly added data into text information, and determining the type of the system newly added data according to the text information;

[0011] S4. Update the counter according to the type of new data added to the system, and calculate the total number of text messages, the total amount of information and the activity index of the new data added to the system according to the text information;

[0012] S5. Update the variable flag according to the total number of text messages, total amount of information and active index of the new data in the system, and judge whether the new data in the system meets the update conditions according to the variable flag. If so, update the digital village language model.

[0013] On the basis of the above technical solutions, preferably, the newly added data of the system includes public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activity data.

[0014] Based on the above technical solution, preferably, the counter includes a total text message counter, a total public security incident counter, a total farming activity counter, a total crop growth and environment counter, and a total farmer leisure and entertainment activity counter, and the variable flag includes a total text message flag, a total amount of information flag, and an activity index flag.

[0015] On the basis of the above technical solution, preferably, the total number of text information, the total amount of information and the activity index of the newly added data according to the text information calculation system specifically include:

[0016] Determine the total number of text messages of the newly added data according to the text messages;

[0017] Construct a comparison matrix according to the newly added data of the system, and normalize the comparison matrix by columns to obtain a normalized matrix;

[0018] Calculate the average value of each row of the normalized matrix to obtain the weights of different types of text information;

[0019] Calculate the activity index of new data added to the system based on the weights of different types of text information;

[0020] The total amount of new data added to the system is determined based on the total number of text messages and the activity index.

[0021] On the basis of the above technical solution, preferably, the calculation formula of the activity index is:

[0022]

[0023] Among them, i represents the number of information, w i Represents the weight coefficient of the i-th information.

[0024] On the basis of the above technical solution, preferably, the calculation formula of the total amount of information is:

[0025] TInfo =Total×H(X)

[0026]

[0027] Among them, Total represents the total number of text messages, H(X) represents information entropy, and P(x i ) represents the probability of the random variable X taking the ith possible value, where X is one of the public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activities.

[0028] More preferably, step S5 specifically includes:

[0029] Determine whether the total number of text messages is greater than the total number threshold, if so, increase the total number of text messages flag by 1;

[0030] If not, determine whether the activity index is greater than the activity index threshold. If so, increase the activity index flag by 1;

[0031] If not, determine whether the total amount of information is greater than the total amount of information threshold. If so, increase the total amount of information flag by 1;

[0032] If not, the sum of the total number of text messages, the total amount of information, and the activity index is compared with the update threshold:

[0033] If it is greater than the update threshold, the digital village language model is updated;

[0034] If it is less than the update threshold, return to step S2.

[0035] In a second aspect, the present invention provides an adaptive incremental update system based on a digital village large language model, using the adaptive incremental update method as described above, including:

[0036] An initialization module configured to set counters and variable flags in the digital village system and initialize the counters and variable flags;

[0037] A data collection module configured to collect new system data in real time using the digital village system;

[0038] A text information classification module, configured to convert the system newly added data into text information, and determine the type of the system newly added data according to the text information;

[0039] A counter update module, configured to update the counter according to the type of new data added to the system, and calculate the total number of text messages, the total amount of information and the activity index of the new data added to the system according to the text information;

[0040] The flag update module is configured to update the variable flag according to the total number of text messages, the total amount of information and the activity index of the new data in the system, and judge whether the new data in the system meets the update conditions according to the variable flag. If so, the digital village language model is updated.

[0041] In a third aspect, the present invention provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein:

[0042] The processor, memory, and communication interface communicate with each other via the bus;

[0043] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the adaptive incremental update method as described above.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the adaptive incremental update method as described above.

[0045] The adaptive incremental update method of the present invention has the following beneficial effects compared with the prior art:

[0046] (1) By considering the total number of text messages, the total amount of information, and the activity index, a comprehensive evaluation of the new data in the system is achieved, avoiding the one-sidedness that may be caused by a single indicator. The update trigger mechanism is more flexible by using the flag accumulation and threshold comparison method. By setting multiple conditions and thresholds, unnecessary frequent updates are avoided, effectively saving computing resources. At the same time, each threshold can be adjusted according to actual conditions, so that the system can adapt to different scenarios and needs.

[0047] (2) The number of new data added to different types of systems is accurately tracked by updating counters, and the weights of different types of text information are calculated using a comparison matrix and normalization processing. The importance of each type of information is dynamically adjusted according to the actual distribution of the data, thus achieving adaptive adjustment of the weights.

[0048] (3) By combining the total number of text messages and the activity index to determine the total amount of information, not only the amount of data but also the quality and importance of the data are taken into account. This can more accurately determine whether the model needs to be updated, avoid unnecessary updates, and optimize the use of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 is a flow chart of the adaptive incremental update method of the present invention;

[0051] Figure 2 It is a flowchart of the adaptive incremental update method of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, the present invention provides an adaptive incremental update method based on a digital village large language model, comprising the following steps:

[0054] S1. Set counters and variable flags in the digital village system and initialize the counters and variable flags; the counters include a total text message counter, a total public security incident counter, a total farming activity counter, a total crop growth and environment counter, and a total farmer leisure and entertainment activity counter; the variable flags include a total text message flag, a total amount of information flag, and an activity index flag.

[0055] S2. Use the digital village system to collect new system data in real time; wherein, the new system data includes public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activity data.

[0056] It is understandable that public security incident data refers to various incidents that affect public safety and social order (such as theft, fighting, illegal occupation of land or damage to public facilities, etc.), which are recorded, monitored and managed through digital means, such as real-time monitoring and recording of abnormal data through cameras, dividing the village into multiple grids and assigning grid members, and reporting problems found by grid members through public service hotlines or reporting platforms.

[0057] Agricultural activity data includes land preparation, sowing, fertilization, weeding, pest control and harvesting, etc., which are detected through regular aerial photography by drones and automatic recording of operation types by smart agricultural machinery.

[0058] Crop growth and environmental data refers to the growth process, growth environment and related influencing factors of crops obtained through modern information technology and digital means, reflecting the growth status of crops and changes in the surrounding environment, such as using Internet of Things sensors (such as soil sensors, meteorological sensors, crop growth sensors, etc.), identifying crop photos, and integrating spatial data such as geographic location, land use and crop distribution.

[0059] The data on farmers’ leisure and entertainment activities reflects the behaviors and movements of farmers during their non-working hours, such as going to the Wenhua Activity Center, participating in sports activities, attending markets or temple fairs, etc., which are identified and recorded through surveillance videos and machine learning technology.

[0060] This application does not impose any specific restrictions on the method of obtaining public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activity data, which can be obtained using specific equipment or devices corresponding to the digital village system.

[0061] S3. Convert the system newly added data into text information, and determine the type of the system newly added data according to the text information.

[0062] It can be understood that the new data added to the system includes structured data and unstructured data. The new data added to the system is first converted into text information, and each text information is converted into a 768-dimensional or 1536-dimensional word vector, and then multiple text information are converted into multiple word vectors, and the type of the new data added to the system is determined based on the word vector.

[0063] Among them, the types of new data added to the system include: public security incidents, agricultural activities, crop growth and environment, and farmers' leisure and entertainment activities.

[0064] In the embodiment of the present application, unstructured data usually comes from surveillance video, and vehicles or people are identified through machine learning technology, and text information is generated. The template of the text information of farmers' leisure and entertainment activities is as follows: {time}, {person or car}, {enter or leave}, {somewhere}. Among them, {time} is the timestamp of the video surveillance identifying the vehicle or person; {person or car} is to identify whether it is a person or a vehicle appearing in the surveillance video. If it is a person, it is further determined whether it is a person who has been entered into the digital village system. If not, it is marked as a stranger; if it is a vehicle, the license plate number of the vehicle is identified. If there is no license plate number, it is marked as a license-free vehicle; {enter or leave} is based on the direction of the surveillance equipment shooting, with the area as the center, the situation of people or vehicles entering or leaving the area; {somewhere} is the location where the surveillance equipment is installed.

[0065] In the embodiment of the present application, structured data usually comes from a database, and text is generated through a template. The template of the text information of agricultural activities generated by agricultural activity data is as follows: {time}, {farmer} in {land number}, using {tools}, consuming {agricultural materials}, and carrying out {duration} of {agricultural activities} to ensure the growth of {crops}. Among them, {time}, {farmer}, {land number}, {tools}, {agricultural materials}, {duration}, {agricultural activities} and {crops} all come from each newly added structured data in the agricultural activity table. For example, the farmer prepared the land on March 1, 0 days after the last activity, the labor per mu was 7 hours / mu, and the consumption of agricultural materials per mu was none; the farmer planted rice seedlings on the land on March 10, 9 days after the last activity, the labor per mu was 2 hours / mu, and the consumption of agricultural materials per mu was 3KG / mu.

[0066] In the embodiment of the present application, the template of the text information of the public security incident is as follows: {time}, an {incident type} occurred at {location}, involving {personnel description}, {tools or methods} were used, the incident lasted {duration}, and caused {impact or loss}. {processing result}. Among them, {time} is the specific time and date of the public security incident, {location} is the specific location where the public security incident occurred, {incident type} is a specific incident, such as theft, fighting, vandalism, etc., {personnel description} is a brief description of the persons involved in the public security incident, such as the number and identity of the persons involved, {tools or methods} are the tools used or methods adopted when the public security incident occurred, {duration} is the duration of the public security incident, {impact or loss} is the impact or economic loss caused by the public security incident, and {processing result} is the processing result of the persons involved, such as being stopped, arrested, or the case is under investigation.

[0067] In the embodiment of the present application, the text information template of crop growth and environment is as follows: {date}, {crop} of {land number} is in {growth stage}, environmental data shows: {temperature}, {humidity}, {pH value}, {light intensity}, growth status: {plant height}. Among them, {date}, {land number}, {crop}, {growth stage}, {temperature}, {humidity}, {pH value}, {light intensity} and {plant height} are all from each new data in the crop growth and environment table. For example, on March 10, the potatoes on land No. 1 were in the seedling stage, the temperature was 18°C, the humidity was 72%, the land pH was 6.2, the light intensity was 32000lux, and the plant height was 12cm.

[0068] S4. Update the counter according to the type of the new data added to the system, and calculate the total number of text information, the total amount of information and the activity index of the new data added to the system according to the text information.

[0069] Specifically, the total number of text messages, the total amount of information, and the activity index of the newly added data of the text information calculation system include:

[0070] Determine the total number of text messages of the newly added data according to the text messages;

[0071] Construct a comparison matrix according to the newly added data of the system, and normalize the comparison matrix by columns to obtain a normalized matrix;

[0072] Calculate the average value of each row of the normalized matrix to obtain the weights of different types of text information;

[0073] Calculate the activity index of new data added to the system based on the weights of different types of text information;

[0074] The total amount of new data added to the system is determined based on the total number of text messages and the activity index.

[0075] By updating the counters, we can accurately track the number of new data added to different types of systems, and use the comparison matrix and normalization processing to calculate the weights of different types of text information. We can dynamically adjust the importance of each type of information according to the actual distribution of the data, and achieve adaptive adjustment of the weights. By combining the total number of text messages and the activity index to determine the total amount of information, we not only consider the amount of data, but also the quality and importance of the data. This can more accurately determine whether the model needs to be updated, avoid unnecessary updates, and optimize the use of computing resources.

[0076] In one embodiment of the present application, public security events, crop growth and environmental information, agricultural activity information, and farmers' leisure and entertainment activities are compared in pairs, and a scale of 1-9 is used to represent their relative importance. The constructed comparison matrix is ​​shown in Table 1:

[0077] Table 1

[0078]

[0079]

[0080] From Table 1, the comparison matrix is ​​as follows:

[0081]

[0082] The weights of different types of text information are:

[0083]

[0084] In the embodiment of the present application, the weights of different types of text information are verified for consistency. Only when the consistency requirement is met, it means that the comparison matrix is ​​reasonable and the calculated weights of different types of text information are appropriate. The maximum eigenvalue λ is obtained by calculation.max =4.1669, the consistency index (CI) is 0.05563, and the consistency ratio (CR) is calculated to be CR = 0.06185. Since CR < 0.1, it means that the comparison matrix has good consistency, and the calculated weights of different types of text information are reasonable. 1 =0.129, farmers' leisure and entertainment activities w 2 =0.087, agricultural activities w 3 =0.334, public security incidents w 4 =0.45.

[0085] Furthermore, the calculation formula of the activity index is:

[0086]

[0087] Among them, i represents the number of information, w i Represents the weight coefficient of the i-th information.

[0088] In one embodiment of the present application, the analytic hierarchy process is used to analyze public security events, agricultural activities, crop growth and environment, and farmers' leisure and entertainment activities to construct a calculation formula for the activity index. Specifically, a comparison matrix is ​​constructed, and the comparison matrix is ​​shown in Table 2:

[0089] Table 2

[0090]

[0091]

[0092] Calculate the sum of each column from Table 1. The sum of the public security incident column is: 1+0.5+0.333+0.25=2.083;

[0093] The total of crop growth and environment is: 2+1+0.2+0.333=3.533; the total of agricultural activities is:

[0094] 3+5+1+0.5=9.5; Total of farmers’ leisure and entertainment activities: 5+2+2+1=10;

[0095] The sum of each column is normalized, and the results after normalization are shown in Table 3:

[0096] Table 3

[0097]

[0098] The weight vector is obtained by summing and averaging the elements in each row of Table 3, where:

[0099] The weight of public security incidents is: w = (0.48 + 0.57 + 0.32 + 0.5) / 4 = 0.4675;

[0100] Agricultural activities are: w = (0.24 + 0.28 + 0.53 + 0.2) / 4 = 0.3125;

[0101] The weights of crop growth and environment are: w = (0.16 + 0.06 + 0.11 + 0.2) / 4 = 0.1325;

[0102] The weight of farmers’ leisure and entertainment activities is: w = (0.12 + 0.09 + 0.05 + 0.1) / 4 = 0.09.

[0103] The weight of public security incidents is 46.75%, the weight of crop growth and environment is 31.25%, the weight of agricultural activities is 13.25%, and the weight of farmers' leisure and entertainment activities is 9%.

[0104] Furthermore, the calculation formula of the total amount of information is:

[0105] T Info =Total×H(X)

[0106]

[0107] Among them, Total represents the total number of text messages, H(X) represents information entropy, and P(x i ) represents the probability of a random variable X taking the ith possible value, where X is one of the public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activities. Information entropy is used to measure the uncertainty or randomness of new data added to the system. The higher the information entropy, the more uncertain the information is and the more information it contains.

[0108] It can be understood that when the entropy of each piece of information is the same over a period of time and all information items are independent of each other, the total amount of information during this period can be calculated.

[0109] In one embodiment of the present application, if there is no public security incident in the new system data of the digital village system in one day, the value range of X is {crop growth and environmental information, farming activities, farmers' leisure and entertainment activities}.

[0110] In an embodiment of the present application, the updating variable flag bit specifically includes:

[0111] When the newly added data in the system belongs to a public security incident, the total number of public security incidents counter will be increased by 1, and the total number of text messages counter will be increased by 1;

[0112] When the newly added data in the system belongs to agricultural activities, the total number of agricultural events counter is increased by 1, and the total number of text information counter is increased by 1;

[0113] When the system adds new data belonging to farmers' leisure and entertainment activities, the total number of farmers' leisure and entertainment activities counter is increased by 1, and the total number of text information counter is increased by 1;

[0114] When the system adds new data that does not belong to public security incidents, agricultural activities, and farmers' leisure and entertainment activities, the total number of crop growth and environment counters will be increased by 1, and the total number of text information counters will be increased by 1.

[0115] By accurately classifying and counting different types of time, the growth of various types of data can be accurately reflected, and unstructured new data can be converted into structured statistical data. Whenever new data is input, the corresponding counters are updated immediately, which enables real-time tracking of data changes and ensures the timeliness of information. By comparing different types of counters, the balance of data can be monitored, and abnormal growth or decrease of a certain type of data can be discovered in a timely manner. This allows for more accurate judgment of when the model needs to be updated, avoiding unnecessary updates and optimizing the use of computing resources.

[0116] S5. Update the variable flag according to the total number of text messages, total amount of information and active index of the new data in the system, and judge whether the new data in the system meets the update conditions according to the variable flag. If so, update the digital village language model.

[0117] Specifically, step S5 specifically includes:

[0118] Determine whether the total number of text messages is greater than the total number threshold, if so, increase the total number of text messages flag by 1;

[0119] If not, determine whether the activity index is greater than the activity index threshold. If so, increase the activity index flag by 1;

[0120] If not, determine whether the total amount of information is greater than the total amount of information threshold. If so, increase the total amount of information flag by 1;

[0121] If not, the sum of the total number of text messages, the total amount of information, and the activity index is compared with the update threshold:

[0122] If it is greater than the update threshold, the digital village language model is updated;

[0123] If it is less than the update threshold, return to step S2.

[0124] By simultaneously considering the total number of text messages, the total amount of information, and the activity index, a comprehensive assessment of new data in the system is achieved, avoiding the one-sidedness that may be caused by a single indicator. The method of flag accumulation and threshold comparison is used to make the update trigger mechanism more flexible. Even if a single indicator does not reach the threshold, the combined effect of multiple indicators may trigger an update. By setting multiple conditions and thresholds, unnecessary frequent updates are avoided, effectively saving computing resources. At the same time, the thresholds can be adjusted according to actual conditions, so that the system can adapt to different scenarios and needs.

[0125] like Figure 2 As shown, step 1, calculate the threshold V Total , V index , V Tinfo : V is determined by counting the number of pieces of information on public security events, agricultural activities, crop growth and environment, and farmers’ leisure and entertainment activities Total , which is the total number of text messages. Then, the activity index V is calculated using the activity index formula and the weights calculated by the hierarchical analysis method. index Next, use the total amount of information calculation formula to calculate the H(X) value first, and then multiply it by the total number of text messages to get V Tinfo These values ​​will be used as thresholds to determine whether the digital rural language model needs to be updated. Once completed, proceed to step 2.

[0126] Step 2, set the Type value of the updated model: According to the three thresholds, consider setting the update Type value from three aspects. If Type = 1, it means that as long as the current information meets any threshold requirement, the model can be updated; if Type = 2, it means that any two threshold requirements must be met before the update; if Type = 3, it means that all three threshold requirements must be met before the digital rural language model can be updated. After completion, proceed to step 3.

[0127] Step 3, set the initial values ​​of N0, N1, N2, N3, N4, T1, T2, T3 to 0: Set the initial values ​​of these variables to 0. Variables N0, N1, N2, N3, N4 are used to record the total number of text messages, the total number of public security events, the total number of agricultural activities, the total number of crop growth and environment, and the total number of farmers' leisure and entertainment activities. If the current total number of text messages exceeds V Total , then the variable T1 is set to 1, otherwise it is 0; if the current activity index exceeds V index , then the variable T2 is set to 1, otherwise it is 0; if the current total amount of information exceeds V Tinfo , then variable T3 is set to 1, otherwise it is set to 0. After completing this step, go to step 4.

[0128] Step 4: Get new data on digital villages: Collect new data on public security incidents, agricultural activities, crop growth and environment, and farmers' leisure and entertainment activities from the digital village system in real time. After collecting the new data from the system, proceed to step 5.

[0129] Step 5: Convert data into text: Convert the newly added data into text information according to the template. After completing this step, proceed to step 6.

[0130] Step 6: Whether the text belongs to a public security incident: Determine whether the text belongs to a public security incident according to the converted template. If yes, proceed to step 7; otherwise, proceed to step 8.

[0131] Step 7, N4=N4+1: Increase the total number of public security incident texts by 1, and proceed to step 13 after completion.

[0132] Step 8: Whether the text belongs to agricultural activities: Determine whether the text belongs to agricultural activities according to the converted template. If yes, proceed to step 9; otherwise, proceed to step 10.

[0133] Step 9, N3=N3+1: Increase the total number of agricultural activity texts by 1, and proceed to step 13 after completion.

[0134] Step 10: Whether the text belongs to farmers' leisure and entertainment activities: Determine whether the text belongs to farmers' leisure and entertainment activities according to the converted template. If yes, proceed to step 11; otherwise, proceed to step 12.

[0135] Step 11, N2=N2+1: Increase the total number of texts on farmers' leisure and entertainment activities by 1, and proceed to step 12 after completion.

[0136] Step 12, N1=N1+1: This indicates that the text belongs to the data of crop growth and environment, and the total number of this type of text is increased by 1. After completion, go to step 13.

[0137] Step 13, N0=N0+1: After the text category determination is completed, the total number of text information is increased by 1. After completing this step, proceed to step 14.

[0138] Step 14: N0>0.8*V Total: In order to improve program efficiency and avoid repeatedly calculating the current activity and the current total amount of information in the early stage of accumulating text information, and comparing them with the corresponding thresholds, this goal can be achieved by judging whether the number of N0 exceeds 80% of the total number of text information. Normally, when the number of N0 is less than 80% of the total number of text information, the current activity and the current total amount of information will be lower than the preset threshold. 80% is just an estimate, which can also be set to 90%, generally ranging from 80% to 90%. If the judgment result shows that the number of N0 exceeds 80%, it means that the currently collected text information is not enough to update the model and data needs to be collected again; otherwise, the current activity and the current total amount of information can be calculated and further analyzed.

[0139] Step 15: Calculate the current activity index C index :Use the current accumulated N1, N2, N3 and N4 data to calculate the current activity index C through the activity index formula index . When finished, proceed to step 16.

[0140] Step 16: Calculate the current amount of information C Tinfo :First, use the currently accumulated N1, N2, N3 and N4 data to calculate the information entropy H(X), and then calculate the current total information C based on the information entropy and the current total text information Tinfo . When finished, proceed to step 17.

[0141] Step 17: N0 ≥ V Total : Compare the total number of current text messages with the text message threshold. If the total number of current text messages is greater than or equal to the text message threshold, proceed to step 18; otherwise, proceed to step 19.

[0142] Step 18, T1=1: Set the variable T1 to 1, indicating that the total number of text messages exceeds the threshold. Then go to step 19.

[0143] Step 19, C index ≥V index : Compare the current activity index with the activity index threshold. If the current activity index is greater than or equal to the activity index threshold, proceed to step 20; otherwise, proceed to step 21.

[0144] Step 20, T2=1: set the variable T2 to 1, indicating that the current activity index exceeds the threshold. Then go to step 21.

[0145] Step 21, C Tinfo ≥V Tinfo : Compare the current total amount of information with the total amount of information threshold. If the current total amount of information is greater than or equal to the total amount of information threshold, proceed to step 22; otherwise, proceed to step 23.

[0146] Step 22, T3=1: set the variable T3 to 1, indicating that the current total amount of information exceeds the threshold. Then go to step 23.

[0147] Step 23, T1+T2+T3≥Type: Determine whether the sum of T1+T2+T3 is greater than or equal to the value of the update mode. If the condition is met, it means that the current text information is sufficient and the large language model can be updated, and then proceed to step 24; otherwise, it means that the information is insufficient to support the update of the large language model and further information needs to be collected, and then proceed to step 4.

[0148] Step 24: Update the large language model with text: Use all the currently collected text to retrain the digital rural large language model. After the training is completed, use the trained model to update the current digital rural large language model. After completion, go to step 3, reset various accumulated parameters, start collecting new text information again, and wait for the next model update.

[0149] This application realizes refined statistics and multi-dimensional analysis of different types of data by collecting and processing new system data in real time and setting multiple counters and variable flags. By setting update conditions and accurately controlling update trigger conditions, unnecessary frequent updates are avoided, accurate classification of data is achieved, and computational complexity is reduced.

[0150] The present invention provides an adaptive incremental update system based on a digital village large language model, which adopts the adaptive incremental update method as described above, and includes:

[0151] An initialization module, which is configured to initialize counters and variable flags;

[0152] A data collection module configured to collect new system data in real time using the digital village system;

[0153] A text information classification module, configured to convert the system newly added data into text information, and determine the type of the system newly added data according to the text information;

[0154] A counter update module, configured to update the counter according to the type of new data added to the system, and calculate the total number of text messages, the total amount of information and the activity index of the new data added to the system according to the text information;

[0155] The flag update module is configured to update the variable flag according to the total number of text messages, the total amount of information and the activity index of the new data in the system, and judge whether the new data in the system meets the update conditions according to the variable flag. If so, the digital village language model is updated.

[0156] The present invention provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the adaptive incremental update method as described above.

[0157] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the adaptive incremental update method as described above.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An adaptive incremental update method based on a digital village large language model, characterized in that: The following steps are involved: S1. Setting counters and variable flags in the digital village system and initializing the counters and variable flags; S2. Use the digital village system to collect new data in real time; S3, converting the system newly added data into text information, and determining the type of the system newly added data according to the text information; S4. Update the counter according to the type of new data added to the system, and calculate the total number of text messages, the total amount of information and the activity index of the new data added to the system according to the text information; S5. Update the variable flag according to the total number of text messages, the total amount of information, and the activity index of the newly added data in the system, and determine whether the newly added data in the system meets the update conditions according to the variable flag. If so, update the digital village language model; The total number of text messages, the total amount of information, and the activity index of the newly added data of the text information calculation system specifically include: Determine the total number of text messages of the newly added data according to the text messages; Construct a comparison matrix according to the newly added data of the system, and normalize the comparison matrix by columns to obtain a normalized matrix; Calculate the average value of each row of the normalized matrix to obtain the weights of different types of text information; Calculate the activity index of new data added to the system based on the weights of different types of text information; Determine the total amount of new data added to the system based on the total number of text messages and the activity index; The calculation formula of the activity index is: Among them, i represents the number of information, w i Represents the weight coefficient of the i-th information; The calculation formula of the total amount of information is: T Info =Total×H(X) Among them, Total represents the total number of text messages, H(X) represents information entropy, and P(x i ) represents the probability of the random variable X taking the ith possible value, where X is one of the public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activities.

2. The adaptive incremental update method based on the digital village large language model according to claim 1, characterized in that: The newly added data of the system include public security incident data, agricultural activity data, crop growth and environmental data, and farmers' leisure and entertainment activity data.

3. The adaptive incremental update method based on the digital village large language model according to claim 2, characterized in that: The counters include a total text message counter, a total public security incident counter, a total farming activity counter, a total crop growth and environment counter, and a total farmer leisure and entertainment activity counter. The variable flags include a total text message flag, a total amount of information flag, and an activity index flag.

4. The adaptive incremental update method based on the digital village large language model as claimed in claim 3, characterized in that: Step S5 specifically includes: Determine whether the total number of text messages is greater than the total number threshold, if so, increase the total number of text messages flag by 1; If not, determine whether the activity index is greater than the activity index threshold. If so, increase the activity index flag by 1; If not, determine whether the total amount of information is greater than the total amount of information threshold. If so, increase the total amount of information flag by 1; If not, the sum of the total number of text messages, the total amount of information, and the activity index is compared with the update threshold: If it is greater than the update threshold, the digital village language model is updated; If it is less than the update threshold, return to step S2.

5. An adaptive incremental update system based on a digital rural large language model, characterized in that: The method for adaptive incremental updating according to any one of claims 1 to 4 is adopted, comprising: An initialization module configured to set counters and variable flags in the digital village system and initialize the counters and variable flags; A data collection module configured to collect new system data in real time using the digital village system; A text information classification module, configured to convert the system newly added data into text information, and determine the type of the system newly added data according to the text information; A counter update module, configured to update the counter according to the type of new data added to the system, and calculate the total number of text messages, the total amount of information and the activity index of the new data added to the system according to the text information; The flag update module is configured to update the variable flag according to the total number of text messages, the total amount of information and the activity index of the new data in the system, and judge whether the new data in the system meets the update conditions according to the variable flag. If so, the digital village language model is updated.

6. An electronic device, characterized in that: include: At least one processor, at least one memory, a communication interface and a bus; wherein, The processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the adaptive incremental update method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement the adaptive incremental update method as described in any one of claims 1-4.

Images