Data management method and system for agricultural planning

Abstract

The invention discloses a data management method and system for agricultural planning, and the method comprises the steps: accessing a webpage comprising first content, the webpage being associated with a first data set related to the first content, the first data set being unable to be accessed through a webpage link or a web crawler, and the first data set comprising agricultural planning data; obtaining a first question input by the user in a dialog box of the webpage; determining a data screening condition; obtaining a second data set from the first data set based on user permission and a data screening condition; questioning the second data set and the first question to a question and answer model according to a first template; and returning the content output by the question and answer model to the dialog box.

Description

Technical Field

[0001] This application relates to the application of artificial intelligence, in particular to a data management method and system for agricultural planning. Background Art

[0002] Traditional resource planning websites usually only set up some web pages with fixed content for the public to view. For relevant professionals or insiders, if they need to query some actual planning data behind the content of a certain web page, they need to use other internal systems. And different users have different access rights to data.

[0003] In the prior art, with the maturity of the large model solution, the large model has demonstrated good material analysis capabilities. There are many systems trying to connect to the large model for applications. However, given that most companies do not have the ability to train large models, or it is difficult to obtain good results under the condition of scarce training resources, resulting in the deployment of the model in the system not being able to have a good coupling with the system. Most deployments are limited to setting up a dialogue entry.

[0004] And if deep access to the model is required, it may be necessary for the model provider to provide a separate solution, which is expensive, and problems such as project collaboration will extend the project cycle.

[0005] At the same time, attempts to use the large model to analyze and apply the data of the system itself have also encountered various problems. On the one hand, excessive data input makes the results output by the model chaotic. On the other hand, it may lead to various data security problems. Especially for some businesses where some data is not suitable for public access by all users, the problem is more prominent. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. For this reason, the present invention provides a data management system for agricultural planning to improve the traditional planning website to achieve rapid intelligence while ensuring data access security.

[0007] On the one hand, an embodiment of the present application provides a data management method for agricultural planning, including:

[0008] Access a web page including first content, the web page is associated with a first data set related to the first content, the first data set cannot be accessed through the web page link or web crawler access, and the first data set includes agricultural planning data;

[0009] Obtain a first question input by the user in the dialogue box of the web page;

[0010] Determine data screening conditions;

[0011] A second data set obtained from the first data set based on user permissions and data filtering conditions;

[0012] Ask the question-and-answer model about the second data set and the first question according to the first template;

[0013] Return the content output by the question-and-answer model to the dialog box.

[0014] In some embodiments, the first template includes content indicating that the question-and-answer model uses the second data set and the first content as the basis for answering, and content for answering the first question.

[0015] In some embodiments, determining the data filtering conditions specifically includes:

[0016] Collect the first filtering condition through the UI components of the user access page, and / or obtain the second filtering condition through the first question, and use the combination of the first filtering condition and / or the second filtering condition as the data filtering condition.

[0017] In some embodiments, based on the first question, ask the question-and-answer model according to the second template, so that the question-and-answer model outputs specific types of information in the manner specified by the second template, and use the information output by the question-and-answer model as the second filtering condition;

[0018] Among them, the specific types of information include time range, geographical range, and / or preset data tags;

[0019] Each data file in the first data set is set with one or more preset data tags.

[0020] In some embodiments, a preset access address is set in the second template, the access address includes all preset data tags, and the second template is set with a statement requiring the question-and-answer model to determine the preset data tags related to the first question and requiring the question-and-answer model to output the related preset data tags in a preset format;

[0021] The preset data tags in the access address are mapped and updated based on the tags of the data files in the first data set.

[0022] In some embodiments, the preset data tags corresponding to each data file are set with the degree of association between the tags and the data files;

[0023] The method further includes the following steps:

[0024] Judge the data volume of the second data set, if the data volume is greater than a preset value;

[0025] Without reducing the number of preset data tags covered, starting from the data file with the smallest sum of relevance of the preset data in the data file, reduce the amount of data in the second data set according to the size of the sum of relevance of the preset data in the data file.

[0026] In some embodiments, the following steps are further included: when the user makes a second question based on the first question, before the third data set selected from the first data set based on the second question is sent to the question and answer model, the data belonging to the second data set in the third data set is removed.

[0027] In some embodiments, when the question and answer model is called for the first time, the question and answer model is required to output a result according to a preset rule through a third template, and the preset rule includes the output length.

[0028] On the other hand, an embodiment of the present application discloses a data management system for agricultural planning, including:

[0029] An access module for accessing a web page including first content, the web page being associated with a first data set related to the first content, the first data set not being accessible through the web page link or web crawler access, and the first data set including agricultural planning data;

[0030] An acquisition module for acquiring a first question input by the user in a dialog box of the web page;

[0031] A determination module for determining data screening conditions;

[0032] A screening module for obtaining a second data set from the first data set based on user permissions and data screening conditions;

[0033] A question module for asking the question and answer model about the second data set and the first question according to a first template;

[0034] A return module for returning the content output by the question and answer model to the dialog box.

[0035] On the other hand, an embodiment of the present application discloses a data management system for agricultural planning, including:

[0036] A memory for storing a program;

[0037] A processor for loading the program to execute the data management method for agricultural planning.

[0038] By establishing the association between the web page and the first data set in the embodiments of the present application, users are allowed to access agricultural planning data that is permitted by their own permissions and of interest through the question-and-answer model, and the question-and-answer model outputs the answers to relevant questions; users can use the window of the web page to ask questions, and the system indirectly asks questions through the user's question content and the data screened based on the user's permissions and filtering conditions, and returns the questions answered by the model to the user; in this way, the deployer does not need to train the model too much, nor does it need to provide the data to the model in advance, realizing the rapid deployment and application of the model, and providing data to the model based on the scope required by the user. On the one hand, the data can be updated in a timely manner, with fewer errors in the output results than letting the model learn these data in advance. On the other hand, by means of permission control and other methods, screening the data can speed up the model response and reduce the risk of data leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments.

[0040] Figure 1 is the flowchart of the method provided by the embodiments of the present application;

[0041] Figure 2 is the schematic diagram of data flow provided by the embodiments of the present application;

[0042] Figure 3 is the system block diagram provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] To make the objectives, technical solutions, and advantages of the present application clearer, the following will, with reference to the drawings in the embodiments of the present application, clearly and completely describe the technical solutions of the present application through the embodiments. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present application fall within the scope of protection of the present application.

[0044] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0046] Question-answering models mainly refer to current mainstream natural language models such as Wenxin Yiyan, GPT, and DeepSeek. These models allow remote calls by purchasing tokens, etc., and some allow users to deploy them locally by themselves. In particular, the local deployment solution makes users' data and privacy more secure.

[0047] Usually, these models are pre-trained and can handle most common-sense questions. At the same time, they can achieve instant learning through web retrieval. However, currently, these models generally need to comply with the web crawler protocol. That is, when the crawler crawls web content, it will ignore the above-mentioned web pages to avoid infringement. Similarly, if the web page is in a non-public state or has permission settings, these models cannot obtain it based on their retrieval function.

[0048] In this case, when it is necessary to quickly deploy the model in an existing website system, it is necessary to use content data or content to train the model, which involves training software and hardware costs and personnel costs. Most companies do not have the above capabilities. And simple deployment usually only leaves an interface for the question-answering model for users to ask questions and answers, and users cannot access some non-public data in the form of documents.

[0049] Refer to Figure 1 , the embodiment of the present application provides a data management method for agricultural planning. In the embodiment, a dialogue window component is added to the existing website as the front-end access, and a third-party question-answering model is called based on the method of this embodiment in the background to answer users' questions. The method includes:

[0050] S1. Access a web page including a first piece of content. The web page is associated with a first data set related to the first piece of content. The first data set cannot be accessed through the web page link or web crawler. The first data set includes agricultural planning data. In this embodiment, the web page is a website related to agricultural planning, and the first piece of content is mainly publicly available information that can be accessed by most users who want to access it. The above content belongs to public content. Of course, it lacks some data or information that is inconvenient or unnecessary to disclose in the text.

[0051] And the background of the web page is associated with a first data set, and the data set includes one or more data tables. For example, the web page content is about introducing the planning situation of a certain place, and the first data set details the monthly yield information of various crops in that place. This information is generally not disclosed to the public, and the website content may only be a general description of the situation. For professional users such as scholars or staff, they may hope to know more detailed information when browsing the web page. The traditional method can only be for users to log in to a professional database to query, and the operation is not convenient. Given that these data cannot be crawled by the crawler and are also difficult to be interpreted by the third-party question-answering model.

[0052] S2. Obtain the first question input by the user in the dialog box of the web page.

[0053] By setting a dialog box component on the web page, the chat window will transmit the content input by the user, that is, the "first question", to the background. The background will call the model for processing based on this information (the first question). It should be noted that the first question will not be directly sent to the third-party Q&A model, but will be first transmitted to the background for processing, that is, embedded in the question template, and then the background will call the Q&A model.

[0054] S3. Determine the data screening conditions.

[0055] In this step, the amount of data associated with some web pages may be from previous years and is relatively large. It may also involve data in various different aspects. Therefore, it is necessary to screen the data to reduce the amount of data. On the one hand, it can make the Q&A model focus on the content of the user's question, and on the other hand, it can also prevent too much data from being input into the model, resulting in the model giving incorrect answers due to limited memory capacity.

[0056] In this step, there are various ways to determine the data screening conditions.

[0057] Among them, some data can be determined through the UI components of the web page itself, such as maps, time selection items, area selection items, etc. At the same time, the ability of the model itself can also be used to determine which information the user hopes to know from the user's first question.

[0058] In some embodiments, determining the data screening conditions specifically includes:

[0059] Collect the first screening condition through the UI component of the page accessed by the user, and / or obtain the second screening condition from the first question, and use the combination of the first screening condition and / or the second screening condition as the data screening condition.

[0060] It can be seen that there are multiple ways to determine the data screening conditions as described above, and they can be combined with each other. Generally, if the UI component collects information and there is an overlap with the information obtained through the first question, the conditions of the first question can be prioritized or combined. For example, when the user selects the map of Area A through the UI component and mentions the information of Area B in the question, the following strategies can be adopted: prioritize the screening condition as Area B; or consider the collected information as a parallel relationship, that is, when screening, select A or B. Among them, time and location are relatively useful screening information for agricultural planning content. Similarly, a similar processing scheme can be adopted for time. For example, the user can select a certain time period on the page and then ask a question. It can be understood that the UI component can be used to select areas in the map. For the UI component, the areas in the map can be pre-set for zoning, and the corresponding place names in the data file can be set for the selected areas. Then, when the user selects an area and then asks a question, it is equivalent to directly obtaining an area that the user may not be able to describe in words.

[0061] Based on the first question, ask the question-and-answer model according to the second template, so that the question-and-answer model outputs specific types of information in the manner specified by the second template, and use the information output by the question-and-answer model as the second screening condition. Specifically, questions can be set in the second template, requiring the question-and-answer model to obtain this information and output it in a specific format. Based on the response of the question-and-answer model in the background, after screening out this formatted data, it is used as the condition for data screening. For example, set a question in the template to let the template output at least one place name mentioned by the user in the manner of XX City or XX District. If not mentioned, do not output. If there are multiple, separate them with commas. Set the "Place Name:" identifier before the output result when outputting. In this way, the model can output the information in the first question obtained through semantic understanding in a specific format.

[0062] Regarding time, a statement can be set to require the question-and-answer model to output the time range that the user wants to know. It can be uniformly required to be converted into a data range. For example, for today, it can be set as January B, 2025 - January B, 2025. If it is 2024, output January 1, 2024 - December 31, 2024. By setting the output requirements in the template, the corresponding data can be formatted.

[0063] Of course, it can be understood that the above multiple screening conditions can be set separately or in combination, and the way to obtain them can be through the UI components of the website itself or through the question-and-answer model. It can be seen that the embodiments include multiple ways, and the ways do not exclude each other.

[0064] Among them, specific types of information include time range, geographical range, and / or preset data tags. It can be understood that the general range of data can be filtered out through the time range and geographical range. What the user is interested in may be certain types of data, which may exist in the data table or may not exist in the data table and need to be calculated through the data table.

[0065] Regarding the filtering of the time range, the following conditions need to be set. When the time period output by the model does not exist in the range of data records, the setting rule provides data of a preset time period close to the relevant time period. For example, if the user's question is to predict the crop output next year, next year obviously does not belong to the time period of the existing data but belongs to the future. And to analyze the data of the future time period, only historical data can be used. Therefore, the data of the previous year can be provided. That is to say, when filtering data using time conditions, rules need to be set. When the time the user is interested in is in the future, filter the data according to the preset time range. For data that did not exist in the past, in the absence of such data, the model will tell the user that there is no such data or that analysis cannot be performed. In the template, it can be told to the model that if the user hopes to analyze future data, specific information is output, so that the background can filter data based on the output result.

[0066] Then, one or more preset data tags can be set for each data file in the first dataset. For example, a data table records the yield information of various crops in a certain area. Then the field information in the table itself belongs to the label of this data table. At the same time, some values that can be indirectly calculated through the data table can be set, such as planting area, planting efficiency, etc. Common data types can be set to be associated with the data tables that can calculate these information.

[0067] To facilitate the management of these tag information and avoid repeatedly modifying the template, a preset access address can be set in the second template. The access address includes all preset data tags. The second template has a statement requiring the question-and-answer model to determine the preset data tags related to the first question and requiring the question-and-answer model to output the relevant preset data tags in a preset format.

[0068] The preset data tags in the access address are mapped and updated based on the tags of the data files in the first dataset. It can be understood that by updating in the above manner, these tags can be updated without frequently rewriting the script. When the data table is deleted, the relevant content can also be quickly deleted. In this embodiment, it can be set that the question instruction requires the question-and-answer model to output the content that the user is interested in and belongs to these preset data tags in a specific format.

[0069] S4. A second data set obtained from the first data set based on user permissions and data filtering conditions.

[0070] Then, the data range can be filtered out by matching these times and locations with tags. Specifically, for the time range, data can be filtered based on the time range. Due to the user's need for data, generally, a certain extension will be made within the data range of interest to the user. Based on the region, the regions involved in data filtering. By matching tags to filter out relevant tags, it is possible to roughly determine which data in which data tables are involved. Thus, a second data set is formed.

[0071] Of course, if the permission requirements of a data table are higher than the user's current permissions, even if this data table is relevant to the content of interest to the user, it will not be filtered out. This also ensures that users cannot bypass permissions to obtain data they should not know through the Q&A model.

[0072] S5. Ask the Q&A model the second data set and the first question according to the first template.

[0073] Among them, the first template includes content indicating that the Q&A model uses the second data set and the first content as the basis for answering, as well as content for answering the first question. It can be understood that this first template sends the filtered data set into the model and instructs the model to answer the question in the first question based on the second data set. For example, "Please answer the average monthly yield of corn in XX region this year in combination with the data table and predict the data for next month". According to the requirements, the model will output "Location: XX District, XX City, XX Province", "Time: January 1, 2025 to December 31, 2025, April 1, 2025 to April 30, 2025". When the time passes through the condition library, a judgment condition is set to judge that there is a time period completely in the future. Therefore, the data for the most recent year will be taken as the filtering condition. If the filtering time is in the past for a period of time, no data will be filtered out. When no data can be filtered out, it is also possible to default to filtering for a period of time, such as the data for the most recent year, or filtering the data that exists and is close to the target time period within one year.

[0074] S6. Return the content output by the Q&A model to the dialog box.

[0075] It can be understood that the data returned based on the Q&A model is returned to the user's chat box component. Of course, it is also possible to perform certain processing on the data before returning it based on the answer mode of the Q&A model.

[0076] In some embodiments, the preset data tags corresponding to each data file are set with the degree of association between the tags and the data files.

[0077] The method further includes the following steps:

[0078] Judge the data volume of the second data set. If the data volume is greater than a preset value;

[0079] Without reducing the number of covered preset data tags, reduce the data volume of the second data set starting from the data file with the smallest sum of relevance degrees of the preset data in the data file, according to the size of the sum of relevance degrees.

[0080] Relevance degrees can be set for tags. For example, some tags have a relatively high degree of association with data files, and a larger relevance degree can be set. For example, the association degree between the fields directly in the data file and the device is relatively high, while as the result of indirect calculation, the association degree can be adjusted lower. At this time, when the matched data volume is large, data files with lower sums of relevance degrees can be preferentially excluded. A lower sum of relevance degrees indicates that the associated tags are few or the associated tags are all indirectly calculated, which will reduce the response speed of the AI. Through this simple rule, although the optimal optimization cannot be obtained, the response speed of the system can be improved, and at the same time, the tonken cost of the model can be reduced by reducing the number of input data. For example, file A contains the monthly output, and file B records the daily output. When the user hopes to understand the monthly output, file A is directly associated, and file B needs to be calculated. Assume that the relevance degree of directly associated tags is set to 1, and the indirect association is 0.5. In this example, when the data volume is relatively large, the data in file B can be excluded first. On the basis of this example, assume that the relevance degree of another tag in file B is 1 when it is also hit. At this time, the sum of the relevance degrees of the tags in file B is greater than that in file A, and at this time, A can be excluded.

[0081] Regarding the issue of tag coverage, assume that the screening conditions include four preset tags T1, T2, T3, and T4. Among them, T1 and T2 are matched in file A, T3 and T4 are matched in file B, and file C matches file T1, T2, and T3. It can be understood that in this example, assume that the data volume has exceeded the threshold. At this time, in order to ensure that the number of covered hit tags does not decrease. File A or file C can be removed, depending on the size of their relevance degrees. Assume that the sum of the relevance degrees of file A is greater than that of file C, then the data in file C is discarded. When the relevance degrees of the two are the same, the actual data volumes of the two can be considered, and the data file with a larger data volume is preferentially discarded. Therefore, the number of covered hit tags not decreasing means that after removing some data files, all the tags that could be hit before removing the data can still be hit.

[0082] In some embodiments, the following steps are further included: when the user makes a second question based on the first question, before the third data set selected from the first data set based on the second question is sent to the Q&A model, the data belonging to the second data set in the third data set is removed. In this solution, in multiple Q&A sessions, data may be sent to the model multiple times. To reduce the consumption of tokens, the data that has been sent to the Q&A model can be removed.

[0083] To further reduce the number of tokens, the system will use a third template to set the answer mode of the Q&A model to reduce the number of output tokens. When the Q&A model is called for the first time, the Q&A model is required to output results according to preset rules through the third template, and the preset rules include the output length. Of course, statement requirements can also be set to require the model to output concisely.

[0084] Refer to Figure 2 , it can be learned about the implementation method of this solution. In this solution, the user can ask questions about the web page content through the chat window in the client (such as a browser). After the user asks a question, the background will, based on the user's question, use the Q&A model to extract specific information, and then continue to use this information as a screening condition to obtain data related to the web page from the database, and combine this data and the user's question to call a third-party model again to complete the answer to the user's question and reply to the user's chat window.

[0085] Refer to FIG. 3. An embodiment of the present application discloses a data management system for agricultural planning, including:

[0086] An access module for accessing a web page including first content, the web page being associated with a first data set related to the first content, the first data set cannot be accessed through the web page link or web crawler, and the first data set includes agricultural planning data;

[0087] An acquisition module for acquiring a first question input by the user in the dialog box of the web page;

[0088] A determination module for determining data screening conditions;

[0089] A screening module for obtaining a second data set from the first data set based on user permissions and data screening conditions;

[0090] A question module for asking the Q&A model about the second data set and the first question according to the first template;

[0091] A return module for returning the content output by the Q&A model to the dialog box.

[0092] An embodiment of the present application discloses a data management system for agricultural planning, including:

[0093] A memory for storing programs;

[0094] A processor for loading the program to execute the data management method for agricultural planning.

[0095] Note that the above is only a preferred embodiment of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments only. Without departing from the concept of the present application, more other equivalent embodiments can be included, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A data management method for agricultural planning, characterized in that: include: Accessing a webpage including first content, the webpage being associated with a first data set related to the first content, the first data set being inaccessible through a link on the webpage or a web crawler, the first data set including agricultural planning data; Obtaining a first question input by a user in a dialog box of a web page; Determine the data screening criteria; A second data set is obtained from the first data set based on user permissions and data screening conditions; Ask the question-answering model questions using the second data set and the first question according to the first template; Return the content output by the question-answering model to the dialog box.

2. The data management method for agricultural planning according to claim 1, characterized in that: The first template includes content that instructs the question-answering model to use the second data set and the first content as a basis for answering, as well as content that answers the first question.

3. The data management method for agricultural planning according to claim 2, characterized in that: Determine the data screening criteria, including: The first filtering condition is collected through the UI component of the user access page, and / or the second filtering condition is obtained through the first question, and the combination of the first filtering condition and / or the second filtering condition is used as the data filtering condition.

4. The data management method for agricultural planning according to claim 3, characterized in that: Based on the first question, ask the question-answering model a question according to the second template, so that the question-answering model outputs a specific type of information in a manner specified by the second template, and uses the information output by the question-answering model as a second screening condition; The specific type of information includes a time range, a geographical range, and / or a preset data tag; Each data file in the first data set is provided with one or more preset data labels.

5. The data management method for agricultural planning according to claim 4, characterized in that: A preset access address is set in the second template, and the access address includes all preset data tags. The second template is provided with a statement requiring the question-answering model to determine the preset data tags related to the first question, and requiring the question-answering model to output the relevant preset data tags in a preset format; The preset data tag in the access address is mapped and updated based on the tag of the data file in the first data set.

6. The data management method for agricultural planning according to claim 2, characterized in that: The preset data tag corresponding to each data file is provided with a correlation degree between the tag and the data file; The method further comprises the following steps: Determine the data volume of the second data set, if the data volume is greater than a preset value; Without reducing the number of preset data label coverages, the data volume of the second data set is reduced starting from the data file with the smallest sum of association degrees according to the sum of association degrees of the preset data of the data files.

7. The data management method for agricultural planning according to claim 1, characterized in that: The following steps are also included: When the user asks a second question based on the first question, the third data set filtered from the first data set based on the second question excludes data belonging to the second data set before being sent to the question-answering model.

8. The data management method for agricultural planning according to claim 3, characterized in that: When the question-answering model is called for the first time, the third template is used to require the question-answering model to output results according to preset rules, where the preset rules include output length.

9. A data management system for agricultural planning, characterized in that: include: An access module, used to access a webpage including a first content, the webpage is associated with a first data set related to the first content, the first data set cannot be accessed through the webpage link or a web crawler, and the first data set includes agricultural planning data; An acquisition module, used for acquiring a first question input by a user in a dialog box of a web page; A determination module, used to determine data screening conditions; A screening module, configured to obtain a second data set from the first data set based on user permissions and data screening conditions; A questioning module, used for asking the question-answering model questions based on the first template using the second data set and the first question; The return module is used to return the content output by the question-answering model to the dialog box.

10. A data management system for agricultural planning, characterized in that: include: Memory, used to store programs; A processor, used for loading the program to execute the data management method for agricultural planning as described in any one of claims 1-9.

Images