A planting regulation method based on multi-source data coupling and related equipment

Abstract

The application discloses a planting regulation method based on multi-source data coupling and related equipment, and the method comprises the following steps: inputting acquired multi-modal input data into a trained multi-modal large model and an identification model for identification analysis to obtain fusion feature information; performing vectorization on the fusion feature information to obtain query vector data; performing retrieval matching on a dynamic knowledge base according to the query vector data to obtain target agricultural technology information; performing reasoning on the target agricultural technology information and a trained large language model to generate structured scheme information; performing format conversion on the structured scheme information according to a preset protocol to obtain target control instructions, and controlling planting equipment according to the target control instructions. According to the embodiment of the application, multi-modal large models and identification models are used for collaborative identification analysis, the retraining and parameter adjustment of the models are reduced, the identification analysis of different crops is supported, and the model implementation cost can be reduced. The application can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] This application relates to the field of smart agriculture technology, and in particular to a planting control method and related equipment based on multi-source data coupling. Background Technology

[0002] With the continuous iterative development of large model technology, existing technologies identify agricultural crops by constructing large models and output planting plans through the inference capabilities of the models, providing guidance for the production and planting of agricultural crops; however, existing large models have low adaptability to different crops, requiring the model parameters to be readjusted and retrained, resulting in high model implementation costs. Summary of the Invention

[0003] The main objective of this application is to propose a planting regulation method and related equipment based on multi-source data coupling, which can reduce the model implementation cost.

[0004] To achieve the above objectives, one aspect of this application proposes a planting regulation method based on multi-source data coupling, the method comprising: Acquire multimodal input data, and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information; The fused feature information is vectorized to obtain query vector data, and the query vector data and dynamic knowledge base are used for retrieval and matching to obtain target agricultural technology knowledge information; Based on the target agricultural technology knowledge information, preset structured prompt words, and a trained large language model, reasoning is performed to generate structured solution information; The structured scheme information is converted into a new format according to a preset protocol to obtain target control instructions, and the planting equipment is controlled according to the target control instructions.

[0005] In some embodiments, the step of performing recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information specifically includes: The multimodal input data is analyzed to determine video data, soil sensor data, and meteorological data; The video data is detected and image features are determined based on the trained multimodal large model and the trained recognition model. Feature extraction is performed on the soil sensor data to obtain soil features, and feature extraction is performed on the meteorological data to obtain meteorological features; Feature fusion is performed based on the image features, soil features, and meteorological features to obtain fused feature information.

[0006] In some embodiments, the step of detecting the video data and determining image features based on the trained multimodal large model and the trained recognition model specifically includes: The video data is sliced ​​to obtain image data, and inference is performed based on the trained multimodal large model and the image data to determine crop status information; wherein, the crop status information includes crop variety and current growth cycle; The trained recognition model and the crop status information are used to filter and determine the target detection model. The target detection model is then used to perform disease detection on the image data to obtain the crop detection result, which is then used as the image feature.

[0007] In some embodiments, the step of retrieving and matching the target agricultural technology knowledge information based on the query vector data and the dynamic knowledge base specifically includes: The dynamic knowledge base is queried based on the query vector data to obtain several knowledge vector data. Based on the query vector data, cosine similarity is calculated between each of the knowledge vector data and the query vector data to determine several cosine similarity values. The knowledge vector data are sorted according to several cosine similarity values ​​to determine the sorting result, and the sorting result is filtered according to a preset threshold to determine the target agricultural technology knowledge information.

[0008] In some embodiments, the dynamic knowledge base is determined by the following method: Collect currently available information according to a preset cycle, and organize the currently available information and preset custom planting information to obtain the original knowledge set; The original knowledge set is sliced ​​to obtain a sliced ​​knowledge set, and the sliced ​​knowledge set is vectorized to obtain the dynamic knowledge base.

[0009] In some embodiments, the method further includes: Data extraction is performed on the multimodal input data to determine meteorological data and soil sensor data. Prediction is made based on the meteorological data to determine the probability of precipitation, and data extraction is performed on the soil sensor data to determine the soil moisture. The candidate irrigation amount is determined by calculating based on the preset coupling equation, the precipitation probability, and the soil moisture; and then compared with the precipitation probability and the preset probability threshold. If the precipitation probability is less than or equal to the preset probability threshold, the candidate irrigation amount is taken as the target irrigation amount, and water and fertilizer are regulated according to the target irrigation amount. If the probability of precipitation is greater than the preset probability threshold, the candidate irrigation amount is adjusted according to the preset irrigation adjustment amount to obtain the target irrigation amount, and water and fertilizer regulation is performed according to the preset time period and the target irrigation amount.

[0010] To achieve the above objectives, another aspect of this application proposes a planting control system based on multi-source data coupling, the system comprising: The feature fusion module is used to acquire multimodal input data and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information. The knowledge retrieval module is used to vectorize the fused feature information to obtain query vector data, and to perform retrieval matching based on the query vector data and the dynamic knowledge base to obtain target agricultural technology knowledge information; The solution generation module is used to generate structured solution information by reasoning based on the target agricultural technology knowledge information, preset structured prompt words and trained large language model; The device driver module is used to convert the structured scheme information according to a preset protocol to obtain target control instructions, and to control the planting equipment according to the target control instructions.

[0011] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0012] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0013] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described above. The embodiments of this application include at least the following beneficial effects: This application provides a planting regulation method, system, electronic device, storage medium, and program product based on multi-source data coupling. This solution acquires multimodal input data and inputs the multimodal input data into a trained multimodal large model and recognition model for recognition and analysis to obtain fused feature information; by vectorizing the fused feature information, query vector data is obtained; the query vector data is used to retrieve and match the dynamic knowledge base to obtain target agricultural technology information; reasoning is performed based on the target agricultural technology information and the trained large language model to generate structured scheme information; the structured scheme information is formatted according to a preset protocol to obtain target control instructions, and the planting equipment is controlled according to the target control instructions; through the collaborative work of the trained multimodal large model and recognition model, multimodal input data is identified and analyzed to obtain fused features and further obtain the target scheme, reducing the need for model retraining and parameter adjustment, supporting the identification and analysis of different crops, and reducing the model implementation cost. Attached Figure Description

[0014] Figure 1 This is a flowchart of a planting regulation method based on multi-source data coupling provided in an embodiment of this application; Figure 2 yes Figure 1 The flowchart of step S101 in the text; Figure 3 yes Figure 2 The flowchart of step S202 in the document; Figure 4 yes Figure 1 The flowchart of step S102 in the document; Figure 5 This is a flowchart illustrating the determination of a dynamic knowledge base in a planting regulation method based on multi-source data coupling, as provided in an embodiment of this application. Figure 6 This is a flowchart illustrating water and fertilizer regulation in a planting regulation method based on multi-source data coupling, as provided in an embodiment of this application. Figure 7 This is a flowchart of a specific embodiment provided in this application; Figure 8 This is a schematic diagram of the structure of a planting control system based on multi-source data coupling provided in an embodiment of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0016] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0017] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0019] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0020] Multimodal large models: Large-scale artificial intelligence models that can process multiple input data such as images and text simultaneously, such as GPT-4V and ERNIE-ViL, and have cross-modal information understanding and generalization capabilities.

[0021] CV small models: Lightweight deep learning models specifically designed for computer vision tasks, such as YOLO and ResNet, optimized for specific agricultural scenarios, and used for the accurate identification of detailed features such as crop diseases, pests, and growth.

[0022] RAG (Retrieval-Augmented Generation): This retrieval-enhanced generation technology enhances the accuracy and professionalism of the content generated by large models by retrieving the latest agricultural technology knowledge from authoritative external knowledge bases (such as the China Agricultural Technology Extension Network).

[0023] MCP (Model Control Protocol): A standardized equipment control interface protocol defined in this invention, supporting the automatic conversion and transmission of AI decision results into agricultural equipment execution commands.

[0024] Multi-source data coupling: refers to the process of spatiotemporal registration and feature fusion of IoT sensor data (soil EC value, pH value, etc.), UAV image data, and meteorological forecast data.

[0025] This application provides a planting regulation method based on multi-source data coupling, relating to the field of information technology. This planting regulation method based on multi-source data coupling can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited thereto; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing a planting regulation method based on multi-source data coupling, but is not limited to the above forms.

[0026] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0027] Figure 1 This is an optional flowchart of a planting regulation method based on multi-source data coupling provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0028] Step S101: Obtain multimodal input data, and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information; Step S102: The fused feature information is vectorized to obtain query vector data, and the query vector data and dynamic knowledge base are used for retrieval and matching to obtain target agricultural technology knowledge information; Step S103: Based on the target agricultural technology knowledge information, preset structured prompt words and trained large language model, reasoning is performed to generate structured solution information; Step S104: Convert the structured scheme information according to the preset protocol to obtain the target control command, and control the planting equipment according to the target control command.

[0029] Steps S101 to S104, as shown in this embodiment, involve collecting multi-source data related to crop planting using a multi-source data acquisition device. This includes acquiring video stream data by accessing multiple monitoring channels, collecting soil sensor data from a constructed IoT sensor array, and accessing meteorological station data corresponding to the crop planting area. A pre-trained multimodal large model and several pre-trained dedicated lightweight small models are used to collaboratively analyze the collected multimodal data, extracting image features from the multimodal data, and extracting corresponding features from other modal data, including soil features and meteorological features. Feature fusion is performed on the extracted multimodal feature data to achieve a multidimensional representation of the crop planting environment. The fused feature data is then processed. Vectorization is performed, and the vectors are used to query and match knowledge in the constructed dynamic knowledge base to obtain agricultural technology knowledge corresponding to the current crop planting environment, providing support for subsequent crop planting regulation. The obtained agricultural technology knowledge is then reasoned using a large language model and preset structured prompts to output structured solution information, making it easy for users to obtain the generated solution information. At the same time, the regulation device or system uses a custom control protocol to convert the data format of the structured solution information output by the large language model, generate corresponding control commands, and send the generated control commands to the IoT devices or planting equipment in crop planting for control, regulating the current crop planting environment, such as disease control, water and fertilizer adjustment, and irrigation cycle adjustment.

[0030] Please see Figure 2In some embodiments, step S101 may include, but is not limited to, steps S201 to S204: Step S201: Analyze the multimodal input data to determine the video data, soil sensor data, and meteorological data; Step S202: Detect video data and determine image features based on the trained multimodal large model and the trained recognition model; Step S203: Extract features from soil sensor data to obtain soil features, and extract features from meteorological data to obtain meteorological features; Step S204: Perform feature fusion based on image features, soil features, and meteorological features to obtain fused feature information.

[0031] In step S201 of some embodiments, the system parses and organizes the collected multimodal data, extracting data of different modes, including multi-channel monitoring video data, soil sensor data collected by IoT sensor array, and meteorological data collected by accessing weather stations; the system performs corresponding data processing on the data of different modes respectively, and extracts feature data corresponding to the data of different modes; for example, image recognition, target detection and other processing are performed on the multi-channel monitoring video data to extract the image features therein.

[0032] In step S202 of some embodiments, the system inputs the extracted multi-channel monitoring video data into a trained multimodal large model and several trained lightweight recognition models for processing, extracting image data, including feature information such as the crop type planted in the video data, the current growth cycle of the crop, and the current growth status of the crop. In this embodiment, a cascaded inference architecture is constructed using the multimodal large model and several lightweight recognition models to perform collaborative analysis on the input modal data, automatically identify the crop type planted, and automatically call the corresponding lightweight recognition model for feature extraction, adapting to multiple crops, reducing the cost of model parameter adjustment and retraining, thereby reducing the model implementation cost.

[0033] In step S203 of some embodiments, the system also performs corresponding feature extraction processing on other extracted modal data, including preprocessing the soil sensor data and performing time-frequency domain analysis to obtain feature data such as soil electrical conductivity characteristics and pH characteristics; and preprocessing the acquired meteorological data such as alignment processing, and extracting features from the processed meteorological data through a feature extraction model, such as a deep learning model, to obtain corresponding meteorological feature data; in this embodiment, the meteorological data can also be used for predictive reasoning to obtain the precipitation probability of the current crop planting area within a certain period of time, as a kind of meteorological feature data.

[0034] In step S204 of some embodiments, the system performs feature fusion on the previously obtained image features, soil features, and meteorological features. For example, feature data of different modalities are spliced ​​together to obtain fused feature data, so that the system can subsequently determine the corresponding planting control strategy based on the fused feature data.

[0035] Please see Figure 3 In some embodiments, step S202 may include, but is not limited to, steps S301 to S302: Step S301: Slice the video data to obtain image data, and perform inference based on the trained multimodal large model and the image data to determine crop status information; wherein, crop status information includes crop variety and current growth cycle; Step S302: Based on the trained recognition model and crop status information, a target detection model is selected, and disease detection is performed on the image data according to the target detection model to obtain the crop detection result, and the crop detection result is used as the image feature.

[0036] In step S301 of some embodiments, the system slices multiple monitoring video data according to a preset duration to obtain several image data. The obtained image data is input into a trained multimodal large model for inference to obtain information on the crop variety planted in the image data and the current growth cycle of the crop. The system uses the crop variety information and the corresponding growth cycle obtained by the multimodal large model as crop status information for further identification and detection of the image data to generate the final image features. In this embodiment, the image data is input into the multimodal large model for inference to determine the crop type in the image data. The system calls the corresponding crop library according to the determined crop type and combines the crop library to infer the image data to determine the current growth cycle of the crop. For example, the multimodal large model infers the input image data and determines that the current crop is winter wheat. The system calls the corresponding winter wheat crop library according to the crop type and combines the winter wheat crop library to further infer the image data, outputting that the growth cycle of winter wheat in the image data is the heading stage.

[0037] In step S302 of some embodiments, the system filters several lightweight recognition models based on the crop status information output by the multimodal large model, matches and calls the corresponding lightweight recognition model to perform further feature extraction on the input image data, including calling the disease monitoring model to extract features of crop diseases in the image data and determine the corresponding feature information, such as disease type, disease development status, etc.; the system combines the output results of the lightweight recognition model and the output results of the multimodal large model to generate image features corresponding to multiple monitoring videos.

[0038] Please see Figure 4In some embodiments, step S101 may include, but is not limited to, steps S401 to S403: Step S401: Query the dynamic knowledge base based on the query vector data to obtain several knowledge vector data. Step S402: Calculate the cosine similarity between the query vector data and several knowledge vector data respectively, and determine several cosine similarity values. Step S403: Sort several knowledge vector data according to several cosine similarity values, determine the sorting results, and filter the sorting results according to a preset threshold to determine the target agricultural technology knowledge information.

[0039] In step S401 of some embodiments, the system performs vectorization processing on the obtained fused feature data and uses the obtained fused feature vector as query vector data. The system performs query matching on the constructed dynamic knowledge base according to the generated query vector, and filters out agricultural technology knowledge related to the query vector. Keyword recognition algorithms can be used to retrieve knowledge vectors stored in the dynamic knowledge base and extract knowledge vectors with keywords, etc. In this embodiment, the dynamic knowledge base periodically accesses authoritative channels according to the set update cycle to obtain the latest agricultural technology knowledge, and updates the dynamic knowledge base according to the obtained agricultural technology knowledge to ensure the timeliness of agricultural technology knowledge and reduce the reliance on the professional knowledge of individual experts.

[0040] In step S402 of some embodiments, the system calculates the cosine similarity between the generated query vector and the knowledge vector extracted from the dynamic knowledge base, determines the degree of correlation between the current knowledge vector and the query vector based on the cosine similarity value calculated for each knowledge vector, and selects the optimal agricultural technology knowledge to support crop planting regulation.

[0041] In step S403 of some embodiments, the system sorts the extracted knowledge vectors according to the previously calculated cosine similarity values ​​and selects several knowledge vectors with the highest cosine similarity values ​​as target agricultural technology knowledge information. In this embodiment, after calculating the cosine similarity values ​​of each knowledge vector, the system selects the five knowledge vectors with the highest cosine similarity values ​​as target agricultural technology knowledge information. For example, the query vector generated based on the fused feature data is "winter wheat - heading stage - rust disease - moderate rain warning". The system performs query matching on the dynamic knowledge base based on this query vector to obtain several knowledge vectors. The system calculates the cosine similarity values ​​between each knowledge vector obtained from the query matching and the query vector, and selects the five knowledge vectors with the highest cosine similarity values ​​as target agricultural technology knowledge information, including agricultural technology knowledge such as pesticide type, pesticide concentration, and safety interval.

[0042] Please see Figure 5In some embodiments, step S101 may include, but is not limited to, steps S501 to S502: Step S501: Collect currently available information according to a preset cycle, and organize the currently available information and preset custom planting information to obtain the original knowledge set; Step S502: The original knowledge set is sliced ​​to obtain a sliced ​​knowledge set, and the sliced ​​knowledge set is vectorized to obtain a dynamic knowledge base.

[0043] In step S501 of some embodiments, the system stores agricultural technology knowledge by constructing a dynamic knowledge base to provide professional knowledge support for crop planting regulation; the system collects the latest agricultural technology knowledge according to the set update cycle and pre-set access path, such as regularly accessing nationally published professional knowledge websites every day; the system integrates the regularly collected latest agricultural technology knowledge with user-defined planting procedures to obtain the original knowledge set; in this embodiment, the system accesses the latest agricultural technology knowledge of the authoritative knowledge base through RAG technology, and constructs and / or updates the dynamic knowledge base based on the latest agricultural technology knowledge.

[0044] In step S502 of some embodiments, the system slices the obtained original knowledge set or the latest agricultural technology knowledge collected in real time to optimize the information granularity of the knowledge and improve the query efficiency of the dynamic knowledge base; and vectorizes the sliced ​​agricultural technology knowledge to obtain knowledge vectors, which facilitates subsequent query matching based on query vectors and screening by calculating cosine similarity; and stores the vectorized knowledge vectors to obtain the dynamic knowledge base.

[0045] Please see Figure 6 In some embodiments, the planting regulation method based on multi-source data coupling provided in this application may also include, but is not limited to, steps S601 to S604: Step S601: Extract data from the multimodal input data to determine meteorological data and soil sensor data; make predictions based on the meteorological data to determine the probability of precipitation; and extract data from the soil sensor data to determine soil moisture. Step S602: Calculate and determine candidate irrigation amounts based on the preset coupling equation, precipitation probability, and soil moisture; and compare the precipitation probability with the preset probability threshold. Step S603: If the probability of precipitation is less than or equal to the preset probability threshold, the candidate irrigation amount is taken as the target irrigation amount, and water and fertilizer are regulated according to the target irrigation amount. Step S604: If the probability of precipitation is greater than the preset probability threshold, adjust the candidate irrigation amount according to the preset irrigation adjustment amount to obtain the target irrigation amount, and regulate water and fertilizer according to the preset time period and the target irrigation amount.

[0046] In step S601 of some embodiments, the system can extract features based on the collected multimodal data by constructing and training a large multimodal model and a lightweight recognition model, and construct a dynamic knowledge base for knowledge query and matching to generate corresponding planting control schemes. It can also analyze and calculate based on the collected multimodal data to generate corresponding water and fertilizer control strategies to regulate crop planting. The system has pre-set corresponding coupling equations, which are constructed and adjusted by the user according to the actual application scenario. The system extracts and parses the collected multimodal data to obtain meteorological data and soil sensor data. The system performs prediction and inference based on the meteorological data and outputs the precipitation probability of the current planting area within a certain period of time, such as the precipitation probability within 48 hours. The system parses the soil sensor data, including time and frequency domain analysis, to obtain soil moisture data.

[0047] In step S602 of some embodiments, the system calculates the corresponding irrigation amount for crop planting based on the multimodal data obtained from the previous analysis and extraction and the pre-set coupling equation. The coupling equation is based on "precipitation probability - soil moisture - irrigation amount" and can be adjusted or modified according to actual application. In this embodiment, the coupling equation is set as "irrigation amount = basic water requirement × (1 - precipitation probability × w) × soil moisture coefficient", where w is the weight of precipitation probability. The system determines whether the calculated irrigation amount needs to be adjusted based on the obtained precipitation probability and the preset probability threshold.

[0048] In step S603 of some embodiments, if the predicted precipitation probability is less than or equal to a preset probability threshold, such as 60%, it means that there will be no additional precipitation to supplement the current planting area within a certain period of time. The system will use the calculated irrigation amount as the target irrigation amount and control the equipment in the planting area to irrigate and fertilize the crops.

[0049] In step S604 of some embodiments, if the predicted probability of precipitation is greater than a preset probability threshold, such as 60%, it indicates that there will be precipitation in the current planting area for a period of time to irrigate the crops. Therefore, the calculated irrigation amount needs to be adjusted to reduce the irrigation amount to obtain the final irrigation amount. Since there is precipitation to irrigate the crops, the utilization rate of fertilizer is low at this time, and some fertilizer will be lost due to precipitation. The system postpones the fertilization time according to a preset time, for example, postpones it to 24 hours after the rain. The system drives the equipment in the current planting area to regulate water and fertilizer according to the adjusted irrigation amount and fertilization time settings.

[0050] The following is a detailed description and explanation of the solutions in the embodiments of the present invention, using specific application examples: Please see Figure 7 , Figure 7This is a flowchart illustrating a planting regulation method based on multi-source data coupling provided in this application embodiment, applied in a specific embodiment. In this embodiment, an intelligent planting system is set up in a smart agricultural irrigation area. This system includes a crop condition analysis module, an agricultural technology knowledge module, and a large-scale model analysis and control module. The system analyzes and processes the collected multimodal data, outputs intelligent planting decisions, and performs end-to-end collaborative control of terminals such as integrated water and fertilizer equipment and drones. Specifically, the crop condition analysis module acquires meteorological data, IoT soil sensor data, and image data through different interfaces. It processes the meteorological data and IoT soil sensor data to obtain corresponding meteorological characteristic data and soil characteristic data, which are then output to the data analysis submodule. The large-scale multimodal model identifies crops and cycles in the collected image data and outputs crop status information. Based on the output crop status information, the small model is matched to determine the current crop growth status as winter wheat - heading stage - rust disease. The agricultural technology knowledge module slices and vectorizes user-defined planting knowledge and the latest authoritative planting knowledge acquired periodically to construct an agricultural technology knowledge base. The knowledge query module within the agricultural technology knowledge module queries the agricultural technology knowledge base based on crop growth data output by the crop condition analysis module, obtaining the optimal knowledge vector as triadimefon wettable powder - 200g / mu - safety interval (15 days). The crop condition analysis module and the agricultural technology knowledge module output feature data and the optimal knowledge vector to the large model analysis and control module. The large model analysis and control module calls a general large model to generate a structured scheme from the feature data and the optimal knowledge vector, resulting in a scheme of "triadimefon wettable powder 40g / mu, drone flight altitude 3m, speed 4m / s, fertilization scheme NPK=15-8-20 (kg / mu)". The generated scheme is then converted into control commands for devices or IoT terminals through the MCP protocol, and the corresponding operation module controls the devices or IoT terminals to execute the generated scheme.

[0051] The embodiments of this application include at least the following beneficial effects: This application provides a planting regulation method, system, electronic device, storage medium, and program product based on multi-source data coupling. This solution acquires multimodal input data and inputs the multimodal input data into a trained multimodal large model and a recognition model for recognition and analysis to obtain fused feature information; by vectorizing the fused feature information, query vector data is obtained; the query vector data is used to retrieve and match the dynamic knowledge base to obtain target agricultural technology information; reasoning is performed based on the target agricultural technology information and the trained large language model to generate structured scheme information; the structured scheme information is formatted according to a preset protocol to obtain target control instructions, and the planting equipment is controlled according to the target control instructions; through the collaborative work of the trained multimodal large model and the recognition model, the multimodal input data is recognized and analyzed to obtain fused features and further obtain the target scheme, reducing the need for model retraining and parameter adjustment, supporting the recognition and analysis of different crops, and reducing the model implementation cost.

[0052] Please see Figure 8 This application also provides a planting control system based on multi-source data coupling, which can implement the above method. The system includes: The feature fusion module is used to acquire multimodal input data and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information. The knowledge retrieval module is used to vectorize the fused feature information to obtain query vector data, and to perform retrieval matching based on the query vector data and the dynamic knowledge base to obtain target agricultural technology knowledge information; The solution generation module is used to generate structured solution information by reasoning based on the target agricultural technology knowledge information, preset structured prompt words and trained large language model; The device driver module is used to convert the structured scheme information according to a preset protocol to obtain target control instructions, and to control the planting equipment according to the target control instructions.

[0053] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0054] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0055] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0056] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0057] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0058] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0059] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0060] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0061] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0062] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0063] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0066] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0069] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0070] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A planting regulation method based on multi-source data coupling, characterized in that, The method includes: Acquire multimodal input data, and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information; The fused feature information is vectorized to obtain query vector data, and the query vector data and dynamic knowledge base are used for retrieval and matching to obtain target agricultural technology knowledge information; Based on the target agricultural technology knowledge information, preset structured prompt words, and a trained large language model, reasoning is performed to generate structured solution information; The structured scheme information is converted into a new format according to a preset protocol to obtain target control instructions, and the planting equipment is controlled according to the target control instructions.

2. The method according to claim 1, characterized in that, The step of performing recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information specifically includes: The multimodal input data is analyzed to determine video data, soil sensor data, and meteorological data; The video data is detected and image features are determined based on the trained multimodal large model and the trained recognition model. Feature extraction is performed on the soil sensor data to obtain soil features, and feature extraction is performed on the meteorological data to obtain meteorological features; Feature fusion is performed based on the image features, soil features, and meteorological features to obtain fused feature information.

3. The method according to claim 2, characterized in that, The step of detecting and determining image features based on the trained multimodal large model and the trained recognition model of the video data specifically includes: The video data is sliced ​​to obtain image data, and inference is performed based on the trained multimodal large model and the image data to determine crop status information; wherein, the crop status information includes crop variety and current growth cycle; The trained recognition model and the crop status information are used to filter and determine the target detection model. The target detection model is then used to perform disease detection on the image data to obtain the crop detection result, which is then used as the image feature.

4. The method according to claim 1, characterized in that, The step of retrieving and matching the target agricultural technology knowledge information based on the query vector data and the dynamic knowledge base specifically includes: The dynamic knowledge base is queried based on the query vector data to obtain several knowledge vector data. Based on the query vector data, cosine similarity is calculated between each of the knowledge vector data and the query vector data to determine several cosine similarity values. The knowledge vector data are sorted according to several cosine similarity values ​​to determine the sorting result, and the sorting result is filtered according to a preset threshold to determine the target agricultural technology knowledge information.

5. The method according to claim 1, characterized in that, The dynamic knowledge base is determined using the following method: Collect currently available information according to a preset cycle, and organize the currently available information and preset custom planting information to obtain the original knowledge set; The original knowledge set is sliced ​​to obtain a sliced ​​knowledge set, and the sliced ​​knowledge set is vectorized to obtain the dynamic knowledge base.

6. The method according to claim 1, characterized in that, The method further includes: Data extraction is performed on the multimodal input data to determine meteorological data and soil sensor data. Prediction is made based on the meteorological data to determine the probability of precipitation, and data extraction is performed on the soil sensor data to determine the soil moisture. The candidate irrigation amount is determined by calculating based on the preset coupling equation, the precipitation probability, and the soil moisture; and then compared with the precipitation probability and the preset probability threshold. If the precipitation probability is less than or equal to the preset probability threshold, the candidate irrigation amount is taken as the target irrigation amount, and water and fertilizer are regulated according to the target irrigation amount. If the probability of precipitation is greater than the preset probability threshold, the candidate irrigation amount is adjusted according to the preset irrigation adjustment amount to obtain the target irrigation amount, and water and fertilizer regulation is performed according to the preset time period and the target irrigation amount.

7. A planting control system based on multi-source data coupling, characterized in that, The system includes: The feature fusion module is used to acquire multimodal input data and perform recognition analysis based on the multimodal input data, the trained multimodal large model, and the trained recognition model to obtain fused feature information. The knowledge retrieval module is used to vectorize the fused feature information to obtain query vector data, and to perform retrieval matching based on the query vector data and the dynamic knowledge base to obtain target agricultural technology knowledge information; The solution generation module is used to generate structured solution information by reasoning based on the target agricultural technology knowledge information, preset structured prompt words and trained large language model; The device driver module is used to convert the structured scheme information according to a preset protocol to obtain target control instructions, and to control the planting equipment according to the target control instructions.

8. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images