Detection scheme generation method and device

Abstract

The invention discloses a detection scheme generation method and device, and the method comprises the steps: obtaining input information which at least comprises a target image set for a target detection task and task description information; analyzing the input information based on a target prompt template of each processing stage for generating the detection scheme, and determining prompt information corresponding to an analysis result of each processing stage; wherein the target prompt template is used for generating prompt information of each processing stage, and the prompt information is used for guiding the processing process of the reasoning detection scheme of the corresponding processing stage; the analysis result represents information having a target relationship with the input information in the processing stage; based on the prompt information corresponding to each processing stage, processing the input information through a large model to obtain a processing result corresponding to each processing stage; and generating a detection scheme corresponding to the target detection task based on each processing result.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically to a method and device for generating a detection scheme. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, more and more processing platforms include multiple functional modules. Each functional module can be implemented through corresponding types of AI software and hardware, thereby providing users with AI-related services to meet people's needs for intelligent data processing.

[0003] Taking image detection application scenarios as an example, different functional modules correspond to different detection stages or detection purposes. If the relevant functional modules are manually combined to determine the corresponding detection plan, the process is rather cumbersome and cannot guarantee the processing efficiency and accuracy of the image detection processing task. Summary of the invention

[0004] In view of this, this application provides the following technical solutions:

[0005] A method for generating a detection scheme, comprising:

[0006] Obtaining input information, wherein the input information at least includes a target image set for a target detection task and task description information;

[0007] The input information is parsed based on the target prompt template of each processing stage of generating the detection scheme, and prompt information corresponding to the parsing result of each processing stage is determined; wherein the target prompt template is used to generate prompt information of each processing stage, and the prompt information is used to guide the processing process of the recommended detection scheme of the corresponding processing stage; the parsing result represents the information having a target relationship with the input information in the processing stage;

[0008] Based on the prompt information corresponding to each processing stage, the input information is processed by the large model to obtain the processing result corresponding to each processing stage;

[0009] Based on each of the processing results, a detection solution corresponding to the target detection task is generated.

[0010] Optionally, the processing stages include a first stage, a second stage, and a third stage in a sequential processing order, wherein determining prompt information corresponding to the parsing results of each processing stage includes:

[0011] Based on the first analysis result corresponding to the first stage, determining the first prompt information of the first stage;

[0012] Determine second prompt information of the second stage based on the second parsing result and the first processing result corresponding to the second stage, wherein the first processing result is information obtained by processing the input information through the large model based on the first prompt information;

[0013] Based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, the third prompt information of the third stage is determined, and the second processing result is the information obtained by processing the input information through the large model based on the second prompt information.

[0014] Optionally, the first stage represents a task planning stage, the target prompt template includes a first prompt template corresponding to the task planning stage, the first parsing result represents context information corresponding to the input information determined based on the first prompt template, and the first prompt information of the first stage is determined based on the first parsing result corresponding to the first stage, including: determining the first prompt information corresponding to the context information, the first prompt information is used to guide the task planning stage to determine the task information based on the context information.

[0015] Optionally, the second stage represents a processing flow or a function template selection stage, and the target prompt template includes a second prompt template corresponding to the processing flow or function module selection stage, wherein the first processing result includes task information, and the second parsing result includes candidate processing flows or candidate function modules; based on the second parsing result and the first processing result corresponding to the second stage, determining the second prompt information of the second stage includes:

[0016] Based on the task information and the candidate processing flows or candidate functional modules, second prompt information for the processing flow or functional module selection phase is determined, wherein the second prompt information is used to guide the processing flow or functional module selection phase to select a target processing flow or target functional module that matches the task information from among the candidate processing flows or candidate functional modules.

[0017] Optionally, the third stage represents a solution generation stage, the target prompt template includes a third prompt template corresponding to the solution generation stage, wherein the second processing result represents the target processing flow or target function module obtained by screening, and the third analysis result represents the detection solution reference information, wherein the third prompt information of the third stage is determined based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, including:

[0018] Based on the detection scheme reference information, the target processing flow or target functional module and / or the task information, third prompt information is determined, and the third prompt information is used to generate a detection scheme that meets the target conditions in the scheme generation stage, wherein the target conditions are determined at least based on the target processing flow or target functional module and / or the task information.

[0019] Optionally, based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including:

[0020] Based on the first prompt information of the task planning stage, the task description information and context information in the input information are processed by the large model to obtain the task information corresponding to the task planning stage.

[0021] Optionally, based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including:

[0022] Based on the second prompt information corresponding to the processing flow or function module selection stage, the image in the target image set in the input information is processed by the large model using the candidate processing flow or candidate function module to obtain an image processing result;

[0023] Based on the image processing result and the task information of the task planning stage, a target processing flow or a target functional module corresponding to the processing flow or functional module selection stage is obtained.

[0024] Optionally, based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including:

[0025] Based on the third prompt information corresponding to the solution generation stage, the task description information in the input information is processed by the large model using the task information to obtain a task list;

[0026] Determine the target processing flow or target functional module corresponding to the task list;

[0027] Processing the target image set in the input information by using the target processing flow or the target function module through the large model to obtain an image reasoning result;

[0028] A detection plan is generated based on the task list, the target processing flow or target functional module and the image reasoning result.

[0029] Optionally, it also includes:

[0030] Based on the detection scheme, the image to be detected is processed to obtain a detection result; the detection result is used to update the detection scheme.

[0031] A device for generating a detection scheme, comprising:

[0032] An acquisition unit, configured to obtain input information, wherein the input information at least includes a target image set and task description information for a target detection task;

[0033] A determination unit, configured to parse the input information based on a target prompt template for each processing stage of generating a detection scheme, and determine prompt information corresponding to the parsing result of each processing stage; wherein the target prompt template is used to generate prompt information for each processing stage, and the prompt information is used to guide the processing process of the inference detection scheme of the corresponding processing stage; and the parsing result represents information having a target relationship with the input information in the processing stage;

[0034] A processing unit, configured to process the input information through a large model based on the prompt information corresponding to each processing stage, and obtain a processing result corresponding to each processing stage;

[0035] A generating unit is used to generate a detection scheme corresponding to the target detection task based on each of the processing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of a flow chart of a method for generating a detection scheme provided in an embodiment of the present application;

[0038] Figure 2 A flowchart of a method for determining prompt information provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of an application scenario for a target detection task provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of the structure of a device for generating a detection scheme provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] The terms "first" and "second" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0043] The embodiment of the present application provides a method for generating a detection scheme, which can automatically generate a detection scheme corresponding to the detection requirements. The method for generating the detection scheme can be executed by an intelligent agent, which is a system that can autonomously perform tasks or make decisions. For example, the intelligent agent can be an electronic device that includes a large model or can call a large model, such as a server or server cluster configured with a large model, or a terminal device that can call a large model. The intelligent agent can accurately analyze the input information corresponding to the target detection task and automatically generate a detection scheme, without the need for manual combination analysis of related processing procedures or functional modules, thereby obtaining a detection scheme. The efficiency and accuracy of the generation of the detection scheme are thereby achieved.

[0044] See also Figure 1 , is a flow chart of a method for generating a detection scheme provided in an embodiment of the present application, the method may include the following steps:

[0045] S101: Obtain input information.

[0046] In the case where the agent executes the method for generating the detection scheme, the input information may be information for the target detection task input by the detection demand party to the agent through the input interface of the agent, and the input information at least includes the detection object and the detection demand, wherein the detection object may be the detection content for the detection scheme finally generated, such as the detection object may be an image or a video stream, etc. The application scenario corresponding to the embodiment of the present application is mainly a scenario for detecting images, such as detecting defects of components included in the image. Therefore, in the embodiment of the present application, the input information at least includes a target image set for the target detection task and task description information. The target detection task refers to the task of the current detection scene, such as the task of detecting defects of components in the image, or the task of detecting image anomalies in the video stream. Correspondingly, each image in the target image set is a series of representative images, that is, images that match the current target detection task, such as images of different angles of the detection object for the target detection task, or images of different detection defect types, so that when the images in the target image set are analyzed and processed later, accurate image features can be obtained. Correspondingly, the input information also includes task description information, which can characterize the current detection requirements, such as what kind of components in the image to detect, what kind of image abnormalities to detect, etc. In this way, the detection requirements can be determined more accurately based on the task description information, so that the subsequent detection plan generated based on the input information is more in line with the current detection scenario.

[0047] S102, parsing the input information based on the target prompt template of each processing stage of generating the detection solution, and determining prompt information corresponding to the parsing result of each processing stage.

[0048] S103. Based on the prompt information corresponding to each processing stage, the input information is processed by the large model to obtain the processing result corresponding to each processing stage.

[0049] S104: Generate a detection solution corresponding to the target detection task based on each processing result.

[0050] If the large model is used to directly process the input information input by the user, the large model may have a deviation in understanding the user's intention, so that the final generated detection plan deviates from the current detection requirements. Therefore, in the embodiment of the present application, the user's input information and related information are analyzed to determine the prompt information, and the input information is processed based on the prompt information to obtain the information that is finally input to the large model, which can improve the accuracy of the large model processing. Among them, the prompt information is an input of the large model, which is used to guide the large model to generate a specific output. Specifically, the prompt information can be an instruction or related guidance information, so that the large model can clearly understand what type of information the user needs or what task the model wants to perform.

[0051] There will be different processing stages in the process of generating the detection scheme, and each processing stage can generate the corresponding processing results for determining the final detection scheme, so as to determine the final detection scheme according to the processing results of each processing stage. In order to improve the processing accuracy of each processing stage, the input information can be parsed based on the target prompt template of each processing stage in the embodiment of the present application, and the prompt information corresponding to the parsing results of each processing stage can be determined. Among them, the target prompt template is used to generate the prompt information of the corresponding processing stage, and the prompt information is used to guide the processing process of the reasoning detection scheme of the corresponding processing stage, and the parsing result characterizes the information with the target relationship with the input information in the processing stage. Specifically, the target prompt template can be a template generated in advance based on the information corresponding to various detection scenes and detection requirements. In one embodiment, the target prompt template can be manually designed and generated by experts in the detection field, and its application in the field of image detection is high in accuracy. Specifically, a prompt template library can be pre-generated according to different detection scenes, different detection requirements or different detection objects, and the prompt template library includes multiple prompt templates, for example, including prompt templates for different detection objects. Thereby, the target prompt template can be obtained in the prompt template library according to the current detection scene or detection object.

[0052] And different prompt templates can be obtained in different processing stages, and then the input information is parsed based on the target prompt template corresponding to each processing stage to obtain information that characterizes the target relationship with the input information in the processing stage. For example, if the processing stage includes a task planning stage, the input information can be parsed according to the target prompt template of the task planning stage to obtain a parsing result, such as the parsing result can include associated information corresponding to the input information, specifically user feature information corresponding to the user who generates the input information, historical dialogue information of the interaction between the intelligent agent and the user, etc., so that when the input information is processed, the prompt information of the processing stage can be determined based on the parsing result and the input information, such as the prompt information can be information that guides the large model to determine the task intention of the current detection task based on the input information and the user's historical conversation information. Then the large model processes the input information based on the prompt information corresponding to each processing stage to obtain the processing result corresponding to the corresponding stage. For example, in the task planning stage, the large model can determine the task intention corresponding to the input information in the user's historical conversation information based on the prompt information corresponding to the stage. Thereby, the large model can more accurately understand the user's current task intention.

[0053] Correspondingly, in the process of determining the prompt information corresponding to each processing stage of the detection scheme, if the first stage is the previous processing stage of the second stage, the second stage can also process the input information according to the processing result of the first stage, so that the correlation relationship of each processing stage can be reflected in the corresponding processing result, making the processing result more accurate. Therefore, a detection scheme corresponding to the target detection task can be generated according to each processing result.

[0054] For example, if the target detection task is to identify image defects, there may be multiple functional modules applied in the detection scenario, and each functional module may perform corresponding detection processing, such as a functional module may include a function for enhancing the image, a functional module for extracting features from the image, a functional module for recognizing the image, etc. There may also be multiple functional modules that perform the same function. If they are directly combined manually, the process is cumbersome and affected by subjective factors, and the accuracy and efficiency will be low. Therefore, in an embodiment of the present application, the processing stage of the detection scheme for generating the detection task can be determined, and then the prompt information of the stage can be determined based on the target prompt template corresponding to each processing stage, so as to guide the processing stage to process the input information according to the prompt information, so that the processing of the large model is more accurate, so that the selected functional modules can be obtained, and these selected functional modules are combined according to the detection requirements to obtain the final detection scheme, so that complex detection tasks can be solved and the efficiency and accuracy of the detection scheme generation can be improved.

[0055] The following describes the method for generating a detection solution according to an embodiment of the present application in conjunction with a specific application scenario.

[0056] The processing stage of generating the detection scheme may correspond to the target detection task. Specifically, the corresponding processing stage may be determined by preliminarily identifying the task description information. In one implementation of the embodiment of the present application, the processing stage includes a first stage, a second stage, and a third stage in a sequential processing order. Figure 2 , is a flow chart of a method for determining prompt information provided in an embodiment of the present application, which may specifically include the following steps:

[0057] S201. Determine first prompt information of the first stage based on a first parsing result corresponding to the first stage.

[0058] S202: Determine second prompt information of the second stage based on the second parsing result and the first processing result corresponding to the second stage.

[0059] S203: Determine third prompt information of the third stage based on the third parsing result, the second processing result and / or the first processing result corresponding to the third stage.

[0060] In this implementation, the first stage, the second stage and the third stage are processed in sequence, such as executing the processing of the first stage first, then executing the processing of the second stage, and finally executing the processing of the third stage. Correspondingly, the processing result of the previous stage will be applied to the processing of the next stage, so that the processing of the input information generated by the user can be more accurate.

[0061] Specifically, the input information generated by the user is analyzed through the target prompt template corresponding to the first stage to determine the first parsing result corresponding to the first stage, and the first parsing result is used to assist in generating the first prompt information. For example, the first parsing result may be the associated information, reference information, or extracted key field information corresponding to the input information generated by the current user, or may be some information used to determine the format of the output information of this stage. For example, the first stage is a stage for identifying user features, and the prompt template of the first stage may be a user feature parsing template, through which the input keywords generated by the user are extracted, and the historical data of the user is obtained according to the keywords, and the historical data is determined as the first parsing result, and the first prompt information is determined according to the historical data and the input information generated by the user, and the first prompt information may be a prompt for the large model to analyze the user's features based on the historical data of the user. In this way, the large model can obtain more user data in the process of determining the user's features, thereby generating a first processing result based on the first prompt information, that is, obtaining user feature data, so that the large model can more accurately output user feature data related to the user based on the prompt information.

[0062] After obtaining the prompt information of the first stage, the large model will process the input information based on the first prompt information to obtain the first processing result. Then, based on the second analysis result and the first processing result corresponding to the second stage, the second prompt information of the second stage is determined. For example, the second stage is the functional module determination stage corresponding to the target detection task. Taking the above-mentioned first stage as the user feature determination stage as an example, the user type can be determined according to the user characteristics. For example, the quality inspection department to which the user belongs can be determined. Different quality inspection departments correspond to different types of components to be detected, and thus the corresponding component defect detection types are different. Therefore, the detection requirements can be determined more accurately according to the user characteristics, and the functional modules used for detection can be further determined more accurately. In the second stage, the second analysis result can be a candidate functional module determined according to the task description information and user characteristics corresponding to the input information, and the corresponding second prompt information can be a prompt for the large model to select the functional modules that constitute the detection scheme from these candidate functional modules, so that the large model processes the input information based on the second prompt information to obtain the target functional module corresponding to the second stage.

[0063] Based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, the third prompt information of the third stage is determined. In the process of determining the third prompt information, in addition to the third analysis result corresponding to the third stage, the processing result corresponding to the previous processing stage can also be used, such as the third prompt information can be determined based on the third analysis result and the second processing result, or the third prompt information can be determined based on the third analysis result and the first processing result, or the third prompt information can be determined based on the third analysis result, the second processing result and the first processing result. In the process of generating the second processing result, the first processing result is used. Therefore, the third prompt information can be directly determined based on the third analysis result and the second processing result, so that the third prompt information also considers the processing intentions of the previous stages, so that the generated retrieval scheme is more accurate. Correspondingly, the third prompt information can also be determined based on the third analysis result and the first processing result, but the influence of the first processing result on the second stage needs to be considered in this process. In order to improve processing efficiency and accuracy, the third stage can determine the third prompt information based on the processing results of the first two stages. For example, the third stage is the detection scheme generation stage, and the third prompt information can be a prompt large model to generate a detection scheme required by the user based on the target function module. In this way, corresponding prompt information can be determined for each stage, making the processing of subsequent large models more accurate at each stage.

[0064] The following takes the target detection task of defect detection of components in an image as an example. The first stage represents the task planning stage, the second stage represents the processing flow or functional module selection stage, and the third stage represents the solution generation stage.

[0065] See also Figure 3 , is a schematic diagram of an application scenario for a target detection task provided by an embodiment of the present application, in Figure 3 In the method for generating the detection scheme by an agent, the information input to the agent includes input information generated by the detection demand side, which includes a target image set and task description information, wherein the target image set can be a group of representative pictures, and the task description information can represent the current detection demand, such as "please provide a detection scheme that can locate the position of defects in this batch of pictures", and at the same time, a prompt template is input to the agent, and the prompt template can be used to limit the processing and output of the agent. Specifically, the prompt template can have multiple forms, such as a prompt template can be a prompt table, in which prompt design information of different processing stages is configured. Then the agent executes the corresponding processing stage based on the user input information and the prompt template to output the final detection scheme. Among them, the processing stage can include a task planning stage, a processing flow or functional module selection stage, and a scheme generation stage.

[0066] The first stage corresponds to the task planning stage, the target prompt template includes the first prompt template corresponding to the task planning stage, the first parsing result represents the context information corresponding to the input information determined based on the first prompt template, wherein the first prompt information of the first stage is determined based on the first parsing result corresponding to the first stage, including: determining the first prompt information corresponding to the context information, the first prompt information is used to guide the task planning stage to determine the task information based on the context information.

[0067] Correspondingly, during the first stage of processing, based on the prompt information of each processing stage, the input information is processed through the big model to obtain the processing results corresponding to each processing stage, including: based on the first prompt information of the task planning stage, the task description information and context information in the input information are processed through the big model to obtain the task information corresponding to the task planning stage.

[0068] Specifically, the first prompt template may include parsing fields for parsing the input information, that is, it is necessary to extract key information based on these parsing fields and determine context information, and use the context information as the first parsing result. Then generate the prompt information for the task planning stage, such as prompting the large model to perform task planning processing based on the context information to obtain the task information for this stage. Specifically, the task information may include task intent and a corresponding task list. Further, the task list may include multiple subtasks and the relationship between each task. Referring to Table 1, the information corresponding to the task planning stage shows the relevant information in the first prompt template, wherein the "prompt design" in Table 1 includes relevant reference information for generating prompt information in the prompt template, and the "parsing results" in Table 1 represent the information used to generate prompt information determined based on the prompt template, such as the parsing results corresponding to the task planning stage may include a task list and context information.

[0069] Table 1

[0070]

[0071]

[0072]

[0073] See Table 2, which shows an example of a parsing result. In this way, the user's input information can be automatically parsed based on the prompt template to obtain the corresponding parsing result to facilitate the subsequent processing of the large model.

[0074] Table 2

[0075]

[0076] See Table 3, which shows a schematic diagram of relevant fields in the prompt template.

[0077] Table 3

[0078]

[0079]

[0080] Correspondingly, the second stage represents the processing flow or function module selection stage, and the target prompt template includes a second prompt template corresponding to the processing flow or function module selection stage, wherein the information included in the second prompt template can refer to the prompt design information of the processing flow or function module selection stage in Table 1. Among them, the first processing result includes task information, and the second parsing result includes candidate processing flows or candidate function modules. Based on the second parsing result and the first processing result corresponding to the second stage, the second prompt information of the second stage is determined, including: based on the task information and the candidate processing flows or candidate function modules, the second prompt information of the processing flow or function module selection stage is determined. Among them, the second prompt information is used to guide the processing flow or function module selection stage to select the target processing flow or target function module that matches the task information from the candidate processing flows or function modules.

[0081] Correspondingly, in the processing flow or function module selection stage, based on the prompt information corresponding to each processing stage, the input information is processed through the large model to obtain the processing results corresponding to each processing stage, including:

[0082] Based on the second prompt information corresponding to the processing flow or function module selection stage, the images in the target image set in the input information are processed by the large model using the candidate processing flow or candidate function modules to obtain the image processing results; based on the image processing results and the task information in the task planning stage, the target processing flow or target function module corresponding to the processing flow or function module selection stage is obtained.

[0083] Among them, the process of determining the target processing flow or target functional module in the processing flow or functional module selection stage may also include task execution, that is, performing relevant detection on the image based on each candidate processing flow or functional module, so as to screen the corresponding processing flow or functional module according to the image processing results obtained by the detection to obtain the target processing flow or target functional module.

[0084] Specifically, the processing flow can be the overall flow for the target detection task, and different processing flows can include sub-processing flows for different detection subtasks. In the process of determining the detection scheme, it is necessary to select the target processing flow that matches the user input information, so as to generate the corresponding detection scheme according to the target processing flow. The functional module is used to perform the corresponding detection subtask, such as the image style function module can be used to perform the image segmentation subtask in the image detection process, the image enhancement module can be used to enhance the image features in the image to be detected in the image detection process, and the image recognition module is used to perform the recognition of defective parts in the image. Specifically, different functional modules can be implemented based on different models, such as the image recognition module can be implemented based on the image recognition model. By selecting the candidate functional module or candidate processing flow corresponding to the task list in the current functional module library or processing flow for the task list or the task intention information determined in the task planning stage, the second prompt information generated for this stage can be "please select the corresponding target functional module from the candidate functional modules below", and correspondingly, the second processing result can be the selected target processing flow or target functional module. That is, in the processing flow or functional module selection stage, based on the user's input information and the information generated in the task planning stage, the most suitable model for each parsing task is selected in a given context.

[0085] The third stage represents the solution generation stage, and the target prompt template includes a third prompt module corresponding to the solution generation stage, wherein the second processing result represents the target processing flow or target functional module obtained by screening, and the third analysis result represents the detection solution reference information. Based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, the third prompt information of the third stage is determined, including: based on the detection solution reference information, the target processing flow or the target functional module and / or the task information, the third prompt information is determined. The third prompt information is used to guide the solution generation stage to generate a detection solution that meets the target conditions, and the target conditions are determined at least based on the target processing flow or the target functional module and / or the task information.

[0086] Correspondingly, based on the prompt information corresponding to each processing stage, the input information is processed by the large model to obtain the processing results corresponding to each processing stage, including:

[0087] Based on the third prompt information corresponding to the solution generation stage, the task description information in the input information is processed by the large model using the task information to obtain a task list; the target processing flow or target function module corresponding to the task list is determined; the target image set in the input information is processed by the large model using the target processing flow or target function module to obtain an image reasoning result; based on the task list, the target processing flow or the target function module and the image reasoning result, a detection solution is generated.

[0088] Among them, the detection scheme reference information corresponding to the third parsing result may be the information format, description method and content of generating the detection scheme. The corresponding third prompt information is used to guide the scheme generation stage to generate a detection scheme that meets the target conditions, such as the third prompt information may be "please output the functional module information corresponding to the target functional module and the task list corresponding to the task information as the detection scheme". Specifically, the detection scheme may be a concise summary of the processing results obtained in the previous processing stages. Specifically, the detection scheme may include a task list, a target processing flow or a target functional module, and an image reasoning result obtained by applying the target processing flow or the target functional module. The image reasoning result may include an image defect detection result. For example, each functional module may be an artificial intelligence model of MVLiTE (virtual machine software). When outputting the detection scheme, the storage path and description information of each selected artificial intelligence model are included, so that the user can obtain detailed information of each selected artificial intelligence model based on the detection scheme, which is convenient for the subsequent application of the detection scheme. Specifically, the description information may include the training information and usage information of the artificial intelligence model. The detailed information of each selected artificial intelligence model can be displayed based on the task list in the task information. See Table 4, in which the information of the task list included in the detection scheme is explained using the functional module as the artificial intelligence model.

[0089] Table 4

[0090]

[0091]

[0092] As shown in Table 4, the corresponding task types may include positioning detection of screws or holes, detecting text in images, image registration, image classification, image segmentation, target detection, anomaly detection, etc. Each task type has a corresponding candidate model, and includes the storage path of the candidate model and text description information. For example, the storage path of the candidate model corresponding to the image registration task type can be expressed as "MVLiTE / reg001.mvl,...", and its corresponding description information can be "based on the template selected by the user as the only standard, used to calculate the rotation matrix of the image", etc. In this way, the functions and features of the selected model can be obtained according to the detailed information in Table 4, which is convenient for subsequent applications.

[0093] In an embodiment of the present application, the input information of the detection demander is input into the intelligent agent, and the prompt information of the corresponding processing stage is generated through the corresponding prompt template, thereby improving the information processing capability of the intelligent agent, so that the intelligent agent analyzes and combines the existing processing flow or functional modules according to the input information and prompt information to obtain the corresponding detection plan, which can solve the problems of low efficiency and poor accuracy of manually generated detection plans, and can be applied to complex artificial intelligence detection tasks, expanding the application scenarios.

[0094] In one implementation of the present application, the detection scheme can also be adjusted according to the generated detection scheme and the feedback information for the detection scheme, so as to achieve the purpose of updating the detection scheme. Specifically, the detection image can be processed based on the detection scheme to obtain the detection result. The detection result is used to update the detection scheme. After the intelligent agent outputs the detection scheme, the user can perform subsequent operations based on the detection scheme, such as verifying the detection scheme, or adjusting the relevant target function modules in the detection scheme. For example, if the user is satisfied with the detection scheme, it can generate an automatic execution instruction corresponding to the detection scheme, so that the intelligent agent can complete subsequent image detection tasks based on the detection scheme, and output the detection result. If the user has fed back the relevant information in the detection result, such as feedback on information such as image clarity or abnormal annotation boxes, the intelligent agent can update the current detection scheme according to the feedback information. For example, the current detection scheme includes a first anomaly detection function module for image anomaly annotation. If the contrast difference between the anomaly annotation box and the image presented in the detection result obtained based on the detection scheme is not obvious, the user can generate feedback information on the problem. At this time, the first anomaly detection function module can be updated to the second anomaly detection function module to obtain an updated detection scheme, so that the updated detection scheme not only meets the detection requirements of the target detection task, but also meets the user's usage needs.

[0095] In another embodiment of the present application, a device for generating a detection scheme is also provided, see Figure 4 , the device may include:

[0096] An acquisition unit 401 is used to obtain input information, where the input information at least includes a target image set and task description information for a target detection task;

[0097] A determination unit 402 is used to parse the input information based on a target prompt template for each processing stage of the detection scheme, and determine prompt information corresponding to the parsing result of each processing stage; wherein the target prompt template is used to generate prompt information for each processing stage, and the prompt information is used to guide the processing process of the inference detection scheme of the corresponding processing stage; and the parsing result represents information having a target relationship with the input information in the processing stage;

[0098] A processing unit 403 is used to process the input information through a large model based on the prompt information corresponding to each processing stage to obtain a processing result corresponding to each processing stage;

[0099] The generating unit 404 is used to generate a detection scheme corresponding to the target detection task based on each of the processing results.

[0100] Optionally, the processing stages include a first stage, a second stage and a third stage in a sequential processing order, wherein the determining unit includes:

[0101] A first determining subunit, configured to determine first prompt information of the first stage based on a first parsing result corresponding to the first stage;

[0102] A second determining subunit is used to determine second prompt information of the second stage based on a second parsing result and a first processing result corresponding to the second stage, wherein the first processing result is information obtained by processing the input information through a large model based on the first prompt information;

[0103] The third determination subunit is used to determine the third prompt information of the third stage based on the third analysis result corresponding to the third stage, the second processing result and / or the first processing result, wherein the second processing result is information obtained by processing the input information through a large model based on the second prompt information.

[0104] Optionally, the first stage represents a task planning stage, the target prompt template includes a first prompt template corresponding to the task planning stage, the first parsing result represents context information corresponding to the input information determined based on the first prompt template, and the first determination subunit is configured to: determine first prompt information corresponding to the context information, and the first prompt information is used to guide the task planning stage to determine task information based on the context information.

[0105] Optionally, the second stage represents a processing flow or a functional module selection stage, the target prompt template includes a second prompt template corresponding to the processing flow or the functional module selection stage, wherein the first processing result includes task information, and the second parsing result includes candidate processing flows or candidate functional modules; the second determining subunit is configured to:

[0106] Based on the task information and the candidate processing flows or candidate functional modules, second prompt information for the processing flow or functional module selection phase is determined, wherein the second prompt information is used to guide the processing flow or functional module selection phase to select a target processing flow or target functional module that matches the task information from among the candidate processing flows or candidate functional modules.

[0107] Optionally, the third stage represents a solution generation stage, the target prompt template includes a third prompt template corresponding to the solution generation stage, wherein the second processing result represents the target processing flow or target function module obtained by screening, and the third parsing result represents the detection solution reference information, wherein the third determination subunit is configured as:

[0108] Based on the detection scheme reference information, the target processing flow or target functional module and / or the task information, third prompt information is determined, and the third prompt information is used to guide the scheme generation stage to generate a detection scheme that meets the target conditions, wherein the target conditions are determined at least based on the target processing flow or target functional module and / or the task information.

[0109] Optionally, the processing unit includes a first processing subunit, and the first processing subunit is configured to:

[0110] Based on the first prompt information of the task planning stage, the task description information and context information in the input information are processed by the large model to obtain the task information corresponding to the task planning stage.

[0111] Optionally, the processing unit includes a second processing subunit, and the second processing subunit is used for:

[0112] Based on the second prompt information corresponding to the processing flow or function module selection stage, the image in the target image set in the input information is processed by the large model using the candidate processing flow or candidate function module to obtain an image processing result;

[0113] Based on the image processing result and the task information of the task planning stage, a target processing flow or a target functional module corresponding to the processing flow or functional module selection stage is obtained.

[0114] Optionally, the processing unit includes a third processing subunit, and the third processing subunit is used to:

[0115] Based on the third prompt information corresponding to the solution generation stage, the task description information in the input information is processed by the large model using the task information to obtain a task list;

[0116] Determine a target processing flow or a target functional module corresponding to the task list;

[0117] Processing the target image set in the input information by using the target processing flow or the target function module through the large model to obtain an image reasoning result;

[0118] A detection plan is generated based on the task list, the target processing flow or target functional module and the image reasoning result.

[0119] Optionally, it also includes:

[0120] The detection unit is used to process the image to be detected based on the detection scheme to obtain a detection result; the detection result is used to update the detection scheme.

[0121] It should be noted that the specific implementation of each unit and sub-unit in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0122] In another embodiment of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating a detection scheme as described above is implemented.

[0123] In another embodiment of the present application, an electronic device is further provided, and the electronic device may include:

[0124] A memory, used to store applications and data generated by the operation of the applications;

[0125] The processor is used to execute the application program to implement the method for generating the detection scheme as described above.

[0126] It should be noted that the specific implementation of the processor in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0127] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0128] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0130] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a detection scheme, comprising: Obtaining input information, wherein the input information at least includes a target image set for a target detection task and task description information; The input information is parsed based on the target prompt template of each processing stage of the generated detection scheme, and prompt information corresponding to the parsing result of each processing stage is determined; wherein the target prompt template is used to generate prompt information of each processing stage, and the prompt information is used to guide the processing process of the inference detection scheme of the corresponding processing stage; the parsing result represents the information having a target relationship with the input information in the processing stage; Based on the prompt information corresponding to each processing stage, the input information is processed by the large model to obtain the processing result corresponding to each processing stage; Based on each of the processing results, a detection solution corresponding to the target detection task is generated.

2. The method according to claim 1, wherein the processing stages include a first stage, a second stage and a third stage in a sequential processing order, wherein: The step of determining prompt information corresponding to the parsing results of each processing stage includes: Determine first prompt information of the first stage based on the first parsing result corresponding to the first stage; Determine second prompt information of the second stage based on the second parsing result and the first processing result corresponding to the second stage, wherein the first processing result is information obtained by processing the input information through the large model based on the first prompt information; Based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, the third prompt information of the third stage is determined, and the second processing result is the information obtained by processing the input information through the large model based on the second prompt information.

3. The method according to claim 1, wherein the first stage represents a task planning stage, the target prompt template includes a first prompt template corresponding to the task planning stage, the first parsing result represents context information corresponding to the input information determined based on the first prompt template, and the first prompt information of the first stage is determined based on the first parsing result corresponding to the first stage, comprising: First prompt information corresponding to the context information is determined, where the first prompt information is used to guide the task planning stage to determine task information based on the context information.

4. The method according to claim 3, wherein the second stage represents a processing flow or a function module selection stage, and the target prompt template includes a second prompt template corresponding to the processing flow or the function module selection stage, wherein: The first processing result includes task information, and the second parsing result includes candidate processing procedures or candidate functional modules; Determining second prompt information of the second stage based on the second parsing result and the first processing result corresponding to the second stage includes: Based on the task information and the candidate processing flows or candidate functional modules, second prompt information for the processing flow or functional module selection phase is determined, wherein the second prompt information is used to guide the processing flow or functional module selection phase to select a target processing flow or target functional module that matches the task information from among the candidate processing flows or candidate functional modules.

5. The method according to claim 4, wherein the third stage represents a solution generation stage, and the target prompt template includes a third prompt template corresponding to the solution generation stage, wherein: The second processing result represents the target processing flow or target functional module obtained by screening, and the third analysis result represents the detection scheme reference information, wherein the third prompt information of the third stage is determined based on the third analysis result, the second processing result and / or the first processing result corresponding to the third stage, including: Based on the detection scheme reference information, the target processing flow or target functional module and / or the task information, third prompt information is determined, and the third prompt information is used to guide the scheme generation stage to generate a detection scheme that meets the target conditions, wherein the target conditions are determined at least based on the target processing flow or target functional module and / or the task information.

6. The method according to claim 3, wherein based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including: Based on the first prompt information of the task planning stage, the task description information and context information in the input information are processed by the large model to obtain the task information corresponding to the task planning stage.

7. The method according to claim 4, wherein based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including: Based on the second prompt information corresponding to the processing flow or function module selection stage, the image in the target image set in the input information is processed by the large model using the candidate processing flow or candidate function module to obtain an image processing result; Based on the image processing result and the task information of the task planning stage, a target processing flow or a target functional module corresponding to the processing flow or functional module selection stage is obtained.

8. The method according to claim 5, wherein based on the prompt information corresponding to each processing stage, the input information is processed by a large model to obtain a processing result corresponding to each processing stage, including: Based on the third prompt information corresponding to the solution generation stage, the task description information in the input information is processed by the large model using the task information to obtain a task list; Determine a target processing flow or a target functional module corresponding to the task list; Processing the target image set in the input information by using the target processing flow or the target function module through the large model to obtain an image reasoning result; A detection plan is generated based on the task list, the target processing flow or target functional module and the image reasoning result.

9. The method according to claim 1, further comprising: Based on the detection scheme, the image to be detected is processed to obtain a detection result; The detection result is used to update the detection scheme.

10. A device for generating a detection scheme, comprising: An acquisition unit, configured to obtain input information, wherein the input information includes at least a target image set and task description information for a target detection task; A determination unit, configured to parse the input information based on a target prompt template for each processing stage of generating a detection scheme, and determine prompt information corresponding to the parsing result of each processing stage; wherein the target prompt template is used to generate prompt information for each processing stage, and the prompt information is used to guide the processing process of the inference detection scheme of the corresponding processing stage; and the parsing result represents information having a target relationship with the input information in the processing stage; A processing unit, configured to process the input information through a large model based on the prompt information corresponding to each processing stage, and obtain a processing result corresponding to each processing stage; A generating unit is used to generate a detection scheme corresponding to the target detection task based on each of the processing results.

Images