An operation interface flow binding method, device, equipment and storage medium

Abstract

The application discloses a running interface flow binding method and device, equipment and storage medium, including: acquiring a visual flow model, each flow containing multiple tool nodes; in response to the configuration operation of the control in the running interface, generating the mapping information of the control, the mapping information containing the to-be-bound attribute, positioning information and binding source of the control, and the target object being a bindable data unit; in response to the loading instruction of the running interface, acquiring the current value of the target object according to the mapping information, and determining the display content of the control according to the current value; in response to the editing operation of the editable control, writing the editing content into the target object; executing the current flow according to the dependency relationship between the tool nodes, and updating the display content of the control after the execution is completed; the structured binding and bidirectional linkage of the running interface and the visual flow can be realized, the configuration mismatch rate is reduced, and the delivery consistency and maintenance efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of industrial machine vision software technology, and more specifically, to a method, apparatus, device, and storage medium for binding a runtime interface process. Background Technology

[0002] Industrial machine vision systems typically involve developers building inspection workflows within a development interface, including tool nodes for image acquisition, image processing, template matching, geometric lookup, measurement, alignment, communication, and result output. Production line operators then view images, results, and statuses through the runtime interface and trigger workflow execution. Existing solutions often separate the workflow development interface from the production line runtime interface. Engineers first establish tool nodes and connections in the workflow canvas, then manually enter variable names, tool names, output names, or control binding expressions in a separate runtime interface or script configuration to display workflow results on the operator interface or write operator input back to workflow parameters. Some systems also maintain image display, result display, command buttons, trigger rules, and variable management in separate configuration items. While the above solutions can achieve basic display and operation, the interface controls and process variables, inputs, tool inputs, and outputs largely rely on manual input or script conventions. Engineers are prone to entering incorrect variable names, tool names, or output names. Later in the project, when node names, output names, or process structures are modified, mismatches can easily occur between the interface and the process. Furthermore, the interface can typically only display results or write a small number of ordinary values, making it difficult to safely write back tool parameters, inputs, or global variables according to the actual data types in the visual process. Also, the run button and the process execution link are easily disconnected, leading to inconsistencies between the verification results during development and the production line results.

[0003] Therefore, existing technologies suffer from high risks of interface and process mismatch and high maintenance costs. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and storage medium for binding a runtime interface to a visual process. This addresses the technical problem in existing technologies where the lack of a structured and unified association mechanism between the runtime interface and the visual process leads to high mismatch risks and high maintenance costs in the configuration, operation, and migration stages of the binding relationship between the interface and the process. The application achieves a structured correspondence between the attributes to be bound and the target objects in the visual process model through the mapping information of controls. Based on the data type and state information of the target objects, it enables bidirectional linkage between the interface and the process, reducing the configuration mismatch rate and improving the delivery consistency and maintenance efficiency of visual projects on the production line.

[0005] To achieve the above objectives, the first aspect of this application proposes a method for binding a runtime interface flow, comprising: Obtain a visual process model; the visual process model contains at least one process, and each process contains multiple tool nodes; Based on the binding source selected by the user, scan the bindable data units in the visual process model corresponding to the binding source, and generate a binding candidate list based on the scan results; In response to the selection operation of the target object in the binding candidate list, the metadata information of the target object is read, and the mapping information of the control in the running interface is generated based on the metadata information; the mapping information includes the attribute to be bound to the control, the positioning information of the target object in the visual flow model, and the binding source, wherein the target object is a bindable data unit, and the target object contains multiple data types and status information for data transfer between the multiple tool nodes; In response to the loading instruction of the running interface, the current value of the target object is obtained according to the mapping information, and the display content of the control is determined according to the current value; In response to an editing operation on an editable control, the edited content is written to the target object; In response to a trigger operation on the command control in the running interface, the current process is executed according to the dependencies between the tool nodes, and the display content of the control is updated after execution is completed.

[0006] In one or more possible embodiments, the data type of the target object includes numeric type, string type, list type, image type, region type, contour type, template type, geometric type, and device reference type; The method further includes: When the data type of the target object is a list type, a single element is obtained from the list type according to the preset index parameter, and the data type of the single element is used as the converted data type.

[0007] In one or more possible embodiments, writing the edited content to the target object includes: The edited content is converted into a value of the corresponding type based on the data type of the target object; Write the converted value into the target object.

[0008] In one or more possible embodiments, the binding sources include: constants, tool node running states, global variables, process inputs, tool inputs, process outputs, tool outputs, and executed commands; The step of obtaining the current value of the target object based on the mapping information includes: When the binding source of the target object is a constant, the constant is used as the current value of the target object; When the binding source of the target object is the running state of a tool node, the running state of the tool node is used as the current value of the target object; When the binding source of the target object is a global variable, process input, tool input, process output, or tool output, the target object is located according to the positioning information, and the current value of the target object is read. When the binding source of the target object is an execution command, the command object corresponding to the execution command is located according to the location information, and the current command state of the command object is read as the current value of the target object; the current command state includes at least one of the following: executable state, disabled state, executing state, execution completed state, and execution failed state; When the binding source of the target object is process output or tool output and no output parameters are specified, the output value with valid status information in the current output set is selected as the current value of the target object.

[0009] In one or more possible embodiments, after writing the edited content to the target object in response to an editable control edit operation, the method further includes: Set the binding source of the target object to a constant, and clear the original upstream node references, variable references, and index object references.

[0010] In one or more possible embodiments, updating the display content of the control based on the updated current value includes: When the control is an image display control, the display object is obtained from the updated current value and displayed. The display object includes at least one of a region object, a contour object, a text object, and a template matching result object.

[0011] Secondly, this application provides a device for binding a runtime interface process, comprising: A model acquisition module is used to acquire a visual process model; the visual process model contains at least one process, and each process contains multiple tool nodes. The candidate acquisition module is used to scan the bindable data units in the visual process model corresponding to the binding source selected by the user, and generate a binding candidate list based on the scanning results; The mapping acquisition module is used to respond to the selection operation of a target object in the binding candidate list, read the metadata information of the target object, and generate mapping information of controls in the running interface based on the metadata information; the mapping information includes the binding attributes of the control, the positioning information of the target object in the visual flow model, and the binding source, wherein the target object is a bindable data unit in the visual flow model; the target object contains multiple data types and status information, which are used for data transfer between the multiple tool nodes; The binding parsing module is used to respond to the loading command of the running interface, obtain the current value of the target object according to the mapping information, and determine the display content of the control according to the current value; The data write-back module is used to respond to editing operations on editable controls by converting the edited content according to the data type of the target object and writing it back to the target object; The process execution module is used to respond to the trigger operation of the command control in the running interface, execute the current process according to the dependency relationship between each tool node, and update the display content of the control after execution.

[0012] Thirdly, this application provides a device for binding a running interface process, the device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any of the first aspects.

[0013] Fourthly, this application provides a storage medium storing a computer program for causing a computer to perform the method described in any of the first aspects.

[0014] The technical solutions provided by the embodiments of this application may include the following beneficial effects: In this application, by generating a set of attributes to be bound to controls, location information of the target object in the visual process model, and mapping information of the binding source, wherein the target object contains multiple data types and state information and is used for data transfer between multiple tool nodes; in response to the loading command of the running interface, the unified type value currently stored in the target object is obtained from the visual process model according to the mapping information, and the display content of the control is determined according to the value and the attributes to be bound; in response to the editing operation of the editable control, the edited content is converted according to the data type of the target object and written to the target object; and in response to the triggering operation of the command control, the current process is executed according to the dependency relationship between each tool node, and the display content of the control is updated after execution, the structured binding and bidirectional linkage between the running interface and the visual process are realized, reducing the configuration mismatch rate and improving the delivery consistency and maintenance efficiency of the production line. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A flowchart of the runtime interface process binding method provided in this application; Figure 2 A module diagram of the device for binding the runtime interface process provided in this application; Figure 3 A block diagram of the electronic device provided in this application; Figure 4 A schematic diagram of the storage medium provided in this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] The following is an explanation of the terminology used in this application: Industrial machine vision platform: A software system used to build, run and maintain visual inspection processes on production lines. It typically includes image acquisition, image processing, positioning and measurement, communication triggering, result display and project management capabilities.

[0019] Runtime Interface: A project-level user interface for production line operators, used to display images, results, status and trigger process execution, which is different from the process development interface used by engineers.

[0020] Process / Workflow: A visual processing link consisting of multiple tool nodes and node connections, which can be scheduled and executed in a directed acyclic graph manner.

[0021] Tool nodes: The smallest unit of execution in the process, such as image source, template matching, Blob analysis (Binary Large Object Analysis), QR code recognition, measurement, alignment, communication transmission, etc.

[0022] Uniform type value: A data carrier with type label and valid status, used to pass numerical values, strings, images, regions, contours, models, geometric objects, or device references between tools, processes, variables, and runtime interfaces.

[0023] Binding definition: Structured data that describes the relationship between the properties of the running interface control and a variable, process parameter, tool input or tool output in the project.

[0024] Binding source: The source type for the running interface control to read and write data or trigger process actions, including constants, program running status, global variables, process input, tool input, process output, tool output, and execution commands; among them, execution commands are used to represent command objects that can be triggered by command controls, and the command objects can provide the current command status such as executable, disabled, executing, execution completed, or execution failed.

[0025] Binding candidates: Optional binding targets automatically generated by the system based on the current project model, such as a global variable, a process input, or a tool output.

[0026] Write-back: The process by which editable controls in the runtime interface convert user input into the target data type and then write it into global variables, process inputs, or tool inputs.

[0027] Result overlay objects: Used to display detection results such as regions, contours, text, crosses, and template matching contours on the image window.

[0028] Project persistence: Save the process structure, runtime interface, control bindings, variables and necessary parameters as project files or project directories, and restore the structure required for runtime state when loaded.

[0029] Existing solutions typically handle the process development interface and the production line operation interface separately. Engineers first create the visual process by dragging and dropping in the process canvas to establish the various tool nodes and their connections. Then, in another set of operation interfaces or script configuration files, they need to manually fill in variable names, tool names, output names, or control binding expressions to associate the images, values, statuses, and other results generated during the process with the display controls on the operation interface, or to send back the data entered by the operator on the operation interface to the process parameters. Some systems also place image display, result display, command buttons, trigger rules, and variable management functions in different configuration items for maintenance. While the above solutions can achieve basic display and operation functions, they still have the following shortcomings in actual production line projects: (1) The relationships between the interface controls and process variables, process inputs, tool inputs, and tool outputs mostly rely on manual input of names or script conventions. Engineers are prone to mistype variable names, tool names, or output names during configuration. Moreover, once the project enters the later maintenance stage, the binding relationships configured in the interface are very likely to become invalid after the node names, output names, or process structures are modified. This requires checking and correcting them one by one, resulting in high maintenance costs.

[0030] (2) The operating interface usually only supports one-way display of process execution results, or only allows operators to write a small number of parameters of ordinary numerical type. However, the tool parameter types in industrial vision processes are rich and diverse. The same control may need to write multiple types of data such as integers, floating-point numbers, booleans, string lists, camera names, light source names or index objects in different scenarios. Existing solutions rely on handwritten scripts to implement parameter writing, which lacks a unified type verification mechanism. When data types do not match, errors are often exposed only at runtime, making debugging difficult and easily leading to production line anomalies.

[0031] (3) The output types of industrial vision tools are complex, including basic types such as numerical values ​​and strings, as well as complex types such as images, regions, contours, template models, measurement geometry, display objects, and device references. The existing operating interface has difficulty uniformly identifying the valid status of each output value and its applicable scenarios. For example, which output value is currently valid, which output is suitable for display on the interface, and which output can be used as input for downstream tools. This results in the need to write adaptation logic separately for different tools for functions such as image display, overlay object drawing, and result status presentation, which leads to redundant code and poor scalability.

[0032] (4) The command buttons and operation controls in the running interface are easily disconnected from the process execution chain. If the start button in the running interface implements a separate set of execution logic independent of the process development environment, the existing DAG (Directed Acyclic Graph) scheduling, cancellation, error handling, statistical recording, log output, and image refresh mechanisms in the process canvas cannot be reused. This will cause inconsistent behavior of the process that has been verified in the development environment when executed in the production line running interface, increasing the complexity of debugging and deployment.

[0033] (5) Regarding project persistence, if the XAML (Extensible Application Markup Language) layout file, control binding definition, process structure, global variables and tool parameters of the running interface are stored separately, problems such as the loss of binding relationship of the control, the inability to locate the target object, and the inability of the running interface to automatically refresh and display with the process output may occur after the project is reloaded, which seriously affects the portability of the project and the efficiency of on-site maintenance.

[0034] To address the aforementioned issues, this application provides a method for binding the runtime interface flow, such as... Figure 1 As shown, it includes: Step 101: Obtain a visual process model; the visual process model includes at least one process, and each process includes multiple tool nodes; In one or more possible embodiments, at least one process is established in the vision software platform, and the process includes multiple tool nodes. Tool nodes can be image sources, image processing, template matching, geometric lookup, measurement, alignment, communication, or display tools. Each tool node includes input parameters, output parameters, tool inputs, and tool outputs; the input parameters are used to record the source of values, which at least include constants, upstream node outputs, process inputs, process outputs, and global variables.

[0035] Step 102: Based on the binding source selected by the user, scan the bindable data units in the visual process model corresponding to the binding source, and generate a binding candidate list based on the scanning results; In one or more possible embodiments, in the runtime interface designer, each control is assigned a stable control identifier that remains unchanged throughout the project lifecycle and is used to uniquely identify the control. When a user selects a control in the runtime interface and triggers a binding configuration operation, the control type and the attributes to be bound are first determined based on the currently selected control. The control type includes text display, image display, result display, number input, text input, command buttons, switch buttons, sub-interface buttons, runtime flow controls, and template matching runtime controls, etc. Then, based on the binding source type selected by the user, matching bindable data units that meet the conditions are automatically scanned and extracted from the current visual flow model to generate a binding candidate list.

[0036] Step 103: In response to the selection operation of the target object in the binding candidate list, read the metadata information of the target object, and generate the mapping information of the control in the running interface based on the metadata information; the mapping information includes the binding attribute of the control, the positioning information of the target object in the visual flow model, and the binding source, wherein the target object is a bindable data unit, and the target object contains multiple data types and status information for data transfer between the multiple tool nodes; In one or more possible embodiments, a uniform type value (target object) is used to represent data passed between tools. The uniform type value contains data type and status information. The uniform type value can carry basic numerical values, strings, lists, images, regions, contours, camera parameters, template models, ROI (Region of Interest), measurement geometry, device references, file paths, and display objects, etc. Data types include numerical types, string types, list types, image types, region types, contour types, template types, geometry types, device reference types, etc.

[0037] When the target object's data type is a list type, a single element is retrieved from the list type according to a preset index parameter, and the data type of the single element is used as the converted data type. Specifically, the list type refers to a collection containing multiple elements of the same type. In industrial vision scenarios, the output parameters of a tool node may output multiple results; for example, a geometric search tool may output multiple sets of coordinates, positions, or scores of matching targets. When a control in the running interface needs to be bound to and display a specific element in the list type, the position of the required element in the list is specified by the preset index parameter. The index parameter is a non-negative integer used to indicate the extraction of the Nth element from the list, where N is the value of the index parameter. For example, when the index parameter is 0, it indicates the extraction of the first element in the list; when the index parameter is 1, it indicates the extraction of the second element in the list, and so on. After the single element is extracted, since the data type of the single element is different from the list type, the data type of the single element needs to be passed to the downstream target object as the converted data type so that the downstream target object can perform subsequent display, storage, or participate in the calculation of subsequent process nodes according to the data type of the single element. For example, in a scenario where a geometric search tool outputs a set of candidate target coordinates, the first coordinate in the coordinate set is extracted using an index parameter, and this coordinate is output as the positioning result to the downstream measurement tool, thus achieving type downgrading from a list of coordinates to a single coordinate.

[0038] The status information is used to indicate whether the data corresponding to the unified type value is in an available state at the current time. The status information includes three states: valid, invalid, and null. When the tool node to which the unified type value belongs has not yet been executed, the status information of the unified type value is null. When the tool node is executed successfully and produces a valid result, the status information of the unified type value is valid. When the tool node fails to execute or does not produce a valid result, the status information of the unified type value is invalid.

[0039] Step 104: In response to the loading instruction of the running interface, obtain the current value of the target object according to the mapping information, and determine the display content of the control according to the current value; In one or more possible embodiments, the binding source includes: constants, tool node running states, global variables, process inputs, tool inputs, process outputs, tool outputs, and executed commands; obtaining the current value of the target object according to the mapping information includes: when the binding source of the target object is a constant, using the constant as the current value of the target object; when the binding source of the target object is a tool node running state, using the tool node running state as the current value of the target object; when the binding source of the target object is a global variable, process input, tool input, process output, or tool output, locating the target object according to the location information and reading the current value of the target object; When the binding source of the target object is an execution command, the target object is a command object. The command object includes at least a command type, a command target, command parameters, and a current command state. The command object is located according to the location information, and the command state currently saved by the command object or the execution context state returned by the process executor is read as the current value of the target object. The current command state includes at least one of the following: executable state, disabled state, executing state, execution completed state, and execution failed state. When the binding source of the target object is process output or tool output and no output parameters are specified, the output value with valid status information in the current output set can be selected as the current value of the target object.

[0040] For example, when the user selects a global variable as the binding source, the system lists all global variables in the current project; when the binding source is a process input, the system lists all input parameters of the current process, and so on. When generating the binding candidate list, it can automatically filter runtime temporary variables and internal cached data, retaining only valid data units that can be bound to the operator interface, thus preventing the operator interface from binding to unstable internal temporary values. After the user selects a target object from the binding candidate list by clicking or touching, the system automatically reads the metadata information of the target object, including the process name, tool name, parameter name, data type, and status information to which the target object belongs, and generates the mapping information of the controls in the runtime interface. The mapping information includes the stable control identifier, the attribute to be bound, the binding source, the process name, the tool name, the parameter name, the data type, and the location information of the target object, etc., thereby realizing the structured association between the control and the visual process model, replacing the configuration method in the traditional solution that requires the user to manually fill in the variable name, tool name, or parameter name, fundamentally avoiding the binding mismatch problem caused by manual input errors or changes in the process structure.

[0041] Step 105: In response to the editing operation on the editable control, write the edited content into the target object; In one or more possible embodiments, writing the edited content to the target object in response to an editing operation on the editable control specifically includes: converting the edited content into a value of the corresponding type according to the data type of the target object; writing the converted value into the target object; and setting the binding source of the target object to a constant, and clearing the original upstream node reference, variable reference, and index object reference. Specifically, when an operator modifies the input content of an editable control in the runtime interface and triggers a submit event (e.g., pressing the Enter key or clicking the Confirm button), the system first retrieves the mapping information currently bound to the editable control. Based on the type label of the target object in the mapping information, the expected data type of the target object is determined. The type label is determined when the target object is created; for example, if the target object is the minimum matching score parameter of a geometry search tool, its type label is a floating-point number. Then, based on the determined data type, the corresponding type conversion engine is called to convert the operator's original text content into a value corresponding to that data type. If the format of the input content does not match the target data type, a type conversion failure message is returned to the runtime interface, and no write operation is performed. If the conversion is successful, the converted value is written to the storage space corresponding to the target object in the current runtime instance, completing the data update. Furthermore, when the target object is a tool input, while writing the new value, the value source attribute of the input parameter is set to a constant source, and the original upstream node reference, variable reference, and index object reference of the input parameter are cleared to prevent data conflicts or overwriting caused by the input parameter being controlled by multiple sources simultaneously. Through the above methods, the unified type conversion and write-back mechanism can ensure that the data entered by the operator in the running interface can be safely and accurately transmitted to the target object of the visual process model. At the same time, by forcibly setting it as a constant source and clearing old references, the parameter uncertainty caused by multi-source control is eliminated, thereby ensuring the stability and predictability of process execution.

[0042] Step 106: In response to the triggering operation of the command control in the running interface, execute the current process according to the dependency relationship between each tool node, and update the display content of the control after execution.

[0043] In one or more possible embodiments, in response to a trigger operation on a command control in the running interface, the mapping information corresponding to the command control is first read, and it is determined whether it is an execution command based on the binding source in the mapping information. If it is an execution command, the command object is obtained based on the positioning information. The command object includes a command type, a command target, and command parameters. When the command type is to execute the current process, the executor of the current process of the platform is invoked, and the executor executes the current process according to the dependency relationship between each tool node. During the execution process, the executor determines the tool node with an in-degree of zero as the starting execution node based on the dependency relationship between each tool node. After each tool node is executed, the in-degree of its downstream tool node is updated, and when the in-degree of the downstream tool node becomes zero, the downstream tool node is added to the execution queue. The executor controls the number of tool nodes executed in parallel through concurrency limit parameters, propagates the stop command through cancellation tokens, and realizes the pause and resumption during the execution process through pause gating. When a branch node is not hit or a tool node executes incorrectly, the branch node or the successor node of the tool node is marked as canceled. After the current process is completed, the current and historical values ​​of the output parameters of each tool node are refreshed, the error list, execution statistics and logs are refreshed, and each control in the running interface is triggered to re-execute the binding resolution according to its own mapping information, obtain the updated current value of the target object from the visual process model, and update the display content of each control; wherein, when the control is an image display control, the image display control reads the current image from the tool output or global variable of the bound tool node, and overlays the display object generated during the execution of the current process onto the current image, the display object including at least one of region object, contour object, text object and template matching result object.

[0044] In one or more possible embodiments, upon completion of execution, in response to a project save operation, the process structure, tool nodes, input / output definitions, global variables, layout files of the running interface, and control mapping information are saved as a whole as a project file. During saving, temporary images, temporary global variables, and current output values ​​generated during runtime are stripped away, retaining only default values, mapping information, and lightweight parameters. The saving process employs temporary file writing and backup file methods to reduce the risk of project corruption due to write failures. In response to a project load operation, the visual process model and mapping information of each control are reconstructed based on the saved project file, restoring the running interface to its binding state before saving and enabling it to continue updating and displaying as the process executes.

[0045] In one or more possible embodiments, within the SmartVision intelligent vision platform project, the runtime interface designer provides a toolbox of display controls and functional controls. Display controls may include text display, image display, result display, numeric input, and text input; functional controls may include buttons, switches, subforms, runtime flow controls, and quick outline matching controls. The designer saves the runtime interface XAML and control binding definitions, and loads them with the project through the runtime interface manager.

[0046] For example, in a typical inspection project, an engineer can first establish an inspection workflow of "image source - fast contour matching - result display". Then, in the running interface, a numeric input control is placed and bound to the minimum matching score input of the fast contour matching tool; a command button is placed and bound to execute the current workflow; an image display control is placed and bound to the workflow output image or the tool's output result display object; and a result display control is placed and bound to the matching score or program running status. When the operator modifies the numeric input control, the system writes it back to the tool input as a floating-point number; when the command button is clicked, the system executes the current workflow; after the workflow ends, the image display control refreshes the current image and overlays the matched contour, and the result display control refreshes the matching score or OK / NG status.

[0047] The fast contour matching tool uses a trained shape template to find matching targets in the input image. It can output the position, angle, score, and index of the first target, as well as a list of multiple target positions, angles, scores, and indices, and generate a matching result display object. The result display object can include the template feature contour and a matching center cross, which can be overlaid on the image display control. If the template is not loaded, the matching score is below a threshold, or no target is found, the tool returns a failure result and sets corresponding error information; the running interface and error panel can be displayed synchronously. The above implementation is not limited to the fast contour matching tool. For QR code recognition, blob analysis, geometric measurement, alignment, communication transmission, light source control, or script tools, as long as their input and output are bound through a unified type value and mapping information, the running interface can use the same mechanism to generate candidates, parse the current value, write back parameters, and refresh according to the process.

[0048] This application also provides a device for binding the runtime interface process, such as... Figure 2 As shown, it includes: The model acquisition module 201 is used to acquire a visual process model; the visual process model includes at least one process, and each process includes multiple tool nodes. The candidate acquisition module 202 is used to scan the bindable data units in the visual process model corresponding to the binding source selected by the user, and generate a binding candidate list based on the scanning results; The mapping acquisition module 203 is used to respond to the selection operation of the target object in the binding candidate list, read the metadata information of the target object, and generate the mapping information of the control in the running interface based on the metadata information; the mapping information includes the attribute to be bound of the control, the positioning information of the target object in the visual process model, and the binding source, wherein the target object is a bindable data unit in the visual process model; the target object contains multiple data types and status information, which are used for data transfer between the multiple tool nodes; The binding parsing module 204 is used to respond to the loading instruction of the running interface, obtain the current value of the target object according to the mapping information, and determine the display content of the control according to the current value; The data write-back module 205 is used to respond to the editing operation on the editable control, convert the edited content according to the data type of the target object and write it to the target object; The process execution module 206 is used to respond to the trigger operation of the command control in the running interface, execute the current process according to the dependency relationship between each tool node, and update the display content of the control after execution.

[0049] Based on the same inventive concept, this application also provides a runtime interface process binding device, including at least one processor; and a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to execute the above-described runtime interface process binding method.

[0050] like Figure 3 As shown, the device includes a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, memory 302, and communication interface 303 are interconnected via the bus 304.

[0051] Processor 301 is used to read instructions from memory 302 and execute them, so that at least one processor can execute the runtime interface flow binding method provided in the above embodiments.

[0052] The memory 302 is used to store various instructions and programs of the running interface flow binding method provided in the above embodiments.

[0053] Bus 304 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0054] Processor 301 can be a central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), or any combination of CPU, NP, and GPU. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0055] In addition, this application also provides a storage medium, such as Figure 4 As shown, the storage medium stores a computer program that enables the computer to perform any of the methods described in the above embodiments.

[0056] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) 401 and / or cache memory 402, and may further include read-only memory (ROM) 403.

[0057] The memory may also include a program / utility 405 having a set (at least one) of program modules 404, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0058] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0062] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for binding runtime interface flow, characterized in that, include: Obtain the visual process model; The visual process model includes at least one process, and each process includes multiple tool nodes; Based on the binding source selected by the user, scan the bindable data units in the visual process model corresponding to the binding source, and generate a binding candidate list based on the scan results; wherein, the binding source includes: constants, tool node running status, global variables, process inputs, tool inputs, process outputs, tool outputs, and execution commands; In response to the selection operation of a target object in the binding candidate list, the metadata information of the target object is read, and mapping information of the controls in the running interface is generated based on the metadata information. The mapping information includes the binding attribute of the control, the positioning information of the target object in the visual flow model, and the binding source. The target object is a data unit with a unified type value that can be bound, containing multiple data types and status information for data transfer between multiple tool nodes. The data types of the target object include numeric, string, list, image, region, contour, template, geometric, and device reference types. When the data type of the target object is a list type, a single element is obtained from the list type according to a preset index parameter, and the data type of the single element is used as the converted data type. The status information of the target object includes three states: valid, invalid, and null, which indicate whether the data corresponding to the unified type value is available at the current moment. In response to the loading instruction of the running interface, the current value of the target object is obtained according to the mapping information, and the display content of the control is determined according to the current value; In response to the editing operation on the editable control, the edited content is written to the target object, the binding source of the target object is set to a constant, and the original upstream node reference, variable reference and index object reference are cleared; In response to a trigger operation on the command control in the running interface, the current process is executed according to the dependencies between the tool nodes, and the display content of the control is updated after execution is completed.

2. The method according to claim 1, characterized in that, Writing the edited content to the target object includes: The edited content is converted into a value of the corresponding type based on the data type of the target object; Write the converted value into the target object.

3. The method according to claim 1, characterized in that, The step of obtaining the current value of the target object based on the mapping information includes: When the binding source of the target object is a constant, the constant is used as the current value of the target object; When the binding source of the target object is the running state of a tool node, the running state of the tool node is used as the current value of the target object; When the binding source of the target object is a global variable, process input, or tool input, the target object is located according to the location information, and the current value of the target object is read. When the binding source of the target object is an execution command, the command object corresponding to the execution command is located according to the location information, and the current command state of the command object is read as the current value of the target object; the current command state includes at least one of the following: executable state, disabled state, executing state, execution completed state, and execution failed state; When the binding source of the target object is process output or tool output and no output parameters are specified, the output value with valid status information in the current output set is selected as the current value of the target object.

4. The method according to claim 1, characterized in that, Update the display content of the control based on the updated current value, including: When the control is an image display control, the display object is obtained from the updated current value and displayed. The display object includes at least one of a region object, a contour object, a text object, and a template matching result object.

5. A device for binding a runtime interface flow, characterized in that, include: The model acquisition module is used to acquire the visual process model; The visual process model includes at least one process, and each process includes multiple tool nodes; The candidate acquisition module is used to scan the bindable data units in the visual process model corresponding to the binding source selected by the user, and generate a binding candidate list based on the scanning results; wherein, the binding source includes: constants, tool node running status, global variables, process inputs, tool inputs, process outputs, tool outputs, and execution commands; The mapping acquisition module is used to respond to the selection operation of the target object in the binding candidate list, read the metadata information of the target object, and generate the mapping information of the controls in the running interface based on the metadata information. The mapping information includes the attribute to be bound to the control, the positioning information of the target object in the visual flow model, and the binding source. The target object is a data unit with a unified type value that can be bound, which contains multiple data types and status information for data transfer between multiple tool nodes. The data types of the target object include numeric type, string type, list type, image type, region type, contour type, template type, geometric type, and device reference type. When the data type of the target object is a list type, a single element is obtained from the list type according to a preset index parameter, and the data type of the single element is used as the converted data type. The status information of the target object includes three states: valid, invalid, and null, which are used to indicate whether the data corresponding to the unified type value is available at the current time. The binding parsing module is used to respond to the loading command of the running interface, obtain the current value of the target object according to the mapping information, and determine the display content of the control according to the current value; The data write-back module is used to respond to the editing operation of the editable control, write the edited content to the target object, set the binding source of the target object to a constant, and clear the original upstream node reference, variable reference and index object reference; The process execution module is used to respond to the trigger operation of the command control in the running interface, execute the current process according to the dependency relationship between each tool node, and update the display content of the control after execution.

6. A device-bound interface process, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-4.

7. A storage medium, characterized in that, The storage medium stores a computer program that enables a computer to perform the method as described in any one of claims 1-4.

Images