Visual software development system, method, apparatus, computer storage medium and computer program

Abstract

The visual software development system, method, apparatus and computer storage medium belong to the machine vision technical field. The application interface layer in the visual software development system is used to display a module add control, and the module add control is used to introduce module data of a third-party module into a data processing layer. Thus, the modules corresponding to the N module display controls displayed in the application interface layer may include third-party modules. As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system, which allows users to develop different third-party modules according to different business needs and mount the developed third-party modules into the software development system provided in the embodiment of the present application. This allows the subsequently developed visual software to be more adapted to the personalized needs of different businesses.

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application bearing application number 202110251760.X and entitled "Visual software development system, method, apparatus and computer storage medium" filed with the State Intellectual Property Office of the People's Republic of China on March 8, 2021, the entire contents of which are hereby incorporated by reference into this application.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the field of machine vision technology, and more particularly to visual software development systems, methods, apparatus, and computer storage media. [Background technology]

[0003] Machine vision is a technology that uses machines to measure or judge instead of human eyes. In the field of machine vision technology, instead of traditional language codes, graphic flow charts realize certain functions as software, so that more users can quickly develop visual software that meets their needs. In the field of machine vision, visual software is also called machine vision software. Visual software needs to be realized based on a visual software development system. Since different visual software meets different needs of users, currently, an important problem to be solved for visual software development systems is how to build a visual software development system so that visual software created based on the visual software development system meets user needs.

[0004] Conventionally, in order for the developed visual software to meet user needs, a visual software development system is usually constructed for each type of visual software. In this way, the visual software development system has a specialization, and the visual software developed based on the visual software development system can meet the specified user needs. However, since the conventional visual software development system does not have versatility, it is necessary to construct different visual software development systems for different types of visual software. Furthermore, when user needs change, it is difficult for the visual software developed based on the existing visual software development system to meet the latest needs of users. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present application provide a visual software development system, method, apparatus and computer storage medium for developing visual software to meet different business needs. [Means for solving the problem]

[0006] In one embodiment, a visual software development system includes an application interface layer and a data processing layer, the application interface layer is used to display N module display controls, the N module display controls respectively corresponding to N modules, and module data for each of the N modules is stored in the data processing layer, the module data including execution logic of the corresponding module; the application interface layer is further used to display a module addition control, the module addition control is used to introduce module data of a third party module into the data processing layer, the N modules include the third party module, and the N is a positive integer greater than or equal to 1. The present invention provides a visual software development system, wherein the application interface layer is further used for displaying a flowchart in response to a user's first operation on one or more module display controls among the N module display controls, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls, and the data processing layer is used for calling and executing module data of modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer, obtaining a data processing result of the flowchart, and returning the data processing result to the application interface layer.

[0007] In one possible embodiment, the application interface layer is used to, in response to a second operation on the module addition control, obtain module data of the third-party module installed by the user through the module addition control and send the module data of the installed third-party module to the data processing layer, and when the data processing layer determines that the module data of the installed third-party module satisfies the reference rule, store the module data of the installed third-party module and return a module loading success message to the application interface layer, where the module loading success message indicates that the loading of the third-party module has been successful.

[0008] In one possible embodiment, the reference rule includes a design method of an input / output port of a third party module and / or a packaging method of module data of a third party module, the design method of the input / output port of the third party module includes a format of input data and / or output data of the third party module, and the packaging method of the module data of the third party module indicates a storage format of the module data of the third party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file.

[0009] In one possible embodiment, the application interface layer is adapted to display a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule.

[0010] In one possible embodiment, the application interface layer is further used for displaying a module configuration interface of a first module corresponding to a first module display control in response to a fourth operation by a user on a first module display control among the N module display controls, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls instructs the user to edit parameters in the execution process of the first module; the application interface layer is further used for obtaining parameters edited by the user through any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls and sending the edited parameters to the data processing layer; the data processing layer is further used for updating module data of the first module based on the edited parameters; and if the flowchart includes the first module, the data processing layer is further used for updating a data processing result of the flowchart based on the updated module data of the first module and returning the updated data processing result to the application interface layer.

[0011] In one possible embodiment, when the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicates the program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

[0012] In one possible embodiment, the data processing layer is further used for generating a first code package based on the flowchart, the first code package instructing the execution logic of each module included in the flowchart and the execution order between each module included, the data processing layer is further used for controlling the application interface layer to display a first code control corresponding to the first code package, the first code control is used for instructing the user to download the first code package, the first code package supports being introduced into a third-party system and displays the flowchart after being executed in the third-party system.

[0013] In one possible embodiment, the application interface layer is further used for displaying an execution interface display control, the execution interface display control instructs displaying an execution interface for the flowchart, the execution interface is used for displaying execution information of the flowchart, the execution information of the flowchart indicates intermediate data in the execution process of the flowchart, and the application interface layer is further used for displaying the execution interface in response to a sixth operation on the execution interface display control, such that the user views the intermediate data in the execution process of the flowchart based on the execution interface.

[0014] In one possible embodiment, the application interface layer is further used for displaying an execution interface configuration control, the execution interface configuration control instructing the display of an execution interface configuration interface, the execution interface configuration interface being used to configure content displayed in the execution interface; the application interface layer is further used for displaying the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, one of the plurality of execution interface configuration sub-controls having a launch option; the application interface layer is further used for displaying a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and for binding the first execution interface configuration sub-control to a module in the flowchart in response to a ninth operation on the content source option, and the execution information displayed in the execution interface includes data of the module bound to the first execution interface configuration sub-control.

[0015] In one possible embodiment, the data processing layer is further used for deriving the program code of the execution interface as a second code package, and the data processing layer is further used for controlling the application interface layer to display a second code control corresponding to the second code package, the second code control is for instructing the user to download the second code package, and the second code package supports being introduced into a third-party system, and displays the execution interface after being executed in the third-party system.

[0016] In one possible embodiment, the data processing layer includes a data scheduling layer and a functional module layer, in which module data of the N modules are stored, and the data scheduling layer is further used for sequentially calling and executing the module data of the modules corresponding to each of the one or more module display controls from the functional module layer according to the execution order between the modules corresponding to each of the one or more module display controls in the flowchart, and obtaining the data processing result.

[0017] In one possible embodiment, the data processing layer is used for storing module data of M modules, where M is a positive integer greater than or equal to N, and the application interface layer is further used for displaying a module registration control, and the application interface layer is further used for displaying the N module display controls in response to the N modules selected by the user from the M modules via the module registration control.

[0018] In one possible embodiment, the N modules are modules that the data processing layer verifies and validates, and the modules that it verifies and validates are corresponding modules that the user has permission to use.

[0019] In one possible embodiment, the application interface layer is further used for displaying a call control for any of the module display controls, the call control indicating that a user allows a third party system to call the module corresponding to any of the module display controls, the application interface layer is further used for displaying a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control, and the application interface layer is further used for obtaining module data of the module corresponding to any of the module display controls from the data processing layer based on the port link when a data call request for the port link sent from the third party system is detected, and transmitting the obtained module data to the third party system.

[0020] In another aspect, a visual software development method is applied to a visual software development system including an application interface layer and a data processing layer, the method comprising: the application interface layer displays N module display controls, the N module display controls respectively corresponding to N modules; module data for each of the N modules is stored in the data processing layer, the module data includes execution logic of the corresponding module; the application interface layer further displays a module add control, the module add control is used to introduce module data of a third party module into the data processing layer, the N modules include the third party module, and the N is 1 a positive integer equal to or greater than N; the application interface layer, in response to a user's first operation on one or more module display controls among the N module display controls, displays a flowchart, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls; and the data processing layer, in response to the flowchart displayed on the application interface layer, calls and executes module data of modules corresponding to each of the one or more module display controls, obtains a data processing result of the flowchart, and returns the data processing result to the application interface layer.

[0021] In one possible embodiment, the method further includes: in response to a second operation on the module addition control, the application interface layer obtains module data of the third party module installed by the user through the module addition control and sends the module data of the installed third party module to the data processing layer; and if the data processing layer determines that the module data of the installed third party module satisfies a reference rule, it stores the module data of the installed third party module and returns a module load success message to the application interface layer, the module load success message indicating that the loading of the third party module was successful.

[0022] In one possible embodiment, the reference rule includes a design method of an input / output port of a third party module and / or a packaging method of module data of a third party module, the design method of the input / output port of the third party module includes a format of input data and / or output data of the third party module, and the packaging method of the module data of the third party module indicates a storage format of the module data of the third party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file.

[0023] In one possible embodiment, the method further includes the application interface layer displaying a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule.

[0024] In one possible embodiment, the method further includes: in response to a fourth operation by the user on a first module display control among the N module display controls, the application interface layer displays a module configuration interface of a first module corresponding to the first module display control, the module configuration interface including a plurality of module configuration controls, and one of the plurality of module configuration controls instructs the user to edit parameters in an execution process of the first module; in response to a fifth operation by the application interface layer on any one of the module configuration controls, the application interface layer obtains parameters edited by the user through any one of the module configuration controls and transmits the edited parameters to the data processing layer; and the data processing layer updates module data of the first module based on the edited parameters, and if the flowchart includes the first module, the data processing layer updates a data processing result of the flowchart based on the updated module data of the first module and returns the updated data processing result to the application interface layer.

[0025] In one possible embodiment, when the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicates a program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

[0026] In one possible embodiment, the method further includes: the data processing layer generates a first code package based on the flowchart, the first code package instructing the execution logic of each module included in the flowchart and the execution order between each module included; the data processing layer controls the application interface layer so that the application interface layer displays a first code control corresponding to the first code package, the first code control is used to instruct the user to download the first code package, the first code package supports being introduced into a third-party system, and displays the flowchart after being executed in the third-party system.

[0027] In one possible embodiment, the method further includes the application interface layer displaying an execution interface display control, the execution interface display control instructing to display an execution interface for the flowchart, the execution interface being used to display execution information of the flowchart, the execution information of the flowchart indicating intermediate data in the execution process of the flowchart, and the application interface layer, in response to a sixth operation on the execution interface display control, displaying the execution interface such that the user views the intermediate data in the execution process of the flowchart based on the execution interface.

[0028] In one possible embodiment, the method further includes: the application interface layer displays an execution interface configuration control, the execution interface configuration control instructs the display of an execution interface configuration interface, the execution interface configuration interface being used to configure content displayed in the execution interface; the application interface layer displays the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, one of the plurality of execution interface configuration sub-controls having a launch option; the application interface layer displays a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and binds the first execution interface configuration sub-control to a module in the flowchart in response to a ninth operation on the content source option, and the execution information displayed in the execution interface includes data of the module bound to the first execution interface configuration sub-control.

[0029] In one possible embodiment, the method further includes: the data processing layer deriving the program code of the execution interface as a second code package; and the data processing layer controlling the application interface layer so that the application interface layer displays a second code control corresponding to the second code package, the second code control being used to instruct the user to download the second code package, the second code package supporting introduction into a third-party system, and displaying the execution interface after being executed in the third-party system.

[0030] In one possible embodiment, the data processing layer includes a data scheduling layer and a functional module layer, in which module data of the N modules is stored, and the data processing layer, in response to the flowchart displayed on the application interface layer, calls and executes module data of modules corresponding to each of the one or more module display controls, and obtains a data processing result of the flowchart, including the data scheduling layer sequentially calling and executing module data of modules corresponding to each of the one or more module display controls from the functional module layer in accordance with an execution order between modules corresponding to each of the one or more module display controls in the flowchart, and obtains the data processing result.

[0031] In one possible embodiment, the method further includes the application interface layer displaying a module registration control, the application interface layer displaying the N module display controls in response to the N modules selected by the user from the M modules through the module registration control, module data of the M modules being stored in the data processing layer, and M being a positive integer greater than or equal to N.

[0032] In one possible embodiment, the N modules are modules that the data processing layer verifies and validates, and the modules that it verifies and validates are corresponding modules that the user has permission to use.

[0033] In one possible embodiment, the method further includes: the application interface layer displays a call control for any of the module display controls, the call control indicating that a user allows a third party system to call the module corresponding to any of the module display controls; the application interface layer is further used for displaying a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control; when the application interface layer detects a data call request for the port link sent from the third party system, the application interface layer obtains module data of the module corresponding to any of the module display controls from the data processing layer based on the port link, and sends the obtained module data to the third party system.

[0034] In another aspect, a visual software development apparatus is provided, which is constructed in a visual software development system including an application interface layer and a data processing layer, the apparatus including a display module and an execution module, the display module is used to control the application interface layer to display N module display controls, each of the N module display controls corresponding to N modules, module data for each of the N modules is stored in the data processing layer, and the module data includes execution logic of the corresponding module, the application interface layer further displays a module addition control, and the module addition control is used to introduce module data of a third party module into the data processing layer, the N modules include the third party module, and the N is a positive integer greater than or equal to 1, the display module is further used for controlling the application interface layer to display a flowchart in response to a user's first operation on one or more module display controls among the N module display controls, the flowchart instructing an execution order among modules corresponding to each of the one or more module display controls, and the execution module is used for controlling the data processing layer to call and execute module data of modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer, obtain a data processing result of the flowchart, and return the data processing result to the application interface layer.

[0035] In one possible embodiment, the device further includes an acquisition module and a determination module, wherein the acquisition module is used to control the application interface layer to acquire module data of a third-party module introduced by a user through the module addition control in response to a second operation on the module addition control, and to send the module data of the introduced third-party module to the data processing layer; and when the determination module determines that the module data of the introduced third-party module satisfies a reference rule, the determination module is used to control the data processing layer to store the module data of the introduced third-party module and return a module load success message to the application interface layer, wherein the module load success message indicates that the loading of the third-party module is successful.

[0036] In one possible embodiment, the reference rule includes a design method of an input / output port of a third party module and / or a packaging method of module data of a third party module, the design method of the input / output port of the third party module includes a format of input data and / or output data of the third party module, and the packaging method of the module data of the third party module indicates a storage format of the module data of the third party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file.

[0037] In one possible embodiment, the display module is further used to control the application interface layer to display a module configuration file in response to a third operation on the module addition control, and the module configuration file indicates a reference rule.

[0038] In one possible embodiment, the display module is further used for controlling the application interface layer to display a module configuration interface of a first module corresponding to a first module display control among the N module display controls in response to a fourth operation by a user on a first module display control, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls is used for instructing the user to edit parameters in the execution process of the first module; the apparatus further includes an acquisition module and an update module, the acquisition module is used for controlling the application interface layer to acquire parameters edited by the user through any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls, and to send the edited parameters to the data processing layer; the update module is used for controlling the data processing layer to update module data of the first module based on the edited parameters; and if the flowchart includes the first module, the data processing layer updates a data processing result of the flowchart based on the updated module data of the first module, and returns the updated data processing result to the application interface layer.

[0039] In one possible embodiment, when the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicates a program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

[0040] In one possible embodiment, the apparatus further includes a generation module, which is used to control the data processing layer to generate a first code package based on the flowchart, and the first code package indicates the execution logic of each module included in the flowchart and the execution order between each module included; the display module is further used to control the data processing layer, which is further used to control the application interface layer to display a first code control corresponding to the first code package, and the first code control is used to instruct the user to download the first code package, and the first code package supports being introduced into a third-party system and displays the flowchart after being executed in the third-party system.

[0041] In one possible embodiment, the display module is further used for controlling the application interface layer to display an execution interface display control, the execution interface display control instructs displaying an execution interface for the flowchart, the execution interface is used for displaying execution information of the flowchart, the execution information of the flowchart indicates intermediate data in the execution process of the flowchart, and the display module is further used for controlling the application interface layer to display the execution interface in response to a sixth operation on the execution interface display control so that the user can view the intermediate data in the execution process of the flowchart based on the execution interface.

[0042] In one possible embodiment, the display module is further used to control the application interface layer to display an execution interface configuration control, the execution interface configuration control instructs the display of an execution interface configuration interface, and the execution interface configuration interface is used to configure content displayed in the execution interface; the display module is further used to control the application interface layer to display the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, one of the plurality of execution interface configuration sub-controls having a launch option; the display module is further used to control the application interface layer to display a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and to bind the first execution interface configuration sub-control to a module in the flowchart in response to a ninth operation on the content source option; and the execution information displayed in the execution interface includes data of the module bound to the first execution interface configuration sub-control.

[0043] In one possible embodiment, the device further includes a derivation module, which is used to control the data processing layer to derive the program code of the execution interface as a second code package; the display module is further used to control the data processing layer, which is further used to control the application interface layer to display a second code control corresponding to the second code package, which is used to instruct the user to download the second code package, and which supports being introduced into the third-party system and displays the execution interface after being executed in the third-party system.

[0044] In one possible embodiment, the data processing layer includes a data scheduling layer and a functional module layer, in which module data of the N modules are stored, and the execution module is used to control the data scheduling layer to sequentially call and execute module data of modules corresponding to each of the one or more module display controls from the functional module layer according to an execution order between modules corresponding to each of the one or more module display controls in the flowchart, and obtain the data processing result.

[0045] In one possible embodiment, the display module is further used for controlling the application interface layer to display a module registration control, and the application interface layer displays the N module display controls in response to the N modules selected by the user from the M modules through the module registration control, and the data processing layer stores module data of the M modules, and M is a positive integer greater than or equal to N.

[0046] In one possible embodiment, the N modules are modules that the data processing layer verifies and validates, and the modules that it verifies and validates are corresponding modules that the user has permission to use.

[0047] In one possible embodiment, the display module is further used to control the application interface layer to display a call control for any of the module display controls, the call control instructing a user to allow a third party system to call the module corresponding to any of the module display controls, and the display module is further used to control the application interface layer to display a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control, and the device further includes an acquisition module, which is used to control the application interface layer to acquire module data of the module corresponding to any of the module display controls from the data processing layer based on the port link when it detects a data call request for the port link sent from the third party system, and to send the acquired module data to the third party system.

[0048] In another aspect, there is provided a visual software development apparatus including a processor and a memory storing instructions executable by the processor, the processor being configured to perform any of the steps of the visual software development method described above.

[0049] In another aspect, there is provided a computer readable storage medium storing instructions which, when executed by a processor, perform any of the steps of the visual software development method described above.

[0050] In another aspect, there is further provided a computer program product comprising commands which, when executed on a computer, cause the computer to perform any of the steps of the visual software development method described above.

[0051] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least the following: The embodiments of the present application include providing a visual software development system, in which an application interface layer is used to display a module add control, and the module add control is used to introduce module data of a third-party module into a data processing layer. In this way, a module corresponding to the N module display controls displayed on the application interface layer may include a third-party module. Therefore, in the embodiments of the present application, a user can introduce a third-party module through the module add control at any time according to business needs, and then display a flow chart configured by the user through the application interface layer based on the third-party module, and the flow chart becomes a visual software configured by the user, and the data processing layer executes the flow chart, thereby completing the development flow of the visual software.

[0052] As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system that allows users to develop different third-party modules according to different business needs and mount the developed third-party modules in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can be more adapted to the personalized needs of different businesses. In this way, it is not necessary to build different visual software development systems for different types of visual software, and different types of visual software can be developed with the visual software development system provided in the embodiment of the present application.

[0053] In addition, even if business needs or user needs change, a user of the visual software development system provided in the embodiment of the present application can develop a new third-party module, and then mount the developed new third-party module in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can quickly adapt to the latest needs of the user. [Brief description of the drawings]

[0054] In order to more clearly explain the technical solutions of the embodiments of the present application and the prior art, the drawings necessary for the embodiments and the prior art will be briefly described below. Of course, the drawings described below are only part of the embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings. [Figure 1] FIG. 1 is a schematic diagram of the architecture of a visual software development system provided in an embodiment of the present application. [Diagram 2] FIG. 2 is a schematic diagram of a main interface displayed on an application interface layer provided in an embodiment of the present application. [Diagram 3] FIG. 3 is a schematic diagram of a flow chart provided in an embodiment of the present application. [Figure 4] FIG. 4 is a schematic diagram of the data scheduling layer provided in the embodiment of the present application invoking module data of each module in the function module layer. [Diagram 5] FIG. 5 is a schematic diagram of a module provided in an embodiment of the present application. [Figure 6] FIG. 6 is a schematic diagram of module loading and module release provided in an embodiment of the present application. [Figure 7] FIG. 7 is a schematic diagram of the format of module data of a third-party module provided in an embodiment of the present application. [Figure 8] FIG. 8 is a schematic diagram of the configuration process and derived code of the execution interface provided in the embodiment of the present application. [Figure 9]FIG. 9 is a flow diagram of module registration provided in an embodiment of the present application. [Figure 10] FIG. 10 is a flow chart of a module loading and module registration method provided in an embodiment of the present application. [Figure 11] FIG. 11 is a flow chart of the method for constructing the flow chart provided in an embodiment of the present application. [Figure 12] FIG. 12 is a schematic diagram of the configuration of a visual software development device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0055] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in more detail below with reference to the drawings and examples. It is clear that the described examples are only some of the embodiments of the present invention, and are not all of the embodiments. All other embodiments that a person skilled in the art can obtain based on the embodiments of the present invention are all included in the protection scope of the present invention.

[0056] The term "multiple" as referred to in this specification is understood to mean two or more than two. In the description of this application, " / " means or unless otherwise specified, for example, A / B may represent A or B. In this specification, "and / or" is merely a related relationship that represents a related symmetry, and may represent three relationships, for example, A and / or B may represent three situations: a situation in which A exists alone, a situation in which A and B exist simultaneously, and a situation in which B exists alone. In addition, in order to clearly explain the technical solutions of the embodiments of this application, terms such as "first" and "second" are used in the embodiments of this application to distinguish between the same or similar items that have basically the same functions and roles. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution procedure, and terms such as "first" and "second" do not necessarily limit the different.

[0057] Before describing the visual software development system provided in the embodiment of the present application in detail, the application scenario of the embodiment of the present application will be described in detail first.

[0058] At present, the main principle of machine vision is to convert the detection target into an image signal by an image sampling device, then perform dedicated image processing on the image signal, analyze and obtain the characteristic information of the detection target based on the image processing result, and further determine the detection result for the target based on the characteristic information of the target.

[0059] Based on the above-mentioned operation principle of machine vision, currently, machine vision technology is widely applied in various fields. In industrial technology, machine vision technology can realize the detection of the contour and surface defects of products, such as the detection of wood processing, the detection of metal surfaces, the detection of printed circuit board defects, and the automatic identification of welding seam defects. In medical field, machine vision technology can assist doctors in diagnosing the size, shape and abnormality of pathogens in the human body, thereby providing evidence for effective diagnosis. In traffic monitoring technology field, machine vision technology can realize the automatic identification of the license plates of vehicles that have committed traffic violations, run in the wrong direction, etc. The above is merely an exemplary description of the application fields of machine vision technology, and the embodiments of the present application are not limited to the specific application fields of machine vision technology.

[0060] The vision software is software that realizes the above machine vision technology, that is, the vision software can realize a specific detection function for the above goal. Furthermore, in order to facilitate developers to develop vision software quickly, currently in machine vision technology, developers can develop vision software by graphical flowcharts instead of traditional language codes. That is, the vision software is graphically displayed in the form of a flowchart, so that developers can easily understand the internal execution logic of the vision software. In the above vision software flowchart, the flowchart is a flowchart in which multiple modules are connected according to a certain connection relationship. Each module is used to realize one specific sub-function. The connection relationship between each module indicates the execution order of each module. Therefore, the flowchart can indicate the internal execution logic of the vision software.

[0061] The visual software development system provided in the embodiment of the present application may be applied to the above-mentioned flowcharted visual software development scenario, with the objective of providing a general-purpose visual software development system that allows developers to further create the required visual software according to business needs.

[0062] For ease of explanation, the visual software development system according to the embodiment of the present application will be described in detail here first.

[0063] 1 is an architecture schematic diagram of a visual software development system provided in an embodiment of the present application. As shown in FIG. 1, the visual software development system 100 includes an application interface layer 101 and a data processing layer 102.

[0064] The application interface layer 101 is used to display N module display controls, where N is a positive integer equal to or greater than 1, and the N module display controls respectively correspond to N modules, and module data for each of the N modules is stored in the data processing layer 102, and the module data includes execution logic for the corresponding module. The application interface layer 101 is further used to display a flowchart in response to a user's first operation on one or more module display controls among the N module display controls, and the flowchart indicates an execution order between the modules corresponding to each of the one or more module display controls. The data processing layer 102 is used to call and execute the module data of the modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer 101, obtain a data processing result of the flowchart, and return the data processing result to the application interface layer 101.

[0065] The first operation may be a drag operation, a click operation, etc. A user (e.g., a developer) may configure a flowchart through module display controls corresponding to the N modules displayed on the application interface layer, and the flowchart becomes visual software that meets a certain business need. The flowchart includes one or more modules among the N modules and the connections between the one or more modules. The connections between the modules in the flowchart indicate the execution order between the modules. For example, the flowchart shows that if module A points to module B, the data processing layer first executes the associated logic based on the module data of module A, and then executes the associated logic based on the module data of module B.

[0066] In the process of configuring a flowchart in the application interface layer, the user obtains a data processing result of the flowchart by calling and executing module data of the modules in the flowchart based on the flowchart displayed on the application interface layer, thereby completing a development flow for the configured flowchart.

[0067] 2 is a schematic diagram of a main interface displayed on an application interface layer provided in an embodiment of the present application. As shown in FIG. 2, the main interface includes functional areas such as a menu area, a visual tool area, a flow editing area, an image rendering area, and a text display area.

[0068] Here, the menu area is used to display various controls for the main interface, such as interface view controls, page area configuration controls, etc.; the visual tool area is used to display each module display control; the flow editing area is used to display a flowchart configured by a user; the user may display the module display control in the visual tool area in the flow editing area by dragging to configure the flowchart; the image rendering area is used to display the image processing result of a module in the flowchart, and the image processing result is displayed in a graphical manner; the text display area is used to display the data processing result of a module in the flowchart, and the data processing result is displayed in a text manner. For example, for a flowchart displayed in the flow editing area, when a user clicks on a module in the flowchart, the image rendering area may display the image processing result of the module in the flowchart, and the text display area may display the data processing result of the module in the flowchart in text format. In addition, the image rendering area and text display area described above are both result display bars for the processing results of a certain module, and the embodiments of the present application are not limited to the display manner of the processing results of a certain module, and may be displayed only in the illustrated manner, only in the text manner, or simultaneously in both the illustrated manner and the text manner.

[0069] As can be seen from the main interface shown in FIG. 2, the application interface layer provided in the embodiment of the present application adopts graphic interaction design to generate a flowchart by dragging the module display control in the visual tool area, and realizes the visual software in the form of a graphical flowchart.

[0070] FIG. 3 is a schematic diagram of a flow chart provided in an embodiment of the present application. As shown in FIG. 3, the general tools on the left side of FIG. 3 are the module display controls in the visual tools area. The right side of FIG. 3 is a user-configured flow chart. The flow chart includes four modules, which are an image source 1 module, a fast feature analysis module, a position correction module, and a circle search module. Here, the image source 1 module is directed to the fast feature analysis module, the fast feature analysis module is directed to the position correction module, and the position correction module is directed to the circle search module. The vision software scheme indicated in the flow chart is to first perform fast feature analysis on the image source 1, then perform position correction, and finally perform circle search. All of these four modules are modules corresponding to the module display controls in the general tools area shown on the left side of FIG. 3.

[0071] It should be noted that the main interface shown in Figures 2 and 3 is merely one example of a main interface provided to the application interface layer. The functional areas displayed in the main interface and the combination manner of different functional areas may be individually created by the user, and will not be described in detail in the embodiments of the present application. In addition, the controls shown in Figures 2 and 3, such as the module display control, have the characteristics of being independently packaged and displayable and configurable. Other controls according to the following embodiments all have this characteristic, and the description will not be repeated here.

[0072] Independent packaging means that the data corresponding to each control is packaged independently in the data processing layer. There is no data interaction between each control, and the data corresponding to all controls is directed to the data processing layer. The data corresponding to each control includes information such as the display size and display color of the control displayed in the application interface layer. In this case, when the data corresponding to a control in the data processing layer changes, the control displayed in the application interface layer changes accordingly. In this way, each control is used independently, and mutual influence between the controls can be avoided.

[0073] The term "configurable display" refers to the fact that the display information, such as the display position, display size, and display color, of each control displayed on the application interface layer can be modified by the user according to the user's own needs. In this way, the controls displayed on the application interface layer can meet the user's needs as much as possible. For example, the user drags a control with a mouse, and the application interface layer can modify the display position of the control in response to the mouse drag event on the control, and store the modified display position of the control in a configuration file corresponding to the control. The configuration file corresponding to the control is stored in the data processing layer. The above-mentioned "configurable display" feature allows the display size, display position, and display color of the controls displayed on the application interface layer to be modified. Optionally, each control has a minimum display size to avoid the display size being too small and affecting the layout of the control in the main interface.

[0074] In the embodiment of the present application, in the system shown in FIG. 1, the application interface layer 101 is used to display a module adding control, and the module adding control is used to introduce module data of a third-party module into the data processing layer. The third-party module may be any functional data packet configured by a user according to business needs. Therefore, based on the system shown in FIG. 1, a user can introduce a third-party module through the module adding control displayed in the application interface layer according to business needs at any time, and then the user can configure a flowchart including the third-party module in the application interface layer and execute the flowchart through the data processing layer, thereby completing the development flow of visual software. That is, based on the system shown in FIG. 1, a user can create visual software according to business needs at any time.

[0075] In one possible embodiment, when the application interface layer detects an operation on the module addition control, it may display a module loading window, and then obtain module data of the third-party module introduced by the user through the module loading window, and store the module data in the data processing layer. Then, the third-party module may be displayed in the application interface layer in the manner of a third-party module display control. That is, the system provided in the embodiment of the present application can support a third-party module developed by a third-party developer to be added to the main interface of the application interface layer. The display of the above-mentioned third-party module in the application interface layer in the manner of a third-party module display control may be realized by a module registration manner. The relevant contents of the module registration will be described in detail in the subsequent embodiment, and will not be described here.

[0076] In addition, when a module display control corresponding to the module data of the data processing layer is added to the main interface, description information for the module corresponding to the added module display control may be further added to the configuration file of the main interface. The description information may include, but is not limited to, information such as a module name, a module icon, and a storage path of the module data corresponding to the module. In this way, when the application interface layer displays a module display control corresponding to the module in the main interface, information such as the module name and module icon of the module may be further displayed.

[0077] As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system that allows users to develop different third-party modules according to different business needs and mount the developed third-party modules in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can be more adapted to the personalized needs of different businesses. In this way, it is not necessary to build different visual software development systems for different types of visual software, and different types of visual software can be developed with the visual software development system provided in the embodiment of the present application.

[0078] In addition, even if business needs or user needs change, a user of the visual software development system provided in the embodiment of the present application can develop a new third-party module, and then mount the developed new third-party module in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can quickly adapt to the latest needs of the user.

[0079] In view of the above, an embodiment of the present application provides an open framework visual software development system, which adopts an open framework design and supports users to carry out multiple layers of secondary development based on this framework according to actual application needs, so that the visual software development system is both flexible and easy to use.

[0080] Also, as shown in Fig. 1, in the system shown in Fig. 1, the data processing layer 102 may further include a data scheduling layer and a function module layer. In this case, in order to facilitate effective management of each module, the functions of the above-mentioned data processing layer are distributed to the data scheduling layer and the function module layer. Here, the function module layer stores module data of N modules, and the data scheduling layer is used to sequentially call and execute the module data of the modules corresponding to one or more module display controls from the function module layer according to the execution order between the modules corresponding to one or more module display controls in the flowchart, and obtain the data processing result.

[0081] That is, the functional module layer is used to package the module data of each module and provide the input / output ports of each module to the outside. The data scheduling layer is used to call the module data of each module according to the input / output ports of each module, and then realize the function of each module in the flowchart. For example, when the data scheduling layer needs to call the module data of module A in the functional module layer, the data scheduling layer inputs the data required for the execution of module A to the functional module layer through the input port of module A, and then the functional module layer executes the execution logic of module A according to the input data, obtains the result data, and sends the result data to the data scheduling layer through the output port of module A. Up to now, the data scheduling layer has completed the call process for module A in the functional module layer.

[0082] In addition, in order to improve the security of the module data of each module in the functional module layer, the module data of each module in the functional module layer is packaged independently. That is, there is no communication between each module in the functional module layer. The data scheduling layer is configured to call the data in the functional module layer in an integrated manner. Figure 4 is a schematic diagram of the data scheduling layer provided in the embodiment of the present application calling the module data of each module in the functional module layer. As shown in Figure 4, the data scheduling layer schedules modules 1, 2, and 3 independently of each other, and there is no crossover between the called modules. That is, each module is executed based on the execution of the scheduling, and the result data of each module is temporarily stored.

[0083] When the data processing layer includes a data scheduling layer and a function module layer, before the data scheduling layer executes a flowchart, it may first record the order relationship of calling each module in the flowchart in a certain data structure, which may be, for example, a tree structure or a graph structure, and then, when the data scheduling layer executes the flowchart, it traverses the data structure according to a certain order and calls the ports of each module in the function module layer in order to complete the development flow for the flowchart.

[0084] Moreover, the modules according to the embodiments of the present application may be modules that realize any function. The modules may include an algorithm module, an image acquisition module, a data calculation module, an external control module, a communication module, etc. The embodiments of the present application are not limited thereto. FIG. 5 is a schematic diagram of modules provided in the embodiments of the present application. As shown in FIG. 5, the modules according to the embodiments of the present application may include the following four types. The first type is a module corresponding to external hardware, such as an image acquisition module, a camera control module, a light source control module, etc. The second type is a module related to image processing, such as a template matching module, a geometric search module, a code reading module, an OCR (Optical Character Recognition) module, etc. The third type is a module related to data processing, such as a data processing module, a logic calculation module, a data counting module, etc. The fourth type is a module related to communication.

[0085] The above describes important functions of the visual software development system provided in the embodiment of the present application, and the following describes other functions of the system shown in Fig. 1 in more detail. Note that some of the following functions are merely exemplary descriptions of functions related to the embodiment of the present application, and do not limit the functions of the system shown in Fig. 1.

[0086] 1. The visual software development system supports the independent design and development of module data for third-party modules.

[0087] Since a third-party module is not an existing module in the system but a module independently designed and developed by a third party, the module data of the third-party module must satisfy the rules specified by the system provided in the embodiment of the present application to ensure that the introduced third-party module can be mounted in the system provided in the embodiment of the present application.

[0088] Thus, in one possible embodiment, the application interface layer is adapted to obtain module data of the third-party module installed by the user through the module add control in response to the second operation on the module add control, and send the module data of the installed third-party module to the data processing layer. In this case, if the data processing layer determines that the module data of the installed third-party module meets the reference rule, it is adapted to store the module data of the installed third-party module and return a module load success message to the application interface layer, where the module load success message indicates that the loading of the third-party module is successful. On the other hand, if the data processing layer determines that the module data of the installed third-party module does not meet the reference rule, it does not store the module data of the installed third-party module, and returns a module load failure message to the application interface layer, where the module load failure message indicates that the loading of the third-party module has failed.

[0089] That is, when a developer introduces a third-party module into the system, the data processing layer in the system judges the module data of the third-party module, and only if it judges that the module data of the introduced third-party module meets the reference rule, the module data of the third-party module is successfully loaded into the data processing layer, thereby ensuring that the third-party module mounted in the visual software development system provided in the embodiment of the present application is compatible with the existing module in the visual software development system itself.

[0090] In addition, the system may externally explain the reference rules that the module data of the third-party module must satisfy, so that the developer can quickly know the rules that the module data of the third-party module must satisfy. Therefore, in one possible embodiment, the application interface layer in FIG. 1 is further used to display a module configuration file in response to a third operation on the module addition control, and the module configuration file indicates the above-mentioned reference rules. The third operation may be a click operation, a voice operation, etc. For example, if the third operation is a right double-click operation, when the user right-double-clicks the module addition control displayed on the main interface with the mouse, the application interface layer can display the module configuration file. The module configuration file indicates the rules that the module data of the introduced third-party module must satisfy. That is, the user can know the rules that the module data of the currently desired third-party module must satisfy by examining the module configuration file.

[0091] Alternatively, a developer who develops a third-party module may also know the reference rules that the module data of the third-party module introduced into the system must satisfy through other methods, and the embodiments of the present application are not limited thereto. For example, the developer may know these reference rules from the related documentation of the visual software development system provided in the embodiments of the present application.

[0092] Specifically, the reference rule may include a design mode of an input / output port of a third-party module and / or a packaging mode of module data of a third-party module, where the design mode of an input / output port of a third-party module includes a format of input data and / or output data, where the format of the input data or output data may specifically include information such as data sensitivity. The packaging mode of module data of a third-party module indicates a storage format of the module data in a data processing layer, where the storage format may include any of a dynamic link library, a static link library, an exe (executable) file, etc.

[0093] The above-mentioned function related to the module addition control can realize loading a module developed by a third party into the data processing layer. Specifically, when the data processing layer includes a data scheduling layer and a function module layer, loading a module developed by a third party into the data processing layer specifically refers to loading the module data of the module developed by a third party into the function module layer. Here, the module data of the module developed by a third party is packaged in the form of a dynamic library (or executable program) and stored in the function module layer, and the function module layer generates a handle of the module developed by a third party (the handle is used to uniquely identify the module), so that the data scheduling layer can easily call the module by the handle in the future.

[0094] In addition, in order to make the module more open, the system shown in FIG. 1 also supports deleting the loaded module, i.e., module release. A specific embodiment is not described in detail here. FIG. 6 is a schematic diagram of module loading and module release provided in an embodiment of the present application. As shown in FIG. 6, the data scheduling layer can load a module into the function module layer in response to an operation performed by a user in the application interface layer, and can also delete a module from the function module layer, i.e., module release, in response to an operation performed by a user in the application interface layer.

[0095] In addition, the first to third operations in the above description are all user operations triggered by the user on the corresponding controls, and the user operations may be click operations, voice operations, slide operations, etc. The first to third operations are merely for distinguishing different operations, and do not limit the specific user operations. The following fourth to tenth operations may all refer to this description, and the description will not be repeated from here on.

[0096] 2. The visual software development system supports users to perform secondary development on the module data of any module.

[0097] For modules mounted in the system provided in the present application, in order to ensure that the modules can respond to changing business needs at any time, the visual software development system further supports secondary development by users of module data of modules mounted in the system, so that the secondary developed modules can meet the latest business needs.

[0098] Therefore, in one possible embodiment, the application interface layer is further used for displaying a module configuration interface of the first module corresponding to the first module display control in response to a fourth operation by the user on a first module display control among the N module display controls, the module configuration interface including a plurality of module configuration controls, and any one of the module configuration controls is used for instructing the user to edit parameters in the execution process of the first module, where the first module display control is any one of the N module display controls. In this case, the application interface layer is further used for obtaining parameters edited by the user through any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls, and sending the edited parameters to the data processing layer. The data processing layer is used for updating module data of the first module according to the edited parameters. This process can realize that the user makes modifications to the module data of any one of the modules, so that the modified module data meets the latest business needs.

[0099] Regarding the timing when the user performs secondary development on the module data of an arbitrary module, the user may perform secondary development on the module data of a certain module included in the flowchart after configuring the flowchart based on the application interface layer. In this case, if the user performs secondary development on the module data of a first module included in the flowchart, the data processing layer is further used to update the data processing result of the flowchart based on the updated module data of the first module, and return the updated data processing result to the application interface layer.

[0100] In other words, after a user configures a flowchart based on the application interface layer, when a secondary development is performed on the module data of a module included in the flowchart, the visual software development system automatically updates the data processing result of the flowchart based on the updated module data.

[0101] The parameters in the execution process of the first module may include input parameters, output parameters, and other related parameters in the execution process, etc. The module configuration control may include a character configuration control, a shaping configuration control, a floating-point number configuration control, an enumeration configuration control, etc.

[0102] The module configuration interface may be an interface that the system pre-configures for all modules, that is, the module configuration interface may be a default module configuration interface in the system. In one possible embodiment, a module configuration interface file for all modules is stored in the system, and the module configuration interface file is used to indicate the relevant content of the module configuration interface, and the application interface layer may display the default module configuration interface after loading the module configuration interface file. For example, the fourth operation is a double-click operation, and when a module display control corresponding to a first module displayed in the main interface is double-clicked, the application interface layer dynamically generates a system default module configuration interface based on a default module configuration interface file defined by XML (eXtensible Markup Language).

[0103] Alternatively, if the first module for which the user has performed secondary development is a third-party module, the module configuration interface of the first module may be independently designed by the third party. In this case, when the module data of the first module is introduced into the system, the introduced module data of the first module may further include a module configuration interface file customized by the third party for the first module, and the module configuration interface file is used to indicate the program code of the module configuration interface corresponding to the first module. In this case, the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module. Here, the module configuration interface file independently designed by the third party may also be a configuration file defined by XML.

[0104] In addition, in the module configuration interface file, the third party may further specify the name of a module configuration control in a specific module configuration control library. In this case, the module data of the third-party module may further include the specific module configuration control library provided by the third party. In this case, when the application interface layer displays the module configuration interface based on the module configuration interface file provided by the third party, the displayed module configuration interface includes the module configuration control in the specific module configuration control library, which makes it easy for the user to complete the configuration work of the third-party module based on the module configuration control in the module configuration control library provided by the third party.

[0105] Here, the application interface layer provides ports for obtaining or setting parameters to module configuration controls provided by third parties, which allows users to easily call or modify related parameters of module configuration controls provided by third parties based on these ports. Here, the ports for obtaining or setting parameters may operate in a key-value format to provide support for module configuration controls of basic types such as integer type (int), floating point type (float), string type, etc. In addition, the ports for obtaining or setting parameters may be provided in a binary format, and the binary operating format allows calling bulk data, so that calling or modification of parameters of bulk data can be realized when the ports for obtaining or setting parameters operate in a binary format.

[0106] From the above, in the embodiment of the present application, for a default module in the system (collectively referred to as a native module in the embodiment of the present application) or a third-party module, when a user triggers the module through a fourth operation, a module configuration interface built in the system can appear, and the module configuration control such as input, output, and execution parameters can be displayed in the module configuration interface, which facilitates the user to modify the relevant parameters of the module. Optionally, for a third-party module, the third party can further introduce a module configuration interface file. This corresponds to the module configuration interface being provided by a third party.

[0107] Fig. 7 is a schematic diagram of the format of module data of a third-party module provided in an embodiment of the present application. As shown in Fig. 7, the data processing layer includes a data scheduling layer and a function module layer. Here, in the visual tool area of ​​the main interface of the application interface layer, a module display control for a native module is displayed, and the module display control is displayed with an icon of the native module. In the visual tool area, a module display control for a third-party module is further displayed, and the module display control is displayed with an icon of the third-party module. The XML corresponding to displaying such an interface is shown in Fig. 7, that is, the code of the module display control of the two modules used for displaying is as follows:

[0108] TIFF0007644830000001.tif29160

[0109] In addition, as shown in Fig. 7, the module data of the third-party module is stored in the function module layer. The module data of the third-party module includes a module configuration interface file for the third-party module, a module dynamic library, and a module configuration control library. Here, in Fig. 7, the code of the module data of the third-party module is as follows:

[0110] disanfangmodule.xml / is the module configuration interface file, disanfangmodule.dll is a module dynamic library. disanfangmoduleConfig.dll is the module configuration control library.

[0111] Here, the internal execution logic of the third-party module may be obtained through the module dynamic library. The module configuration control library and the module configuration interface file have been described in detail in the above-mentioned embodiments, so the description will not be repeated here.

[0112] In addition, in order to be compatible with the system shown in FIG. 1, the module configuration interface file introduced by a third party and the system default module configuration interface file should satisfy the same rules. This makes the module configuration interface provided by a third party compatible with the default module configuration interface of the embodiment of the present application. This may be realized by the relevant content of the rules indicated by the module configuration file described above. In other words, the rules contained in the module configuration file include the rules that the module configuration interface file introduced by a third party must satisfy, and will not be described in detail here.

[0113] 3. The visual software development system supports converting flowcharts into the underlying code, which makes it easier for developers to do further secondary development.

[0114] After a user configures a flowchart, other users can further develop the visual software indicated by the flowchart, so that the system provided in the embodiment of the present application further supports converting the flowchart into the underlying code, which makes it easier for developers to further develop. Therefore, in one possible embodiment, the data processing layer is further used to generate a first code package based on the flowchart, and the first code package indicates the execution logic of each module included in the flowchart and the execution order between each module included in the flowchart, that is, the first code package indicates the internal execution logic of the flowchart. In this case, the data processing layer is further used to control the application interface layer to display a first code control corresponding to the first code package, and the first code control is used to instruct the user to download the first code package, and the first code package supports introduction into a third-party system, and displays the flowchart after being executed in the third-party system.

[0115] Here, after the data processing layer generates the first code package, the first code package may be further stored in a specified location, for example, in the function module layer. When the application interface layer detects a selection operation on the first code control, the application interface layer obtains the first code package and then downloads the first code package to a specified location, for example, a specified location selected by the user, so that the user can easily introduce the first code package into the third party system. Thus, after the first code package is executed in the third party system, the flowchart constructed in the system provided in the embodiment of the present application is displayed in the third party system. In addition, the user may further perform secondary development on the flowchart based on the first code package in the third party system to generate new visual software. That is, the embodiment of the present application further provides a derivation model. The derivation model can convert the module calls and related data read / write relationships in the flowchart into code. The code can be provided to the user for reference or integrated into the user's own software.

[0116] In addition, after the data processing layer generates the first code package, the data processing layer may further send the first code package to the application interface layer, and the application interface layer displays the first code package, which facilitates a developer to make online modifications to the first code package in the system provided in the embodiment of the present application. In this case, after the application interface layer detects a modification command for the first code package, the application interface layer further updates the displayed flowchart according to the modification made by the user to the first code package.

[0117] 4. The visual software development system provides an execution interface, through which the developer can learn important images, graphics, characters, etc. related to the execution process of the visual software.

[0118] Since the flowchart only displays the basic flow and does not display the back-end execution information of the visual software, in the embodiment of the present application, the visual software development system further supports displaying the back-end execution information of the flowchart for the configured flowchart, so as to facilitate the developer to know the execution process of the visual software. In the embodiment of the present application, the execution information of the flowchart indicates intermediate data in the execution process of the flowchart. The execution information of the flowchart is the back-end execution information. Therefore, in one possible embodiment, the application interface layer in the system shown in FIG. 1 is further used for displaying an execution interface display control, the execution interface display control instructs displaying an execution interface for the flowchart, the execution interface is used for displaying the execution information of the flowchart, and the execution information of the flowchart indicates intermediate data in the execution process of the flowchart. In this case, the application interface layer is used for displaying the execution interface in response to the sixth operation on the execution interface display control, so that the user can view the intermediate data in the execution process of the flowchart based on the execution interface.

[0119] For example, after a user composes a flowchart, when a click operation on the execution interface display control is detected, the application interface layer displays an execution interface and displays execution information of the flowchart in the execution interface, which may be a function area of ​​the current main interface, and the specific content displayed in the execution interface may be system default information.

[0120] Alternatively, the specific content displayed in the execution interface may be specified to be displayed according to a basic business need of the user. In this case, the application interface layer is further used for displaying an execution interface configuration control, the execution interface configuration control instructs displaying the execution interface configuration interface, and the execution interface configuration interface is used for configuring the content displayed in the execution interface. In this case, the application interface layer is further used for displaying an execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface is used for the user to configure which information should be displayed in the execution interface and the source of the displayed information.

[0121] In one possible embodiment, the execution interface configuration interface includes a plurality of execution interface configuration sub-controls, and one of the plurality of execution interface configuration sub-controls has a launch option. In this case, the application interface layer is further used for displaying a content source option in response to an eighth operation on the launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and binding the first execution interface configuration sub-control with a module in the flowchart in response to a ninth operation on the content source option. After binding the first execution interface configuration sub-control with a module in the flowchart, execution information corresponding to the first execution interface configuration sub-control may be displayed on the execution interface, and the execution information is derived from data of the module bound with the first execution interface configuration sub-control.

[0122] That is, the system shown in FIG. 1 provides an editable execution interface, which makes it easy for users to design the execution interface of a flowchart according to actual application needs. In this way, when a user creates a visual software, certain important pictures, images, and characters related to the visual software can be displayed by the execution interface. The execution interface configuration sub-controls may include image controls, text display boxes, parameter setting boxes, status lamps, buttons, tables, etc. When configuring the execution interface, these execution interface configuration sub-controls are used to configure which information should be displayed on the execution interface, and then the information to be displayed is bound to the module data of the module in the flowchart, i.e., the source of the displayed information is indicated. In this way, an execution interface that meets business needs is configured.

[0123] In addition, after a user configures an execution interface, in order to allow other users to perform further secondary development on the execution interface, the system provided in the embodiments of the present application further supports converting the execution interface into the underlying code, thereby making it easier for developers to perform further secondary development.

[0124] Therefore, in one possible embodiment, the data processing layer is further used for deriving the program code of the execution interface as a second code package, in which the data processing layer is further used for controlling the application interface layer to display a second code control corresponding to the second code package, and the second code control is used for instructing a user to download the second code package, which supports to be introduced into a third party system, and displays the execution interface after being executed in the third party system.

[0125] Here, after the data processing layer generates the second code package, the second code package may be stored in a specified location, for example, in the function module layer. Then, when the application interface layer detects a selection operation on the second code control, the application interface layer obtains the second code package and downloads the second code package to a specified location, for example, a specified location selected by the user, so that the user introduces the second code package into the third party system, and after the second code package is executed in the third party system, the execution interface built in the system provided in the above-mentioned embodiment of the present application is displayed. Furthermore, the user may perform secondary development on the visual software based on the second code package in the third party system to generate a new execution interface.

[0126] As can be seen, in the embodiment of the present application, by translating the execution interface into code, a code derivation process based on the interface development SDK (Software Development Kit) is generated, and the user can continue to develop other functions based on this process, or integrate visual parts into the user's software. That is, by translating the execution interface into code, the execution interface is opened to the user.

[0127] FIG. 8 is a schematic diagram of the configuration process and derived code of the execution interface provided in the embodiment of the present application. The upper block of FIG. 8 indicates the configured execution interface, and as shown in FIG. 8, an image is displayed on the execution interface, which is an image obtained by capturing an image with a camera and then undergoing a circle search process. The connection status of the camera is displayed on the execution interface, and the radius of the determined circle after performing a circle search on the image captured by the camera is further displayed on the execution interface. Here, the image displayed on the execution interface is determined according to the data after the camera module is executed and the data after the circle search module is executed. The radius of the circle determined by performing the circle search is determined according to the data after the circle search module is executed. In addition, the camera connection status is determined according to the data in the execution process of the camera module. In addition, the execution interface may further display one execution or multiple executions (i.e., continuous execution). One execution indicates that the flowchart is executed only once, and multiple executions indicates that the flowchart is executed multiple times. The content displayed on the execution interface and the source of the displayed content are both realized by the related execution interface configuration sub-control in the execution interface configuration interface. Here, we will not provide examples and explanations for each of them. The dotted block at the bottom of Figure 8 indicates the code corresponding to the displayed execution interface, and this code is displayed in a text manner, that is, the code allows a user to display the execution interface at the top of Figure 8 in his / her own software, thereby realizing the opening of the execution interface to the outside.

[0128] 5. The visual software development system supports module registration and legitimacy verification.

[0129] In order to improve the versatility and specialization of the visual software development system provided in the embodiment of the present application, in the data processing layer, module data of a large number of modules can be further stored. When a user uses the system provided in the embodiment of the present application, the user can select modules required for the current business from the large number of modules and load them into the application interface layer. That is, each user can select and register some modules from the large number of modules according to his / her business needs, thereby developing a flowchart that meets his / her business needs and reducing the data storage and processing load of the application interface layer of the system.

[0130] Therefore, in one possible embodiment, the data processing layer is used to store module data of M modules, where M is a positive integer equal to or greater than N. In this case, the application interface layer is further used to display a module registration control, and the application interface layer is used to display N module display controls in response to the N modules selected by the user from the M modules through the module registration control. That is, at the time of system initialization, the user may select some modules through the module registration control to register them in the application interface layer. In this way, the application interface layer does not need to display all the modules in the data processing layer, which facilitates simplifying the main interface currently displayed in the application interface layer.

[0131] In addition, in order to improve the security of module data and prevent malicious users from arbitrarily changing the module data corresponding to the modules, the modules registered in the application interface layer are modules that have passed the legitimacy verification. That is, the N modules corresponding to the N module display controls displayed in the application interface layer shown in FIG. 1 are modules that are verified and deemed legitimate by the data processing layer. Here, the verified and deemed legitimate modules are the corresponding modules that the user has the authority to use. In this case, the authorization authority of each module may be configured in advance for each module configured in the data processing layer of the system, and the authorization authority indicates which user is allowed to use the corresponding module. In this way, when a user registers a module according to the above method, the system verifies the authority of the user, and if the user does not have the authorization authority of the module, the system refuses the user from registering the module.

[0132] FIG. 9 is a flow diagram of module registration provided in an embodiment of the present application. As shown in FIG. 9, after the system is initialized, based on the user's operation in the application interface layer, when the application interface layer detects that the user currently wants to register a certain module, the application interface layer sends a registration request to the data scheduling layer. After the data scheduling layer receives the registration request (i.e., receives information), it verifies the user's authority, and if the verification is passed, it adds the module to the registered module list and sends a registration success result to the application interface layer, so that the application interface layer displays a module display control for the module when the registration is successful. On the other hand, if the verification is not passed, it sends a registration failure result to the application interface layer. In this case, the application interface layer does not display a module display control for the module. Note that the verification flow shown in FIG. 9 is realized when registering a module, and optionally, the verification flow may be realized in the module loading process shown in FIG. 6, which will not be described in detail here.

[0133] 6. The visual software development system supports third parties to call modules mounted in the system.

[0134] In an embodiment of the present application, the visual software development system further allows a user to call the module mounted in the system in other software. This makes it easier for a user to quickly develop other modules based on the module mounted in the system. For this reason, in one possible embodiment, the application interface layer is further used for displaying a call control for any of the module display controls, and the call control indicates that the user is allowed to allow a third party system to call the module corresponding to any of the module display controls. In this case, the application interface layer is further used for displaying a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control. When the application interface layer detects a data call request for the port link sent from the third party system, it is further used for obtaining module data of the module corresponding to any of the module display controls from the data processing layer based on the port link, and sending the obtained module data to the third party system.

[0135] That is, after displaying a call control for any module display control, when the application interface layer detects a selection operation for the call control, it can provide a port link of the module data corresponding to the module, which makes it easy for the user to call the module data of the module in other software based on the port link.

[0136] From the above, an embodiment of the present application provides a visual software development system, in which an application interface layer is used to display a module adding control, and the module adding control is used to introduce module data of a third-party module into a data processing layer. In this way, the modules corresponding to the N module display controls displayed on the application interface layer may include third-party modules. Therefore, in the embodiment of the present application, a user can introduce a third-party module through the module adding control at any time according to business needs, and then the application interface layer displays a flowchart configured by the user based on the third-party module, and the flowchart becomes a visual software configured by the user, and the data processing layer executes the flowchart, thereby completing the development flow of the visual software.

[0137] As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system that allows users to develop different third-party modules according to different business needs and mount the developed third-party modules in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can be more adapted to the personalized needs of different businesses. In this way, it is not necessary to build different visual software development systems for different types of visual software, and different types of visual software can be developed with the visual software development system provided in the embodiment of the present application.

[0138] In addition, even if business needs or user needs change, a user of the visual software development system provided in the embodiment of the present application can develop a new third-party module, and then mount the developed new third-party module in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can quickly adapt to the latest needs of the user.

[0139] Based on the functions of the visual software development system provided above, the embodiment of the present application further provides a visual software development method applied to a visual software development system including an application interface layer and a data processing layer. It should be noted that, as can be seen from the above-mentioned system related contents, the functions of the system provided in the embodiment of the present application specifically include three things, namely, module loading, module registration, and flow chart configuration. The above three things will be respectively described below by two embodiments. Figure 10 is a flow diagram of module loading and module registration provided in the embodiment of the present application. As shown in Figure 10, the method includes the following steps:

[0140] Step 1001: In response to a second operation on the module addition control, the application interface layer obtains module data of a third-party module installed by a user through the module addition control, and sends the module data of the installed third-party module to the data processing layer.

[0141] Step 1002: If the data processing layer determines that the module data of the installed third-party module satisfies the reference rule, it stores the module data of the installed third-party module and returns a module loading success message to the application interface layer, where the module loading success message indicates that the third-party module has been successfully loaded.

[0142] Step 1003: The application interface layer displays a module registration control, and the application interface layer displays N module display controls in response to N modules selected by the user from the M modules through the module registration control, where the data processing layer stores module data of the M modules, and M is a positive integer greater than or equal to N.

[0143] After completing the module loading and module registration according to the embodiment shown in Fig. 10, the flow chart can be configured in the system provided in the embodiment of the present application to realize the development of visual software. Fig. 11 is a flow chart of the method for configuring the flow chart provided in the embodiment of the present application, and as shown in Fig. 11, the method includes the following steps:

[0144] Step 1101: The application interface layer displays N module display controls and a module adding control, the N module display controls respectively corresponding to N modules, module data of each of the N modules is stored in the data processing layer, and the module data includes the execution logic of the corresponding module, the application interface layer further displays a module adding control, which is used for introducing module data of a third-party module into the data processing layer, the N modules including a third-party module, and N is a positive integer greater than or equal to 1.

[0145] Step 1102: The application interface layer displays a flowchart in response to a first user operation on one or more of the N module display controls, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls.

[0146] Step 1103: In response to the flowchart displayed on the application interface layer, the data processing layer calls and executes module data of modules corresponding to each of one or more module display controls, obtains the data processing result of the flowchart, and returns the data processing result to the application interface layer.

[0147] In one possible embodiment of the method provided in Figures 10 and 11, the reference rule includes a design method of an input / output port of a third-party module and / or a packaging method of module data of the third-party module, where the design method of the input / output port of the third-party module includes a format of input data and / or output data of the third-party module, and the packaging method of the module data of the third-party module indicates a storage format of the module data of the third-party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file.

[0148] In one possible embodiment, the method further includes the application interface layer displaying a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule.

[0149] In one possible embodiment, the method further includes: in response to a fourth operation by the user on a first module display control among the N module display controls, the application interface layer displays a module configuration interface of the first module corresponding to the first module display control, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls is used to instruct the user to edit parameters in the execution process of the first module; in response to a fifth operation by the application interface layer on any one of the module configuration controls, the application interface layer obtains parameters edited by the user through any one of the module configuration controls and sends the edited parameters to the data processing layer; and the data processing layer updates module data of the first module based on the edited parameters, and if the flowchart includes the first module, the data processing layer updates a data processing result of the flowchart based on the updated module data of the first module and returns the updated data processing result to the application interface layer.

[0150] In one possible embodiment, when the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, where the module configuration interface file indicates the program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

[0151] In one possible embodiment, the method further includes: the data processing layer generates a first code package based on the flowchart, the first code package instructing the execution logic of each module included in the flowchart and the execution order between each module included; the data processing layer controls the application interface layer so that the application interface layer displays a first code control corresponding to the first code package, the first code control is used to instruct a user to download the first code package, the first code package supports being introduced into a third-party system, and displays the flowchart after being executed in the third-party system.

[0152] In one possible embodiment, the method further includes: the application interface layer displays an execution interface display control, the execution interface display control instructs displaying an execution interface for the flowchart, the execution interface is used to display execution information of the flowchart, the execution information of the flowchart indicates intermediate data in the execution process of the flowchart; and the application interface layer, in response to a sixth operation on the execution interface display control, displays the execution interface so that the user can view the intermediate data in the execution process of the flowchart based on the execution interface.

[0153] In one possible embodiment, the method further includes: the application interface layer displays an execution interface configuration control, the execution interface configuration control instructs displaying the execution interface configuration interface, the execution interface configuration interface being used to configure content displayed in the execution interface; the application interface layer displays the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, one of the plurality of execution interface configuration sub-controls having a launch option; the application interface layer displays a content source option in response to an eighth operation on the launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and binds the first execution interface configuration sub-control to a module in the flowchart in response to a ninth operation on the content source option, wherein the execution information displayed in the execution interface includes data of the module bound to the first execution interface configuration sub-control.

[0154] In one possible embodiment, the method further includes: the data processing layer deriving the program code of the execution interface as a second code package; the data processing layer controlling the application interface layer so that the application interface layer displays a second code control corresponding to the second code package, the second code control being used to instruct a user to download the second code package, the second code package supporting introduction into a third-party system, and displaying the execution interface after being executed in the third-party system.

[0155] In one possible embodiment, the data processing layer includes a data scheduling layer and a functional module layer, in which module data of N modules is stored, and the data processing layer, in response to a flowchart displayed on the application interface layer, calls and executes the module data of the modules corresponding to each of one or more module display controls, and obtains a data processing result of the flowchart, including the data scheduling layer sequentially calling and executing the module data of the modules corresponding to each of the one or more module display controls from the functional module layer in accordance with the execution order between the modules corresponding to each of the one or more module display controls in the flowchart, and obtaining a data processing result.

[0156] In one possible embodiment, the N modules are modules that the data processing layer verifies and validates, and the modules that it verifies and validates are corresponding modules that the user has the authority to use.

[0157] In one possible embodiment, the method further includes: the application interface layer displays a call control for any of the module display controls, the call control indicating that the user allows the third party system to call the module corresponding to any of the module display controls; the application interface layer displays a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control; and when the application interface layer detects a data call request for the port link sent from the third party system, retrieves module data of the module corresponding to any of the module display controls from the data processing layer based on the port link, and sends the retrieved module data to the third party system.

[0158] Any embodiment of any step in the above-provided method may refer to the description related to the above-mentioned system, and the description will not be repeated here.

[0159] From the above, the embodiment of the present application provides a visual software development method. In the visual software development system to which the method is applied, the application interface layer is used to display a module add control, and the module add control is used to introduce module data of a third-party module into the data processing layer. In this way, the modules corresponding to the N module display controls displayed on the application interface layer may include third-party modules. Therefore, in the embodiment of the present application, a user can introduce a third-party module through the module add control at any time according to business needs, and then the application interface layer displays a flow chart configured by the user based on the third-party module, and the flow chart becomes visual software configured by the user, and the data processing layer executes the flow chart, thereby completing the development flow of the visual software.

[0160] As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system that allows users to develop different third-party modules according to different business needs and mount the developed third-party modules in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can be more adapted to the personalized needs of different businesses. In this way, it is not necessary to build different visual software development systems for different types of visual software, and different types of visual software can be developed with the visual software development system provided in the embodiment of the present application.

[0161] In addition, even if business needs or user needs change, a user of the visual software development system provided in the embodiment of the present application can develop a new third-party module, and then mount the developed new third-party module in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can quickly adapt to the latest needs of the user.

[0162] Fig. 12 is a schematic diagram of a visual software development device provided in an embodiment of the present application, which is constructed as a visual software development system including an application interface layer and a data processing layer. As shown in Fig. 12, the visual software development device 1200 includes the following modules:

[0163] The display module 1201 is used for controlling the application interface layer to display N module display controls, each of which corresponds to N modules; module data of each of the N modules is stored in the data processing layer, and the module data includes an execution logic of the corresponding module; the application interface layer further displays a module adding control, which is used for introducing module data of a third party module into the data processing layer, the N modules including a third party module, and N is a positive integer greater than or equal to 1; the display module 1201 is further used for controlling the application interface layer to display a flowchart in response to a user's first operation on one or more module display controls among the N module display controls, and the flowchart indicates an execution order among the modules corresponding to each of the one or more module display controls; and the execution module 1202 is used for controlling the data processing layer to call and execute the module data of the modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer, obtain a data processing result of the flowchart, and return the data processing result to the application interface layer.

[0164] In one possible embodiment, the apparatus further includes an acquisition module and a determination module, where the acquisition module is used to control the application interface layer to acquire module data of the third-party module introduced by the user through the module addition control in response to a second operation on the module addition control, and send the module data of the introduced third-party module to the data processing layer; when the determination module determines that the module data of the introduced third-party module satisfies the reference rule, the determination module is used to control the data processing layer to store the module data of the introduced third-party module and return a module loading success message to the application interface layer, where the module loading success message indicates that the loading of the third-party module is successful.

[0165] In one possible embodiment, the reference rule includes a design method of an input / output port of a third-party module and / or a packaging method of module data of the third-party module, the design method of the input / output port of the third-party module includes a format of input data and / or output data of the third-party module, and the packaging method of the module data of the third-party module indicates a storage format of the module data of the third-party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file.

[0166] In one possible embodiment, the display module is further used for controlling the application interface layer to display a module configuration file in response to a third operation on the module addition control, where the module configuration file indicates the reference rule.

[0167] In one possible embodiment, the display module is further used for controlling the application interface layer to display a module configuration interface of the first module corresponding to the first module display control in response to a fourth operation by the user on a first module display control among the N module display controls, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls is used for instructing the user to edit parameters in the execution process of the first module, and the apparatus further includes an acquisition module and an update module, the acquisition module is used for controlling the application interface layer to acquire parameters edited by the user by any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls, and send the edited parameters to the data processing layer, the update module is used for controlling the data processing layer to update module data of the first module based on the edited parameters, and when the flowchart includes the first module, the data processing layer updates a data processing result of the flowchart based on the updated module data of the first module and returns the updated data processing result to the application interface layer.

[0168] In one possible embodiment, when the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, where the module configuration interface file indicates the program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

[0169] In one possible embodiment, the apparatus further includes a generation module, which is used for controlling the data processing layer to generate a first code package based on the flowchart, and the first code package indicates the execution logic of each module included in the flowchart and the execution order between each module included; the display module is further used for controlling the data processing layer, and the data processing layer is further used for controlling the application interface layer to display a first code control corresponding to the first code package, and the first code control is used for instructing a user to download the first code package, and the first code package supports being introduced into a third-party system and displays the flowchart after being executed in the third-party system.

[0170] In one possible embodiment, the display module is further used for controlling the application interface layer to display an execution interface display control, the execution interface display control instructs displaying an execution interface for the flowchart, the execution interface is used for displaying execution information of the flowchart, the execution information of the flowchart indicates intermediate data in the execution process of the flowchart, and the display module is further used for controlling the application interface layer to display the execution interface in response to a sixth operation on the execution interface display control, such that the user views the intermediate data in the execution process of the flowchart based on the execution interface.

[0171] In one possible embodiment, the display module is further used to control the application interface layer to display an execution interface configuration control, the execution interface configuration control instructs displaying the execution interface configuration interface, and the execution interface configuration interface is used to configure the content displayed in the execution interface; the display module is further used to control the application interface layer to display the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, and one of the plurality of execution interface configuration sub-controls has a launch option; the display module is further used to control the application interface layer to display a content source option in response to an eighth operation on the launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and to bind the first execution interface configuration sub-control to a module in the flowchart in response to a ninth operation on the content source option; and the execution information displayed in the execution interface includes data of the module bound to the first execution interface configuration sub-control.

[0172] In one possible embodiment, the apparatus further includes a derivation module, which is used for controlling the data processing layer to derive the program code of the execution interface as a second code package; the display module is further used for controlling the data processing layer, which is further used for controlling the application interface layer to display a second code control corresponding to the second code package, which is used for instructing a user to download the second code package, and the second code package supports being introduced into a third-party system and displays the execution interface after being executed in the third-party system.

[0173] In one possible embodiment, the data processing layer includes a data scheduling layer and a functional module layer, in which module data of N modules are stored, and the execution module is used to control the data scheduling layer to sequentially call and execute the module data of modules corresponding to each of one or more module display controls from the functional module layer according to the execution order between the modules corresponding to each of one or more module display controls in the flowchart, and obtain a data processing result.

[0174] In one possible embodiment, the display module is further used for controlling the application interface layer to display a module registration control, the application interface layer displays N module display controls in response to N modules selected by a user from the M modules through the module registration control, and the data processing layer stores module data of the M modules, where M is a positive integer greater than or equal to N.

[0175] In one possible embodiment, the N modules are modules that the data processing layer verifies and validates, and the modules that it verifies and validates are corresponding modules that the user has the authority to use.

[0176] In one possible embodiment, the display module is further used to control the application interface layer to display a call control for any of the module display controls, the call control instructing a user to allow a third party system to call the module corresponding to any of the module display controls, and the display module is further used to control the application interface layer to display a port link of module data of the module corresponding to any of the module display controls in response to a tenth operation on the call control, and the apparatus further includes an acquisition module, which is used to control the application interface layer to acquire module data of the module corresponding to any of the module display controls from the data processing layer based on the port link when it detects a data call request for the port link sent from the third party system, and to send the acquired module data to the third party system.

[0177] From the above, the embodiment of the present application provides a visual software development device. The application interface layer of the visual software development system configured by the device is used to display a module add control, and the module add control is used to introduce module data of a third-party module into the data processing layer. In this way, the modules corresponding to the N module display controls displayed on the application interface layer may include third-party modules. Therefore, in the embodiment of the present application, a user can introduce a third-party module through the module add control at any time according to business needs, and then the application interface layer displays a flowchart configured by the user based on the third-party module, and the flowchart becomes visual software configured by the user, and the data processing layer executes the flowchart, thereby completing the development flow of the visual software.

[0178] As can be seen, the visual software development system provided in the embodiment of the present application is a general-purpose open visual software development system that allows users to develop different third-party modules according to different business needs and mount the developed third-party modules in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can be more adapted to the personalized needs of different businesses. In this way, it is not necessary to build different visual software development systems for different types of visual software, and different types of visual software can be developed with the visual software development system provided in the embodiment of the present application.

[0179] In addition, even if business needs or user needs change, a user of the visual software development system provided in the embodiment of the present application can develop a new third-party module, and then mount the developed new third-party module in the software development system provided in the embodiment of the present application, so that the subsequently developed visual software can quickly adapt to the latest needs of the user.

[0180] In addition, the visual software development apparatus provided in the above embodiments is described as an example of only partitioning each of the above functional modules when developing visual software, but in practice, the above functions can be distributed and completed to different functional modules as necessary, that is, the internal structure of the device can be partitioned into different functional modules to complete all or part of the functions described above. In addition, the visual software development apparatus and the visual software development method provided in the above embodiments correspond to the same technical idea, and the specific implementation steps thereof are referred to in the method embodiments and will not be described again here.

[0181] Those skilled in the art will understand that all or part of the steps for realizing the above embodiments may be completed by hardware, or may be completed by instructing related hardware with a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0182] In another aspect, there is provided a visual software development apparatus including a processor and a memory storing instructions executable by the processor, the processor being configured to perform any of the steps of the visual software development method described above.

[0183] In another aspect, there is provided a computer readable storage medium storing instructions which, when executed by a processor, perform any of the steps of the visual software development method described above.

[0184] In another aspect, there is further provided a computer program product comprising commands which, when executed on a computer, cause the computer to perform any of the steps of the visual software development method described above.

[0185] The above description is merely a preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A visual software development system including an application interface layer and a data processing layer, The application interface layer is used for displaying N module display controls, each of the N module display controls corresponding to N modules, and module data of each of the N modules is stored in the data processing layer, and the module data includes an execution logic of the corresponding module; the application interface layer is further used for displaying a module addition control, and the module addition control is used for introducing the module data of a third-party module into the data processing layer, and the N modules include the third-party module, and the N is a positive integer greater than or equal to 1; The application interface layer is further used for displaying a flowchart in response to a first user operation on one or more of the N module display controls, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls; the data processing layer is used for calling and executing the module data of the modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer, obtaining a data processing result of the flowchart, and returning the data processing result to the application interface layer; A visual software development system.

2. the application interface layer is used to obtain, in response to a second operation on the module addition control, the module data of the third party module installed by the user through the module addition control, and transmit the module data of the installed third party module to the data processing layer; When the data processing layer determines that the module data of the introduced third-party module meets the reference rule, the data processing layer is used to store the module data of the introduced third-party module and return a module loading success message to the application interface layer, where the module loading success message indicates that the third-party module is successfully loaded. The system of claim 1 .

3. The reference rule includes a design method of an input / output port of the third-party module and / or a packaging method of the module data of the third-party module; The design manner of the input / output port of the third-party module includes a format of the input data and / or output data of the third-party module; the packaging manner of the module data of the third-party module indicates a storage format of the module data of the third-party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file; The system of claim 2.

4. the application interface layer is used for displaying a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule; 4. A system according to claim 2 or 3.

5. the application interface layer is further used for displaying a module configuration interface of a first module corresponding to a first module display control in response to a fourth operation of the user on a first module display control among the N module display controls, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls instructs the user to edit a parameter in an execution process of the first module; the application interface layer is further used for obtaining a parameter edited by the user through any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls, and transmitting the edited parameter to the data processing layer; the data processing layer is further used for updating the module data of the first module based on the edited parameters; When the flow chart includes the first module, the data processing layer is further used for updating the data processing result of the flow chart according to the updated module data of the first module, and returning the updated data processing result to the application interface layer. The system of claim 1 .

6. If the first module is the third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicating a program code of the module configuration interface of the first module; the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module; The system of claim 5.

7. The data processing layer is further used for generating a first code package according to the flowchart, the first code package indicating an execution logic of each module included in the flowchart and an execution order between each module included in the flowchart; The data processing layer is further used for controlling the application interface layer to display a first code control corresponding to the first code package, and the first code control is used for instructing the user to download the first code package, the first code package supports being introduced into a third party system, and displays the flowchart after being executed in the third party system. The system of claim 1 .

8. The application interface layer is further used for displaying an execution interface display control, the execution interface display control instructs to display an execution interface for the flowchart, the execution interface is used for displaying execution information of the flowchart, and the execution information of the flowchart indicates intermediate data in the execution process of the flowchart; the application interface layer is further used for displaying the execution interface in response to a sixth operation on the execution interface display control, so that the user can view intermediate data in the execution process of the flowchart based on the execution interface. The system of claim 1 .

9. the application interface layer is further used for displaying an execution interface configuration control, the execution interface configuration control instructing to display an execution interface configuration interface, the execution interface configuration interface being used for configuring content displayed in the execution interface; the application interface layer is further adapted to display the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, one of the execution interface configuration sub-controls having a launch option; the application interface layer is further used for displaying a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and binding the first execution interface configuration sub-control with a module in the flowchart in response to a ninth operation on the content source option; the execution information displayed on the execution interface includes data of a module bound to the first execution interface configuration sub-control; The system of claim 8.

10. The data processing layer is further used for deriving the program code of the execution interface as a second code package; The data processing layer is further used for controlling the application interface layer to display a second code control corresponding to the second code package, the second code control is for instructing the user to download the second code package, the second code package supports to be introduced into a third party system, and displays the execution interface after being executed in the third party system. The system of claim 8.

11. the data processing layer includes a data scheduling layer and a function module layer; The module data of the N modules is stored in the functional module layer, The data scheduling layer is further used for sequentially calling and executing the module data of the modules corresponding to each of the one or more module display controls from the function module layer according to an execution order between the modules corresponding to each of the one or more module display controls in the flowchart, and obtaining the data processing result. A system according to any one of claims 1 to 10.

12. The data processing layer is used to store the module data of M modules, where M is a positive integer equal to or greater than N; the application interface layer is further adapted to display a module registration control, and the application interface layer is further adapted to display the N module display controls in response to the N modules selected by the user from the M modules through the module registration control. The system of claim 1 .

13. The N modules are modules that the data processing layer verifies and determines to be valid, and the modules that the data processing layer verifies and determines to be valid are corresponding modules that the user has the authority to use. The system of claim 12.

14. The application interface layer is further adapted to display an invocation control for any of the module display controls, the invocation control indicating that a user is authorized to allow a third party system to invoke a module corresponding to any of the module display controls; the application interface layer is further used for displaying a port link of the module data of a module corresponding to any one of the module display controls in response to a tenth operation on the call control; The application interface layer is further used for obtaining the module data of the module corresponding to any one of the module display controls from the data processing layer according to the port link when detecting a data call request for the port link sent from the third party system, and sending the obtained module data to the third party system. The system of claim 1 .

15. 1. A visual software development method applied to a visual software development system including an application interface layer and a data processing layer, the method comprising: The application interface layer displays N module display controls, each of which corresponds to N modules, and module data of each of the N modules is stored in the data processing layer, and the module data includes an execution logic of the corresponding module; the application interface layer further displays a module addition control, and the module addition control is used to introduce the module data of a third-party module into the data processing layer, and the N modules include the third-party module, and the N is a positive integer equal to or greater than 1; the application interface layer displays a flowchart in response to a first user operation on one or more of the N module display controls, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls; The data processing layer, in response to the flowchart displayed on the application interface layer, calls and executes the module data of the module corresponding to each of the one or more module display controls, obtains a data processing result of the flowchart, and returns the data processing result to the application interface layer. Visual software development methods.

16. The method comprises: the application interface layer, in response to a second operation on the module addition control, obtains the module data of the third party module installed by the user through the module addition control, and transmits the module data of the installed third party module to the data processing layer; When the data processing layer determines that the module data of the installed third-party module satisfies a reference rule, the data processing layer stores the module data of the installed third-party module and returns a module load success message to the application interface layer, the module load success message indicating that the third-party module is successfully loaded. The method of claim 15.

17. The reference rule includes a design method of an input / output port of the third-party module and / or a packaging method of the module data of the third-party module; The design manner of the input / output port of the third-party module includes a format of the input data and / or output data of the third-party module; the packaging manner of the module data of the third-party module indicates a storage format of the module data of the third-party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file; 17. The method of claim 16.

18. The method comprises: the application interface layer further includes displaying a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule.

18. The method according to claim 16 or 17.

19. The method comprises: the application interface layer, in response to a fourth operation by the user on a first module display control among the N module display controls, displays a module configuration interface of a first module corresponding to the first module display control, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls instructs the user to edit a parameter in an execution process of the first module; the application interface layer, in response to a fifth operation on any one of the module configuration controls, acquires a parameter edited by the user through any one of the module configuration controls, and transmits the edited parameter to the data processing layer; The data processing layer further includes updating the module data of the first module based on the edited parameters; When the flow chart includes the first module, the data processing layer updates the data processing result of the flow chart based on the updated module data of the first module, and returns the updated data processing result to the application interface layer. The method of claim 15.

20. If the first module is the third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicating a program code of the module configuration interface of the first module; the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module; 20. The method of claim 19.

21. The method comprises: The data processing layer generates a first code package based on the flowchart, the first code package indicating an execution logic of each module included in the flowchart and an execution order between each module included in the flowchart; The data processing layer further includes controlling the application interface layer so that the application interface layer displays a first code control corresponding to the first code package, the first code control is used to instruct the user to download the first code package, the first code package supports being introduced into a third party system, and displaying the flowchart after being executed in the third party system. The method of claim 15.

22. The method comprises: the application interface layer displays an execution interface display control, the execution interface display control instructs to display an execution interface for the flowchart, the execution interface is used to display execution information of the flowchart, and the execution information of the flowchart indicates intermediate data in the execution process of the flowchart; the application interface layer further includes, in response to a sixth operation on the execution interface display control, displaying the execution interface so that the user can view intermediate data in the execution process of the flowchart based on the execution interface. The method of claim 15.

23. The method comprises: the application interface layer displays an execution interface configuration control, the execution interface configuration control instructs displaying an execution interface configuration interface, the execution interface configuration interface is used to configure content displayed in the execution interface; the application interface layer displays the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, and any execution interface configuration sub-control of the plurality of execution interface configuration sub-controls has a launch option; the application interface layer further includes: displaying a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls; and binding the first execution interface configuration sub-control to one module in the flowchart in response to a ninth operation on the content source option; the execution information displayed on the execution interface includes data of a module bound to the first execution interface configuration sub-control; 23. The method of claim 22.

24. The method comprises: the data processing layer deriving the program code of the execution interface as a second code package; The data processing layer further includes controlling the application interface layer so that the application interface layer displays a second code control corresponding to the second code package, the second code control is used to instruct the user to download the second code package, the second code package supports being introduced into a third party system, and after being executed in the third party system, displays the execution interface.

23. The method of claim 22.

25. the data processing layer includes a data scheduling layer and a function module layer; The module data of the N modules is stored in the functional module layer, the data processing layer, in response to the flowchart displayed on the application interface layer, calls and executes the module data of the module corresponding to each of the one or more module display controls, and obtains the data processing result of the flowchart; The data scheduling layer includes sequentially calling and executing the module data of the modules corresponding to each of the one or more module display controls from the function module layer according to an execution order between the modules corresponding to each of the one or more module display controls in the flowchart, and obtaining the data processing result.

25. The method according to any one of claims 15 to 24.

26. The method comprises: The application interface layer displays a module registration control, and the application interface layer displays the N module display controls in response to the N modules selected by the user from the M modules through the module registration control, and the module data of the M modules is stored in the data processing layer, and the M is a positive integer equal to or greater than the N. The method of claim 15.

27. The N modules are modules that the data processing layer verifies and determines to be valid, and the modules that the data processing layer verifies and determines to be valid are corresponding modules that the user has the authority to use.

27. The method of claim 26.

28. The method comprises: the application interface layer displays an invocation control for any of the module display controls, the invocation control indicating that a user is authorizing a third party system to invoke a module corresponding to any of the module display controls; the application interface layer is further used for displaying a port link of the module data of a module corresponding to any one of the module display controls in response to a tenth operation on the call control; The application interface layer further includes, when detecting a data call request for the port link sent from the third party system, acquiring the module data of the module corresponding to any one of the module display controls from the data processing layer based on the port link, and sending the acquired module data to the third party system. The method of claim 15.

29. A visual software development apparatus configured in a visual software development system including an application interface layer and a data processing layer, The device includes a display module and an execution module; The display module is used to control the application interface layer to display N module display controls, the N module display controls respectively corresponding to N modules, and module data of each of the N modules is stored in the data processing layer, and the module data includes an execution logic of the corresponding module; the application interface layer further displays a module addition control, and the module addition control is used to introduce the module data of a third-party module into the data processing layer, the N modules include the third-party module, and the N is a positive integer greater than or equal to 1; The display module is further used for controlling the application interface layer to display a flowchart in response to a first user operation on one or more module display controls among the N module display controls, the flowchart indicating an execution order among modules corresponding to each of the one or more module display controls; The execution module is used for controlling the data processing layer to call and execute the module data of the modules corresponding to each of the one or more module display controls in response to the flowchart displayed on the application interface layer, obtain a data processing result of the flowchart, and return the data processing result to the application interface layer. A visual software development tool.

30. The apparatus further includes an acquisition module and a determination module; the acquiring module is used for controlling the application interface layer to acquire the module data of the third party module introduced by the user through the module adding control in response to a second operation on the module adding control, and to transmit the module data of the introduced third party module to the data processing layer; When the determination module determines that the module data of the introduced third-party module meets the reference rule, the determination module is used to control the data processing layer to store the module data of the introduced third-party module and return a module loading success message to the application interface layer, and the module loading success message indicates that the loading of the third-party module is successful.

30. The apparatus of claim 29.

31. The reference rule includes a design method of an input / output port of the third-party module and / or a packaging method of the module data of the third-party module; The design manner of the input / output port of the third-party module includes a format of the input data and / or output data of the third-party module; the packaging manner of the module data of the third-party module indicates a storage format of the module data of the third-party module in the data processing layer, and the storage format includes any one of a dynamic link library, a static link library, and an executable exe file; 31. The apparatus of claim 30.

32. the display module is further used for controlling the application interface layer to display a module configuration file in response to a third operation on the module addition control, the module configuration file indicating the reference rule; 32. Apparatus according to claim 30 or 31.

33. The display module is further used for controlling the application interface layer to display a module configuration interface of a first module corresponding to a first module display control in response to a fourth operation of the user on a first module display control among the N module display controls, the module configuration interface including a plurality of module configuration controls, and any one of the plurality of module configuration controls is used for instructing the user to edit a parameter in an execution process of the first module; The apparatus further includes an acquisition module and an update module; the acquiring module is used for controlling the application interface layer to acquire parameters edited by the user through any one of the module configuration controls in response to a fifth operation on any one of the module configuration controls, and to transmit the edited parameters to the data processing layer; the update module is used to control the data processing layer to update the module data of the first module based on the edited parameters; When the flow chart includes the first module, the data processing layer updates the data processing result of the flow chart based on the updated module data of the first module, and returns the updated data processing result to the application interface layer.

30. The apparatus of claim 29.

34. If the first module is a third-party module, the module data of the first module introduced by the user through the module addition control further includes a module configuration interface file, the module configuration interface file indicates a program code of the module configuration interface of the first module, and the module configuration interface of the first module displayed on the application interface layer is displayed according to the module configuration interface file in the module data of the first module.

34. The apparatus of claim 33.

35. The apparatus further includes a generating module; The generating module is used to control the data processing layer to generate a first code package according to the flowchart, the first code package instructing an execution logic of each module included in the flowchart and an execution order between each module included in the flowchart; The display module is further used for controlling the data processing layer, and the data processing layer is further used for controlling the application interface layer to display a first code control corresponding to the first code package, and the first code control is used for instructing the user to download the first code package, and the first code package supports being introduced into a third party system, and displays the flowchart after being executed in the third party system.

30. The apparatus of claim 29.

36. The display module is further used for controlling the application interface layer to display an execution interface display control, the execution interface display control instructs to display an execution interface for the flowchart, the execution interface is used for displaying execution information of the flowchart, and the execution information of the flowchart indicates intermediate data in the execution process of the flowchart; the display module is further used for controlling the application interface layer to display the execution interface in response to a sixth operation on the execution interface display control, so as to allow the user to view intermediate data in the execution process of the flowchart based on the execution interface.

30. The apparatus of claim 29.

37. The display module is further used for controlling the application interface layer to display an execution interface configuration control, the execution interface configuration control instructs displaying an execution interface configuration interface, and the execution interface configuration interface is used for configuring content displayed on the execution interface; The display module is further used for controlling the application interface layer to display the execution interface configuration interface in response to a seventh operation on the execution interface configuration control, the execution interface configuration interface including a plurality of execution interface configuration sub-controls, and any execution interface configuration sub-control of the plurality of execution interface configuration sub-controls has a launch option; The display module is further used for controlling the application interface layer to display a content source option in response to an eighth operation on a launch option of a first execution interface configuration sub-control in the plurality of execution interface configuration sub-controls, and to bind the first execution interface configuration sub-control and a module in the flowchart in response to a ninth operation on the content source option; the execution information displayed on the execution interface includes data of a module bound to the first execution interface configuration sub-control; 37. The apparatus of claim 36.

38. The apparatus further includes a derivation module; The deriving module is used for controlling the data processing layer to derive the program code of the execution interface as a second code package; The display module is further used for controlling the data processing layer, and the data processing layer is further used for controlling the application interface layer to display a second code control corresponding to the second code package, and the second code control is used for instructing the user to download the second code package, and the second code package supports being introduced into a third party system, and displays the execution interface after being executed in the third party system.

37. The apparatus of claim 36.

39. the data processing layer includes a data scheduling layer and a function module layer; The module data of the N modules is stored in the functional module layer, The execution module is used for controlling the data scheduling layer to sequentially call and execute the module data of the modules corresponding to each of the one or more module display controls from the function module layer according to an execution order between the modules corresponding to each of the one or more module display controls in the flowchart, and obtain the data processing result.

39. Apparatus according to any one of claims 29 to 38.

40. The display module is further used for controlling the application interface layer to display a module registration control, and the application interface layer displays the N module display controls in response to the N modules selected by the user from the M modules through the module registration control, and the data processing layer stores the module data of the M modules, and the M is a positive integer equal to or greater than the N.

30. The apparatus of claim 29.

41. The N modules are modules that the data processing layer verifies and determines to be valid, and the modules that the data processing layer verifies and determines to be valid are corresponding modules that the user has the authority to use.

41. The apparatus of claim 40.

42. The display module is further used to control the application interface layer to display an invocation control for any of the module display controls, the invocation control indicating that the user is authorized to allow a third party system to invoke a module corresponding to any of the module display controls; The display module is further used for controlling the application interface layer to display a port link of the module data of a module corresponding to any one of the module display controls in response to a tenth operation on the call control; The apparatus further includes an acquisition module; The acquisition module is used to control the application interface layer to acquire the module data of the module corresponding to any one of the module display controls from the data processing layer according to the port link when detecting a data call request for the port link sent from the third party system, and to send the acquired module data to the third party system.

30. The apparatus of claim 29.

43. An apparatus comprising a processor and a memory storing instructions executable by the processor, the processor being configured to perform the steps of the method according to any one of claims 15 to 28. A visual software development tool.

44. A computer readable storage medium storing instructions which, when executed by a processor, perform the steps of the method of any one of claims 15 to 28. A computer-readable storage medium.

45. A computer program comprising instructions, which when executed on a computer, cause the computer to carry out the steps of the method according to any one of claims 15 to 28. Computer program.

Images