Method, device, medium, terminal and program product for generating a visual inspection program

Abstract

The application provides a visual detection program generation method, device, medium, terminal and program product. The method comprises the following steps: receiving a dragging operation of dragging an algorithm module from a pre-configured algorithm library to a current working area, and taking each algorithm module in the current working area as an algorithm node; receiving a connection operation of connecting the algorithm nodes in the current working area, and generating an algorithm flow based on the connection operation, wherein the connection order represents the execution order of the algorithm nodes in the current working area; receiving a parameter adjustment of the algorithm modules in the current working area, and saving the adjusted parameters of each algorithm module to a configuration file; performing camera configuration, and saving the configuration parameters of the camera obtained after the camera configuration to the configuration file; and generating a visual detection program based on the generated algorithm flow and the configuration file. The scheme provided by the application can effectively improve the code reuse rate and reduce the development cost.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and more particularly to a method, apparatus, medium, terminal, and program product for generating a visual inspection program. Background Technology

[0002] With the rapid development of intelligent manufacturing and Industry 4.0, industrial vision inspection technology is playing an increasingly important role in product quality control, defect identification, and dimensional measurement.

[0003] However, most industrial vision systems adopt a customized development model of one algorithm per project, resulting in low algorithm reuse rate, long development cycle, and complex system integration, making it difficult to meet the needs of rapid deployment in multiple scenarios and production lines. Summary of the Invention

[0004] The main objective of this invention is to overcome the deficiencies of the aforementioned related technologies and provide an air conditioner and its control method, device, storage medium, and computer program product to solve the problems of low algorithm reuse rate and long development cycle caused by the customized development mode of industrial vision systems in the related technologies.

[0005] This invention provides a method for generating a visual inspection program, comprising: receiving a drag-and-drop operation from a pre-configured algorithm library to a current working area, wherein each algorithm module dragged to the current working area is considered an algorithm node; receiving a connection operation to connect the algorithm nodes in the current working area, and generating an algorithm flow based on the connection operation, wherein the order of the connections represents the execution order of the algorithm nodes in the current working area; receiving parameter adjustments to the algorithm modules in the current working area, and saving the adjusted parameters of each algorithm module to a configuration file; configuring a camera, and saving the camera configuration parameters obtained after camera configuration to a configuration file; and generating a visual inspection program based on the generated algorithm flow and the configuration file.

[0006] Optionally, the algorithm modules may be categorized into at least two of the following: image processing, localization, recognition, deep learning, calibration, communication, and logic.

[0007] Optionally, each of the algorithm modules is encapsulated as an independent class that inherits from an abstract base class.

[0008] Optionally, the various algorithm modules may interact with each other using a preset serialization format.

[0009] Optionally, it also includes: each of the algorithm modules has a built-in decorator to capture exceptions in the execution function and return the captured exception data in a custom format.

[0010] Optionally, camera configuration includes: acquiring SDKs for two or more different cameras and encapsulating the acquired SDKs for the two or more different cameras; acquiring a predefined unified abstract interface for different cameras and calling the APIs of the SDKs for the two or more different cameras through the unified abstract interface to capture images of the object to be detected using different cameras.

[0011] Optionally, it also includes: configuring the PLC device, including: encapsulating different PLC device communication protocols into abstract classes as a unified interface, and creating implementation classes for each PLC device communication protocol; the unified interface implements at least one of PLC connection, register read / write, instruction issuance, and disconnection.

[0012] Another aspect of the present invention provides a visual inspection program generation apparatus, comprising: a first receiving unit, configured to receive drag-and-drop operations of algorithm modules from a pre-configured algorithm library to a current working area, wherein each algorithm module dragged to the current working area is considered an algorithm node; a second receiving unit, configured to receive connection operations of connecting algorithm nodes in the current working area, and generate an algorithm flow based on the connection operations, wherein the order of the connections represents the execution order of the algorithm nodes in the current working area; a third receiving unit, configured to receive parameter adjustments of algorithm modules in the current working area, and save the adjusted parameters of each algorithm module to a configuration file; a camera configuration unit, configured to perform camera configuration, and save the camera configuration parameters obtained after camera configuration to a configuration file; and a generation unit, configured to generate a visual inspection program based on the generated algorithm flow and the configuration file.

[0013] Optionally, the algorithm modules may be categorized into at least two of the following: image processing, localization, recognition, deep learning, calibration, communication, and logic.

[0014] Optionally, each of the algorithm modules is encapsulated as an independent class that inherits from an abstract base class.

[0015] Optionally, the various algorithm modules may interact with each other using a preset serialization format.

[0016] Optionally, it also includes: each of the algorithm modules has a built-in decorator to capture exceptions in the execution function and return the captured exception data in a custom format.

[0017] Optionally, the camera configuration unit performs camera configuration, including: acquiring SDKs for two or more different cameras and encapsulating the acquired SDKs for the two or more different cameras; acquiring a predefined unified abstract interface for different cameras and calling the APIs of the SDKs for the two or more different cameras through the unified abstract interface to capture images of the object to be detected by different cameras.

[0018] Optionally, it further includes: a device configuration unit for configuring PLC devices, including: encapsulating different PLC device communication protocols into abstract classes as a unified interface, and creating implementation classes for each PLC device communication protocol; the unified interface implements at least one of PLC connection, register read / write, instruction issuance, and disconnection.

[0019] In another aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0020] In another aspect, the present invention provides a terminal including a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0021] In another aspect, the present invention provides a terminal including the apparatus for generating the visual inspection program as described above.

[0022] In another aspect, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0023] According to the technical solution of the present invention, code reuse rate can be effectively improved and development costs can be reduced; drag-and-drop visual inspection solution development can be realized, the development cycle from requirements to deployment can be greatly shortened, and the project response speed can be improved. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0025] Figure 1 This is a schematic diagram of an embodiment of the method for generating a visual inspection program provided by the present invention;

[0026] Figure 2 A schematic diagram of the main interface for configuring a visual inspection program according to an embodiment of the present invention is shown;

[0027] Figure 3An example of an algorithm library according to an embodiment of the present invention is shown;

[0028] Figure 4 An example diagram of the algorithm module parameter adjustment interface is shown;

[0029] Figure 5 A schematic diagram of the generation algorithm flowchart interface is shown;

[0030] Figure 6 A schematic diagram of the communication equipment configuration is shown;

[0031] Figure 7 This is a structural block diagram of an embodiment of the visual inspection program generation device provided by the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] This invention provides a method for generating a visual inspection program. The visual inspection program of this invention can be developed using the Python language, the front-end interface can be built using PyQt5, and the back-end logic adopts a modular design, transforming complex visual inspection algorithms into simple modules.

[0035] Figure 1 This is a schematic diagram of an embodiment of the visual inspection program generation method provided by the present invention.

[0036] like Figure 1As shown, according to an embodiment of the present invention, the method for generating the visual inspection program includes at least steps S110, S120, S130 and S140.

[0037] Step S110: Receive drag-and-drop operations from a pre-configured algorithm library to the current work area, where each algorithm module dragged to the current work area is treated as an algorithm node.

[0038] In one specific implementation, the visual detection algorithm modules are divided into two or more categories. In another specific implementation, the categories of algorithm modules include at least two of image processing, localization, recognition, deep learning, calibration, communication, and logic. Each category includes two or more algorithm modules. Algorithms from the algorithm library can be dragged (e.g., dragged with the mouse) to the current work area as needed. The work area is used to visually display the selected algorithm modules and the set algorithm execution order; the work area can be, for example, a canvas. Figure 2 A schematic diagram of the main interface for configuring a visual inspection program according to an embodiment of the present invention is shown.

[0039] For example, Figure 3 An example of an algorithm library according to an embodiment of the present invention is shown. References Figure 3 As shown, image processing includes algorithm modules such as affine transformation, edge detection, channel fusion, gamma transformation, and geometric transformation; localization includes algorithm modules such as region detection, circle detection, contour detection, line detection, line fitting, and template matching; recognition includes algorithm modules such as barcode recognition, counterweight detection, missing object recognition, and character recognition; calibration includes algorithm modules such as calibration mapping, chessboard calibration, icon calibration, and translation calibration; communication includes algorithm modules such as camera triggering, communication receiving, communication sending, and HTTP communication; and logic includes conditional branching, conditional detection, and output mapping.

[0040] Preferably, each algorithm module is encapsulated as an independent class that inherits from a preset abstract base class. For example, each algorithm module is an independent class that inherits from the abstract base class `base_algorithm_class`. Encapsulating each algorithm module as an independent class ensures isolation between algorithm modules, facilitating maintenance and expansion. Since each algorithm module inherits from the `base_algorithm_class` abstract base class, through inheritance, all algorithm modules possess the basic attributes and methods defined in the base class, and can implement their own specific functional logic through method overriding. Through class inheritance and overriding, the basic attributes of the algorithm (such as node status, process number, etc.) are preserved.

[0041] Preferably, the various algorithm modules interact with each other using a preset serialization format. Specifically, the various algorithm modules interact with each other using a standardized serialization format (e.g., JSON) (following the RFC 8259 specification) to achieve cross-module communication and ensure seamless communication and data exchange between different algorithm modules.

[0042] Preferably, the development parameters of each algorithm module in the algorithm library can be pre-tuned. Specifically, the development parameters of each algorithm module can provide single-time and multiple-time debugging operations for users to perform debugging. Users can determine the optimal parameters through visualization results and detection technology requirements. Figure 4 A sample diagram of the module parameter debugging interface is shown. For example... Figure 4 As shown, parameters can be debugged for each algorithm module. For example... Figure 4 The parameter debugging interface of the image fusion module, contour search module, and template matching module is shown.

[0043] Preferably, each algorithm module has a built-in decorator to capture exceptions in the executed function and return the captured exception data (error data) in a custom format. Specifically, by judging other anomalies such as incorrect input data format and incorrect algorithm processing results, and returning descriptive information, parameter input correction can be achieved, realizing a stable detection scheme. Returning error information from all algorithms in a standard format unifies the error output format, facilitating subsequent processing or logging. This standardizes the algorithm input / output data format and algorithm execution interface.

[0044] Step S120: Receive connection operations to connect algorithm nodes in the current working area, and generate an algorithm flow based on the connection operations.

[0045] Each algorithm module dragged to the current work area is treated as an algorithm node. The system receives connection operations that link algorithm nodes within the current work area and generates the algorithm flow based on the order of these connections. Specifically, nodes can be linked by clicking with the mouse to create directed connections and record the node execution order. The order in which algorithm nodes are linked indicates the execution order of the algorithm nodes within the current work area.

[0046] The connection operation can specifically include: a click operation, where the first clicked algorithm node serves as the starting node. For example, the required algorithm module can be dragged to the current work area (e.g., a canvas) as an algorithm node, and a connection can be made from the first clicked algorithm node (starting node) to the next algorithm node using the mouse. The order in which the nodes are connected determines the execution order of the algorithm nodes in the current work area. Based on the received connection operations, the algorithm nodes in the current work area are associated, that is, each algorithm node is associated vertically according to the connection operations. Figure 5 A schematic diagram of the generation algorithm flowchart interface is shown. For example... Figure 5 As shown, the lines are directional arrows. Based on these line connections, the algorithm nodes in the current working area are connected to form a directed acyclic graph (DAG) to manage the execution logic. That is, the backend uses a DAG to manage the execution logic, using a DAG data structure to store nodes and connections, ensuring that there are no loops in the flowchart (i.e., nodes cannot be cyclically dependent), thus avoiding infinite loops in the process. When executing the algorithm flow, the process can be executed by triggering the root node. For example, the trigger node for the vision platform is the input image, triggered by a local image or a camera capture.

[0047] Step S130: Receive parameter adjustments made to the algorithm modules in the current working area, and save the adjusted parameters of each algorithm module to the configuration file.

[0048] Specifically, the parameters of each algorithm module in the current working area can be adjusted in real time. The parameters of each algorithm module can be configured for both single and multiple debugging operations. Users can determine the optimal parameters by visualizing the results and considering the technical requirements of the detection. Figure 4 A sample diagram of the algorithm module parameter adjustment interface is shown. For example... Figure 4 As shown, the parameters of the algorithm module can be debugged. The parameters of the algorithm module can be adjusted in real time through the adjustment interface on the left and saved to the configuration file.

[0049] Optionally, when executing the algorithm flowchart, if there is an independent algorithm node that is not connected to other algorithm nodes, the algorithm node is skipped.

[0050] Specifically, each algorithm node is interconnected with its preceding and following nodes, executing according to the algorithm flowchart structure. Independent nodes are dynamically skipped and are called invalid nodes. The conditional branching algorithm module can dynamically execute debugging nodes by setting judgment conditions, such as the number of detected targets, and executing different branches based on the number of detection results. This is called inserting debugging nodes.

[0051] Step S140: Configure the camera and save the camera configuration parameters obtained after configuration to the configuration file.

[0052] In one specific implementation, camera configuration may include: acquiring SDKs for two or more different cameras and encapsulating the acquired SDKs for the two or more different cameras; acquiring a predefined unified abstract interface for different cameras and calling the APIs of the SDKs for the two or more different cameras through the unified abstract interface to capture images of the object to be detected using different cameras.

[0053] Specifically, an abstract class `BaseCamera` is created as a unified abstract interface. This `BaseCamera` abstract class implements at least one of the following functions: camera IP configuration, trigger mode setting, exposure frame rate setting, and image acquisition. It imports the API (Application Programming Interface) interfaces from the development documentation of the SDKs (Software Development Kits) of the two or more different cameras, and overrides the functionalities (paraphrasing the provided SDKs into platform-style code, such as unified log and interface interfaces). That is, the development documentation of the SDKs of the two or more different cameras is converted into API interface descriptions recognizable by the (current platform), establishing API mapping relationships. Based on the created `BaseCamera` abstract class, specific implementation classes are created for each of the two or more different cameras, and the corresponding camera's SDK API is called within these implementation classes. According to the API interfaces of the SDKs of the two or more different cameras, the abstract methods defined in the abstract class are overridden, converting the unified abstract interface into SDK calling code, thereby allowing the SDKs of the two or more different cameras to be called through the unified abstract interface.

[0054] Preferably, the method may further include configuring a PLC device. The PLC device is used to connect a terminal executing the vision inspection program and a vision inspection result response device. Specifically, the terminal executing the vision inspection program can be a computer. The vision inspection result response device can specifically be an industrial robot or a sorting device. The PLC device is used to control the vision inspection result response device. For example, controlling the industrial robot or sorting device to process defective workpieces, such as selecting out defective workpieces. The terminal (e.g., a computer) where the vision inspection program (vision software) resides is the host computer, and the PLC (Programmable Logic Controller) device is the slave computer, communicating with the host computer to convert the host computer's inspection results into logical actions on the slave computer.

[0055] In one specific implementation, PLC device configuration may include: encapsulating different PLC device communication protocols into abstract classes as a unified interface, and creating corresponding implementation classes for each PLC device communication protocol; wherein the unified interface implements at least one of PLC connection, register read / write, instruction issuance, and disconnection.

[0056] Specifically, an abstract class `BasePLC` is created. Based on this `BasePLC` abstract class, corresponding concrete implementation classes are created for each type of PLC device communication protocol. These different PLC device communication protocols include at least one of Modbus TCP, Siemens S7, and Mitsubishi MC protocols. For example, the Modbus TCP implementation class can be encapsulated using the pymodbus library, and the Siemens S7 implementation class can be encapsulated using the snap7 library. Modbus TCP protocol communication is implemented using the pymodbus library, and Siemens S7 protocol communication is implemented using the snap7 library. Network communication is implemented through third-party libraries to complete the data exchange between the PLC device and the host computer.

[0057] Furthermore, the camera configuration parameters obtained after camera configuration are saved to a configuration file. These configuration parameters may specifically include: camera model, IP address, and port. Preferably, the PLC device configuration parameters obtained after PLC device configuration can also be saved to a configuration file. These PLC device configuration parameters may specifically include TCP communication and PLC-related parameters. Specifically, configuration parameters can be stored using a JSON or YAML configuration file. For example, by uniformly storing camera model, IP address, port, TCP communication-related parameters (e.g., IP address, port, and automatic reconnection parameters) and PLC-related parameters (e.g., brand, protocol, and at least one of the PLC point logic parameters) in a JSON or YAML configuration file, configurable startup of the device can be achieved. Figure 6 A schematic diagram of the communication equipment configuration is shown.

[0058] Step S150: Based on the generated algorithm flow and the configuration file, generate a visual inspection program.

[0059] Preferably, a visual inspection program is generated based on the generated algorithm flow and the camera and PLC equipment configurations.

[0060] This invention generates program code, which is then packaged in an isolated environment for cross-platform use. Specifically, PyInstaller is used to package the program in an isolated venv environment, including only necessary dependencies (such as OpenCV, NumPy, PyQt5, PyPylon, etc.), third-party libraries, modules, binary files, and configuration files, separating them from the project code for easy and rapid software updates. The algorithm module is embedded in encrypted form and dynamically loaded at startup. Furthermore, startup signature verification can be added to prevent program tampering. Using a Python virtual environment (venv) to create an independent runtime environment avoids conflicts with other Python packages, ensuring the purity of the packaging environment. This allows for software packaging in less than 10 minutes and can be used on Windows 7, Windows 10, and Windows 11 systems, significantly improving deployment efficiency.

[0061] This invention also provides a device for generating a visual inspection program. The visual inspection program of this invention can be developed using the Python language, the front-end interface can be built using PyQt5, and the back-end logic adopts a modular design, transforming complex visual inspection algorithms into simple modules.

[0062] Figure 7 This is a structural block diagram of an embodiment of the visual inspection program generation device provided by the present invention. Figure 7 As shown, the visual inspection program generation device 100 includes: a first receiving unit 110, a second receiving unit 120, a third receiving unit 130, a camera configuration unit 140, and a generation unit 150.

[0063] The first receiving unit 110 is configured to receive drag-and-drop operations from a pre-configured algorithm library to the current working area, wherein each algorithm module dragged to the current working area is considered as an algorithm node.

[0064] In one specific implementation, the visual detection algorithm modules are divided into two or more categories. In another specific implementation, the categories of algorithm modules include at least two of image processing, localization, recognition, deep learning, calibration, communication, and logic. Each category includes two or more algorithm modules. Algorithms from the algorithm library can be dragged (e.g., dragged with the mouse) to the current work area as needed. The work area is used to visually display the selected algorithm modules and the set algorithm execution order; the work area can be, for example, a canvas. Figure 2 A schematic diagram of the main interface for configuring a visual inspection program according to an embodiment of the present invention is shown.

[0065] For example, Figure 3 An example of an algorithm library according to an embodiment of the present invention is shown. References Figure 3As shown, image processing includes algorithm modules such as affine transformation, edge detection, channel fusion, gamma transformation, and geometric transformation; localization includes algorithm modules such as circle detection, contour detection, and line detection; recognition includes counterweight detection and character recognition; calibration includes calibration mapping, chessboard calibration, icon calibration, and translation calibration; communication includes camera triggering, communication receiving, communication sending, and PLC receiving; and logic includes conditional branching, conditional detection, and output mapping.

[0066] Preferably, each algorithm module is encapsulated as an independent class that inherits from a preset abstract base class. For example, each algorithm module is an independent class that inherits from the abstract base class `base_algorithm_class`. Encapsulating each algorithm module as an independent class ensures isolation between algorithm modules, facilitating maintenance and expansion. Since each algorithm module inherits from the `base_algorithm_class` abstract base class, through inheritance, all algorithm modules possess the basic attributes and methods defined in the base class, and can implement their own specific functional logic through method overriding. Through class inheritance and overriding, the basic attributes of the algorithm (such as node status, process number, etc.) are preserved.

[0067] Preferably, the various algorithm modules interact with each other using a preset serialization format. Specifically, the various algorithm modules interact with each other using a standardized serialization format (e.g., JSON) (following the RFC 8259 specification) to achieve cross-module communication and ensure seamless communication and data exchange between different algorithm modules.

[0068] Preferably, the development parameters of each algorithm module in the algorithm library can be pre-tuned. Specifically, the development parameters of each algorithm module can provide single-time and multiple-time debugging operations for users to perform debugging. Users can determine the optimal parameters through visualization results and detection technology requirements. Figure 4 A sample diagram of the module parameter debugging interface is shown. For example... Figure 4 As shown, parameters can be debugged for each algorithm module. For example... Figure 4 The parameter debugging interface of the image fusion module, contour search module, and template matching module is shown.

[0069] Preferably, each algorithm module has a built-in decorator to capture exceptions in the executed function and return the captured exception data (error data) in a custom format. Specifically, by judging other anomalies such as incorrect input data format and incorrect algorithm processing results, and returning descriptive information, parameter input correction can be achieved, realizing a stable detection scheme. Returning error information from all algorithms in a standard format unifies the error output format, facilitating subsequent processing or logging. This standardizes the algorithm input / output data format and algorithm execution interface.

[0070] The second receiving unit 120 is used to receive connection operations that connect algorithm nodes in the current working area, and generate an algorithm flow based on the connection operations, wherein the order of the connections represents the execution order of the algorithm nodes in the current working area.

[0071] Each algorithm module dragged to the current work area is treated as an algorithm node. The system receives connection operations that link algorithm nodes within the current work area and generates the algorithm flow based on the order of these connections. Specifically, nodes can be linked by clicking with the mouse to create directed connections and record the node execution order. The order in which algorithm nodes are linked indicates the execution order of the algorithm nodes within the current work area.

[0072] The connection operation can specifically include: a click operation, where the first clicked algorithm node serves as the starting node. For example, the required algorithm module can be dragged to the current work area (e.g., a canvas) as an algorithm node, and a connection can be made from the first clicked algorithm node (starting node) to the next algorithm node using the mouse. The order in which the nodes are connected determines the execution order of the algorithm nodes in the current work area. Based on the received connection operations, the algorithm nodes in the current work area are associated, that is, each algorithm node is associated vertically according to the connection operations. Figure 5 A schematic diagram of the generation algorithm flowchart interface is shown. For example... Figure 5 As shown, the lines are directional arrows. Based on these line connections, the algorithm nodes in the current working area are connected to form a directed acyclic graph (DAG) to manage the execution logic. That is, the backend uses a DAG to manage the execution logic, using a DAG data structure to store nodes and connections, ensuring that there are no loops in the flowchart (i.e., nodes cannot be cyclically dependent), thus avoiding infinite loops in the process. When executing the algorithm flow, the process can be executed by triggering the root node. For example, the trigger node for the vision platform is the input image, triggered by a local image or a camera capture.

[0073] The third receiving unit 130 is used to receive parameter adjustments made to the algorithm modules in the current working area and save the adjusted parameters of each algorithm module to the configuration file.

[0074] Specifically, the parameters of each algorithm module in the current working area can be adjusted in real time. The parameters of each algorithm module can be configured for both single and multiple debugging operations. Users can determine the optimal parameters by visualizing the results and considering the technical requirements of the detection. Figure 4 A sample diagram of the algorithm module parameter adjustment interface is shown. For example... Figure 4 As shown, the parameters of the algorithm module can be debugged. The parameters of the algorithm module can be adjusted in real time through the adjustment interface on the left and saved to the configuration file.

[0075] Optionally, when executing the algorithm flowchart, if there is an independent algorithm node that is not connected to other algorithm nodes, the algorithm node is skipped.

[0076] Specifically, each algorithm node is interconnected with its preceding and following nodes, executing according to the algorithm flowchart structure. Independent nodes are dynamically skipped and are called invalid nodes. The conditional branching algorithm module can dynamically execute debugging nodes by setting judgment conditions, such as the number of detected targets, and executing different branches based on the number of detection results. This is called inserting debugging nodes.

[0077] The camera configuration unit 140 is used to configure the camera and save the camera configuration parameters obtained after configuration to the configuration file.

[0078] In one specific implementation, camera configuration may include: acquiring SDKs for two or more different cameras and encapsulating the acquired SDKs for the two or more different cameras; acquiring a predefined unified abstract interface for different cameras and calling the APIs of the SDKs for the two or more different cameras through the unified abstract interface to capture images of the object to be detected using different cameras.

[0079] Specifically, an abstract class `BaseCamera` is created as a unified abstract interface. This `BaseCamera` abstract class implements at least one of the following functions: camera IP configuration, trigger mode setting, exposure frame rate setting, and image acquisition. The API interfaces from the development documentation of the SDKs for the two or more different cameras are imported, and the functionalities are overridden (paraphrasing the provided SDKs into platform-style code, such as unified log and interface interfaces). That is, the development documentation of the SDKs for the two or more different cameras is converted into API interface descriptions recognizable by the (current platform), establishing API mapping relationships. Based on the created `BaseCamera` abstract class, specific implementation classes are created for each of the two or more different cameras, and the corresponding camera's SDK API is called within these implementation classes. According to the API interfaces of the SDKs for the two or more different cameras, the abstract methods defined in the abstract class are overridden, converting the unified abstract interface into SDK calling code, thereby allowing the SDKs for the two or more different cameras to be called through the unified abstract interface.

[0080] Preferably, the method may further include configuring a PLC device. The PLC device is used to connect a terminal executing the vision inspection program and a vision inspection result response device. Specifically, the terminal executing the vision inspection program can be a computer. The vision inspection result response device can specifically be an industrial robot or a sorting device. The PLC device is used to control the vision inspection result response device. For example, it controls the industrial robot or sorting device to process defective workpieces, such as selecting out defective workpieces. The terminal (e.g., a computer) where the vision inspection program (vision software) resides is the host computer, and the PLC device is the slave computer, communicating with the host computer to convert the host computer's inspection results into logical actions on the slave computer.

[0081] In one specific implementation, PLC device configuration may include: encapsulating different PLC device communication protocols into abstract classes as a unified interface, and creating corresponding implementation classes for each PLC device communication protocol; wherein the unified interface implements at least one of PLC connection, register read / write, instruction issuance, and disconnection.

[0082] Specifically, an abstract class `BasePLC` is created. Based on this `BasePLC` abstract class, corresponding concrete implementation classes are created for each type of PLC device communication protocol. These different PLC device communication protocols include at least one of Modbus TCP, Siemens S7, and Mitsubishi MC protocols. For example, the Modbus TCP implementation class can be encapsulated using the pymodbus library, and the Siemens S7 implementation class can be encapsulated using the snap7 library. Modbus TCP protocol communication is implemented using the pymodbus library, and Siemens S7 protocol communication is implemented using the snap7 library. Network communication is implemented through third-party libraries to complete the data exchange between the PLC device and the host computer.

[0083] Furthermore, the camera configuration parameters obtained after camera configuration are saved to a configuration file. These configuration parameters may specifically include: camera model, IP address, and port. Preferably, the PLC device configuration parameters obtained after PLC device configuration can also be saved to a configuration file. These PLC device configuration parameters may specifically include TCP communication and PLC-related parameters. Specifically, configuration parameters can be stored using a JSON or YAML configuration file. For example, by uniformly storing camera model, IP address, port, TCP communication-related parameters (e.g., IP address, port, and automatic reconnection parameters) and PLC-related parameters (e.g., brand, protocol, and at least one of the PLC point logic parameters) in a JSON or YAML configuration file, configurable startup of the device can be achieved. Figure 6 A schematic diagram of the communication equipment configuration is shown.

[0084] The generation unit 150 is used to generate a visual inspection program based on the generated algorithm flow and the configuration file.

[0085] That is, the generated algorithm flow and the configuration file are used to generate a visual inspection program.

[0086] This invention's device can generate program code and package it in an isolated environment for cross-platform use. Specifically, PyInstaller is used to package the program of this invention in an isolated venv environment, including only necessary dependencies (such as OpenCV, numpy, pyqt5, pypylon, etc.), third-party libraries, modules, binary files, and configuration files, separating them from the project code for easy and rapid software updates. The algorithm module is embedded in encrypted form and dynamically loaded by the interface upon startup. Simultaneously, startup signature verification can be added to prevent program tampering. Using a Python virtual environment (venv) to create an independent runtime environment avoids conflicts with other Python packages and ensures the purity of the packaging environment.

[0087] The present invention also provides a storage medium corresponding to the generation method of the visual inspection program, wherein a computer program is stored thereon, which, when executed by a processor, implements the steps of any of the aforementioned methods.

[0088] The present invention also provides a terminal corresponding to the generation method of the visual inspection program, including a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the aforementioned methods.

[0089] The present invention also provides a terminal corresponding to the generation device of the visual inspection program, including any of the aforementioned generation devices of the visual inspection program.

[0090] The present invention also provides a computer program product corresponding to the generation method of the visual inspection program, comprising a computer program that, when executed by a processor, implements the steps of any of the aforementioned methods.

[0091] Accordingly, the solution provided by this invention can effectively improve code reusability and reduce development costs; it enables the development of drag-and-drop visual inspection solutions, significantly shortens the development cycle from requirements to deployment, and improves project response speed.

[0092] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and embodiments are within the scope and spirit of this invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, the functional units can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0094] The units described as separate components may or may not be physically separate. Similarly, the components of the control device may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0096] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for generating a visual inspection program, characterized in that, include: Receive drag-and-drop operations from a pre-configured algorithm library to the current work area, wherein each algorithm module dragged to the current work area is treated as an algorithm node; Receive connection operations to connect algorithm nodes in the current working area, and generate an algorithm flow based on the connection operations, wherein the order of the connections represents the execution order of the algorithm nodes in the current working area; Receive parameter adjustments made to the algorithm modules in the current working area, and save the adjusted parameters of each algorithm module to the configuration file; Configure the camera and save the resulting camera configuration parameters to a configuration file; Based on the generated algorithm flow and the configuration file, a visual inspection program is generated.

2. The method according to claim 1, characterized in that, The algorithm modules are categorized into at least two of the following: image processing, localization, recognition, deep learning, calibration, communication, and logic.

3. The method according to claim 1, characterized in that, Each of the algorithm modules is encapsulated as an independent class that inherits from an abstract base class.

4. The method according to any one of claims 1-3, characterized in that, The various algorithm modules interact with each other using a preset serialization format.

5. The method according to any one of claims 1-3, characterized in that, Also includes: Each algorithm module has a built-in decorator to capture exceptions in the executed function and return the captured exception data in a custom format.

6. The method according to any one of claims 1-3, characterized in that, Configure the camera, including: Obtain SDKs for two or more different cameras, and encapsulate the obtained SDKs for the two or more different cameras; Obtain a predefined unified abstract interface for different cameras, and call the API of the SDK of the two or more different cameras through the unified abstract interface to capture images of the object to be detected by different cameras.

7. The method according to any one of claims 1-3, characterized in that, Also includes: PLC device configuration includes: Different PLC device communication protocols are encapsulated into abstract classes as a unified interface, and implementation classes are created for each PLC device communication protocol. The unified interface enables at least one of the following: PLC connection, register read / write, instruction issuance, and disconnection.

8. An apparatus for generating a visual inspection program, characterized in that, include: The first receiving unit is configured to receive drag-and-drop operations from a pre-configured algorithm library to the current working area, wherein each algorithm module dragged to the current working area is considered as an algorithm node. The second receiving unit is used to receive connection operations that connect algorithm nodes in the current working area, and generate an algorithm flow based on the connection operations, wherein the order of connection indicates the execution order of algorithm nodes in the current working area. The third receiving unit is used to receive parameter adjustments made to the algorithm modules in the current working area and save the adjusted parameters of each algorithm module to the configuration file. The camera configuration unit is used to configure the camera and save the camera configuration parameters obtained after configuration to the configuration file. The generation unit is used to generate a visual inspection program based on the generated algorithm flow and the configuration file.

9. A storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-7.

10. A terminal, characterized in that, It includes a processor, a memory, and a computer program stored in the memory that can run on the processor, wherein the processor executes the program to implement the steps of any of the methods of claims 1-7, or includes a means of generating a visual inspection program as described in claim 8.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-7.

Images