Visual software framework system and control method

Abstract

The application discloses a kind of visual software framework systems, including visual software framework and visual software workflow, visual software framework includes camera driving unit, loading unit, communication unit and data storage unit;Visual software workflow includes image processing procedure;Camera driving unit is used to open call to the transmission package function of trigger mode commonly used to camera, network cable and support the reconnection of camera line break, image queue cache and AOI setting function;Loading unit is used to load preset CamController control when camera application program starts;Communication unit is used to realize the communication between units;Image processing procedure is used to obtain the image in image queue, process image using image processing program, and output standard result;Data storage unit is connected with PLC system signal through communication unit;The application also includes a kind of control method.The beneficial effects of the application are: it can be compatible with multiple cameras, and be suitable for multiple visual secondary development platform input / output standardization.

Description

Technical Field

[0001] This invention relates to a vision software framework system and control method. Background Technology

[0002] In recent years, with the increasing demand for visual applications, there are various visual development platforms on the market. These platforms have differences, but existing visual software framework systems cannot be compatible with multiple camera brands. Therefore, a software system is needed to meet the compatibility requirements of multiple development platforms. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a vision software framework system that is compatible with multiple camera brands and various vision development and secondary development platforms. This software is built on a Windows operating system, and drivers for each camera brand must be installed when using that brand.

[0004] The present invention discloses a vision software framework system, characterized in that it includes a vision software framework and a vision software workflow. The vision software framework includes a camera driving unit, a loading unit, a communication unit, and a data storage unit. The loading unit is signal-connected to the camera driving unit through the communication unit, and the data storage unit is signal-connected to the loading unit and the PLC system of the power lithium battery automatic grading system through the communication unit. The vision software workflow includes an image processing workflow and a data processing workflow.

[0005] The camera driver unit connects to multiple camera signals to enable access to commonly used camera trigger modes and network cable transmission packet functions, and supports camera disconnection reconnection, image queue caching, and AOI setting functions.

[0006] The loading unit sets up the program and updates related configurations by reading and writing files; the loading unit is signal-connected to the camera drive unit, and loads the preset camera controller when the camera application starts; wherein the camera controller controls the camera drive unit, displays the camera result screen, manages the communication function between the camera and the PLC system, and displays the status of the camera, communication, and signal acquisition; wherein the communication function between the camera and the PLC system includes the camera using communication for soft triggering;

[0007] The communication unit is based on Socket and supports UDP and TCP / IP server communication. It is used to realize communication between the camera controller (CamController control) and the camera drive unit, between the camera controller (CamController control) and the robot, and between the camera controller and the PLC system.

[0008] The data storage unit can store information such as images and communication logs in a specified format;

[0009] The image processing flow described is a multi-threaded process written in a vision library. All operations and data results are performed in this current thread. It retrieves images from the image queue, processes the images using an image processing program, and outputs standard results.

[0010] The data processing flow analyzes the results of the image processing flow, determines the meaning of each data point, processes the data, and finally sends the returned results to the next operating structure (such as a module or robot).

[0011] The communication unit is connected to the camera drive unit via signals, for example, to soft-trigger the camera to take pictures via communication.

[0012] The data storage unit determines whether to save the data based on the configuration file obtained from the loading unit; and modifies the camera settings, such as trigger mode, exposure brightness, etc., by using the camera controller (CamController control) to modify the camera parameters in the configuration file.

[0013] The control method for a vision software framework system according to the present invention is characterized by comprising the following steps:

[0014] Step 1: Start the vision software framework system. The vision software framework system will automatically read the local parameter file and load the corresponding camera controller; the corresponding camera controller defines the operation of the camera driver unit.

[0015] Step 2: Set and initialize the Socket parameters, camera parameters, and project parameters. The first time you open the application, default initialization parameters will be generated. After you modify and save them, the existing configuration will be loaded during initialization.

[0016] Step 3: The Socket-based communication unit operates, waiting to receive a robot trigger signal; the camera is started and is in a ready-to-connect state.

[0017] Step 4: When the battery is in the photo-taking position, the robot (module) sends a communication request, and the Socket accepts the robot's trigger photo-taking request. At this time, the vision software framework system calls the camera driver unit to make the camera take a picture.

[0018] Step 5: After the communication unit triggers the capture, the camera driver unit sends the image to the image queue and stores the communication command in the command queue. The order of the communication command and the image in the queue is kept consistent to ensure that one command corresponds to one image, and different commands process the image differently.

[0019] Step 6: Obtain images from the image queue using a visual processing algorithm, and then select the specified image detection program for processing according to the instructions in the instruction queue;

[0020] Processing algorithms include: image calibration, template matching, software triggering, battery location, and offline testing.

[0021] Step 7: Obtain the results of the visual processing algorithm, including OK, NG, and other data;

[0022] Step 8: Feedback the results of the vision processing algorithm to the PLC system and simultaneously transmit them to the display area of ​​the camera controller (CamController control);

[0023] Step 9: Feedback the results from step 8 to the display interface, which will show the battery's detection parameters, results, and other information.

[0024] Step 10: Save the operation results as an image and record the operation.

[0025] Step 11 is now complete.

[0026] The beneficial effects of this invention are: it is compatible with multiple cameras and provides a standardized visual software framework for input and output across multiple vision secondary development platforms. The entire software system supports cameras such as Hikvision, Basler, and Huarui, and the development platform supports Halcon, OpenCV, and VisionMaster algorithm libraries. It is a simple to operate and a fully functional solution; each part of the system has undergone long-term stability testing to ensure the stability of the framework. Attached Figure Description

[0027] Figure 1 This is the flow chart of the ji control method of the visual software framework system of the present invention.

[0028] Figure 2 This is a unit architecture diagram of the visual software framework system of the present invention.

[0029] Figure 3 This is a block diagram showing the connection of each unit in the visual software framework of the present invention. Detailed Implementation

[0030] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0032] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0033] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0034] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0035] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0036] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0037] The present invention discloses a vision software framework system, comprising a vision software framework and a vision software workflow. The vision software framework includes a camera driving unit, a loading unit, a communication unit, and a data storage unit. The loading unit is signal-connected to the camera driving unit through the communication unit, and the data storage unit is signal-connected to the loading unit and the PLC system of the power lithium battery automatic grading system through the communication unit. The vision software workflow includes an image processing workflow and a data processing workflow.

[0038] The camera driver unit is connected to multiple camera signals, and allows for the use of commonly used camera trigger modes and network cable transmission packet functions. It also supports camera disconnection reconnection, image queue caching, and AOI setting functions.

[0039] The loading unit sets up the program and updates the relevant configuration by reading and writing files; the loading unit is signal-connected to the camera driver unit and is used to load the preset configuration file and camera controller when the camera application starts. The camera controller is used to control the camera driver unit, display the camera result screen, manage the communication functions of the camera and other device systems, and display the status of the camera, communication and signal acquisition.

[0040] The communication unit is based on Socket and supports UDP and TCP / IP server communication to realize communication between the camera controller and the camera drive unit, between the camera controller and the robot, and between the camera controller and the PLC system.

[0041] The data storage unit is connected to the PLC system of the power lithium battery automatic grading system via a communication unit. The data storage unit stores information such as pictures and communication logs in a specified format.

[0042] The image processing flow is a multi-threaded process written in a vision library. All operations and data results are performed in this current thread. It retrieves images from the image queue, processes the images using an image processing program, and outputs standard results.

[0043] The data processing flow analyzes the results of the image processing flow, determines the meaning of each data point, processes the data, and finally sends the returned results to the next running structure.

[0044] The data storage unit supports saving communication data, processing results, original images, and image processing effect images; tracking image processing progress; determining whether to save data based on the configuration file obtained from the loading unit; and modifying camera settings by using the camera controller to access camera parameters from the configuration file.

[0045] Figure 1 This describes the implementation flow of a control method for a vision software framework system as described in this invention. It is applicable to devices running vision software framework systems, including but not limited to automatic grading systems for power lithium batteries. The above method flow includes the following steps:

[0046] Step 1: Start the vision software framework system. The vision software framework system automatically reads the local parameter file through the loading unit and loads the corresponding camera controller (CamController control). The camera controller (CamController control) defines the operation of the camera driver unit.

[0047] Step 2: Set and initialize the Socket parameters, camera parameters, and project parameters. The first time you open the application, default initialization parameters will be generated. After you modify and save them, the existing configuration will be loaded during initialization.

[0048] Socket parameters: communication protocol, communication address, communication port, etc.;

[0049] Camera parameters: camera trigger mode, camera exposure brightness, camera image acquisition format, camera gain, etc.

[0050] Project parameters: whether to save images, whether to save logs, battery location parameters, etc.

[0051] Step 3: The Socket-based communication unit operates, waiting to receive a robot trigger signal; the camera is started and is in a ready-to-connect state.

[0052] Step 4: When the battery is in the camera position, the robot (module) sends a communication request, and the Socket accepts the robot's trigger photo-taking request. At this time, the vision software framework system calls the camera driver unit to instruct the camera to take a picture.

[0053] Step 5: After the communication unit triggers the capture, the camera driver unit sends the image to the image queue and stores the communication command in the command queue. The order of the communication command and the image in the queue is kept consistent to ensure that one command corresponds to one image, and different commands process the image differently.

[0054] Step 6: Obtain images from the image queue using a visual processing algorithm, and then select the specified image detection program for processing according to the instructions in the instruction queue;

[0055] Processing algorithms include: image calibration, template matching, software triggering, battery location, offline testing, etc.

[0056] Step 7: Obtain the results of the visual processing algorithm, including OK, NG, and other data;

[0057] Step 8: Feedback the results of the visual processing algorithm to other operating systems, and simultaneously transmit them to the display area of ​​the camera controller (CamController control);

[0058] Step 9: Feedback the results from step 8 to the display interface, which will show the battery's detection parameters, results, and other information.

[0059] Step 10: Save the operation results as an image and record the operation.

[0060] Step 11 is now complete.

[0061] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A vision software framework system, characterized in that: The system includes a vision software framework and a vision software workflow. The vision software framework includes a camera driving unit, a loading unit, a communication unit, and a data storage unit. The loading unit is connected to the camera driving unit via the communication unit, and the data storage unit is connected to the loading unit and the PLC system of the power lithium battery automatic grading system via the communication unit. The vision software workflow includes an image processing workflow and a data processing workflow. The camera driver unit is connected to multiple camera signals, and allows for the use of commonly used camera trigger modes and network cable transmission packet functions. It also supports camera disconnection reconnection, image queue caching, and AOI setting functions. The loading unit sets up the program and updates the relevant configuration by reading and writing files; the loading unit is signal-connected to the camera driver unit and is used to load the preset configuration file and camera controller when the camera application starts. The camera controller is used to control the camera driver unit, display the camera result screen, manage the communication functions of the camera and other device systems, and display the status of the camera, communication and signal acquisition. The communication unit is based on Socket and supports UDP and TCP / IP server communication to realize communication between the camera controller and the camera drive unit, between the camera controller and the robot, and between the camera controller and the PLC system. The data storage unit is connected to the PLC system of the power lithium battery automatic grading system via a communication unit. The data storage unit stores information such as pictures and communication logs in a specified format. The image processing flow is a multi-threaded process written in a vision library. During this process, all operations and data results are performed in the current thread. Images are acquired from the image queue, and the image processing program is used to process the images and output standard results. The data processing flow analyzes the results of the image processing flow, determines the meaning of each data point, processes the data, and finally sends the returned results to the next running structure.

2. The visual software framework system as described in claim 1, characterized in that: The data storage unit supports saving communication data, processing results, original images, and image processing effect images; tracking image processing progress; determining whether to save data based on the configuration file obtained from the loading unit; and modifying camera settings by using the camera controller to access camera parameters from the configuration file.

3. The control method for a vision software framework system according to claim 2, characterized in that, Includes the following steps: Step 1: Start the vision software framework system. The vision software framework system will automatically read the local parameter file and load the corresponding camera controller; the corresponding camera controller defines the operation of the camera driver unit. Step 2: Set and initialize the Socket parameters, camera parameters, and project parameters. The first time you open the application, default initialization parameters will be generated. After you modify and save them, the existing configuration will be loaded during initialization. Step 3: The Socket-based communication unit operates, waiting to receive a robot trigger signal; the camera is started and is in a ready-to-connect state. Step 4: When the battery is in the photo-taking position, the robot module sends a communication request and the Socket receives the robot's photo-taking request. At this time, the vision software framework system calls the camera driver unit to make the camera take a picture. Step 5: After the communication unit triggers the photo capture, the camera driver unit sends the image to the image queue and stores the communication command in the command queue. The order of the communication command and the image in the queue is kept consistent to ensure that one command corresponds to one image, and different commands process the image in different ways. Step 6: Obtain images from the image queue using a visual processing algorithm, and then select the specified image detection program for processing according to the instructions in the instruction queue; Step 7: Obtain the results of the visual processing algorithm, including OK, NG, and other data; Step 8: Feedback the results of the vision processing algorithm to the PLC system and simultaneously transmit them to the display area of ​​the camera controller (CamController control); Step 9: Feedback the results of step 8 to the display interface, which will show the battery's detection parameters, results, and other information. Step 10: Save the operation results as an image and record the operation. Step 11 is now complete.

Images