Development system, method, and program

Abstract

Provided is a technique capable of reducing the difficulty in developing a system such as an image inspection system and the workload of a user's work. A computer system as a development system generates and provides a GUI screen 30 for supporting development for a user. A GUI screen (30) is provided with: a module selection unit (31) in which a plurality of processing modules (BL), which are components for constituting a flow line of an image inspection system, are disposed; a module arrangement unit 32 that configures and displays a process line FL during production by arranging the processing modules BL selected from the module selection unit 31 in accordance with the user's GUI operation and connecting the arranged processing modules to each other by a line; and a preview display unit 33 that displays a preview image of the operating state of the flow line FL on the basis of the GUI operation by the user with respect to the flow line FL that is being created and that is disposed in the module disposition unit 32.

Description

Technical Field

[0001] The present invention relates to a technology for developing and manufacturing a computer system (sometimes referred to as a development system) and the like such as an image inspection system. Background Art

[0002] Various image inspection systems have been developed. For example, image inspection systems are used in factories with production lines. The image inspection system uses cameras / sensor devices such as line sensors and area sensors to capture images of objects such as parts and products flowing through the production line. The image inspection system performs processing such as detection / determination of the captured images based on image processing implemented by a computer. In this way, image inspection of the object is achieved. As an example of image inspection, appearance inspection can be listed, which is to identify / detect the object reflected in the image to determine the existence, quantity, appearance quality, etc. of the object.

[0003] In the past, the development of image inspection systems was performed by engineers and other personnel with specialized skills and knowledge (sometimes referred to as developers, users, etc.), requiring advanced programming skills (such as programming based on C language, etc.). In addition, image inspection systems may be constructed using image processing technologies such as AI (artificial intelligence) such as machine learning (such as deep learning), in which case developers are required to have knowledge related to AI.

[0004] As an example of the prior art related to the system development as described above, Japanese Patent Publication No. 2009-141413 (Patent Document 1) can be cited. Patent Document 1 describes the following contents, etc.: an image processing device is constructed so that an operator can efficiently perform operations; one of a plurality of pre-set workflows is specified, and the workflow editing unit is used to edit the workflow by adding or reducing processing steps, changing the order of processing steps, etc., and the workflow selection unit is used to manually select the workflow saved after editing and execute it.

[0005] Prior art literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2009-141413 Summary of the invention

[0007] For example, although Patent Document 1 is a technology different from the development of an image inspection system, it is about editing of a workflow related to image processing in a photo printing device, such as Figure 5 , 0045 and thereafter. In Patent Document 1, Figure 6 In such a screen, operate the edit button to display Fig.10Such a workflow editing screen. In the workflow editing screen, a pre-set workflow is displayed in the normal display area, and the processing steps that constitute the workflow are displayed as icons in the editing area. The workflow editing unit has a GUI that allows icons of the processing steps of the workflow in the normal display area to be dragged and dropped with a cursor and displayed in the editing area. When the icon in the editing area is specified and operated, the processing process information associated with the processing step is displayed, and the processing process to be executed can be selected. When the save button is operated, the name of the edited workflow is input and saved. In addition, the following content is recorded: It is possible to use the recording mode ( Fig.11 ) or drag and drop icons to set the sequence of processing processes.

[0008] In addition, the "preview screen" ( Figure 7 ) is a screen that displays thumbnail images of photo images, etc., and is an element of a processing step that constitutes a part of the workflow, and is different from the "preview" described in this manual.

[0009] The development and production of systems such as image inspection systems (in other words, software, applications, etc.) requires developers to have advanced technical knowledge, so the difficulty is relatively high and the workload of programming, debugging, etc. is also large. In order to reduce such difficulty and workload, a system (sometimes referred to as a development system) that can support system development and make it easier and more efficient is required.

[0010] In the development system of the prior art example, when developing / producing the system, the user temporarily produces the entire flow line corresponding to the development target system, and then performs the operation confirmation of the entire produced flow line. As a result of the confirmation, when debugging or changing a part of the development target system, the user performs operations such as changing a part of the flow line. After that, the user performs the operation confirmation of the entire flow line again. The development work including such duplication of work sometimes increases the workload of the user and prolongs the development time.

[0011] The present disclosure aims to provide a technology capable of reducing the difficulty of system development and the workload of the user regarding a computer system (development system) or the like for system development / production of the above-mentioned image inspection system or the like.

[0012] A representative embodiment of the present disclosure has a structure shown below. The embodiment is a development system that supports the development of an image inspection system, wherein the development system has a computer system having a processor and a memory, and the computer system generates and provides a graphical user interface (GUI) screen for supporting the development for a user who performs the development, and the GUI screen has: a module selection unit configured with a plurality of processing modules, which are components for constituting a flow line of the image inspection system; a module configuration unit, which configures the processing modules selected from the module selection unit according to the GUI operation of the user, and connects the configured processing modules with each other by lines, thereby constituting and displaying the flow line being produced; and a preview display unit, which displays a preview image of the action status of the flow line configured in the module configuration unit according to the GUI operation of the user.

[0013] According to the representative embodiments of the present disclosure, the technology of computer systems (development systems) for system development / production of the above-mentioned image inspection system, etc. can reduce the difficulty of system development and the workload of the user. The specific embodiments show the problems, structures and effects other than the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram showing the relationship between the image inspection system as a development target and the development system of the first embodiment.

[0015] Figure 2 This is a diagram showing a first configuration example of a development system according to the first embodiment.

[0016] Figure 3 This is a diagram showing a second configuration example of the development system according to the first embodiment.

[0017] Figure 4 This is a diagram showing a configuration example of a computer system as a development system according to the first embodiment.

[0018] Figure 5 This is a diagram showing a configuration example of a GUI screen in the development system according to the first embodiment.

[0019] Figure 6 This is a diagram showing the process flow in the development system according to the first embodiment.

[0020] Figure 7 This is a diagram showing an operation example of flow line creation related to the flow line editing function in the first embodiment.

[0021] Figure 8 This is a diagram showing an example of module deletion operation in the first embodiment.

[0022] Fig. 9 This is a diagram showing an operation example of adding (inserting) a module in the first embodiment.

[0023] Fig.10 This is a diagram showing an example of module replacement operation in the first embodiment.

[0024] Fig.11 This is a diagram showing an operation example related to the preview function in the first embodiment.

[0025] Fig.12 This is a diagram showing various functions related to the preview function in the first embodiment.

[0026] Fig.13 This is a diagram showing function 1 (preview display switching function) in the first embodiment.

[0027] Fig.14 This is a diagram showing an object selection method related to Function 1 in Embodiment 1.

[0028] Fig.15 This is a diagram showing an example of displaying a video as a preview display of content details in accordance with Function 1 in Embodiment 1.

[0029] Fig.16 This is a diagram showing an example of preview display using a sample input image in accordance with the first embodiment.

[0030] Fig.17 This is a diagram showing an example of the internal setting of a processing module related to Function 1 in Embodiment 1.

[0031] Fig.18 This is a diagram showing a preview display example of the internal settings of the processing module related to Function 1 in Embodiment 1.

[0032] Fig.19 This is a diagram showing a display example of detailed information on the internal settings of a processing module related to Function 1 in Embodiment 1.

[0033] Fig. 20 This is a diagram showing an example of a virtual application preview display related to Function 1 in Embodiment 1.

[0034] Fig.21 This is a diagram showing a GUI example of a preview display unit related to function 1 in the first embodiment.

[0035] Fig. 22 This is a diagram showing function 2 (multi-preview function 1) in the first embodiment.

[0036] Fig.23This is a diagram showing a modified example of function 2 (multi-preview function 1) in the first embodiment.

[0037] Fig.24 This is a diagram showing function 3 (multi-preview function 2) in the first embodiment.

[0038] Fig.25 This is a diagram showing the preview mode in the first embodiment.

[0039] Fig.26 This is a diagram showing function 4 (warning) in the first embodiment.

[0040] Fig. 27 This is a diagram showing function 5 (color information extraction) in the first embodiment.

[0041] Fig.28 This is a diagram showing a first example of function 6 (comparison method) in the first embodiment.

[0042] Fig.29 This is a diagram showing a second example of function 6 (comparison method) in the first embodiment.

[0043] Fig.30 This is a diagram showing function 6 (comparison method) in the first embodiment.

[0044] Fig.31 This is a diagram showing a modified example of function 6 in the first embodiment.

[0045] Fig.32 This is a diagram showing the creation of a plurality of candidate flow lines in the first embodiment.

[0046] Fig.33 This is a diagram showing a modified example of displaying the flow line in the first embodiment.

[0047] Fig.34 This is a diagram showing function 7 (line sensor installation support) in the first embodiment.

[0048] Fig.35 This is a diagram showing the internal settings of the camera capture module in the first embodiment.

[0049] Fig.36 This is a diagram showing function 8 (template matching setting support) in the first embodiment.

[0050] Fig.37 This is a diagram showing an example of a flow line confirmation screen in the first embodiment.

[0051] Fig.38 A configuration example of a conventional image inspection system and a development system is shown.

[0052] Fig.39This is a diagram showing the preview display presence / absence switching function in Embodiment 2.

[0053] Fig.40 This is a diagram showing an example of another GUI in the preview display presence / absence switching function in Embodiment 2.

[0054] Fig.41 This is a diagram showing an example of another GUI in the preview display presence / absence switching function in Embodiment 2.

[0055] Fig.42 This is a diagram showing an example of another GUI in the preview display presence / absence switching function in Embodiment 2.

[0056] Fig.43 This is a diagram showing an example of another GUI in the preview display presence / absence switching function in Embodiment 2.

[0057] Fig.44 This is a diagram showing an example of another GUI in the preview display presence / absence switching function in Embodiment 2.

[0058] Fig.45 This is a diagram showing the preview display resolution setting function in Embodiment 2.

[0059] Fig.46 This is a diagram showing an example of another GUI in the preview display resolution setting function in Embodiment 2.

[0060] Fig.47 This is a diagram showing the preview display rotation function in Embodiment 2.

[0061] Fig.48 This is a diagram showing a setting example of an image inspection system related to the preview display rotation function in Embodiment 2.

[0062] Fig.49 This is a diagram showing a setting example of an image inspection system related to the preview display rotation function in Embodiment 2.

[0063] Fig.50 This is a diagram showing the display function availability of the flow line module in Embodiment 2.

[0064] Fig.51 This is a diagram showing an example of another GUI in the display function availability of the flow line module in Embodiment 2.

[0065] Fig.52 This is a diagram showing an example of another GUI in the display function availability of the flow line module in Embodiment 2.

[0066] Fig.53This is a diagram showing an example of setting conditions in the flow line module placement and display function in the second embodiment.

[0067] Fig.54 This is a diagram showing an example of setting properties in the flow line module placement display function in the second embodiment.

[0068] Fig.55 This is a diagram showing a specific example of the configuration in the flow line module configuration availability display function in the second embodiment.

[0069] Fig.56 This is a diagram showing a specific example of the configuration in the flow line module configuration availability display function in the second embodiment.

[0070] Fig.57 This is a diagram showing that the unset module display function must be set in the second embodiment.

[0071] Fig.58 This is a diagram showing a display example in the must-set unset module display function in the second embodiment.

[0072] Fig.59 This is a diagram showing the function of setting the input and output images of the process line in the second embodiment.

[0073] Fig.60 This is a diagram showing an example in which the setting of the input and output images in the unset module display function must be set in the second embodiment.

[0074] Fig.61 This is a diagram showing an example in which the setting of the input and output images in the unset module display function must be set in the second embodiment.

[0075] Fig.62 This is a diagram showing an example in which the setting of the input and output images in the unset module display function must be set in the second embodiment.

[0076] Fig.63 This is a diagram showing an example in which the setting of the input and output images in the unset module display function must be set in the second embodiment.

[0077] Fig.64 This is a diagram showing a control process flow in the must-set unset module display function in the second embodiment.

[0078] Fig.65 This is a diagram showing the module display function in the flow line processing in the second embodiment.

[0079] Fig.66 This is a diagram showing a detailed display example of the module display function in the flow line processing in the second embodiment.

[0080] Fig.67 This is a diagram showing a display example of the internal setting column of the processing module in the second embodiment. DETAILED DESCRIPTION

[0081] In this embodiment, by providing the following technology, it contributes to the creation of a work environment that is easy to work in and the realization of a smart factory. By realizing this image inspection system, it contributes to the Sustainable Development Goals (SDGs: Sustainable Development Goals) advocated by the United Nations, namely "8. Decent work and economic growth" and "9. Infrastructure construction for industrial and technological innovation".

[0082] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals are attached to the same parts in principle, and repeated descriptions are omitted. In the accompanying drawings, the actual position, size, shape, range, etc. of the constituent elements are sometimes not shown in order to facilitate the understanding of the invention.

[0083] In terms of description, when describing program-based processing, sometimes the program, function, processing unit, etc. are used as the main body for description, but the main body of their hardware is a processor, or a controller, device, computer, system, etc. composed of the processor, etc. The computer uses resources such as memory and communication interface appropriately through the processor, and executes the processing according to the program read out to the memory. Thus, the predetermined function, processing unit, etc. are realized. The processor is composed of semiconductor devices such as CPU / MPU and GPU, etc. The processing is not limited to software program processing, and even a dedicated circuit can be installed. FPGA, ASIC, CPLD, etc. can be applied as a dedicated circuit.

[0084] The program can be pre-installed as data on the target computer, or it can be distributed as data from the program source to the target computer. The program source can be a program distribution server on a communication network, or it can be a non-temporary computer-readable storage medium, a non-volatile recording medium, such as a memory card or a disk. The program can also be composed of multiple components. The computer system can also be composed of multiple devices. The computer system can also be composed of a client-server system, a cloud computing system, an IoT system, etc. Various data and information are composed of structures such as tables and lists, but are not limited to this. Expressions such as identification information, identifiers, IDs, names, numbers, etc. can be interchangeable.

[0085] <Topics, etc.>

[0086] Supplementary explanation of the subject, etc. First, an image inspection system which is an example of a system to be developed will be explained.

[0087] Fig.38 The following is a configuration example of an image inspection system and a development system of a comparative example. The image inspection system 2 is a development target system. The development system 3 is a computer system for developing the image inspection system 2. The image inspection system 2 is a system for performing image inspection such as appearance inspection using image processing on an object 20.

[0088] The image inspection system 2 uses a camera / sensor device 21 such as a line sensor or an area sensor to capture an object 20, such as a component or product on a production line in a factory, and inputs the captured image 50 (in other words, an image signal, etc.). The image inspection system 2 is a system that performs a predetermined image inspection process on the image 50 and outputs the result. The image inspection system 2 is a system including software including necessary programs, hardware, data, circuits, etc.

[0089] A line sensor (in other words, a line scan camera) is, for example, a device that includes a CCD image sensor element, a lens, and a drive circuit arranged linearly, and can generate an image of an object by, for example, combining multiple images obtained by linearly photographing a moving object. In addition, an area sensor is, for example, a device that includes multiple sensors arranged for a rectangular area and can generate an image of an object corresponding to the area. The camera / sensor device 21 is not limited to a line sensor or an area sensor, and other types of devices that can photograph images can also be applied.

[0090] The predetermined image inspection processing of the image inspection system 2 includes, for example, camera input processing 201, image processing 202, and determination processing 203. Each of these processes is implemented by processing based on a computer or circuit program processing, etc. In other words, the flow line of the image inspection system 2 is formed by connecting the processing modules or processing steps corresponding to each of these processes.

[0091] For example, the camera capture process 201 is implemented as the first process by the first program. The image process 202 is implemented as the second process by the second program. The determination process 203 is implemented as the third process by the third program. The program refers to a computer program or code.

[0092] The camera input processing 201 is a processing for inputting the image 50 (in other words, the image signal) output from the camera 21 and taking it in as image data (in other words, the image input processing), and outputting the camera input image 51 (also recorded as the first image) as the processing result. As the image processing 202, there are various processings such as binarization and image shape conversion. In the image processing 202, for example, the camera input image 51 is input and binarized, and as the processing result, the binarized result image 52 (also recorded as the second image) is output. The inspection processing 203 is, for example, a processing in which the result image 52 is input, the object 20 reflected in the image is identified / detected, and the existence, quantity, appearance quality, etc. of the object 20 are determined. The inspection processing 203 outputs the processing result information (in other words, the inspection result information, the determination result image) 53. In addition, depending on the image inspection system 2 to be developed, the inspection processing 203 may not be included.

[0093] In the example shown in the figure, as the content of the first image 51, an object area 20A having the characters "ABC" written thereon is shown in a circular shape with respect to the object 20. The first image 51 is, for example, a full-color image, but is schematically shown in black and white in the figure. As the content of the second image 52, a binarized object area 20B is shown.

[0094] The image inspection system 2 is composed of a plurality of processing modules such as a camera input processing 201, an image processing 202, and an inspection processing 203, and can also be expressed as follows: Fig.38 Such a processing flow chart. The configuration of such an image inspection system 2 may be described as a flow line.

[0095] Conventionally, the development and production of such an image inspection system 2 (and its corresponding process line) is performed by engineers and other personnel with professional skills and knowledge (sometimes referred to as users, developers, etc.). The work of engineers and other personnel includes the preparation and programming of a programming environment, and the preparation and debugging of a debugging environment.

[0096] In addition, such an image inspection system 2 may be constructed using a model based on machine learning (such as deep learning), or may be constructed using general image processing technology instead of a model based on machine learning. For example, any processing module (in other words, processing step) on the process line of the image inspection system 2 may also apply a model based on machine learning. For example, in a certain processing module, CNN (convolutional neural network) is applied as a model, and the model is trained in advance in the learning stage. In the inference stage of actual application, the processing module is used to output an inference result based on CNN for the input image. The development system of the implementation method can be applied in any case.

[0097] On the other hand, the development system 3 is a computer system including a development environment for developing / producing such an image inspection system 2 as an object. A user such as an engineer uses a computer system installed with software for development and performs development work of the image inspection system 2. The computer system can be composed of a common PC, server, database, communication interface, input device, output device, etc. The engineer performs the following work: develop / produce the first program of the camera input processing 201, the second program of the image processing 202, the third program of the determination processing 203, etc. through programming.

[0098] In the conventional development system 3, as described above, after the user temporarily completes the flow line of the image inspection system 2 to the end (e.g., inspection process 203), the user performs overall action confirmation of the image inspection system 2. After confirmation, the user changes a part of the flow line according to design changes, debugging, etc. After that, the user performs overall action confirmation of the image inspection system 2 again. In the conventional development system 3, even if there is a preview function for action confirmation, the object that can be previewed is the processing result of the overall flow line of the image inspection system 2 for the input image 50. Therefore, in such development work, the workload of the user is large.

[0099] <Solutions, etc.>

[0100] Based on the above-mentioned problems, the development system of the embodiment has the following solution means.

[0101] The development system of the embodiment is a computer system that constitutes an environment for developing / producing software for the image inspection system 2. The development system of the embodiment is a system that constitutes a GUI-based development environment including a graphical user interface (GUI) for developing / producing the image inspection system 2.

[0102] Figure 1 The development system 1 of the embodiment 1 shown in the figure is a computer system including an application program that supports the development / production of a process line corresponding to the image inspection system 2. The development system 1 generates a GUI-based screen (sometimes referred to as a GUI screen) having a unique graphical user interface (GUI) and display content not found in the prior art, and provides it to the user U1. The GUI screen is displayed on the screen of an arbitrary display device. For example, the GUI screen can also be provided in the form of a web page.

[0103] The user U1, such as an engineer, who performs development work using the development system 1, operates the GUI screen while viewing the GUI screen, thereby utilizing the functions of the development system 1. The user U1 performs work of developing / creating programs for each process (e.g., camera capture process 201, image process 202, and determination process 203) constituting the flow line of the image inspection system 2 based on the GUI.

[0104] <Implementation Method 1>

[0105] use Figure 1 etc. to illustrate the development system and method of implementation mode 1.

[0106] [Image inspection system and development system]

[0107] Figure 1 The following is a configuration example of an image inspection system 2 as a development target system and a development system 1 according to Embodiment 1. The image inspection system 2 has Fig.38 The image inspection system 2 uses a camera / sensor device 20 such as a line sensor or an area sensor to photograph an object 20 and inputs the photographed image 50. The image inspection system 2 performs, for example, camera input processing 201 on the input image 50, and obtains a camera input image (first image) 51 as a processing result. The image inspection system 2 performs, for example, image processing 202 such as binarization on the camera input image 50, and obtains a result image (second image) 52 as a processing result. The image inspection system 2 performs inspection processing 203 such as determination processing on the result image 52, and obtains inspection result information, in other words, determination result information 53.

[0108] The development system 1 is a system that supports the work of developing / producing a process line corresponding to the image inspection system 1, for example, a process line composed of connection of processing modules such as the camera capture process 201, the image process 202, and the inspection process 203. The development system 1 is a system that supports the work of developing software of the image inspection system 1, for example, a first program constituting the camera capture process 201, a second program constituting the image process 202, and a third program constituting the inspection process 203. The development system 1 supports these development work based on GUI operations performed by the user U1 on the GUI screen.

[0109] The development system 1 has at least a flow line editing function F1 and a preview display function F2.

[0110] The flow line editing function F1 is a function that allows a user to create and edit a flow line by operating processing modules prepared in advance as components by dragging and dropping or the like on a GUI screen and connecting the modules.

[0111] In particular, in the GUI screen, a preview screen implemented by the preview display function F2 can be displayed in association with the flow line created by the flow line editing function F1. The preview display function F2 is a function for controlling the preview display of the flow line in the preview screen. The preview display function F2 takes the flow line or processing module, etc., which are being created by the flow line editing function F1 as an object, and implements preview display related to their operation status and processing results. In the preview screen, an image showing the operation status of the provisional flow line being created can be previewed and displayed.

[0112] [Development System]

[0113] Figure 2 and Figure 3 An example of an implementation configuration of the development system 1 according to the first embodiment is shown. Figure 2 The first installation example is shown. Figure 3 The second installation example is shown. In the first embodiment, Figure 2 The first installation example is used as the basis for the description, but as a modified example, it can also be Figure 3 The second installation example.

[0114] exist Figure 2 In the embodiment 1, the development system 1 is a system mainly composed of connecting a computer 101 and an image processing device 102. The computer 101 can be composed of, for example, a common PC. The computer 101 has a general OS, middleware (file system, Web server, etc.), a communication interface, etc., and has an image inspection system development environment 500 as software. The image inspection system development environment 500 is equivalent to the software of the development system 1 (including computer programs, databases, etc.). The computer 101 and the image processing device 102 are connected through a LAN, for example, by cable connection, and each has a communication interface corresponding to the LAN.

[0115] Furthermore, it is obvious that the implementation example of the development system 1 is not limited to a configuration consisting of one PC, and for example, a client server system (for example, a Web system) may also be applied.

[0116] The computer 101 is externally connected to a computer video display 103 as a display device by cable connection. The computer 101 transmits a video signal s1 to the computer video display 103 , so that the GUI screen 30 generated by the development environment 500 is displayed on the screen of the computer video display 103 .

[0117] Although not shown in the figure, an input device such as a mouse is externally connected to the computer 101. In addition, the computer video display 103 may be a touch panel.

[0118] The image processing device 102 is a part that functions as an image input engine for the software (development environment 500) of the development system 1. The image processing device 102 is connected to a digital signal output device 105 by cable connection. The digital signal output device 105 is, for example, a camera or a sensor device. For example, the digital signal output device 105 as a camera has a camera function, and as a Figure 1 The corresponding part of the camera / sensor device 21 in the development environment 500 functions. The digital signal output device 105 of the camera sends the image signal s4 of the captured image to the image processing device 102. The image processing device 102 generates an image 15 (image data) based on the image signal s4 and sends it to the computer 101 as a signal s3. The computer 101 receives the image 15 and uses it as a processing target image in the development environment 500.

[0119] In addition, the image processing device 102 is connected to an auxiliary storage device 106 (for example, a USB memory) by means of a cable connection. The image processing device 102 can read and write data / information (signal s6) such as images to the auxiliary storage device 106. In addition, the image processing device 102 is connected to a monitoring display 104 by means of a cable connection. The monitoring display 104 is an image display or a touch panel for an image processing device. The image processing device 102 sends an image signal s2 corresponding to an image captured by the digital signal output device 105 to the monitoring display 104. The monitoring display 104 displays an image on the screen based on the image signal s2. In the case where the monitoring display 104 or the computer image display 103 is a touch panel, the user can input instructions to the computer 101 or the image processing device 102 by performing a touch operation on the screen of the touch panel.

[0120] The development environment 500 of the computer 101 is installed with Figure 1 The process line editing function F1, the preview display function F2, etc. of the image inspection system 2 are maintained, and the data FD of the process line created is maintained.

[0121] exist Figure 3 In the embodiment 1, the development system 1 is mainly composed of an image processing device 102 (102B) operating independently, for example, a video camera device. In the image processing device 102 (102B), a Figure 2 The image inspection system development environment 500 of the computer 101 in FIG. Figure 3 In the installation example, it can be omitted Figure 2The computer 101 in the image processing apparatus 102B may be connected to the auxiliary storage apparatus 107 via a LAN in a cable connection manner. The image processing apparatus 102B may send data of the captured image 15 and the like as a signal s3 to the auxiliary storage apparatus 107 and store the data in the auxiliary storage apparatus 107.

[0122] [Computer System]

[0123] Figure 4 As shown in the development system 1 (for example Figure 2 ) of a computer system 1. Figure 4 , a computer 101, an input device 1005 and an output device 1006 externally connected to the computer 101, and a client terminal device 1030 connected via a communication network 1020 (e.g., LAN) are illustrated. The client terminal device 1030 is used when a client server system is used, but is not essential.

[0124] The computer 101 includes a processor 1001, a memory 1002, a communication interface device 1003, an input / output interface device 1004, etc., and these components are connected to each other via a bus. The input / output interface device 1004 is externally connected to an input device 1005 and an output device 1006. The input device 1005 and the like may also be built into the computer 101. Examples of the input device 1005 are a mouse, a keyboard, a microphone, etc. Examples of the output device 1006 are a display, a printer, a speaker, etc. The communication interface device 1003 is installed with a computer 1004. Figure 2 A communication interface for the image processing device 102 and a communication interface for the LAN 1020 .

[0125] The processor 1001 executes processing according to the program read out to the memory 1002. Thus, predetermined functions (such as the above-mentioned flow line editing function F1) are realized as execution components.

[0126] The memory 1002 stores a control program 1011, setting information 1012, DB 1013, flow line structure information 1014, GUI screen data 1015, image data 1016, etc. The memory 1002 can be composed of a non-volatile storage device, etc., but is not limited thereto. A memory resource (such as a DB server) outside the computer 101 can also be used.

[0127] In DB1013, as a component Figure 2 The database of the software of the development environment 500 includes data / information such as processing modules that are components for constituting the flow line. The processing modules constituting the flow line are program components and associated data / information (eg, files, tables, lists, etc.) as entities.

[0128] The process line structure information 1014 is the structure information of the process line of the image inspection system 2. Figure 2 The flow line structure information 1014 corresponds to the data of the flow line created and saved by the user. The flow line structure information 1014 indicates the structure and setting status of the processing modules in the flow line. The development system 1 can reproduce the flow line by reading the flow line structure information 1014.

[0129] The GUI screen data 1015 is data for providing the GUI screen 30 described above to the user, and includes GUI component data for constituting a predetermined GUI, data of an image displayed on the GUI screen 30, and the like.

[0130] Image data 1016 is from Figure 1 Camera 21 in Figure 2 The image data 1016 may be image data obtained by the image processing device 102 and the digital signal output device 105 in the development environment 500, or image data generated as a process or a result of a process in the creation of a flow line in the development environment 500. The image data 1016 may also be image files or other data including attribute information for each image.

[0131] In addition, in the memory resources such as the memory 1002, data / information generated during the processing in the development system 1 is appropriately stored. Each data / information can also be saved to Figure 2 Auxiliary storage devices shown in the above, etc.

[0132] In addition, the user U1 can also use the client terminal device 1030 to remotely access the computer system 1 as a server to use the functions of the development system 1. In addition, when the development system 1 is in the form of a client server system, for example, the following operations are performed. The user operates the client terminal device 1030, and the client terminal device 1030 accesses the computer 101 of the development system 1 to send a request. In response to the request, the computer 101 sends a response based on a web page to the client terminal device 1030. The client terminal device 1030 receives the response and displays the above-mentioned GUI screen 30 on its own display. The user observes the GUI screen 30 and inputs instructions, settings, etc. as appropriate. The client terminal device 1030 sends the information such as the instructions to the computer 101. The computer 101 executes the processing realized by the function of the development environment 500 according to the instructions, etc., and stores the processing results (such as flow lines and preview images). The computer 101 sends the processing result information to the client terminal device 1030. The client terminal device 1030 displays the GUI screen 30 on which the processing result information is displayed. The user can check the processing result information and the like on the GUI screen 30 .

[0133] [Development system GUI screen]

[0134] exist Figure 5 , an example of a GUI screen 30 is shown as a simplified configuration of a GUI in a development environment 500 of the development system 1 . Figure 5 The GUI screen 30 has a module selection section 31 (in other words, a component bar), a module configuration section 32 (in other words, a flow line bar), and a preview display section 33 (in other words, a preview bar). In addition, although not shown, the GUI screen 30 also has general GUI components such as menus, scroll bars, and command buttons.

[0135] The user U1 operates an input device such as a mouse or a keyboard, and operates a cursor 39 or the like on the GUI screen 30. The user U1 selects a desired processing module BL from a plurality of processing modules BL set as components in the module selection unit 31, and moves the processing module BL to the module configuration unit 32 by, for example, dragging and dropping, and configures the processing module BL. Thus, the user U1 creates a desired flow line FL by connecting the processing modules BL to each other in the module configuration unit 32.

[0136] In this example, in the module selection unit 31, as processing modules BL, there are a "camera input" module b1, a "determination" module b2, a "processing A" module, a "processing B" module, a "processing C" module, etc. "Processing A" and the like are various image processing, for example, and symbols are added to schematically illustrate. In this example, the following situation is shown: the user U1 selects the "processing C" module in the module selection unit 31 with the cursor 39, and connects it with the subsequent (in other words, behind, to the right) of the "processing B" module in the module configuration unit 32 by dragging and dropping, and configures it.

[0137] For example, a flow line FL is constructed and displayed in a form in which the processing modules BL are connected by lines. In this example, as a provisional flow line FL configured in the production process of the module configuration unit 32, the "camera input" module b1, the "processing A" module, the "processing B" module, and the "processing C" module are sequentially configured from the left side as processing modules BL. Tentatively, the last module is the "processing C" module, and no module is configured after the "processing C" module. Such a state in which the next module is not configured, in other words, a state in which the next module can be connected is displayed as a blank module BS shown in a dotted frame. When the user U1 wants to connect the next module to the flow line FL, the next module can be configured at the position of the blank module BS.

[0138] In a modified example, the blank module BS may not be provided, but when the next processing module BL is moved into the module arrangement section 32, the processing module BL may be connected to the end of the flow line FL.

[0139] In the flow line editing function F1 , which will be described in detail later, the user U1 can create a flow line FL of a desired structure by using the process modules BL of the module selection unit 31 and adding, deleting, or replacing the process modules BL in the module arrangement unit 32 .

[0140] In addition, the user U1 can use the preview display function F2 to make the preview display unit 33 preview and display the action status of the provisional flow line FL configured / displayed in the module configuration unit 32. The user performs a predetermined preview execution operation on the flow line FL of the module configuration unit 32. Based on this operation, the development system 1 displays a preview image of the action status of the flow line FL in the preview display unit 33. The action status of the flow line FL displayed in the preview is the action status of the object and range specified by the operation of the user U1. The object and range are at least one processing module in the flow line FL, and the details will be described later.

[0141] In addition, Figure 5 In the example, the direction of the flow line FL is set to be from left to right and is configured / displayed, but it is obviously not limited to this, and it can also be set to be from top to bottom, etc. Figure 5 In the GUI screen 30, a module selection unit 31, a module configuration unit 32, and a preview display unit 33 are sequentially arranged from the top, but the present invention is not limited thereto. In the GUI screen 30, each column can be arranged in a position and size that is easy for the user U1 to operate. In addition, each column can also be set in the form of an independent window, etc., and is not limited to being always displayed, but can also be set in the form of being displayed according to an operation, etc.

[0142] In addition, the module selection unit 31 may also arrange and configure a plurality of processing modules BL according to types, categories, hierarchies, etc. Examples of types and categories are "input and output", "image processing", etc. In the example of hierarchies, under the "image processing" type, "binarization", "image shape transformation", "filtering", "feature quantity extraction", "arithmetic operation", "logical operation", "rotation", "enlargement / reduction", "graphic drawing", "pattern matching / template matching", "size measurement", "object detection", etc. are set as subtypes. In addition, hierarchies may be further set under the subtypes.

[0143] In addition, various processing modules BL may be provided in different forms such as colors and shapes for easy identification. In addition, the module selection unit 31 may also provide a search column to enable searching of the processing module BL. In addition, as described later, the module selection unit 31 may also provide an input image module as a component. The input image module in this case is a module that specifies an image input to the processing module BL such as the "camera input" module b1.

[0144] When the user U1 wishes to save the flow line FL created by the module configuration unit 32, he performs a predetermined saving operation (e.g., inputting a flow line saving command). In response to this operation, the development system 1 saves the structure of the flow line FL of the module configuration unit 32 as data such as a file ( Figure 2 The data of the flow line in FD, Figure 4 The process line structure information 1014) is saved to the memory resource.

[0145] In addition, in the GUI screen 30, a GUI (e.g., a general file selection dialog, etc.) is provided for selecting the flow line FL to be edited from the data of the saved flow line FL. Alternatively, as described later, a dedicated flow line confirmation screen is provided. In such a GUI or screen, the user U1 selects the flow line FL to be edited from the data of the saved flow line FL. The development system 1 configures the selected flow line FL to the module configuration unit 32 and reproduces it. Thus, the user U1 can edit the flow line FL again. In addition, as described later, multiple flow lines FL can also be created / edited in the GUI screen 30.

[0146] [Process flow of developing system]

[0147] Figure 6 The main processing flow of the development system 1 is shown, which includes steps S1 to S5. In step S1, the development system 1 displays a GUI screen 30 (for example, Figure 5 ). In other words, the user U1 activates the application of the development system 1. In addition, on the GUI screen 30, settings by the user U1, selection of a mode described later, selection of a processing target file, and the like can be performed.

[0148] In step S2, the development system 1 selects a process module BL from the module selection unit 31 according to the operation of the user U1 on the GUI screen 30, and arranges the selected process module BL in the module arrangement unit 32, thereby creating / editing the flow line FL. The development system 1 updates / holds the state of the structure of the flow line FL corresponding to the operation as data.

[0149] In step S3 , the development system 1 previews and displays the operating state of the tentative flow line FL (particularly the designated object / range) being created on the preview display unit 33 according to the operation of the user U1 on the GUI screen 30 , particularly the preview execution operation.

[0150] In step S4, the user U1 confirms whether the flow line FL is completed, whether changes are required, etc., and confirms whether to save the completed flow line FL and end, or whether to save the flow line FL in the process of being created and end. If it is saved in step S4 (Yes), in step S5, the development system 1 saves the state of the structure of the flow line FL at this time as data such as a file, and ends the application of the development system 1. If it is not saved in step S4 (No), the process returns to step S2, and the user U1 repeats the development work in the same way.

[0151] As an example of step S4, the user U1 performs a predetermined saving operation (e.g., inputting a flow line saving command) on the GUI screen 30, and the development system 1 displays a confirmation message such as "Save the flow line being edited?" The user U1 presses the "OK (Save)" button, and the development system 1 saves the flow line FL at this time as data. In addition, the data of the flow line FL can also be automatically saved at a predetermined timing according to the user's setting.

[0152] [Method 1: Flow line editing function]

[0153] The development system 1 of the first embodiment has a flow line editing function F1 as a first mode. The flow line editing function F1 is a function that Figure 5 ) in which the user U1 connects the processing modules BL prepared in advance in the module selection section 31 through GUI operations (such as drag and drop, etc.) and configures them in the module configuration section 32, thereby being able to freely edit / create the flow line FL of the image inspection system 2.

[0154] [Process line creation / editing (1)]

[0155] Replenish Figure 5 Creation / Editing of flow lines FL in. Figure 7 to Figure 10 Show production / editing as Figure 5 This is an example of detailed operation and GUI in the case of such flow line FL. Figure 7 The flowchart shows the flow of sequentially arranging the process modules BL in the module arranging section 32 when the process line FL is produced.

[0156] Figure 7(A) shows a state where the processing module BL is not configured as the initial state of the module configuration unit 32. It is also possible that a blank module BS is displayed within the module configuration unit 32 at this time. The blank module BS becomes the target for configuring the first or the next processing module BL, and is illustrated as a rectangle with a dashed line frame, for example. For example, it is also possible to set up a GUI such that the processing module BL can be added to the flow line FL only when the processing module BL is configured at the position of the blank module BS. In a form where the blank module BS is not used, it is also possible to set up a GUI such that the processing module can be added to the flow line FL when the processing module BL is configured at a position other than the existing processing module BL within the module configuration unit 31.

[0157] In (A), the user U1 configures the processing module BL selected from the module selection unit 31, for example, the "camera input" module b1, at the position of the blank module BS within the module configuration unit 32. Thereby, the "camera input" module b1 is configured as the first module of the flow line FL.

[0158] (B) is a state where the "camera input" module b1 is configured. The blank module BS moves behind the configured "camera input" module b1 and is displayed. The user U1 configures the next processing module BL selected from the module selection unit 31, for example, the "processing A" module, at the position of, for example, the blank module BS within the module configuration unit 32.

[0159] (C) is a state where the "processing A" module is configured behind the "camera input" module b1. The blank module BS moves behind the configured "processing A" module and is displayed. The user U1 configures the next processing module BL selected from the module selection unit 31, for example, the "processing B" module, at the position of, for example, the blank module BS within the module configuration unit 32.

[0160] (D) is a state where the "processing B" module is configured behind the "processing A" module. The blank module BS moves behind the configured "processing B" module and is displayed. The user U1 configures the next processing module BL selected from the module selection unit 31, for example, the "processing C" module, at the position of, for example, the blank module BS within the module configuration unit 32.

[0161] Thereby, a flow line FL as shown in Figure 5 can be created. The operation of adding a module can be implemented not limited to the above example. For example, the user U1 selects a desired processing module BL with the module selection unit 31, and for this module, inputs a module addition instruction, for example, by a right-click operation. In response to this operation, the development system 1 configures the processing module BL to be added to the end of the flow line FL within the module configuration unit 32.

[0162] [Production / Editing of Flow Line (2)]

[0163] in addition, Figure 8 The following is an example of an operation for deleting a processing module BL in a flow line FL. When deleting a desired processing module BL in a flow line FL, the user U1 selects the desired processing module BL and performs a predetermined deletion operation. In response to this operation, the development system 1 deletes the processing module BL on the flow line FL of the module configuration unit 32.

[0164] Figure 8 (A) shows a state in which the "camera input" module b1 to the "processing C" module are configured as a tentative flow line FL in the module configuration unit 32. In (B), when the user U1 deletes, for example, the "processing B" module in the flow line FL, the user U1 selects the "processing B" module with the cursor 39 and performs a predetermined deletion operation. The predetermined deletion operation may be, for example, a deletion instruction input as shown in the figure. For example, candidate instructions are displayed in response to a right-click, and a deletion instruction is specified from the candidate instructions. As another example of a predetermined deletion operation, an operation of dragging and dropping the processing module BL selected by the user U1 to the outside of the module configuration unit 32 may also be provided. For example, it may be an operation of returning to the module selection unit 31 or an operation of configuring to a predetermined trash can bar.

[0165] (C) shows the state of the flow line FL after the deletion operation of (B). In this state, the "processing A" module located in front of the deleted "processing B" module and the "processing C" module located behind the deleted "processing B" module are connected by a line.

[0166] Furthermore, the development system 1 can also perform an undo (retract) operation by storing a history of operations performed by the user U1 on the flow line FL of the module configuration unit 32. For example, when the user U1 wishes to retract the module deletion from (B) to (C), the user U1 performs a predetermined undo operation (e.g., inputs an undo command). This allows the state to return from (C) to (B).

[0167] In addition, Fig. 9 In FIG. 1 , as a variation of the module adding operation, the user U1 inserts a processing module BL in the middle of the flow line FL. Fig. 9In (A), the flow line FL is in a state where the "camera input" module b1, the "processing A" module, and the "processing B" module are configured. In this state, when the user U1 wishes to insert "processing D" between "processing A" and "processing B", for example, as in (B), the "processing D" module is selected from the module selection unit 31, and is configured near the line between the "processing A" module and the "processing B" module in the module configuration unit 32 by dragging and dropping. In response to this operation, the development system 1 is set to the following state as in (C): in the flow line FL, the "processing D" module is added by insertion between the "processing A" module and the "processing B" module, and they are connected by lines.

[0168] In addition, the development system 1 may display the position where the processing module BL can be configured in the flow line FL of the module configuration unit 32 in an easy-to-understand manner and in a predetermined form for each processing module BL selected by the user U1 using the module selection unit 31. For example, when the "processing D" module is selected in (B), the position where the "processing D" module can be configured includes not only the position of the blank module BS behind the "processing B" module, but also the position of the line between the "processing A" module and the "processing B" module. In this case, the development system 1 displays these configurable positions in the form of areas of predetermined colors, etc. The configurable positions may also include the module replacement positions described later.

[0169] [Process line creation / editing (3)]

[0170] in addition, Fig.10 FIG. 2 shows an operation example in which a process module BL in a flow line FL is replaced with another process module BL. Fig.10 In (A), in the module configuration section 32, the "Processing C" module is configured as an example of the flow line FL of the state before replacement. Assume that from this state, the user U1 wishes to replace the "Processing A" module with the "Processing D" module. The user U1 selects the "Processing D" module of the object after replacement from the module selection section 31, and configures it to the position of the "Processing A" module of the replacement object (in other words, before replacement) in the module configuration section 32 by dragging and dropping. In response to this operation, the development system 1 replaces the "Processing A" module with the "Processing D" module as in (B). In addition, the development system 1 may also display the part of the replaceable processing module BL in a predetermined color or other form according to the processing module BL selected by the user U1 during the operation of (A).

[0171] In addition, as an example of other permutations, assume that in the state of (B), user U1 wishes to swap the configurations of the "Process D" module and the "Process B" module. In this case, user U1 selects the "Process D" module as in (C) and moves it to the position of the "Process B" module by means of an operation such as drag and drop. In response to this operation, development system 1 swaps the "Process B" module with the "Process D" module as in (D), and swaps the "Process D" module at the source of the movement with the "Process B" module.

[0172] [Second Mode: Preview Display Function]

[0173] In development system 1 of Embodiment 1, as the second mode, it has a preview display function F2. The preview display function F2 is a function of displaying a preview of the action state of the flow line FL produced / edited by the flow line editing function F1 of the first mode on the preview display unit 33. This preview display function F2 displays a preview image representing the action state of a specified object / range (for example, the range up to the specified processing module) in the tentative flow line FL of the module configuration unit 32. This preview image is, for example, a processing result image of each point (i.e., the output value) behind each module in the range from the processing module at the beginning of the flow line FL to the specified processing module.

[0174] With the preview display function F2, user U1 can appropriately preview and confirm the action state of the tentative flow line FL, so it can make the action confirmation, debugging, etc. of the flow line FL easier. The preview display function F1 has various types of preview display functions described later that can assist in the development work implemented by user U1.

[0175] Return to Figure 5 , in the GUI screen 30, for example, when a predetermined preview execution operation (such as preview display instruction input) is performed without user U1 performing any special operation or when the preview display condition on the control is satisfied, development system 1 displays a preview image of the action state of the tentative flow line FL at this time in the preview display unit 33. The preview display object / range at this time can be specified in various styles. As an example, it is set to all the modules from the beginning to the end of the flow line FL at this time. In this case, the preview image displayed on the preview display unit 33 becomes a video or animation that continuously displays the processing result images of each module in time series.

[0176] In addition, when the user U1 specifies a desired processing module BL using the module configuration unit 32 and performs a preview execution operation, as an example, the preview display object / range becomes the range from the beginning of the flow line at that time to the specified processing module. Alternatively, in other embodiments, when the user U1 specifies a desired processing module BL using the module configuration unit 32 and performs a preview execution operation, as an example, the preview display object / range becomes only the specified processing module.

[0177] The predetermined preview execution operation can be specified in various styles, for example, the following can be listed. An example of a preview execution operation is a specific operation such as double-clicking a blank column or a desired processing module in the module configuration unit 32. Another example of a preview execution operation is inputting a preview display instruction. For example, the user U1 can select a preview display instruction from candidate instructions by right-clicking, etc. and input it. In addition, for example, a preview display button in the GUI screen 30 can also be pressed. Another example of a preview execution operation may also be an operation such as double-clicking on an area of ​​the preview display unit 33. In other examples, the user U1 may select a desired module using the module configuration unit 32 and configure it to the preview display unit 33 by dragging and dropping.

[0178] When there is no designation / selection of a specific processing module BL as the preview execution operation, the development system 1 may determine the preview object / range to be the entirety of the tentative flow line FL (from the beginning to the end). In addition, when there is designation / selection of a specific processing module BL as the preview execution operation, the development system 1 may determine the preview object / range to be the object / range including the designated processing module in the tentative flow line FL (for example, the range from the beginning to the designated module).

[0179] The regulations regarding the GUI, operation, preview display object / range, etc. can be pre-installed in the software of the development system 1 and defined according to each function. In addition, these can also be selected / switched according to the mode or user setting described later.

[0180] [Preview display (1)]

[0181] Fig.11Action examples related to the preview display function F1 are shown. (A) shows the first example of preview display, and (B) shows the second example. In (A), in the module configuration unit 32, the "camera input" module b1 to the "processing C" module are configured as a temporary flow line FL. In the first example, in the flow line FL at this time, the user U1 selects the desired processing module BL, such as the "camera input" module b1 (or the part of the line immediately after the module), and performs a predetermined preview execution operation. In response to this operation, the development system 1 previews and displays the image of the processing result from the beginning of the flow line FL at this time to the selected processing module BL in the preview display unit 33. In the first example of (A), only the "camera input" module b1 becomes the range, and the image of the processing result as the output of the "camera input" module b1 becomes the object. The development system 1 displays the image of the processing result as the output of the "camera input" module b1 (with Figure 1 The camera captured image 51 is equivalent to being displayed as a preview image 1101.

[0182] In the second example of (B), the user U1 selects the "Processing A" module (or the part of the line immediately following the module) in the module configuration unit 32 and performs a predetermined preview execution operation. In response to this operation, the development system 1 sets the preview object / range to the image of the processing result of each module in the range from the "Camera Intake" module b1 at the beginning of the flow line FL to the selected "Processing A" module. The development system 1 displays the processing result image of the "Camera Intake" module b1 and the processing result image of the "Processing A" module (their video) as preview images 1102 in the area of ​​the preview display unit 33.

[0183] The processing result image of the "Processing A" module is an image obtained by inputting the camera input image, which is the output of the "Camera Input" module b1, into the "Processing A" module and processing it. For example, in the case where the "Processing A" module is a binary image processing, the image after the binary image (with Figure 1 The resulting image 52 is equivalent to ).

[0184] Similarly, although not shown, by the user U1 selecting a desired processing module BL on the flow line FL and performing a preview execution operation, the processing results up to the selected module can be previewed and displayed.

[0185] In the above example, the preview object / range is set to the module at the beginning of the flow line FL to the specified module. In this case, the processing result image of each module becomes the preview display object, and the plurality of processing result images are previewed and displayed as a video, animation, etc. in a time series in the preview display unit 33 (described later). Fig.15 ).

[0186] Furthermore, as described later, according to the definition of the preview display function F2 , the object / range of preview can be set to only one processing module selected / designated by the user U1 .

[0187] [About the camera import module]

[0188] Supplement the input and output of the "camera input" module b1. The development system 1 has a camera / sensor device 21 ( Figure 1 ) or the camera / sensor device of development system 1 ( Figure 2 The digital signal output device 105 in the image / video is captured / input in real time. Figure 1 The input image 50 of the camera input processing 201, Figure 2 Such an image 50 is equivalent to the image 15 in FIG. Figure 5 The "camera input" module b1 on the process line FL of FIG. 1 is an image input by the "camera input" module b1 (sometimes referred to as an input image or a processing target image). In other words, the "camera input" module b1 is an "image input" module. The "camera input" module b1 outputs an image ( Figure 1 The camera-captured image 51 in the image capture process is, for example, a process in which a signal of an image 50 corresponding to the camera / sensor device 21 is sampled according to a predetermined imaging cycle or the like (internal setting to be described later) and converted into image data having pixel values.

[0189] The input image input to the "camera capture" module b1 is not limited to the camera image captured in real time by an actual camera, but may be other types of images. Examples of other types of images include image data arbitrarily created, set, or specified in advance by the operator or user U1.

[0190] [Preview display function and mode]

[0191] Fig.12 The development system 1 may also display the preview function F2 in the GUI screen 30 through a GUI such as a menu. Fig.12 The various functions such as the example above enable the user U1 to select and execute, and user settings. Fig.12 , it is obvious that a form in which only a single function among these functions is installed, or a form in which a predetermined plurality of functions are installed can also be set.

[0192] The development system 1 switches the preview display function selected by the user U1 in the GUI screen 30, for example, in a valid state. The development system 1 prepares a control mode for each preview display function, for example. When the user U1 specifies a function to be used, the development system 1 switches the control mode to a mode in which the function can be used. In accordance with the mode, the GUI in the GUI screen 30 is also switched. The GUI of each mode includes, for example, a system that interprets what command or process will be interpreted when a predetermined operation (such as single click, double click, etc.) is performed on an operation object such as a processing module or a preview bar.

[0193] Hereinafter, various functions of the preview display function F2 will be described in sequence.

[0194] [Function 1: Preview display switching function]

[0195] exist Fig.13 , an explanatory diagram of a preview display switching function implemented by module selection is shown as function 1. Function 1 is a function that enables selection of a preview display object according to an operation of the user U1, for example, switching of preview display content according to selection of the processing module BL. Function 1 may be set as a normal preview function that is basic in the first embodiment. The development system 1 may also control the state in which function 1 is enabled to be a predetermined mode (set to mode 1) in terms of control.

[0196] exist Fig.13 In the example of the GUI screen 30 of FIG. 3 , in the module configuration section 32, as a tentative flow line FL, the "camera input" module b1, the "processing A" module, the "processing B" module, and the "processing C" module are sequentially configured. It is assumed that from this state, the user U1 performs a preview of the flow line FL according to a predetermined preview execution operation (for example, double-clicking the processing module BL, or inputting a preview display instruction). In function 1, the following processing actions (including the operation of the user U1) are performed when executing the preview. In detail, function 1 includes the following function 1a, etc.

[0197] (Function 1a) When there is no special operation by the user U1 other than the predetermined preview execution operation, the development system 1 uses the processing module BL arranged at the end of the flow line FL at that time, for example, the "processing C" module, as the preview object. The development system 1 displays the processing result image of the "processing C" module of the object as the preview image 1301 on the preview display unit 33. In this example, the preview object is the same as the preview image 1301. Fig.11 Different from the previous one, it is a single processing module, and the preview image becomes a still image. Fig.13 This situation is shown as Example 1.

[0198] Furthermore, when there is no predetermined preview execution operation itself as the operation of the user U1 , the development system 1 may automatically display the preview of the last processing block BL of the flow line FL on the preview display unit 33 .

[0199] Furthermore, when the preview object is set to a single processing module as described above, not only the processing result image as the output of the processing module but also the image as the input of the processing module may be displayed in the area of ​​the preview display unit 33. In the area of ​​the preview display unit 33, for example, these input images and processing result images may be displayed side by side, or these images may be displayed while switching in time series.

[0200] In addition, regarding function 1a, it can also be Fig.11 Similarly, the object / range of preview display is set to the top of the flow line FL to the designated processing module, and the preview image is set to a video based on the processing result image of each module.

[0201] Whether the object / range of the preview display is set as the output image of only one processing module BL specified by the user U1 as in the above example, or as the input / output image, or as the beginning of the flow line FL to the specified module, etc., is determined by the installation of the software of the development system 1, and the user U1 can distinguish and use it according to the function and GUI operation.

[0202] (Function 1b) The user U1 selects, for example, the "Process A" module in the flow line FL of the module configuration unit 32 by clicking or the like. In this case, in response to this operation, the development system 1 switches the preview display object from the previous "Process C" module to the "Process A" module. The development system 1 displays the processing result image of the "Process A" module as a preview image 1302 on the preview display unit 33. Fig.13 This situation is shown in Figure 2 as Example.

[0203] In function 1b, the user U1 can select other desired processing modules BL in the flow line FL (e.g., "camera capture" module b1, "processing B" module) to preview the processing modules. Through this function 1b, the user U1 can confirm the processing content of the desired processing module BL in the flow line FL. Regarding function 1b, as with function 1a, the preview object can be specified in various styles.

[0204] (Function 1c) In addition to functions 1a and 1b, function 1c is also provided. In function 1c, the user U1 performs other predetermined operations (such as a second click, a double click, or other command input) on the "Processing A" module selected by function 1b. In response to this operation, the development system 1 switches the internal setting (details will be described later) on the "Processing A" module, and previews the processing result image of the "Processing A" module under the switched internal setting. For example, the initial internal setting of the "Processing A" module is set to the first internal setting. When the user U1 performs the first click as an operation of the above-mentioned function 1b, the result image of "Processing A" under the first internal setting is previewed. Next, when the user U1 performs the second click as an operation of the above-mentioned function 1c, the second internal setting is switched to, and the result image of "Processing A" under the second internal setting is previewed. Similarly, when the same processing module BL is repeatedly clicked, the internal settings of the "processing A" module are switched in sequence, and the processing result image corresponding to the state of the internal settings is previewed. Similarly, by reselecting another processing module BL, a preview of the switched internal settings can be realized. Through the above function 1c, the user U1 can confirm the processing content of each difference in the internal settings of each processing module BL of the flow line FL.

[0205] (Function 1d) It also has functions 1d and 1e. In function 1d, the user U1 operates, for example, the "Processing A" module as a component in the module selection unit 31 by clicking or the like. In this case, in response to this operation, the development system 1 uses the preview object as the "Processing A" module monomer specified by the selection, and switches the preview display content in the preview display unit 33 to the processing result image of the selected "Processing A" module (described later). In this case, in order to obtain the processing result image of the "Processing A" module, an input image is also required, so the development system 1 uses a pre-set sample input image as the input image. The development system 1 obtains an image of the result of inputting the sample input image to the "Processing A" module and processing it, and displays it as a preview image. In this function 1d, the user U1 can confirm the processing content of each processing module BL that is a component constituting the flow line FL through the preview.

[0206] (Function 1e) In addition, when the same "Processing A" module is continuously operated by clicking, etc. in addition to the operation in the above-mentioned function 1d, the development system 1 previews and displays the processing result image in the state where the internal settings of the "Processing A" module are switched, in the same way as the above-mentioned function 1c.

[0207] As described above, according to function 1, the user U1 can easily judge and confirm which processing module BL should be arranged next in the flow line FL of the module arrangement unit 32 by appropriately using the preview display. Even if the user U1 does not understand the detailed processing content (e.g., program code) of the processing module, he can visually confirm it based on the preview display result, so it is easy to create the flow line FL.

[0208] [How to select the module to be previewed]

[0209] exist Fig.14 In, as Fig.13 In addition, a plurality of methods related to GUI operation for selecting or designating one or more processing modules BL to be the target or range of preview display are shown. At least one of these methods is implemented as a function of the development system 1.

[0210] (A) shows the first method, which is a case where the preview object is set to only one process module BL selected by the user U1. In the figure, the process module BL to be the preview object is shown surrounded by a dotted frame. The user U1 selects a desired process module BL, such as the "process B" module, from the flow line FL of the module configuration unit 32 using a cursor, and performs a predetermined preview execution operation. As a result, a preview of the specified process module BL is displayed.

[0211] (B) shows the second method, which is a case where the preview object is set to the range from the processing module BL at the beginning of the flow line FL to one processing module BL (also recorded as the end module) selected by the user U1. The user U1 selects a desired end module, such as the "processing B" module, from the flow line FL of the module configuration unit 32 using a cursor, and performs a predetermined preview execution operation. As a result, the processing result images of the multiple modules in the range from the beginning module of the flow line FL to the specified end module are previewed and displayed.

[0212] (C) shows a third method, which is a case where the preview object is set to the range from one processing module BL (also recorded as a starting module) selected by the user U1 of the flow line FL to the last processing module BL of the flow line FL. The user U1 selects a desired starting module, such as the "processing B" module, from the flow line FL of the module configuration unit 32 using a cursor, and performs a predetermined preview execution operation. As a result, each processing result image of a plurality of modules in the range from the specified starting module to the last module of the flow line FL is previewed and displayed.

[0213] (D) shows a fourth method, which is a case where the preview object is set to the range from the first module (starting module) selected by the user U1 to the second module (end module) selected by the user U1 on the flow line FL. The user U1 selects the first module desired to be operated, such as the "processing A" module, from the flow line FL of the module configuration unit 32, and then selects the second module desired to be operated, such as the "processing B" module, and performs a predetermined preview execution operation. As a result, the processing result images of the multiple modules in the range from the first module to the second module of the flow line FL are previewed and displayed.

[0214] Although not shown, as another method, as described above, even when no processing module in the flow line FL is selected in the module arrangement unit 32, the preview target / range can be set to all modules or the last module in the flow line FL.

[0215] [Preview display of multiple processing modules]

[0216] exist Fig.15 In, as Fig.13 In the supplementary description, the case where the processing result images of each of the plurality of processing modules BL as the target are sequentially displayed in the preview display unit 33 like a video / animation is shown. It is assumed that, for example, the "processing C" module is selected in the module configuration unit 32, and a preview execution operation is performed. In response to this operation, the development system 1 sets the target / range of preview display to the range of four processing modules BL from the "camera capture" module b1 to the "processing C" module. The development system 1 displays the processing result images of each of the four processing modules BL in the area of ​​the preview display unit 33 while switching sequentially on the time axis, for example, in a loop. The preview images 1500 displayed in the preview display unit 33 are sequentially switched and displayed like a camera capture result image 1501, a processing A result image 1502, a processing B result image 1503, a processing C result image 1504, and a camera capture result image 1501, for example, as a video / animation composed of changes in four still images.

[0217] In addition, when the object / range of preview is a video involving a plurality of processing modules, the user U1 can also designate a desired time point in the video and preview it as a still image.

[0218] [Preview display using sample input image]

[0219] exist Fig.16 In, as Fig.13In addition, an explanatory diagram related to the preview display function of the processing module BL of the sample input image using the above-mentioned function 1d is shown. The user U1 selects a desired processing module BL in the module selection unit 31 differently from the current flow line FL of the module configuration unit 32 and performs a predetermined preview execution operation, thereby realizing a preview (also described as a sample preview) of the selected processing module BL.

[0220] exist Fig.16 In the example, the user U1 uses the module selection unit 31 to select the "processing C" module as the object, and performs a preview execution operation in the sample image mode (for example, sample preview instruction input). In response to this operation, the development system 1 applies a pre-set sample image (sample input image) as an input image for the selected "processing C" module. The development system 1 obtains the processing result image of the "processing C" module for the sample input image as a sample processing result image. In addition, the development system 1 displays the sample input image 1601 and the sample processing result image 1602 as preview images in the area of ​​the preview display unit 33. As a result, the user U1 can confirm the function / effect of the selected processing module BL through the preview.

[0221] The sample input image is associated with the processing module BL and is preset / saved in the DB of the development system 1. In addition, the user U1 may be allowed to select and specify the sample input image to be input to the processing module BL. As the GUI at this time, a general file reading GUI may be applied. As other GUIs, Fig.16 The module selection unit 31 is used as shown in the figure below. In the module selection unit 31, a plurality of set sample images are arranged in the form of modules. For example, the type of sample images can also be managed as one of the tags. The user U1 can select a desired sample image module from the module selection unit 31 and apply it.

[0222] Furthermore, the display of the sample input image 1601 in the preview display unit 33 may be omitted. In addition, sample processing result images using sample input images such as the sample preview image 1602 may be associated and stored in advance. Thus, when previewing the sample, the processing of the processing module BL may be omitted, and the stored sample preview image may be read and displayed for preview.

[0223] [Internal settings of processing module]

[0224] exist Fig.17 In, as Fig.13In addition to the above, an explanatory diagram of the internal setting of each processing module BL is shown. As an example of a processing module BL that is a component, a "binarization" module 1700 in image processing is provided. Regarding the binarization, as detailed processing contents, there are various known binarization methods, and these various binarization methods can be specified by using parameter values ​​of the internal setting 1710 of the "binarization" module 1700. For example, a case where the internal setting 1710 has internal settings 1711, 1712, and 1713 is shown. The internal setting 1711 is "normal binarization", the internal setting 1712 is "range binarization", and the internal setting 1713 is "Otsu binarization". For example, these three types of "binarization" can be switched by an operation such as a click by the user U1. For example, in the case of the first click on the "binarization" module 1700, the internal setting 1711 of "normal binarization" is applied. In the case of the second click, the internal setting 1712 of "range binarization" is applied. In the case of the third click, the internal setting 1713 of "Otsu Binarization" is applied. In the case of the fourth click, the internal setting 1711 of "Normal Binarization" is applied again. In this way, for example, the internal setting value is switched cyclically.

[0225] [Display of internal setting information of processing module]

[0226] in addition, Fig.18 The following is a display example of a preview display related to the above internal settings in the GUI screen 30 (the above functions 1d and 1e). First, as an embodiment, the internal setting information of each of the above processing modules BL may be displayed in the GUI screen 30. The development system 1 displays the internal setting information of the processing module selected by the user U1 in the module selection unit 31 or the module configuration unit 32. Fig.18 In the example, for example, information about the internal settings of a module such as "Process A" selected by the user U1 by clicking or the like in the flow line FL of the module configuration unit 32 is displayed using a balloon-shaped GUI. For example, a balloon image is displayed near the clicked position, and identification information of a plurality of internal settings of the "Process A" module and identification information of the currently applied internal setting are displayed in an easy-to-understand manner. For example, there are internal settings A1, A2, and A3, and the currently applied internal setting is internal setting A1.

[0227] Moreover, in Fig.18In the example, according to the state of the internal setting of the processing module BL in the module configuration unit 32, the preview of the processing module selected / designated by the user U1 can be displayed as a preview of the internal setting state at that time in the preview display unit 33. In this example, the processing result image 1801 corresponding to the state of the internal setting A1 of the "processing A" module is displayed as the preview image. Information describing the preview image, such as internal setting information, may also be displayed in the preview display unit 33.

[0228] [Confirmation / change of internal settings of processing modules]

[0229] In addition, Fig.19 The following display example is shown in which detailed information on the internal settings of the processing module BL can be displayed so that the user U1 can confirm or set / change parameter values. For example, it is assumed that there is a "binarization" module as a "processing A" module configured in the module selection unit 31 or the module configuration unit 32. The user U1 selects the "binarization" module of the operation module configuration unit 32 by, for example, clicking. In response to this operation, the development system 1 uses a predetermined GUI, such as a pop-up format, to display detailed information on the internal settings of the "binarization" module. In this example, a "binarization" column 1901 is displayed in a pop-up format. The module name can be set in the "binarization" column 1901, and a plurality of internal settings (such as function selection) can be selected as the internal setting (in other words, function selection). Fig.17 ) and set it. In addition, as a detailed parameter value, for example, a threshold value for binarization can be set.

[0230] Not limited to the example of the above-mentioned "binarization" module, there are cases where there are multiple internal settings depending on the processing module BL, and similarly, the internal settings can be selected and changed. In other examples, in the case of the "filtering" module, multiple types of filtering processes and detailed parameter values ​​are prepared as multiple internal settings. In other examples, in the case of the "binary image shape transformation" module, "contraction", "expansion", "noise removal", "contour extraction" and the like are prepared as internal settings. In other examples, in the case of the "ellipse drawing" module, drawing mode, center coordinates, radius, color and the like are prepared as internal settings.

[0231] In addition, it is not limited to the case where a plurality of internal settings corresponding to the processing details are provided in one processing module BL. In order to realize the same function, different processing modules may be prepared for each internal setting. For example, a "normal binarization" module, a "range binarization" module, etc. may be provided in the module selection unit 31. In addition, the user U1 may be enabled to change the parameter values ​​of the internal settings to various values ​​and register them in a deformed manner based on the processing module BL (e.g., "binarization" module) prepared in advance as a basic component in the module selection unit 31.

[0232] [Variation of Function 1: Hypothetical App Preview]

[0233] As a modification example of the above-mentioned function 1, the following function may be provided: When the user U1 selects a desired process module BL using the module selection unit 31, a preview of a case where the process module BL is applied to the flow line FL is displayed.

[0234] exist Fig. 20 , an explanatory diagram of a hypothetical application preview function of this modification is shown. It is assumed that in the module configuration unit 32, for example, the "camera capture" module, the "processing A" module, and the "processing B" module are configured as a tentative flow line FL. The user U1 discusses the next processing module BL that he wants to connect after the "processing B" module, and selects it from the components in the module selection unit 31. At this time, it is assumed that, for example, the "processing D" module and the "processing E" module are candidates for the next processing module BL.

[0235] The user U1 first performs a selection operation (e.g., click, hover the cursor, etc.) on the "Processing D" module. According to this operation, the development system 1 assumes that the "Processing D" module is configured at the end of the flow line FL of the module configuration unit 32 (in this example, it is next to the "Processing B" module), obtains the processing result image of the "Processing D" module in this assumed state, and previews it in the preview display unit 33. The processing result image of such a hypothetical application case is an image of the result obtained by inputting the processing result image output by the "Processing B" module into the "Processing D" module and processing it. In this case, the processing result image of the "Processing D" module is displayed in the area of ​​the preview display unit 33 as shown in the preview display 2001.

[0236] Similarly, when the user U1 selects the "Processing E" module as another candidate, the development system 1 assumes that the "Processing E" module is arranged at the end of the flow line FL, obtains the processing result image of the "Processing E" module in this assumed state, and performs a preview display in the preview display unit 33. In this case, the processing result image of the "Processing E" module is displayed in the area of ​​the preview display unit 33 as in the preview display 2002. In addition, Fig. 20 The above two examples are summarized in FIG.

[0237] In the case where the preview function is applied in this modification, the user U1 can view the preview even if he does not place the processing module BL on the flow line FL by dragging and dropping or the like.

[0238] In addition, the preview display unit 33 may not be limited to one area, but may be provided with a plurality of preview display areas. Fig. 20 In this way, the first preview image is displayed in the first preview display area, and the second preview image is displayed in the second preview display area, and they are displayed side by side so as to be comparable.

[0239] [GUI example of preview display section]

[0240] Fig.21 An example of the GUI configuration of the preview display unit 33 is shown. Fig.21 An example is when the preview display object / range is a video including multiple processing modules (e.g. Fig.15 ) is a GUI capable of performing operations such as playback and display of the video. The preview display unit 33 has, in addition to the preview display area 2100, a video playback button 2101, a video stop button 2102, a start button 2103, an end button 2104, a playback time bar 2105, a zoom-in button 2106, a zoom-out button 2107, and a label 2108 as GUI components.

[0241] For example, the preview display can be performed by pressing the video playback button 2101, double-clicking the preview display area 2100, dragging and dropping to the preview display area 2100, etc. As an example of other GUI components, a slider bar can also be provided to select a desired time point in the video for playback display.

[0242] [Function 2: Multi-preview function 1]

[0243] exist Fig. 22, an explanatory diagram of the multi-preview function 1 is shown as the function 2. The multi-preview function 1 is, for example, a function of displaying a plurality of processing result images of a plurality of processing modules BL side by side on the preview display unit 33. In the function 2, when the preview is executed, the following processing actions are performed. The function 2 includes the following function 2a and the like.

[0244] (Function 2a) The user U1 enables the multi-preview function and specifies a plurality of processing modules BL (for example, the above-mentioned processing modules BL) of the flow line FL in the module arrangement unit 32 as the preview object / range. Fig.11 The development system 1 displays the multiple processing result images of the multiple processing modules BL specified by the user U1 as preview display objects. The development system 1 displays the multiple processing result images of the object side by side as multiple previews in the divided multiple areas in the area (one screen) of the preview display unit 33.

[0245] For example, the multi-preview function is enabled by a predetermined operation. For example, the development system 1 may also transfer the control mode to the multi-preview mode (mode 2) according to a predetermined operation. In this mode, a predetermined preview execution operation (e.g., preview display instruction input) is interpreted as a predetermined multi-preview execution operation (e.g., multi-preview display instruction input).

[0246] (Function 2b) In addition, in the above-mentioned function 2a, the user U1 can select the number of divisions for multi-preview. For example, it is also possible to select "none / 4 / 9" as the number of divisions together with the multi-preview execution operation. In other words, it is also possible to select a multi-preview mode for each number of divisions. It is also possible to allow the user to pre-set the number of divisions in the multi-preview mode. The case of "none" division is equivalent to the normal preview display of the above-mentioned function 1.

[0247] exist Fig. 22 In the figure, two examples of multi-preview display are summarized and illustrated when four processing modules BL, namely, the "camera input" module b1 to the "processing C" module, are configured as a temporary flow line FL in the module configuration unit 32. In the first example (A), a multi-preview display 2211 is shown when a multi-preview execution operation 2201 with a division number of 9 is performed on the target flow line FL. In the second example (B), a multi-preview display 2212 is shown when a multi-preview execution operation 2202 with a division number of 4 is performed on the target flow line FL.

[0248] In the preview display unit 33, multiple previews are displayed with the selected number of divisions. In the first example of (A), the area of ​​the preview display unit 33 (a square in this example) is divided into 3×3 division areas by the number of divisions 9. In the 3×3 division areas, 4 division areas are used to display the processing result images of the 4 processing modules of the flow line FL as preview images. In the second example of (B), the area of ​​the preview display unit 33 (a square in this example) is divided into 2×2 division areas by the number of divisions 4. In the 2×2 division areas, 4 division areas are used to display the processing result images of the 4 processing modules of the flow line FL as preview images.

[0249] For example, the display content of the preview display unit 33 in the case of multiple previews is controlled as follows. In (C), for example, as in (B), the multiple preview display content in the case of 4 divisions is schematically illustrated. Regarding the multiple preview display, in the area of ​​the preview display unit 33, for example, the display starts with the upper left division area as the starting point and ends with the lower right division area as the end point. Each division area is initially a predetermined background color. In addition, text such as "Camera Intake Preview" indicating the corresponding associated processing module may be displayed in the division area.

[0250] Development system 1 based on the operation of user U1 and the above (for example Fig.11 ) Similarly, the object / range of the multi-preview is determined. In addition, for example, the user U1 can also select / designate a module for starting the multi-preview, such as the "camera capture" module b1, and a module for ending the multi-preview, such as the "processing C" module, from the flow line FL of the module configuration unit 32 by clicking or other operations when performing the multi-preview operation. The development system 1 obtains respective processing result images for each processing module BL from the starting module to the end module in the object / range of the multi-preview according to the above operation. The development system 1 automatically sets these multiple processing result images as multi-previews in sequence and displays them in the divided areas of the preview display unit 33.

[0251] In the example of (C), the development system 1 first displays the processing result image of the "Camera Intake" module b1 in the upper left segmented area 2221 as a Camera Intake Preview. The development system 1 then displays the processing result image of the "Processing A" module in the upper right segmented area 2222 as a Processing A Preview. The development system 1 then displays the processing result image of the "Processing B" module in the lower left segmented area 2223 as a Processing B Preview. The development system 1 then displays the processing result image of the "Processing C" module in the lower right segmented area 2224 as a Processing C Preview. Thereafter, the images of the four segmented areas are continuously displayed until the multi-preview ends. Multi-preview display can also be performed in the case where the number of segments is 9.

[0252] As a modified example, when performing the multi-preview operation, the user U1 selects the processing module BL (for example, the "processing C" module) to be the end point from the flow line FL of the module configuration unit 32 as the display target of the multi-preview by clicking or other operations. According to this operation, the development system 1 sequentially obtains the processing result images of each module from the specified end point module in the opposite direction to the above, that is, in the direction toward the beginning of the flow line FL, and displays them as a multi-preview on the preview display unit 33.

[0253] In addition, when the number of processing blocks BL is less than the number of divisions (e.g., 4) in the preview display unit 33, a part of the division area becomes the background display, and the processing result image is not displayed. The number of divisions is not limited to 4 or 9, and the number of divisions can be set to 2 or more.

[0254] According to the multi-preview function 1, the user U1 can check the processing contents of each processing module BL side by side on the GUI screen 30. According to this function, the progress of a plurality of image processing can be checked at once, so the efficiency of debugging by preview can be further improved.

[0255] In addition, the development system 1 may automatically determine the number of divisions of the multi-preview according to the number of processing modules BL selected as the object of the multi-preview. For example, when the number of processing modules BL of the object selected by the operation of the user U1 is 1, it is set to normal preview display. When the number of modules is 2, 3, or 4, it is set to multi-preview with a division number of 4. When the number of modules is 2 or 3, 2 or 1 of the 4 division areas become unused. When the number of modules is 5, 6, 7, 8, or 9, it is set to multi-preview with a division number of 9.

[0256] In addition, the development system 1 may automatically determine the number of divisions to be the same as the number of processing modules BL in the range specified by the operation of the user U1. For example, the development system 1 sets the number of divisions to 2 when the number of target processing modules BL is 2, and sets the number of divisions to 3 when the number of target processing modules BL is 3. Fig.12 2 shows a case where the divided regions are also square when the area of ​​the preview display unit 33 is square, but the present invention is not limited to this.

[0257] [Variation of multi-preview function 1]

[0258] Fig.23 A modification example related to the multi-preview function 1 is shown. This modification example is an example in which multiple previews are displayed in a row horizontally in accordance with the arrangement of the flow lines FL. The arrangement of the multiple divided areas in the area of ​​the preview display unit 33 is not limited to Fig. 22In the example, it can be a predetermined configuration, such as a horizontal row and a vertical column. Fig.23 2 shows an example in which the number of divisions is automatically determined in accordance with the number of target processing modules BL.

[0259] exist Fig.23 2 shows a case where the user U1 designates the start module as "Processing A" and the end module as "Processing C" as the target of multi-preview, and performs a multi-preview execution operation 2301 for the state of the provisional flow line FL in the module configuration unit 32. In this case, the targets of the multi-preview are three process modules BL, "Processing A", "Processing B", and "Processing C". The development system 1 displays three divided areas corresponding to the three target process modules BL in a horizontal row in the area of ​​the preview display unit 33 in accordance with the configuration direction of the flow line FL (from left to right in this example), and displays the processing result images of the three modules side by side in these divided areas. In this example, a preview image 2311 of Processing A, a preview image 2312 of Processing B, and a preview image 2313 of Processing C are displayed in sequence.

[0260] [Function 3: Multi-preview function 2]

[0261] exist Fig.24 In the figure, the multi-preview function 2 is shown as the function 3. The multi-preview function 2 is a function for displaying the processing results of the case where multiple internal settings of the processing module BL are switched as multiple previews. The development system 1 can also control the state of enabling the function 3 to a predetermined mode (set to mode 3). The concept related to the internal settings of the processing module BL is as described above. Fig.17 As shown in the following table.

[0262] The following processing actions are performed by the multi-preview function 2. The multi-preview function 2 includes the following function 3a and the like. As a premise, for example, Fig. 22 The multi-preview display 2212 with the number of divisions 4 as shown in (B) is shown. From this state, the following processing operations are performed according to a predetermined operation.

[0263] (Function 3a) The user U1 performs a selection operation 2401 on, for example, the "Processing A" module of the module configuration unit 32 by clicking or the like. In response to this operation, the development system 1 switches the preview display content of the preview display unit 33 so that the processing result image of the "Processing A" module is displayed. The preview display unit 33 has divided areas 2411 to 2414 with a division number of 4. The development system 1 previews and displays the result image of "Processing A" in the first divided area 2411. No preview display is performed in the other divided areas.

[0264] (Function 3b) Next, in addition to the operation of the above-mentioned function 3a, the user U1 continuously operates the same "Process A" module by clicking, etc. In response to this operation, the development system 1 regards the internal setting of the "Process A" module as being switched, and displays the processing result image under the internal setting as a multi-preview on the preview display unit 33.

[0265] exist Fig.24 In the lower part of , there is shown a multiple preview display example of the case where the "Processing A" module switches the internal settings, such as internal settings 1, 2, 3, and n (n=4). By clicking the "Processing A" module for the first time, the result image of Processing A in the state of "Internal setting 1" is displayed in the segmented area 2411. By clicking the second time, the result image of Processing A in the state of "Internal setting 2" is displayed in the second segmented area 2412. By clicking the third time, the result image of Processing A in the state of "Internal setting 3" is displayed in the third segmented area 2413. By clicking the fourth time, the result image of Processing A in the state of "Internal setting n=4" is displayed in the fourth segmented area 2414. These four images are displayed side by side.

[0266] (Function 3c) After operating the above functions 3a and 3b, the user U1 operates each internal setting segment area in the preview display unit 33 by, for example, clicking. In response to this operation, the development system 1 selects the internal setting to be applied to the "processing A" module. For example, as shown in the figure, when the segment area 2412 is operated by clicking, the development system 1 applies "internal setting 2" to the "processing A" module in the flow line FL of the module configuration unit 32. In addition, the operation at this time may also be, for example, an internal setting application instruction input as shown in the figure.

[0267] According to the above function 3, the user U1 can compare and confirm the processing results under the internal settings of the processing module BL side by side. For example, the user U1 can confirm the processing result images under multiple detailed parameter setting values ​​of the image processing at once. Therefore, the efficiency of debugging based on the preview function can be further improved.

[0268] [Preview mode]

[0269] exist Fig.25In the figure, an explanatory diagram about the mode (preview mode) of the preview display function F1 is shown as a supplement. Regarding the preview display function F1 in the GUI screen 30, as in the above example, there are multiple functions in detail. For example, there is mode 1 of the above function 1 (normal preview function), mode 2 that can select the number of divisions of the multi-preview function 1, mode 3 that can select the internal settings of the multi-preview function 2, etc. Therefore, the development system 1 can also prepare a mode corresponding to each function, and switch the control and display modes according to the function that the user U1 wants to use and the GUI operation of the user U1. The development system 1 sets a GUI in the GUI screen 30 that can switch these modes according to the operation of the user U1. The user U1 can select the mode of the desired function through the GUI.

[0270] exist Fig.25 In the example of the GUI screen 30, there is a preview mode bar 2501. The preview mode bar 2501 displays a plurality of modes as options through a GUI such as a list box, and the mode can be selected by the operation of the user U1. The development system 1 applies the mode selected by the operation of the user U1. For example, when "Mode 2" is selected, the development system 1 switches to the state of Mode 2 in which the multi-preview function 1 is effective. That is, the GUI screen 30 including the preview display unit 33 is switched to a state of accepting GUI operations in Mode 2. Even the same operation (such as clicking on the processing module BL) can be interpreted as different instructions corresponding to the mode (such as multiple previews of the entire flow line FL, and multiple previews of the internal settings of the processing module BL alone).

[0271] [Function 4: Warning function]

[0272] As function 4, the warning function (in other words, the determination result support function) is described. The warning function is a function of displaying information indicating OK, NG, etc. related to the content and result of the image inspection on the preview display unit 33, and a function of highlighting the determination result, etc. In particular, the warning function is a function of displaying the image frame in a different form according to the content and result of the image displayed on the preview display unit 33.

[0273] Fig.26 The following diagram shows an explanation of the warning function. Fig.26 In the flowchart FL of the module configuration unit 32, the "camera input" module b1 to the "determination" module b2 are configured. The "determination" module b2 is a module that uses the processing result image output from the previous processing module BL, such as the "processing B" module, as an input image, performs a predetermined determination process, and outputs a determination result image. The determination process is, for example, determination on the presence, quantity, and appearance quality of the object 20 reflected in the input image. The output of the "determination" module b2 includes a determination result image and determination result information.

[0274] The user U1 selects, for example, the "determination" module b2 of the module configuration unit 32 and performs a preview execution operation. When the preview is executed in a state where the warning function is enabled, the following processing operations are performed. Function 4 includes the following function 4a and the like.

[0275] (Function 4a) The development system 1 displays the preview image (i.e., the judgment result image) displayed on the preview display unit 33 in a different color based on the judgment result image and judgment result information (e.g., "OK" / "NG" / other information) as the output of the "judgment" module b2.

[0276] exist Fig.26 In the example of , three examples are summarized and illustrated as display examples in the preview display unit 33. (A) is the first example, in which a judgment result image 2601 is preview-displayed in the preview display unit 33. In the judgment result image 2601, a text image indicating the judgment result, such as "OK", is superimposed on the processing result image of "Processing B". "OK" means, for example, that the presence, quantity, and appearance quality of the object 20 meet a predetermined standard. In this function, the color of the outer frame line 2611 of the judgment result image 2601 is determined based on the text image of the judgment result. In the first example, the color of the outer frame line 2611 is displayed as a color indicating "OK", such as blue, corresponding to the text image of the judgment result "OK".

[0277] (B) is a second example, in which a judgment result image 2602 is preview-displayed in the preview display unit 33. In the judgment result image 2602, a text image indicating the judgment result, such as "NG", is superimposed on the result image of "Processing B". "NG" means, for example, that the presence, quantity, or appearance quality of the object 20 does not meet a predetermined standard. In this function, the color of the outer frame line 2612 is displayed in a color indicating "NG", such as yellow, corresponding to the text image "NG" of the judgment result.

[0278] (C) is a third example, in which a judgment result image 2603 is previewed and displayed in the preview display unit 33. In the judgment result image 2603, a text image indicating the judgment result, such as "Warning", is superimposed on the result image of "Processing B". "Warning" indicates, for example, a warning issued because the presence, quantity, and appearance quality of the object 20 meet the predetermined conditions / results. In this function, the color of the outer frame line 2613 is displayed in a color indicating "Warning", such as red, corresponding to the text image "Warning" of the judgment result.

[0279] According to this function, the user U1 can visually grasp the determination result related to the image inspection in the preview display, thereby improving the efficiency of debugging and the like in system development.

[0280] [Function 5: Color information extraction function]

[0281] exist Fig. 27 , an explanatory diagram of a color information extraction function is shown as function 5. This function 5 is a function in which the user U1 selects a designated pixel position on the preview image of the preview display unit 33 by operating the mouse cursor or the like, thereby extracting and displaying the color information of the selected pixel. In this function 5, the following processing actions are performed when the preview operation is executed. Function 5 includes the following function 5a and the like.

[0282] (Function 5a) The development system 1 extracts the pixel value of the portion where the cursor 39 is hovered by the mouse on the preview image of the preview display unit 33, and displays the pixel value as color information in the preview display unit 33.

[0283] exist Fig. 27 In the figure, two examples are shown in summary in the case of preview display of the determination module b2. In the first example (A), when the pixel position 2721 indicated by the arrow cursor 39 is on the determination result image 2701 of the preview display unit 33, the color information 2711 of the pixel value (R, G, B) at the pixel position 21 is extracted and displayed. In the second example (B), when the pixel position 22 indicated by the arrow cursor 39 is on the determination result image 2702, the color information 2712 of the pixel value at the pixel position 22 is extracted and displayed.

[0284] According to function 5, the user U1 can use the color information displayed together with the preview image as reference information for making the flow line FL. For example, there is a "binarization" module that uses the pixel value as a threshold to process the image, and a "color judgment" module that judges whether the pixel value is good or not based on whether it is within a certain range. In this case, the preview color information can be used as a reference value related to the setting of the detailed parameter values ​​set inside these processing modules BL. Fig.25 In the example, the user U1 can refer to the previewed color information and change the "processing A" module, "processing B" module, and "determination" module b2 to other processing modules BL, or change the internal settings of the processing module BL to change the flow line FL.

[0285] [Function 6: Comparison method]

[0286] exist Fig.28, an explanatory diagram of a processing module selection function in a comparison mode is shown as function 6. In this function 6, the processing result images of each processing module BL are compared in a comparison mode in the preview display, and the user can select a processing module to be adopted. In this function 6, the following processing actions are performed when executing the preview. Function 6 includes the following function 6a and the like.

[0287] (Function 6a) The development system 1 infers the candidates of the processing module BL to be configured next based on the state of the tentative flow line FL of the module configuration unit 32, and previews and displays these candidate modules in a comparison manner in the preview display unit 33, so that the user U1 can select the candidate modules to be configured / adopted in the flow line FL in sequence. The development system 1 automatically determines / extracts the candidate modules (e.g., the first candidate and the second candidate) to be displayed in the preview display unit 33. Alternatively, in a modified example, the candidate module may be specified by the user U1.

[0288] The user U1 observes and compares the processing result images of the candidate modules (e.g., the first candidate and the second candidate) displayed on the preview display unit 33, determines which module is suitable for use in the flow line FL, and performs a predetermined operation such as selecting a processing module BL that is considered to be the winner and retained based on the judgment. The development system 1 retains the candidate module of one party selected by this operation as the winning candidate module, and in the case of the next candidate module, similarly sets it as the candidate module of the opponent for comparison, and similarly performs preview display. The development system 1 repeats such comparison according to the number of candidate modules, and configures the one processing module BL finally selected by the user U1 to a specified position (e.g., the end) of the flow line FL of the module configuration unit 32.

[0289] In addition, in this function 6, the comparison objects in the comparison may be set as comparison between different types of processing modules, such as comparison between "processing A" module and "processing B" module. Alternatively, in this function 6, it may be set as comparison between different internal settings (for example, internal settings that change detailed parameter values) of the same processing module, such as comparison between "processing A (internal setting 1)" and "processing A (internal setting 2)".

[0290] The user U1 can select the winning processing module BL by performing a predetermined operation such as double-clicking on a preview display area corresponding to one of the candidate modules displayed in preview.

[0291] exist Fig.28 In the first example, a comparison is shown between different internal settings in the same processing module. Fig.29 In the second example, a comparison between different types of processing modules is shown. Fig.28 The situation of automatically extracting candidate modules is shown in FIG. Fig.29 FIG. 4 shows a case where the user U1 selects an operation candidate module.

[0292] exist Fig.28 In the module selection unit 31, various processing modules BL such as a "processing A" module and a "processing B" module are prepared. For example, the "processing A" module is binarization, and the "processing B" module is image processing different from binarization. In the "processing A" module, for example, "internal setting A1", "internal setting A2", "internal setting A3" and the like are provided as internal settings. In the module configuration unit 32, as a provisional flow line FL, the "camera input" module b1 is configured first. The user U1 performs a preview execution operation in a comparison mode (particularly an internal setting comparison mode). In response to this operation, the development system 1 automatically determines / extracts a module that can be configured after the "camera input" module b1, and extracts a predetermined number of processing modules BL (set to the comparison number) or internal settings used for comparison display in the comparison mode as candidates from the extracted processing modules BL. The comparison number in the comparison mode can be preset to a fixed value (for example, 2), or can be automatically changed within a maximum number. The user U1 can also be allowed to specify / set the comparison number.

[0293] In this example, it is assumed that the development system 1 extracts the "processing A" module as a candidate module to be configured next. In addition, multiple internal settings of the "processing A" module are set as candidates for comparison. When comparing the internal settings, a preview execution operation in a comparison mode may be performed for the processing module BL selected by the user U1 using the module selection unit 31. Alternatively, a preview execution operation in a comparison mode may be performed for the last processing module BL configured in the flow line FL of the module configuration unit 32.

[0294] exist Fig.28In the first example, the comparison number in the comparison mode is set to 2. The development system 1 sets comparison areas 2801 and 2802 corresponding to the comparison number 2 in the area 2800 (in this example, a square) for preview display in the comparison mode in the preview display unit 33. The development system 1 extracts the internal settings that become candidates for comparison in the "processing A" module. Initially, for example, "internal setting A1" and "internal setting A2" become candidates. The development system 1 obtains the processing result image of "processing A" under each candidate internal setting for the camera input image. The development system 1 displays the processing result image under each internal setting as a preview image in each comparison area. In this example, the development system 1 displays the processing result image 2811 (also referred to as "processing A1") under "internal setting A1" in the comparison area 2801, and displays the processing result image 2812 (also referred to as "processing A2") under "internal setting A2" in the comparison area 2802. In this case, the preview display unit 33 may display the camera-captured image as the processing target image, or may display explanatory information of each internal setting and the like.

[0295] The user U1 observes and compares the two processing result images 2811 and 2812 displayed in the preview, determines which one is suitable as the internal setting of the processing module to be configured next, and performs a predetermined selection operation to select a module with a certain internal setting as a winning module to be retained. For example, when the user U1 selects "internal setting A1", he double-clicks the comparison area 2801 of the processing result image 2811. According to this operation, the development system 1 regards the "processing A" module of "internal setting A1" as a winning candidate and retains it, and regards the "internal setting A2" of the unselected party as a losing candidate and removes it from the candidate. Thus, first, one candidate internal setting is selected from the two candidate internal settings. When there is no other candidate, it is decided to adopt the internal setting of the one candidate. The development system 1 arranges the "processing A" module of "internal setting A1" determined above at a specified position of the flow line FL of the module arrangement unit 32.

[0296] Furthermore, the present invention is not limited to this, and based on the comparison result, the user U1 may select the "Processing A" module of the "Internal Setting A1" from the module selection unit 31 and configure it in the module configuration unit 32. In the first example above, the development system 1 automatically selects the candidate internal setting for comparison, but the present invention is not limited to this, and the candidate internal setting for comparison may be selected and operated by the user U1.

[0297] exist Fig.29In the second example, various processing modules BL such as "Processing A", "Processing B" and "Processing C" are prepared in the module selection unit 31. For example, "Processing A" is binarization, and "Processing B" and "Processing C" are image processing different from binarization. "Processing B" is, for example, rotation processing. In the module configuration unit 32, as a temporary flow line FL, the state in which the "Camera Intake" module b1 is configured first. The user U1 performs a preview execution operation in a comparison mode (particularly, an inter-module comparison mode). In addition, as comparison candidate modules in the processing modules BL that can be configured after the "Camera Intake" module b1, the user U1 specifies two modules, such as the "Processing A" module and the "Processing B" module. In response to this operation, the development system 1 extracts candidate modules under a predetermined comparison number (for example, 2) for comparison display in the comparison mode. In addition, the operation in the case where the user U1 specifies multiple processing modules BL is not limited to clicking on each module, and may also be an operation of enclosing them with a frame using the cursor 39.

[0298] exist Fig.29 In the example of , the development system 1 sets comparison areas 2901 and 2902 corresponding to the comparison number 2 in the area 2900 (a square in this example) in the preview display unit 33 for preview display in comparison. The development system 1 obtains a processing result image based on the "processing A" module and a processing result image based on the "processing B" module for the camera input image as the processing target image. The development system 1 displays the processing result image 2911 based on the "processing A" module in the comparison area 2901, and displays the processing result image 2912 based on the "processing B" module in the comparison area 2902. In addition, the preview display unit 33 may display the processing target image 2920 as the camera input image and the explanation information of each processing module.

[0299] The user U1 observes and compares the two processing result images 2911 and 2912 displayed in preview, determines which one is suitable as a module to be configured on the flow line FL, and performs a predetermined operation to select an arbitrary processing module BL to be considered as the winner and retained. For example, when the user selects the "Processing A" module, he double-clicks the comparison area 2901 of the processing result image 2911. According to this operation, the development system 1 regards the "Processing A" module as the winner and retains it, and removes the "Processing B" module that is not selected from the candidates. In the case where there are no other candidate modules, the "Processing A" module is determined as the module to be adopted. The development system 1 configures the selected "Processing A" module at a designated position of the flow line FL of the module configuration unit 32.

[0300] In the second example above, the user U1 selects candidate modules for comparison, but the present invention is not limited thereto, and the development system 1 may automatically determine / extract candidate modules for comparison. For example, the development system 1 may extract all modules of the type that can be arranged after the "camera input" module b1 based on the structure information of all types of processing modules BL prepared in advance. Furthermore, the development system 1 may generate a combination of comparison candidates corresponding to the number of comparisons based on the extracted multiple modules.

[0301] In the first and second examples described above, the case where two candidates are compared first is shown. However, if there are other candidates, the winning candidate can be compared with the other candidates in the same manner. Fig.30 An example is shown as Fig.28 The subsequent situation is equivalent to the situation where there are 3 candidate internal settings. Fig.28 After the "Processing A" module of "Internal Setting A1" wins and is selected, Fig.30 2 shows a comparison between the module of the "internal setting A1" and the module of the "internal setting A3" (also shown as "processing A3" here) which is a third candidate as other candidates.

[0302] The development system 1 displays the preview area 2800 of the preview display unit 33 and the comparison area 2801. Fig.28 The display of the processing result image 2811 after "Processing A1" in the comparison area 2802 is retained, and the display of the processing result image 2811 after "Processing A1" in the comparison area 2802 is removed. Fig.28 The processing result image 2812 after "processing A2" is displayed, and the processing result image 2813 after "processing A3" is displayed.

[0303] The user U1 observes and compares the two processing result images 2811 and 2813 displayed in preview, determines which one is suitable as the internal setting module adopted in the flow line FL, and selects one of them. For example, when the user U1 selects the "Processing A" module of "Internal Setting A3", he double-clicks the comparison area 2802 of the processing result image 2813. According to this operation, the development system 1 regards the "Processing A" module of "Internal Setting A3" as the winner, and determines it as the module to be adopted when there is no other candidate. The development system 1 arranges the selected "Processing A" module of "Internal Setting A3" at the designated position of the flow line FL of the module arrangement unit 32.

[0304] According to the preview function of the comparison method, the selection of the processing module BL of the flow line FL, the selection of the internal setting, the detailed parameter adjustment, etc. can be assisted to the user U1, and even a user with little experience can develop the system. According to the comparison method, the user U1 can compare two images and select the module of the appropriate one in turn, so it is easy to compare and explore.

[0305] Fig.31 A modification example of the above-mentioned function 6 is shown. Fig.31 In the example of , the three internal settings of "internal setting A1", "internal setting A2" and "internal setting A3" in the "processing A" module are compared simultaneously. This method is not a comparison method (a method in which the winners are compared in sequence), but it is the same in that the processing module BL can be selected based on the comparison in the preview.

[0306] exist Fig.31 In the example, the "camera input" module b1 and the "processing A" module are configured as the provisional flow line FL of the module configuration unit 32. For example, when the user U1 wishes to discuss and adopt the internal setting of the "processing A" module in a comparative manner under preview, the user U1 performs a predetermined preview execution operation. In response to this operation, the development system 1 extracts a plurality of candidate internal settings (in this example, "internal setting A1", "internal setting A2", and "internal setting A3") from the candidate module (in this example, the "processing A" module), and displays preview images of the plurality of candidate internal settings side by side in the area 3100 of the preview display unit 33. In this example, the processing result images 3111, 3112, and 3113 of the "processing A" module under the respective internal settings are displayed side by side in the area 3100, for example, in a horizontal row.

[0307] The user U1 observes and compares these preview images, determines one or more internal settings left as candidates, and performs a predetermined selection operation. This operation may also be a double-click of the preview image, etc., as described above. In this example, a selection button 3130 (e.g., a button capable of inputting a check mark) is further provided for the preview image of each internal setting in the area 3100 of the preview display unit 33. The selection button 3130 is a button for selecting / designating a candidate module or a module to be adopted. For example, when the user U1 determines that the "internal setting A3" shown in the image 3113 is the most suitable, he presses the selection button 3130 corresponding to the image 3113. The development system 1 adopts the module of the internal setting corresponding to the operated selection button 3130. In response to this operation, the development system 1 adopts the "Processing A" module under the selected "Internal Setting A3" in the flow line FL.

[0308] The selection button 3130 in the preview display unit 33 may be set to a mode in which only one can be selected, but in other modes, it may be set to allow multiple selections. That is, the user U1 may leave more than two candidates as the preview result. For example, if the user U1 has not narrowed down the internal settings to be adopted to one, it is also possible to use the selection button 3130 to select multiple options and keep them as candidates, and then decide to select one after thinking about it. The above example is a comparison between internal settings, but it can also be applied to the comparison between different types of modules.

[0309] [Variation: Multiple flow lines]

[0310] exist Fig.32 As a modification example of the above-mentioned function 6, a function is shown in which a plurality of flow lines FL can be created in the module configuration unit 32 according to the selection of the processing module BL by the user U1. Fig.28 When the processing module BL to be arranged on the flow line FL is selected by the function 6 of the comparison method such as the above, if it is impossible to determine one processing module BL (including the difference in internal settings), it is also considered to create two or more flow line structures corresponding to the two or more processing modules BL selected by the user U1. The development system 1 creates a virtual flow line FL according to the type of the selected processing module BL or the difference in internal settings, for example, and displays it on the module arrangement unit 32.

[0311] exist Fig.32 For example, it is assumed that there are "internal setting 1" and "internal setting 2" in the "processing A" module. As a provisional flow line FL of the module configuration unit 32, it is assumed that the user U1 considers / selects which module of "internal setting A1" and "internal setting A2" should be configured as the "processing A" module after the "camera capture" module b1. For this purpose, the development system 1 compares and displays the preview image 3211 of "internal setting A1" and the preview image 3212 of "internal setting A2" in the preview display unit 33. When the user U1 cannot decide on one of the two internal settings, for example, the user U1 selects the two to be turned on (ON) (checked) using the selection button 3230 and leaves them as candidates. The development system creates and displays the first candidate flow line 3241 (FL1) when "internal setting A1" is selected and the second candidate flow line 3242 (FL2) when "internal setting 2" is selected in the module configuration unit 32.

[0312] Furthermore, for example, the development system 1 may divide the area of ​​the module arrangement section 32 by flow lines FL (for example, FL1 and FL2), for example, with dotted lines, and display each candidate flow line FL in each divided area.

[0313] The user U1 can also try out each candidate flow line FL by connecting other processing modules BL thereafter. As described above, the status of each flow line structure can be confirmed through the preview function. The user U1 can also delete the candidate flow lines based on the trial. The user U1 can decide one flow line FL to be adopted based on the trial. For example, the user U1 can also decide one flow line FL to be adopted, or one or more flow lines FL to be retained and saved as candidates, by operating the selection button 3250 provided for each flow line FL using the module configuration unit 32.

[0314] In addition, the above example is not limited to the above example, and a plurality of candidate flow lines FL may be created in the module configuration unit 32 from the beginning. For example, the user U1 virtually creates the first flow line in the module configuration unit 32, and then performs a predetermined operation (e.g., flow line addition instruction input) for adding a second flow line. The development system 1 follows this operation, for example, with Fig.32 Similarly, the area of ​​the module configuration unit 32 is divided, an area for creating / editing the second flow line is created, and a blank module BS is configured. The user U1 can create a second virtual flow line by performing the same operation on these two areas. The user U1 can perform an operation to save the selected flow line as data (for example, input a save command).

[0315] [When there are branches in the flow line]

[0316] Fig.33 Other configuration examples of the flow line FL in the module arrangement unit 32 and other GUI examples are shown. Fig.33 The flow line FL in FIG. 1 shows an example of a structure having branches, in other words, a parallel processing module. The flow line FL is not limited to the simple series structure described above, and can also be a parallel processing structure based on branches as needed. Fig.33 2 shows a case where the direction of the flow line FL on display is set to the vertical direction in the screen. The user U1 can create a branch by a predetermined operation (for example, input of a branch instruction) in the module arrangement unit 32 .

[0317] In addition, it is shown Fig.33The GUI of the flow line FL in the example has a case where there is a node ND between the processing modules BL. The node ND represents the input and output data (image, etc.) for the processing module BL. The user U1 can also use the module configuration unit 32 to perform operations on the node ND. For example, the desired processing module (such as the "processing X" module) is configured by dragging and dropping from the module selection unit 31 to the vicinity of the desired node ND in the module configuration unit 32, so that the processing module BL can be added, etc. Alternatively, it is also possible to configure the desired image file for the desired node ND, thereby specifying it as an input image to the processing module BL behind the node ND. Alternatively, when the user U1 specifies the object / range of the preview display, it can also be set as an operation for the node ND.

[0318] In addition to the branching structure described above, the flow line FL can also have a loop structure, etc., similar to a general information processing flowchart, etc.

[0319] [Function 7: Line sensor setting support]

[0320] exist Fig.34 , an explanatory diagram of a line sensor setting support function (in other words, a camera / sensor device setting support function) is shown as function 7. Function 7 is an explanation of Figure 1 The above-mentioned camera / sensor device 21 ( Figure 2 A function that supports the settings of the digital signal output device 105).

[0321] To start the image inspection process in the image inspection system 2, as shown in FIG. Figure 1 In that way, there is usually a camera input process 201 of the input image 50 at the beginning, and correspondingly, in the GUI screen 30 of the development system 1, a "camera input" module b1 is set at the beginning of the flow line FL. The user U1 needs to perform operations such as selecting internal settings related to the "camera input" module b1 and setting detailed parameter values ​​of the internal settings. In order to create / determine a suitable flow line FL for the image inspection system 2, it is necessary to determine suitable parameter setting values ​​in the "camera input" module b1. Function 7 is a function that uses a preview function, etc. to assist in procedures such as setting detailed parameters of the internal settings of the "camera input" module b1. This function 7 includes the following function 7a, etc. In addition, in Fig.34 In the lower part of FIG. 1 , two examples of preview displays based on each function are summarized.

[0322] (Function 7a) The development system 1 has a function of arranging the standard image and the camera captured image in the preview display unit 33 for preview display. Fig.34(A) shows the function 7a. The development system 1 displays in parallel in the area 3400 of the preview display unit 33 a standard image 3401 set in advance related to the "camera capture" module b1 and a camera capture image 3402 which is an output image of a case where a camera capture process is performed according to the internal setting of the "camera capture" module b1 on the flow line FL. The camera capture image 3402 is an image of the result of the camera capture process based on the input of the image 50 captured in real time from the camera / sensor device 21. The user U1 can observe and compare these images on the preview display unit 33 and confirm whether the camera capture image 3402 obtained by the internal setting of the "camera capture" module b1 of the flow line FL being created is suitable for the standard image 3401. The user U1 can decide to use the parameter value set internally or change it to another parameter value set internally according to the confirmation result.

[0323] (Function 7b) The development system 1 has the following function: when comparing the standard image 3401 and the camera-captured image 3402 under the above-mentioned function 7a, a warning is displayed when the deviation between the camera-captured image 3402 and the standard image 3401 is large. (B) This function 7b is shown. The development system 1 evaluates the degree of deviation of the content of the camera-captured image 3402 in the area 3400 of the preview display unit 33 with respect to the standard image 3401 based on image processing. When the degree of deviation is, for example, greater than a threshold value, the development system 1 displays a warning indicating that the deviation is large in the preview display unit 33. In this example, when a warning is displayed, the above-mentioned Fig.26 Similarly, the development system 1 emphasizes and displays at least the rectangular outer frame 3410 surrounding the camera image 3402 in the area 3400 of the preview display unit 33 using a predetermined color or the like. In addition, in the development system 1, not limited to the color of the outer frame 3410, a text image (e.g., "different") indicating a warning may be displayed. By observing the warning, the user U1 can easily recognize that the deviation from the standard image 3401 is large, and can adjust the parameter value set inside the "camera capture" module b1.

[0324] (Function 7c) The development system 1 has the following function: in the setting of the above-mentioned "camera input" module b1, by clicking the module multiple times, it is possible to change the detailed parameter setting value (for example, the camera cycle) of the internal setting of the module while performing a preview display. When the user U1 operates the "camera input" module b1 of the module configuration unit 32 or the camera input image 3402 of the preview display unit 33 by clicking, etc., the development system 1 cyclically changes the internal setting of the "camera input" module b1 (with the same settings as the preview display unit 33). Fig.17For example, for each click operation, the internal settings are switched to "internal setting 1", "internal setting 2", "internal setting 3", "internal setting 1", etc. In this example, the imaging cycle is made different for each of these internal settings as parameter setting values. In addition, the development system 1 displays a preview image corresponding to the state of the changed internal settings on the preview display unit 33. For example, the development system 1 displays a preview image corresponding to the state of the changed internal settings on the preview display unit 33. Fig.34 When the preview display state as shown in (A) is clicked once, the "internal setting 1" of the "camera input" module b1 is changed to the "internal setting 2", and the processing result image under the "internal setting 2" is displayed as the camera input image 3402. Thus, the user U1 can easily compare and confirm with the standard image 3401 while switching the internal settings.

[0325] According to the above-mentioned function 7, the process line FL is provided with a "camera input" module b1 that can change the internal settings, and a preview function that is realized by comparing with the standard image related to the "camera input" module b1. Therefore, the setting of the camera / sensor device 21 such as the line sensor, which was previously complicated and labor-intensive, can be simplified. Along with this, according to function 7, the internal settings of the "camera input" module b1 can be easily changed and the image can be confirmed by comparing with the standard image through simple operations such as clicking. As a result, the difficulty of setting up / setting the camera / sensor device 21 such as the line sensor of the image inspection system 2 can be greatly reduced, which can contribute to reducing the burden on users.

[0326] [Influence of the internal settings of the camera input module]

[0327] For example, in Figure 1 When a line sensor is used as the camera / sensor device 21 of the image inspection system 2, the aspect ratio of the line sensor image may affect the accuracy of inspection / determination. As a parameter affecting the aspect ratio, there is, for example, an imaging cycle (in other words, a trigger interval). Depending on the value of the imaging cycle, the image of the object 20 may be stretched in the image captured by the camera, resulting in an inappropriate aspect ratio and reduced accuracy of determination.

[0328] So, for example, in Fig.34In the development system 1, the imaging cycle is selected as an internal setting in the above-mentioned "camera capture" module b1, and the camera captured image 3402 in the case of changing to the selected imaging cycle is displayed in the area of ​​the camera captured image 3402 of the preview display unit 33. As a result, the user U1 can easily set a suitable imaging cycle of the "camera capture" module b1 so that the aspect ratio of the camera captured image 3402 becomes suitable, for example, by comparing with the standard image 3401 of the aspect ratio standard.

[0329] As a modified example of the above-mentioned function 7, the following may be used. The above-mentioned multi-preview function may be used in combination with the function 7. The development system 1 displays a plurality of camera-imported images for each difference set in the "camera-imported" module b1 in parallel as multi-previews in the preview display unit 33.

[0330] [GUI for internal settings of camera import module]

[0331] In the above-mentioned "camera input" module b1, as an internal setting, a camera / sensor device 21 ( Figure 2 The development system 1 can also select, set, and save detailed parameter values ​​set in the "camera input" module b1 according to the operation of the user U1 on the GUI screen 30.

[0332] exist Fig.35 , an example of user settings for the internal settings of the "camera input" module b1 in the GUI screen 30 is shown. In the module selection unit 31, the user U1 selects the "camera input" module b1 and selects a new creation operation related to the internal settings. In response to this operation, the development system 1 displays an internal settings column 3500 as a GUI example for the internal settings of the "camera input" module b1. The user U1 can confirm and set the parameter values ​​of the internal settings of the "camera input" module b1 in the internal settings column 3500, and register them as new internal settings. In the internal settings column 3500, for example, the name of the internal settings, the camera / sensor device to be used, the shooting cycle, and other parameter values ​​can be input. After registering the internal settings, when operating the internal settings of the camera input module b1, the internal settings can be displayed as options and used.

[0333] [Function 8: Template matching setting support function]

[0334] exist Fig.36, an explanatory diagram of the template matching setting support function is shown as function 8. In the past, when the template matching method is applied, the camera image of the camera / sensor device is compared with the comparison image (in other words, the template image, the reference image, etc.), and the inspection / judgment, etc. are performed based on the degree of their consistency. In order to register such a template image, it is necessary to specify the coordinate information of the image data. Function 8 is a function to assist / support the setting of the template image, etc. in such a template matching method. This function 8 includes the following function 8a, etc.

[0335] (Function 8a) The development system 1 has a function of allowing the user U1 to operate a mouse or the like on the preview display unit 33 to designate a template area during setting of a template image.

[0336] (Function 8b) The development system 1 has a function of previewing and displaying template candidates in a plurality of sizes in response to an operation such as clicking a template by the user U1.

[0337] (Function 8c) The development system 1 has a function of determining a template image in accordance with an operation such as double-clicking a preview image by the user U1 in addition to the operation of the above-mentioned function 8b.

[0338] exist Fig.36 In the example of (A), the user U1 selects the "camera input" module b1 in the module selection unit 31 and performs the registration procedure for the template image. The development system 1 displays the camera input image 3601 obtained by the "camera input" module b1 based on the actual camera image in the area of ​​the preview display unit 33. In this area, the user U1 performs a selection operation on a desired area in the camera input image 3601 that is suitable as a template. The selection operation at this time includes, for example, the designation of a rectangular range by clicking or dragging the mouse cursor 39. In the example of (A), the user U1 designates a rectangular range 3602 that includes / circumscribes a circle corresponding to an object 20 such as a product as a template area. The development system 1 temporarily registers the template area selected by the operation as a candidate for the template image (set to Candidate 1).

[0339] Similarly, examples of template regions in which other candidates are specified are shown in (B) and (C). In the example of (B), a rectangular range 3603 enclosing a portion of the text image "ABC" is specified as candidate 2 for the template region. In the example of (C), a rectangular range 3604 enclosing a portion of the text image "B" is specified as candidate 3 for the template region. In addition, when determining / registering a candidate for a template image, a "register as a candidate for a template image" command or the like may be used as a GUI example.

[0340] In addition, as described above, the user U1 can also specify candidates for individual template areas such as (A), (B), and (C) through predetermined GUI operations, but in other methods, the development system 1 can also automatically make the size of the template area variable and generate candidates such as (B) and (C) based on the specification of a template area performed by the user U1's operation such as (A).

[0341] exist Fig.36 , an example of the above-mentioned function 8b is shown in the lower part. The user U1 performs a selection operation on the candidate of the above-mentioned registered template image. For example, in the preview display unit 33, the user U1 selects the template area corresponding to the template image by clicking, etc., and performs a predetermined preview execution operation. In response to this operation, the development system 1 previews and displays the selected one or more candidates of the template image in the area of ​​the preview display unit 33. In this example, when three candidates, namely, candidates 1, 2, and 3, are generated, these three candidates are selected as objects. In this case, the development system 1 displays these three candidate template images side by side as a multi-preview in a predetermined number of divided areas (for example, 4) in the same manner as the above-mentioned multi-preview function. In this example, the image of candidate 1 is displayed in the upper left divided area, the image of candidate 2 is displayed in the upper right divided area, and the image of candidate 3 is displayed in the lower left divided area.

[0342] The user U1 compares these candidate images on the preview screen, selects a candidate suitable as a template image, and performs a predetermined operation to determine it as a template image. For example, when the user U1 selects candidate 2, he operates the segmented area of ​​candidate 2 by double-clicking. Thus, the development system 1 determines the candidate image as the template image of the "camera input" module b1 and saves it as data. In addition, as shown in the figure, as a GUI example, a "register as template image" command or the like can also be used.

[0343] According to the above-mentioned function 8, the user U1 can easily register the template image of the template matching method on the flow line FL of the image inspection system 2 with less work. In addition, in the above-mentioned example, the case where the template image is registered in association with the "camera input" module b1 is described, but the template image used in the determination process can also be registered in association with the "determination" module b2.

[0344] [Process line confirmation screen]

[0345] exist Fig.37 In the example of another GUI screen, an example of a GUI screen 60 is shown in which one or more flow lines FL created and saved by the user U1 can be checked using the above functions. Figure 5 In addition to the GUI screen 30 for creating such a flow line, Fig.37 The GUI screen 60 allows the user to check the created flow line. Fig.37 The GUI screen 60 is a flow line confirmation screen, and has a flow line information column 3701, a preview column 3702, a check result column 3703, and the like.

[0346] The information of one or more created and saved flow lines FL is organized and displayed in the flow line information column 3701. In this example, the information of one flow line FL includes a row of the flow line name and a row of the names of each processing module BL constituting the flow line FL, and the information of the processing modules BL is arranged in the order constituting the flow line FL.

[0347] The preview column 3702 can preview and display the flow line FL selected by the user U1 in the flow line information column 3701 .

[0348] In the inspection result column 3703, the inspection result information of the inspection performed on the object 20 by the image inspection system 2 according to the image inspection process corresponding to the selected flow line FL is displayed. The inspection at this time is not limited to the actual inspection using the completed flow line, but can also be a trial inspection using the flow line under production. In the inspection result column 3703, for example, information such as the inspection date and time, the inspector, the total number of inspections, the number of OKs, and the number of NGs is displayed.

[0349] exist Fig.37 In the example of the flow line FL created / saved, that is, the example of the image inspection process of the image inspection system 2 is as follows. The flow line FL1 is composed of a "camera input" module BL1, a "binarization" module BL2, a "binary image shape conversion" module BL3, an "ellipse drawing" module BL4, and a "determination" module BL5 in sequence. In addition, details of the internal settings of each module can also be displayed and confirmed by predetermined button operations.

[0350] [Effects of Embodiment 1, etc.]

[0351] As described above, according to the development system 1 of the first embodiment, the difficulty of development and the workload of the user in the development of the image inspection system 2 can be reduced. According to the development system of the first embodiment, by using the editing and preview functions of the flow line of the image inspection system 2, the action confirmation of the flow line of the image inspection system 2 can be realized more easily and efficiently than before. According to the development system of the first embodiment, the structure of the flow line can be tried in various ways while changing the processing module and internal settings in the GUI screen. According to the development system of the first embodiment, the action status of the flow line, module, etc. can also be confirmed by previewing the flow line in the process of production, so the development can be made easier and more efficient. According to the development system of the first embodiment, compared with the existing technology that the action confirmation cannot be performed by previewing if the flow line of the system is not completed as a whole, the development can be made more efficient. According to the development system of the first embodiment, by supporting the user in the GUI screen, even users without high technical knowledge can develop / produce the flow line of the image inspection system 2.

[0352] <Implementation Method 2>

[0353] use Fig.39 The development system and method of Embodiment 2 are described below. The basic structure of Embodiment 2 is the same as that of Embodiment 1, and the following mainly describes the structural parts of Embodiment 2 that are different from those of Embodiment 1. The development system of Embodiment 2 has the following functions in addition to the functions described in Embodiment 1.

[0354] 1. Preview shows whether there is a switch function

[0355] 2. Preview display resolution setting function

[0356] 3. Preview display rotation function

[0357] 4. Can the flow line module be configured with display function?

[0358] 5. The display function of the unset module must be set

[0359] 6. Module display function in process line processing

[0360] Regarding these functions, Fig.12 Similarly to the description in , the selection execution and user setting by the user U1 can also be realized. It is also possible to realize a mode in which only a single function is installed or a mode in which a plurality of functions are installed.

[0361] [2-1. Whether the preview display has a switching function]

[0362] Fig.39This is an explanatory diagram related to the preview display switching function, showing an example of a GUI screen 30. The GUI screen 30 has a module selection unit 31, a module configuration unit 32, and a preview display unit 33. In addition, the module selection unit 31 and the preview display unit 33 do not need to be displayed on the GUI screen 30 all the time, and can also be popped up and displayed as needed. As a subject, when performing image inspection, it is not necessarily necessary to preview the processing results of all processing modules of the process line. When previewing the processing results of all processing modules including the intermediate process of image inspection, it may become cumbersome. For example, in the above-mentioned Fig.15 In this way, the processing results of multiple processing modules are previewed and displayed in sequence according to the time series. Fig. 22 When the processing results of the plurality of processing modules are previewed and displayed side by side, it may be troublesome depending on the production status of the flow line or the user U1 may not need to confirm the intermediate process.

[0363] Therefore, in Fig.39 In the preview display switching function, a function is provided for each processing module of the flow line that can set the on / off function related to the preview display of the processing result of the module. Fig.39 In the specific example, each module BL in the flow line bar (module configuration part) 32 of the GUI screen 30 of the development system 1 has a GUI 3901 such as a switch for setting on / off of preview display. The user U1 selects the desired module BL (in Fig.39 In the example of "Processing A" module), the preview display can be switched on / off by clicking the switch. For example, all processing modules BL are turned on by default, and the user U1 only needs to operate the processing module BL that he wants to turn off.

[0364] In response to the above-mentioned preview execution operation, a preview of the processing result of the flow line FL is displayed in the preview column 33. At this time, in the case where it becomes Fig.15 In the case of preview display in such a time series, the preview display is not performed for the processing module BL that is set to be turned off in the GUI 3901. In other words, the development system 1 does not perform the operation processing for previewing the processing result of the processing module BL. In the specific example, as shown in the figure, the camera input result image, the "processing B" result image and the "processing C" result image are previewed, and the "processing A" result image is not displayed (becomes turned off display). In addition, the turned off display can be set to a predetermined display such as a black background image, or it can be set to a predetermined display indicating that the preview display is set to be turned off (for example, a "turn off" text image display).

[0365] This function allows you to preview and display only the processing results of any processing module of a process line. You can preview and display only the part of the processing results that is necessary for the user U1, such as a developer or an inspector. Therefore, compared with the preview display of all the processing modules of the process line, confusion and cumbersomeness can be avoided. In addition, depending on the development / execution environment, the computational load when previewing and displaying the processing result image affects the processing time. By previewing and displaying only the necessary part, the computational load can be reduced, which can contribute to shortening the processing time.

[0366] Fig.40 This shows another example of GUI for this function. In the process line column 32, the user U1 can specify a desired range by clicking or dragging the mouse cursor to specify a processing module BL to be the starting point and a processing module BL to be the end point. For the specified range, the GUI 4001 such as a switch can collectively set the on / off of the preview display of the processing results for the processing modules BL within the range.

[0367] Fig.41 This is an example of another GUI. For example, in the flow line column 32, a GUI 4101 can be provided that can collectively set a switch for turning on / off the preview display for all processing modules BL of the flow line FL being edited. For example, by setting the GUI 4101 to off, the user U1 can temporarily turn the preview display off for all processing modules BL of the flow line FL. After that, the user U1 can Fig.39 In the GUI of the present invention, only the individual desired processing module BL is set to the preview display on setting.

[0368] In addition, as another example of collectively setting all the processing modules BL of the flow line FL, a GUI 4102 may be provided in which only the first processing module BL of the flow line FL is set as a button for turning on the preview display, and a GUI 4103 may be provided in which only the last processing module BL of the flow line FL is set as a button for turning on the preview display, etc. In addition, a GUI may be provided in which the first processing module BL and the last processing module of the flow line FL are set as buttons for turning on the preview display, etc.

[0369] These functions can reduce the operations related to the preview display setting even when the number of processing modules BL constituting the flow line FL increases, thereby reducing the workload and working time of the user U1.

[0370] Fig.42 This is another example of the GUI for this function. Fig.39 For example, it is possible to use the internal setting column ( Fig.19etc.) to confirm / set the on / off of the preview display using the corresponding GUI 3901, but is not limited to this. For example, it can also be set as follows Fig.42 .exist Fig.42 In the flowchart column 32, an item 4201 for displaying the on / off setting state of the preview display is provided for each processing module BL. In addition, by the user U1 clicking the item 4201 of the desired processing module BL, the on / off setting of the preview display of the processing module BL can be switched. The display of the row of the item 4201 for displaying the on / off setting state of the preview display can also be turned on / off.

[0371] With this function, the user U1 can easily check the preview display setting status of each process module BL of the flow line FL at a glance.

[0372] In addition, regarding the item 4201 that displays the on / off setting state of the preview display, only the items that are set to be on or only the items that are set to be off may be displayed on the screen. In other examples, according to the above-mentioned preview display on / off setting state, it is also possible to realize a display in which the color of the processing module BL is changed, a display with an icon / mark, etc.

[0373] Fig.43 Another example of the GUI showing this function. In the process line bar 32, for each processing module BL, for the processing module BL for which the preview display is set to on, an icon 4301 (e.g., an "on" mark) is given as a predetermined display indicating that the preview display is set to on. In addition, the processing module BL for which the preview display is set to off may be displayed in a darkened color.

[0374] Fig.44 Another example of the GUI of this function is shown. In the flow line column 32, there is also a case where the flow line FL is composed of a large number of processing modules BL. In the example (A) shown in the figure, it is assumed that the flow line FL includes a camera input module, a "processing A" module to a "processing G" module. When the preview display is turned on / off for a large number of processing modules BL, the screen display may become complicated due to the increase in workload.

[0375] Therefore, in this function, according to the operation performed on the desired processing module BL of the flow line FL, the processing module is set to the display omitted state as shown in (B). At the same time, in this function, the preview display of the processing module is set to the off setting. In the example shown in the figure, it is assumed that the user U1 selects the range of the "processing B" module to the "processing F" module and performs a predetermined display omission operation. In this case, the development system 1 collapses the range and sets it to the display omitted state, and automatically sets the processing modules BL in the range to the preview display off setting. In (B), the flow line FL is shortened. The range becomes a predetermined display indicating that it is in the display omitted state (for example, "omitted" display, gray display, etc.). In addition, when the user U1 operates (for example, double-clicks, etc.) a part of the predetermined display that is the display omitted state, the development system 1 expands the range as shown in (C) and returns to the original display state without display omission, and automatically sets the range to the preview display on setting.

[0376] [2-2. Preview display resolution setting function]

[0377] Fig.45 This is an explanatory diagram related to the preview display resolution setting function. As a problem, when previewing the processing result image of the processing module of the flow line, the computing load affects the processing time depending on the development / execution environment. When the resolution when previewing the processing result image is high, high-definition confirmation can be achieved, but the computing load required for the preview display processing becomes higher accordingly, and the processing time may be longer.

[0378] Therefore, in the preview display resolution setting function, the user U1 is allowed to set the resolution for previewing the processing result image of the processing module of the process line. In the preview column, the preview display resolution can be selected for each processing module. The development system 1 executes processing so that the processing result image of the processing module of the process line is previewed in the preview column according to the set resolution.

[0379] This allows the preview to be displayed at the resolution desired by the user U1. By adjusting the resolution, the computation load can be adjusted, thereby contributing to shortening the processing time associated with the preview display, improving the accuracy of confirmation / visual recognition of the preview display, and the like.

[0380] exist Fig.45 In the example of , a GUI 4501 for setting the resolution of the preview display image 4500 is provided in the preview display column 33. In the example of the GUI 4501, one of three levels of resolution, namely "high", "medium", and "low" can be selected. Fig.45In the example of , the default setting is set to "high" resolution, for example. The "high" resolution is a predetermined resolution, for example, the highest resolution in the present system. In the case of this setting, according to the preview execution operation, the processing result images of each processing module BL of the preview display object of the flow line FL are displayed in the preview column 33 as the preview display image 4500 with "high" resolution. In the case where the user U1 sets the GUI 4501 to "low" resolution, for example, according to the preview execution operation, the processing result images of the processing module BL of the object are displayed in the preview column 33 with "low" resolution.

[0381] Fig.45 The example is the setting of the preview display resolution for the entire flow line FL (all processing blocks BL), but the present invention is not limited thereto, and the preview display resolution may be set for each processing block BL.

[0382] Fig.46 An example of a GUI for setting the resolution of the preview display of each processing module BL is shown. In the process line column 32, a GUI 4601 for setting the resolution of the preview display is provided for each processing module BL. In this example, for each processing module, the resolution of the preview display can be selected and set for the displayed GUI 4601 by a predetermined operation such as clicking or an internal setting column.

[0383] By utilizing this function, for example, only the image of the final processing result of the process line can be previewed and displayed at a high resolution, and the processing results in the middle can be previewed and displayed at a relatively low resolution.

[0384] In this example, the resolution is set to three levels, but it is obviously not limited to this, and it can also be set to two levels, or it can be set to four or more levels. In addition, this function is expressed as resolution on the GUI, but it can also be set as preview display speed as another expression. For example, low resolution is equivalent to high speed from the perspective of preview display speed, so it can also be displayed as "high speed" as an option on the GUI.

[0385] like Fig.39 The preview display can be turned on or off, and Fig.45 Such functions for setting the preview display resolution may also be combined into one function. Fig.46 The GUI 4601 of FIG. 4601 displays an option for turning on the resolution setting and an option for turning off the resolution setting. Fig.39 etc.) Similarly, various GUIs, operation methods, display methods, etc. can be applied.

[0386] [2-3. Preview display rotation function]

[0387] Fig.47 This is an explanatory diagram related to the preview display rotation function. This function is a function that can realize the operation / setting of rotating or inverting the image / video displayed in the preview area.

[0388] As a subject, in the above-mentioned image inspection system 2 ( Figure 1 In the preview display of the GUI screen 30, the image display content may be displayed in a state that is difficult to recognize when the user U1 observes it, depending on the installation / setting status of the camera / sensor device 21 relative to the object 20 on the production line. For example, depending on the installation direction of the camera 21 relative to the object 20, the object 20 is displayed upside down in the preview display image, making it difficult to recognize the text recorded on the object 20. As a result, it is difficult for the user U1 such as a developer or an inspector to perform operations such as adjusting the position of the object 20 while confirming the preview display, and efficiency is reduced.

[0389] Fig.48 Shows illustrations related to the topic. Fig.48 In (A), the Figure 1 The state of placing the object 20 flowing through the production line 4800 or the workbench 4800 corresponding to the image inspection system 2 is schematically illustrated in a longitudinal section (YZ plane view). A camera 21 is arranged vertically above the object 20. An inspector 4801 is arranged on the lower side (front side) relative to the object 20. In the illustrated spatial coordinate system (X, Y, Z), the X direction is the direction in which the object 20 flows and is the left-right direction when observed from the inspector 4801, the Y direction is the up-down direction (in other words, the front-back direction) when the inspector 4801 observes the object 20, and the Z direction is the vertical direction and the height direction. The black arrow indicates the up-down direction (in other words, the front-back direction) observed from the inspector 4801. The white arrow indicates the up-down direction of the camera 21.

[0390] exist Fig.48 In the example of the setting of the image inspection system, the camera 21 is arranged so that the upper side in the camera coordinate system becomes the lower side for the inspector 4801, due to the wiring of the camera 21 relative to the production line 4800 or the workbench 4800. The direction of the optical axis of the camera 21 is vertically downward.

[0391] The XY top view shown in (B) shows an overview of the surface of the object 20 when the inspector 4801 observes the object 20 in the configuration of (A). In the coordinate system of the object 20, there is up and down as indicated by the gray arrows, and this coincides with the up and down in the coordinate system observed by the inspector 4801. When the inspector 4801 observes the surface of the object 20, for example, the characters "ABC" written on the surface are correctly arranged up and down in the Y direction, so the inspector 4801 can easily recognize the characters "ABC".

[0392] In (C), the following example is shown: when the object 20 is photographed by the camera 21 in the setting state of (A), the process line processing result image, such as the camera input result image, is displayed as a preview display image 4803 in the preview column 33 of the GUI screen 30. In this way, corresponding to the setting state of (A), the preview display image 4803 of the preview column 33 corresponds to the top and bottom of the camera 21, so when the user U1 observes, the image of the object 20 is reflected in a reversed manner, for example, the text "ABC" is difficult to recognize. In this state, it is also difficult to perform operations such as adjusting the position of the object 20. For example, when the inspector 4801 moves the object 20 to the right, the image of the object 20 is moved to the left and reflected in the preview display image 4803 as in (C). Furthermore, for example, when the inspector 4801 moves the object 20 downward (front), the image of the object 20 is moved upward (backward) and displayed in the preview display image 4803 as shown in (C).

[0393] Therefore, if Fig.47 As shown in the GUI screen 30, the user U1 can use the preview display rotation function to rotate and reverse the preview display image. Fig.47 In the example, a GUI 4701 including buttons corresponding to operations for rotating or reversing the preview display image is provided in the preview bar 33. In this example, the GUI 4701 includes a "left-right reversal" button, a "up-down reversal" button, a "rotate right" button, and a "rotate left" button. The user U1 can reverse or rotate the preview display image 4700 in the preview bar 33 by selecting and operating from these buttons. For example, the "rotate right" button is a button for instructing an operation of rotating the preview display image 4700 to the right by a predetermined angle, such as 90 degrees. Without being limited to this, the user U1 may also be allowed to specify the rotation angle.

[0394] exist Fig.47 The specific example is shown in the lower part of FIG. This specific example is the image ( Figure 1 The camera takes in the image 51) for preview display, and Fig.48 The setting example corresponds to . Fig.47 The example of (A) is a state in which the preview is initially displayed by performing the preview operation, and the preview display image 4703 is the same as Fig.48 The preview display image 4803 of (C) is the same. It is difficult for user U1 to observe the content of the preview display image 4703 and perform operations, so the user uses GUI4701 to rotate / reverse the preview. For example, user U1 operates the "rotate right" button twice. As a result, the development system 1 displays the preview display image 4703 in a state where it is rotated to the right by 90 degrees × 2 times = 180 degrees. According to this rotation operation / processing, it becomes as follows Fig.47 The preview display image 4704 is in a state where the top and bottom of the object 20 are consistent with the top and bottom of the GUI screen 30, and the user U1 can easily recognize the text "ABC". Fig.48 Such a setting state also makes it easy to observe the preview display image, so the operation becomes easier.

[0395] The reverse operation can also be used in the same way. Fig.47 In the state of the preview display image 4703 of (A), the user U1 may operate the "rotate up and down" button and the "reverse left and right" button, for example. As a result, the development system 1 displays the preview display image 4703 in a state of being reversed up and down and reversed left and right. According to the reversal operation, the state of the preview display image 4704 is as shown in (B).

[0396] According to this function, the restriction of the installation environment of the image inspection system 2 can be solved on the application software, and the user U1 can intuitively operate while checking the preview display, which can facilitate the work.

[0397] The above example is a premise for setting up / configuring a certain image inspection system 2 ( Fig.48 ) is matched with the method of using the preview display status appropriately changed, but is not limited to this. Even in the case of a method of using the image inspection system 2 while changing / adjusting the settings / settings while appropriately changing / confirming the preview display status, it can also be applied in the same way.

[0398] As an example of changing the settings / settings of the image inspection system 2, the following is described. Fig.48 The position of the inspector 4801 (in other words, the working position), the orientation of the camera 21, etc. are temporary. In the above example, the configuration of the camera 21 is fixed and restricted, but in this example, the configuration of the camera 21 can be changed. Fig.47While confirming the preview display image 4700 on the GUI screen 30, it is considered what kind of settings / settings are suitable for the image inspection system 2.

[0399] Fig.49 Shown from Fig.48 The status is an example of the status after the settings / settings of the image inspection system 2 have been changed. Fig.49 The structure of (A) is Fig.48 The configuration of the camera 21 is different from that of the inspection device 4801. The camera 21 is arranged in an adjusted direction so that the top and bottom of the camera 21 are aligned with the top and bottom of the object 20 and the top and bottom viewed from the inspection personnel 4801. In this case, the initial preview display image becomes a preview display image 4903 as shown in (B). Fig.47 (B) is similarly easy to recognize when viewed from the user U1. As a result of the study, the installation / setting of the image inspection system 2 can also be determined as such a structure.

[0400] In addition, as another example, suppose that Fig.49 The state of (A) changes the position (working position) of the inspector 4801. For example, assume that the inspector 4801 moves to the right side in the X direction relative to the production line 4800 or the workbench 4800 as shown in the XY top view of (C), and observes the object 20 from the position on the right side to perform the work. In this case, the state of the inspector 4801 observing the object 20 as in (C) and the state of the preview display image 4903 as in (B) are different, so it is difficult to confirm the preview display image and to perform the work. In this case, if the state of the preview display image 4903 from (B) is different from that of the inspector 4801 observing the object 20, it is difficult to confirm the preview display image and perform the work. Fig.47 For example, if the GUI 4701 of the embodiment is rotated right, the preview display image 4904 is in the state shown in (D). The preview display image 4904 is consistent with the state of the object 20 observed by the inspector 4801. In other words, the preview display image 4904 can be made into an assumed preview in accordance with the position of the inspector 4801. The user U1 can confirm the assumed preview.

[0401] As described above, this function can also be used to support the hardware installation of the image inspection system 2 and to support the inspection work.

[0402] [2-4. Can the flow line module be configured with a display function?]

[0403] Fig.50This is an explanatory diagram related to the display function of whether the flow line module can be configured. As a subject, regarding the processing modules that constitute the flow line, there are modules with conditions related to whether they can be connected among the modules that are connected. In other words, when configuring the processing modules for the flow line, whether the object processing module can be configured behind or in front of a certain module, whether the object processing module can be inserted between certain modules for configuration, etc., there are situations where configuration is possible and situations where configuration is not possible. These situations of whether configuration is possible are sometimes difficult for the user U1 to know.

[0404] The user U1 has to grasp the internal information about the conditions of whether or not the placement is allowed in advance to develop the flow line. This is a heavy burden on the user U1 and cannot be considered as a design suitable for the user.

[0405] Therefore, in this function, in the operation of the processing module selected by the user U1, the location where the processing module can be arranged and the location where the processing module cannot be arranged are determined in the flow line of the flow line column 32 and displayed as a predetermined display.

[0406] In this function, first, the conditions related to the arrangement / connection of the processing modules as described above can be set as parameters (attributes described later) set inside each processing module, for example. Alternatively, the application software corresponding to this function describes / manages the conditions in a table (described later) or the like. The user U1 can set the conditions in the table on the GUI screen 30.

[0407] In this function, the development system 1 automatically determines whether the processing modules can be arranged / connected to each other based on the above-mentioned conditions, and performs a predetermined display indicating whether the arrangement is possible or not according to the determination result.

[0408] exist Fig.50 In the process line column 32 of the GUI screen 30, the "camera input" module b1, the "processing A" module, the "processing B" module, and the "processing C" module are sequentially connected as the process line FL in the process of production. In this state, the user U1 selects and operates the "processing D" module 5001 as a certain target process module BL from the module selection unit 31. In this function, the development system 1 automatically determines the part of the process line FL where the selected "processing D" module 5001 can be configured and the part where the process module cannot be configured, and performs a predetermined display indicating the part that can be configured and / or the part that cannot be configured. Whether the process module BL can be configured is determined based on the conditions described later.

[0409] Fig.50The example is as follows: in the flow line column 32, at the position where the "processing D" module 5001 can be configured, the highlight area 5002 indicating that it can be configured is displayed as the configurable display 5002. In this example, the position of the connecting line behind the "camera input" module b1, the position of the connecting line behind the "processing A" module, and the space position behind the "processing C" module are configurable positions, and the highlight area 5002 is displayed respectively. In addition, conversely, the position where the highlight area 5002 is not displayed (for example, the position behind the "processing B" module) is equivalent to a non-configurable position.

[0410] By observing the highlighted area 5002, the user U1 can easily know the candidate positions where the selected "Processing D" module 5001 can be configured. The user U1 can configure the "Processing D" module 5001 in the position selected from these candidates. For example, the user U1 moves the "Processing D" module 5001 to the position of the highlighted area 5002 between the "Processing A" module and the "Processing B" module, and performs a configuration execution operation (such as release) at this position. Then, the development system 2 accepts the configuration execution operation and connects the "Processing D" module 5001 between the "Processing A" module and the "Processing B" module. That is, the flow line FL is updated.

[0411] exist Fig.51 As another example of display related to whether configuration is possible, Fig.50 On the contrary, the following example is given: in the non-configurable part, a predetermined display indicating that the selected processing module BL is not configurable is displayed as a non-configurable display 5003. In this example, the non-configurable display 5003 as the predetermined display is an × mark area. By observing the non-configurable display 5003, the user U1 can easily know the part where the selected "processing D" module 5001 is not configurable. The user U1 can configure the "processing D" module 5001 in the selected part other than the non-configurable part. In addition, in other words, the part where the non-configurable display 5003 is not displayed is equivalent to the configurable part.

[0412] In addition, assume that the user U1 moves the "process D" module 5001 to the position of the non-configurable display 5003 between the "process B" module and the "process C" module, and performs a configuration execution operation (e.g., release) at this position. In this case, the development system 2 does not accept the configuration execution operation and does not configure / connect the process module BL. That is, the flow line FL is not updated and maintains the original state.

[0413] The predetermined display of whether or not the configuration is possible is not limited to a display of a highlight or a mark, and may be a display of various types such as changing the color of a connection line or an area. Both the display of the configuration being possible and the display of the non-configuration being impossible may be performed simultaneously.

[0414] in addition, Fig.52 As a modified example, when the user U1 wants to move the selected processing module BL to a desired location in the flow line FL by dragging and dropping, the development system 1 can also display whether the configuration is allowed only for the specific location. Fig.50 The determination of whether the arrangement is possible is performed in the same manner, but the display of whether the arrangement is possible is temporarily not performed until the position is moved to the desired position.

[0415] exist Fig.52 In the example, it is assumed that the user U1 selects the "processing D" module as the target processing module. After the "processing D" module is selected, the development system 1 performs the above Fig.50 The judgment on whether the configuration is possible is not made, but the display of whether the configuration is possible in the flow line FL is not performed temporarily. Next, it is assumed that the "Processing D" module is moved to the desired position in the flow line FL, such as the position between the "Processing B" module and the "Processing C" module, by dragging. At the time point after the movement, the development system 1 displays whether the "Processing D" module can be configured at that position. The development system 1 displays the predetermined display indicating that the configuration is possible in an overlapping manner when the configuration is possible, and displays the predetermined display indicating that the configuration is not possible in an overlapping manner when the configuration is not possible. Fig.52 In the example, the part is not configurable, so a non-configurable display 5203 (e.g., an × mark) is superimposed on the "Process D" module of the part. In this way, in the case of non-configurable, the development system 1 does not accept the module configuration operation (e.g., release operation) at the part. The flow line FL is not updated.

[0416] In another example, as shown in the lower part, when the "Process D" module is moved to a position between the "Process A" module and the "Process B" module, the development system 1 overlaps and displays a configurable display 5202 (e.g., a circle mark). In the case of configurability, the development system 1 accepts the module configuration operation (e.g., release operation) at the position and connects the process module. The flow line FL is updated.

[0417] exist Fig.53 In the example of the conditions related to the above-mentioned configuration permission / connection permission, an example of a table (referred to as a condition table) in which the conditions are set is shown. The development system 1 sets / manages the storage resources. Fig.53 Such a condition table. In addition, Fig.54 An example of a table (referred to as an attribute table) in which attributes of a processing module are set is shown. The development system 1 sets / manages the attributes in the storage resource. Fig.54 Such an attribute table. Development system 1 in Fig.53In such a condition table, set / manage Fig.54 Such conditions regarding whether or not modules of each attribute can be connected to each other.

[0418] exist Fig.54 In the attribute table, the attributes (in other words, the type, etc.) of each processing module are set as the components constituting the process line. For example, it is assumed that there are four attributes A, B, C, and D in the candidate processing modules. This attribute can also be set as one of the parameter values ​​in the internal setting column of each processing module. Fig.54 , four attributes, namely, attributes A, B, C, and D, are illustrated. Attribute A indicates a process of outputting a monochrome image regardless of any input, attribute B indicates a process of taking a monochrome image as input and converting it into a color image and outputting it, attribute C indicates a process of taking a color image as input and converting it into a monochrome image and outputting it, and attribute D indicates a process of taking a monochrome image as input and outputting it. For example, attribute A is set for the "camera input" module. Attribute B is set for the "Bayer transform" module. Attribute C is set for the "RGBLUT" module. Attribute D is set for the "binarization" module. In this way, predetermined attributes are provided in advance to each processing module through the attribute table. Here, to simplify the explanation, the type of image input and output of each processing is set as an attribute, but it is not limited to this, and other attributes can also be set.

[0419] exist Fig.53 In the condition table, the combination of the attributes of the pre-module and the attributes of the post-module is used to set whether the modules can be connected to each other. The pre-module is the module configured before the object module, and the post-module is the module configured after the object module. Depending on the configuration situation, the object module selected by the user U1 becomes the pre-module or the post-module. Examples of pre-modules and post-modules will be described later.

[0420] Based on Fig.54 Example of setting properties of Fig.54 In the setting example of the conditions, in row #1, when the front module is attribute A, and the rear module configured thereafter (next) is attribute A, whether the connection is possible is "yes" (○). A specific example is the "camera input" module → the "camera input" module. In row #2, when the front module is attribute A, and the rear module is attribute B, whether the connection is possible is "yes" (○). A specific example is the "camera input" module → the "Bayer transform" module. In row #3, when the front module is attribute A, and the rear module is attribute C, whether the connection is possible is "no" (×). A specific example is the "camera input" module → the "RGBLUT" module. Fig.54The camera input result output is a monochrome image, and the input of RGBLUT of attribute C requires a color image, so the combination of #3 is not connectable. In the case of the front module of #4, when the attribute A is the front module and the rear module is the attribute D, the connection is "yes" (○). A specific example is "camera input" module → "binarization" module. Similarly, all combinations related to all attributes (in this example, 16 combinations #1 to #16) are used to determine whether the connection is possible.

[0421] In addition, each processing module can select and set an internal setting parameter value as described above. It is also possible to set whether or not to connect based on the difference in the internal setting parameter value of the processing module.

[0422] Fig.55 A specific example of whether or not the configuration is possible is shown corresponding to the example of the above condition. It is assumed that in the flow line column 32, a "camera input" module (attribute A) and a "Bayer transformation" module (attribute B) are configured as the flow line FL being edited. In the example shown in the figure, for ease of explanation, the attributes are also recorded in the processing module BL. Next, it is assumed that the object processing module 5501 selected by the user U1 is the "camera input" module (attribute A). The object processing module 5501 of the "camera input" module becomes the second "camera input" module. In this case, according to Fig.53 Conditions, first, in the case of configuring the object processing module 5501 (attribute A) at the position between the "camera input" module (attribute A) and the "Bayer transform" module (attribute B), assuming the arrangement of attributes AAB, about the "camera input" module as the front module and the "camera input" module as the rear module, and the "camera input" module as the front module and the "Bayer transform" module as the rear module, use Fig.53 The condition table is used to determine whether the connection is possible. In the former case, the attribute AA is arranged, so it is determined to be configurable according to the condition #1. In the latter case, the attribute AB is arranged, so it is determined to be configurable according to the condition #2. For the object processing module 5501, the condition is satisfied in the relationship between the three processing modules BL including the previous and next processing modules BL, so the part is determined to be configurable object processing module 5501 (attribute A). Therefore, the development system 2 displays a highlighted area 5002 indicating that the part is configurable.

[0423] In addition, regarding the space after the "Bayer transform" module (attribute B), when the object processing module 5501 (attribute A) is configured, assuming the arrangement of attributes BA, the "Bayer transform" module as the front module and the "camera input" module as the rear module are used. Fig.53The condition table is used to determine whether the connection is possible. At this time, it becomes an array of attributes BA, so it is determined to be configurable according to the condition #5. Since the condition is met, the part is determined to be a configurable object processing module 5501 (attribute A). Therefore, the development system 2 displays a highlighted area 5002 indicating that the part is configurable.

[0424] Fig.56 Here is another example. Assume that in the flow line column 32, the "Camera Input" module (attribute A), the "Bayer Transform" module (attribute B), the "RGBLUT" module (attribute C), the "Binarization" module (attribute D), and the "Bayer Transform" module (attribute B) are configured as the flow line FL being edited. Next, assume that the object processing module 5601 selected by the user U1 is the "Binarization" module (attribute D). In this example, two "Bayer Transform" modules appear, and two "Binarization" modules appear. In order to distinguish between the two, they are recorded as "Number 1" and "Number 2". When multiple processing modules BL of the same type appear at the same time, the development system 1 may also display them as "Number 1", "Number 2", etc. in order to distinguish them.

[0425] In this case, according to the conditions, when the object processing module 5601 (attribute D) is first configured between the "camera input" module (attribute A) and the "Bayer transformation" module (attribute B), the arrangement of attributes ADB is assumed, as a combination of the attributes of the front module and the attributes of the rear module, and the attributes AD and DB are used respectively. Fig.53 The condition table is used to determine whether the connection is possible. Regarding the arrangement of attribute AD, it is configurable according to condition #4, and regarding the arrangement of attribute DB, it is configurable according to condition #14. Since both conditions are met, the part is determined to be configurable. Therefore, the development system 2 displays a highlighted area 5002 at the part.

[0426] Similarly, for the area between the "Bayer Transform" module (attribute B) and the "RGBLUT" module (attribute C), the area between the "RGBLUT" module (attribute C) and the "Binarization" module (attribute D), the area between the "Binarization" module (attribute D) and the "Bayer Transform" module (attribute B), and the space after the "Bayer Transform" module (attribute B), use Fig.53The condition table is used to determine whether configuration is possible. As a result, the development system 2 regards the space between the "Bayer transform" module (attribute B) and the "RGBLUT" module (attribute C) as non-configurable and displays a non-configurable display 5003, regards the space between the "RGBLUT" module (attribute C) and the "binarization" module (attribute D) as configurable and displays a highlighted area 5002, regards the space between the "binarization" module (attribute D) and the "Bayer transform" module (attribute B) as configurable and displays a highlighted area 5002, and regards the space after the "Bayer transform" module (attribute B) as non-configurable and displays a non-configurable display 5003.

[0427] In addition, in this example, both a highlighted display indicating that the configuration is possible and a non-configurable display indicating that the configuration is not possible are displayed, but either one may be displayed. In addition, here, the user U1 is notified whether the configuration is possible by visual display, but it is not limited to this, and a warning sound may be emitted or a warning may be displayed, as long as any means is used to inform the user whether the configuration is possible.

[0428] In the above example, the determination of whether or not the configuration is permitted is determined based on the attribute table and the condition table for each processing module. However, the determination method of whether or not the configuration is permitted is not limited thereto and can be implemented using other known techniques.

[0429] In this function, when the user U1 drags and drops the selected object processing module to the flow line, the development system 2 Fig.53 The condition table is used to determine whether the target processing module can be connected / configured, and a predetermined display indicating whether it can be configured is performed on the process line. According to this function, the user U1 can easily know whether the processing module can be configured, and can also reduce useless operations such as moving the processing module to a non-configurable position.

[0430] [2-5. The unset module display function must be set]

[0431] Fig.57 This is an explanatory diagram related to the display function of modules that must be set but have not been set. As a subject, in the processing modules that constitute the flow line, there are sometimes parameter items that must be set through the above-mentioned internal settings in order to complete the flow line. For the sake of convenience, they are recorded as necessary settings, necessary setting items, etc. For the processing modules that constitute the flow line being produced, if the user U1 executes the flow line without completing the necessary settings, it will result in a processing error due to the incomplete necessary settings of the processing modules. The task of confirming whether the necessary settings are completed for each processing module, or resetting the processing module because the flow line ends due to a processing error, is a heavy burden for the user U1 as a developer.

[0432] In addition, if user U1 has no means of knowing that an error occurs in the processing module during the execution of the flow line (in other words, it ends due to a processing error) or that the required settings of the processing module are not completed, user U1 as a developer may sometimes be unable to complete the flow line or it may take a long time to complete the flow line.

[0433] Therefore, in this function, the development system 1 performs a predetermined display indicating that the required setting is not completed (in other words, the required setting item is not set) for the processing module (also recorded as an unset module, etc.) that has been configured in the process line but the required setting is not completed. Fig.57 In the example of , a predetermined icon 5701 is provided as a predetermined display (also described as a not set display, a must set incomplete display, etc.) 5701. This icon 5701 is provided as a ! mark in this example.

[0434] For example, the "Processing B" module shown in the figure is a "binarization" module having required setting items. Fig.19 The same). In the internal setting column 5700, for example, the "binarization threshold" item 5703 is set as a must-set item. In this example, the threshold value as a value is not set for the "binarization threshold" item 5703. That is, the "binarization" module has not been set. In this case, the development system 1 determines the incomplete state, and displays an icon 5701 indicating that the must-set is not completed on the "binarization" module as the "processing B" module in the flow line FL of the flow line column 32. Similarly, the "processing C" module displays the icon 5701 when the must-set is not completed.

[0435] By observing the icon 5701, the user U1 can easily recognize that the necessary settings of the processing module are not completed, and can make the necessary settings. In addition, the display of the icon 5701 is not limited to the above example, and can be in various forms. In addition, for example, a predetermined display (such as the icon 5701) indicating that the necessary settings are not completed can also be made in the internal setting column of the processing module BL at the location of the necessary setting items (such as the "binarization threshold" item 5703).

[0436] In addition, Fig.57In the example, when the user U1 performs a predetermined flow line execution operation, the development system 1 executes the process of the flow line FL. Alternatively, the development system 1 executes the process of the flow line FL for preview display according to the preview execution operation. In this function, in the flow line FL, a predetermined display is performed to indicate the status of the process result of the processing module BL being executed in the flow line FL, for example, as illustrated as "execution" in the lower row. In this example, as a predetermined display (also described as a module execution status display, etc.) 5702, there are indicator icon displays 5702a, 5702b, and 5702c for each processing module BL. There are multiple types of indicator icon displays, which are displayed in colors corresponding to the status. As for the indicator icon display, as the status, there are, for example, an indicator icon display 5702a of "normal", an indicator icon display 5702b of "error", and an indicator icon display 5702c of "not executed". The "normal" status (for example, a blue icon) indicates that the processing of the processing module BL has been completed normally. The "error" state (eg, a red icon) indicates that an error occurred during the processing of the processing module BL and the processing ended in error. The "not executed" state (eg, a white icon) indicates that the processing of the processing module BL was not executed.

[0437] The user U1 can easily recognize that an error has occurred in the processing module BL during the execution of the flow line FL by observing the indicator icon display in the module execution status display 5702. As a result, the user U1 can efficiently complete the flow line FL. In the case of using the display of the icon 5701 indicating an unset module and the module execution status display 5702 together, the user U1 can recognize that the processing module BL that has an error indicated by the module execution status display 5702 must be set as the cause of the error.

[0438] In addition, regarding the method of determining whether an error occurs in a processing module when executing a flow line, for example, any one of the following two methods may be applied. One method is a method in which the processor of the development system 1 executes processing of a processing module under a certain internal setting (for example, a required setting item is not set), and when the processing results in an error end, the processing module is determined to be in an "error" state. Another method is a method in which the processor of the development system 1 checks whether the required setting item of the processing module is set before executing processing of the processing module under a certain internal setting (for example, a required setting item is not set), and does not execute the processing if it is not set, and determines the processing module to be in an "error" state.

[0439] The functions of icon 5701 related to the above-mentioned required settings also include the following functions. Fig.58 This is an explanatory diagram regarding the function of icon 5701 related to the required setting. Fig.58 In (A), the first state is Fig.57 The same state of the flow line FL. Icon 5701 is displayed in the "Processing B" module where the required settings are not completed. User U1 completes the required settings of the "Processing B" module according to icon 5701. In addition, the development system 1 can automatically display the required setting items that have not been set by the user U1 clicking on icon 5701. When the development system 1 determines that the required settings of the "Processing B" module are completed, it removes icon 5701. Thus, it transitions to the second state (B).

[0440] As a result of confirming the preview of the processing result of the flow line FL, the user U1 is set to further edit the flow line FL, for example, to delete the "Processing A" module. Thus, it becomes the third state (C). In the third state, the processing module connected to the input side (front side) of the "Processing B" module is changed to the "Camera Input" module. Therefore, for example, the internal settings of the "Processing B" module may need to be changed. In this example, it is set that there is a required setting item that matches the output (camera input result image) of the "Camera Input" module in the internal settings of the "Processing B" module, and the required setting is not set. With the deletion of the "Processing A" module, the development system 1 determines that the required setting as described above has not been completed, and displays the icon 5701 for the "Processing B" module again. Thus, it is transferred to the fourth state (D). Afterwards, the user U1 performs the required setting according to the icon 5701 in the same way, and finally the desired flow line FL can be completed.

[0441] In the above example, when the "processing B" module that has been set must be affected by the deletion of the processing module located in front (upstream) and needs to be set again, icon 5701 is displayed to support resetting. Not limited to this, when a processing module is added to the front (upstream) side, or the setting of an existing processing module is changed, the processing module on the rear (downstream) side is also affected, and there is a situation where the status related to the required setting changes. In this case, the function of icon 5701 can also be used.

[0442] [Functions related to input image and output image settings]

[0443] Fig.59Another example related to the above-mentioned required settings is shown. In the flow line FL of the flow line column 32, each processing module BL has an input image and an output image, but as a function of the development system 1, it is also possible to freely select and set the input image for each processing module BL. The development system 1 also has the following function: in the flow line column 32, the input image ID, the output image ID, etc. are clearly displayed in a manner that makes it easy to know the input image and output image of each processing module BL. First, the function related to the setting of the input image and the output image is described.

[0444] In the above-mentioned Embodiment 1, the description is based on the case where the input image / input image of a certain processing module BL in a certain flow line FL is the processing result image / output image of the previously connected processing module BL. In the absence of special settings, such an input and output structure can be set. In the following example of Embodiment 2, the input image / input image of a certain processing module BL in a certain flow line FL is not limited to the processing result image / output image of the previously connected processing module BL, but can be set to the output image of any other processing module BL of the flow line FL.

[0445] exist Fig.59 In the example of FIG. 5 , functions such as setting of input images, display of input image IDs, and display of output image IDs of processing modules BL in flow line FL are shown. In this function, the flow line column 32 has an “input image” display 5901 and an “output image” display 5902. In the “input image” display 5901, the ID of the input image of the corresponding processing module BL on the upper side is displayed in each item 5901a. In the “output image” display 5902, the ID of the output image of the corresponding processing module on the upper side is displayed in each item 5902a.

[0446] First, the development system 1 automatically assigns output image IDs to the output images of the processing modules BL in order from the beginning of the flow line FL. In this example, the output image ID of the "camera capture" module is 001, the output image ID of the "processing B" module is 002, the output image ID of the "processing C" module is 003, and the output image ID of the "processing D" module is 004. The user U1 can also assign desired output image IDs individually by name or the like. In this example, the IDs are assigned automatically, so the ID setting work performed by the user U1 can be reduced.

[0447] Next, the development system 1 sets the input image of each processing module BL in order from the beginning of the flow line FL, and assigns an input image ID. In the case of the default setting, the output image of the previous processing module BL can also be automatically set as the input image of the processing module BL as in the first embodiment. In this example, the user U1 freely selects and sets the input image of each processing module BL. The setting of the input image is one of the required setting items for each processing module BL.

[0448] exist Fig.59 In the example of (A), the user U1 selects and sets the input image ID in the setting item 5903 of the input image in the internal setting column (for example, a GUI such as a list box) or the corresponding item 5901a of the display 5901 of the "input image" for the "processing A" module. As for the option of the input image ID, the output image ID in the display 5902 of the "output image" can be used. In addition, the setting of the input image ID in the item 5901a of the display 5901 can also be set by dragging and dropping the output image ID from the item 5902a of the display 5902 (described later). In this example, 001 is set as the input image ID of the input image of the "processing A" module. Similarly, the user U1 sets 003 as the input image ID for the "processing C" module. Regarding the "processing B" module, the setting of the input image ID is not completed, and the corresponding item 5901a is blank.

[0449] Fig.59 (B) is another example. In the same process line FL as (A), for the input image of the "Processing B" module, the output image ID = 002 is set as the output image of the "Processing A" module. Similarly, for the input image of the "Processing C" module, the output image ID = 002 is set as the output image of the "Processing A" module. The output image of the "Processing A" module is set to the two input images of the "Processing B" module and the "Processing C" module. In addition, regarding the structure of such a process line, it is also possible to create / display it as described above ( Fig.33 ) is an explicitly branched flow line, but in this example it can be displayed as a single flow line without branches.

[0450] like Fig.59As in the example of , in this function, the user U1 can set the output image selected from the output images of the desired processing module as the input image of the desired processing module, and the relationship between the input and the output can be clearly indicated by display 5901 and display 5902. The user U1 can easily understand and confirm the relationship between the input and the output, which is one of the items that must be set. In addition, as a modified example, the setting of the input image ID and the output image ID can be set to a processing module ID indicating which processing module the output is, instead of the image ID. For example, in the display 5901 of the input image, the processing module ID can also be displayed / set for the item 5901a.

[0451] like Fig.59 For example, when a processing module BL (e.g., "processing B" module) selected by the user U1 is configured / connected in the flow line FL, it is necessary to select and set the input image (input image ID) of the processing module BL from the output image of the processing module BL that is ahead (upstream) of the processing module BL. If the required setting of the input image is not completed, the development system 1 and Fig.57 Likewise, an icon 5701 indicating that the setting is not yet completed is displayed in the processing module BL. Thus, the user U1 can set the input image of the processing module BL without omission.

[0452] Fig.60 This is a specific example of the change of the input and output image settings and the display of the icon 5701 accompanying the addition / insertion of the processing module BL to the flow line FL. First, in the state (A), the "camera input" module, the "processing A" module, and the "processing C" module are connected to the flow line FL. The output image ID is automatically assigned. The user U1 sets the input image ID of the "processing A" module to 001, which is the output image ID of the "camera input" module, and sets the input image ID of the "processing C" module to 002, which is the output image ID of the "processing A" module.

[0453] Next, assume that user U1 selects the "Processing B" module from the component column 31 and configures it between the "Processing A" module and the "Processing C" module. Thus, the state is transferred to (B). First, since 001 to 003 have been used, the development system 1 assigns 004 to the output image ID of the added "Processing B" module and sets it as a consecutive number. In addition, regarding the output image ID of each processing module, when it is set as a consecutive number, it can also be reallocated to match the arrangement in the flow line FL (described later). In this case, the output image ID of the "Processing B" module becomes 003, and the output image ID of the "Processing C" module becomes 004.

[0454] Next, regarding the "Processing B" module, since the input image ID is not set (item 5901a is displayed as a blank column), the development system 1 displays the icon 5701 in the "Processing B" module. As a modified example, the icon 5701 may be displayed in the item 5901a where the input image ID is a blank column. In addition, since the "Processing B" module is inserted, the module in front of the "Processing C" module is changed to the "Processing B" module. Regarding the "Processing C" module, since the input image ID is set to 002, the icon 5701 is not displayed. The user U1 may also change the setting of the input image ID of the "Processing C" module along with the insertion of the "Processing B" module.

[0455] As a variation example, regarding the "Processing C" module to which the "Processing B" module is added on the front side, in order to achieve the intention of alerting the user U1, that is, to indicate whether the input and output image settings should be re-evaluated due to the change in the front and back relationships, the "Processing C" module can also be displayed as icon 5701 or a predetermined display different from icon 5701.

[0456] From the state (B), the user U1 sets the input image ID of the "Processing B" module. Thus, for example, the state is transferred to (C). In this example, the user U1 sets the input image ID of the "Processing B" module to 002, which is the output image ID of the "Processing A" module. In addition, the user U1 re-evaluates the input image ID of the "Processing C" module following the "Processing B" module, and sets it to 004, which is the output image ID of the "Processing B" module. When the required settings of the "Processing B" module are completed, the development system 1 removes the icon 5701.

[0457] exist Fig.61 5701 as another example, the change of the setting of the input and output images accompanying the deletion of the processing module BL in the flow line FL and the specific example of the display of the icon 5701. First, in the state (A), the "camera input" module, the "processing A" module, the "processing B" module, and the "processing C" module are connected in the flow line FL as shown in the figure. For the input image of each processing module BL, the output image of the previous processing module BL is set as shown in the figure.

[0458] Next, as shown in the state of (B), user U1 deletes the "Process A" module. The corresponding item 5901a of the display 5901 of the "input image" and the corresponding item 5902a of the display 5902 of the "output image" are also removed. Along with this, as an option for the output image ID, 002, which is the ID of the output image of the "Process A" module, disappears. However, for the input image ID of the "Process B" module, 002, which is the ID of the output image of the "Process A" module, is still set. The development system 1 determines that there is no output image (output image ID) indicated by the input image ID of the "Process B" module within this flow line FL. As a result, in the development system 1, regarding the "Process B" module, although the input image ID, which is a mandatory setting item, has been set, since it needs to be reset, the icon 5701 of the "Process B" module is displayed. Thus, it becomes the state of (C).

[0459] Alternatively, it can also be set as follows. Along with the deletion of the "Process A" module, the development system 1 deletes all settings associated with the output image ID = 002 of the "Process A" module from within this flow line FL. That is, in the above example, 002, which is the set value of the input image ID of the "Process B" module, is restored to a blank field (item 5901a in (C) becomes a blank field). As a result, since the input image ID of the "Process B" module in the development system 1 becomes unset, the icon 5701 of the "Process B" module is displayed.

[0460] Fig.62 An other specific example is shown. In Fig.62 the example, in the display of a certain flow line FL1 in the flow line column 32, as shown in the figure, there are displays of rows for "input image A", "input image B", "processing result", and "execution". The display of the row for "processing result" corresponds to the display 5902 of the "output image" such as the above Fig.59 etc. When the development system 1 configures the processing module BL in the flow line FL1, it automatically displays the row for "processing result", and for each item of the processing module BL, it automatically assigns and displays the ID of the processing result (output image). This ID is, for example, a consecutive number. The ID of this processing result (output image) can be processed as the input image ID of any processing module BL configured later than this processing module BL.

[0461] In addition, along with the creation of the flow line FL1, the development system 1 automatically displays the row for "input image A". In this row, regarding the ID of the input image of each processing module BL, various inputs / settings can be made as described above. In Fig.62In the example, when the "binarization B" module is configured, the input image ID of the processing module is not set and becomes a blank. For example, the user U1 can set the input image ID by dragging and dropping. For example, as shown by the dotted arrow, if the item of the output image ID = 003 of the "binarization A" module is dragged and dropped to the item of the input image of the "binarization B" module, 003 can be set as the input image ID of the "binarization A" module. In addition, in other examples, if the item of the output image ID = 003 of the "binarization A" module is dragged and dropped to the item of the input image of the "pixel value conversion" module further away, 003 can be set as the input image ID of the "pixel value conversion" module.

[0462] This function makes it easy to understand the input-output relationship, such as which processing module's output image is used as which processing module's input image in the process line, and can intuitively set the input-output relationship. If the input-output relationship is not set, support can be provided through the warning output of icon 5701.

[0463] exist Fig.63 , an example of reallocation of the consecutive numbering of the output image ID is shown as another example. Initially, it is set to the state (A). In the illustrated flow line FL, the output image ID of the processing result (output image) of each processing module BL is automatically numbered as a consecutive number, and 001 to 005 are set. In addition, when the processing result of the previous processing module BL is set as the input image of the next processing module BL as the default automatic numbering of the input image ID, 001 to 004 are set as the input image ID as shown in the figure.

[0464] Next, assume that user U1 deletes the "binarization A" module as shown in the state (B). Fig.61 Similarly, the output image ID of the "binarization A" module is deleted. As a transitional state for explanation, it becomes as shown in (B). Along with this, in this example, the input image ID of the "binarization B" module is changed from 002 to a blank column. Since the required settings of the "binarization B" module are not completed, the development system 1 displays the icon 5701 as shown in (C).

[0465] Moreover, in this example, as shown in the state (C), the development system 1 reallocates the output image ID. The development system 1 reallocates the output image ID so that it becomes a continuous number from the beginning in order to match the structure of the current flow line FL1. As a result, the output image IDs become 001 to 004 as shown in the figure. In addition, when the development system 1 reallocates the output image ID, it also automatically updates the setting of the input image ID in the display of the input image as shown in the figure in order to match the reallocated output image ID. For example, the input image ID of the "Pixel Value Conversion" module is updated from 004 to 003.

[0466] This function makes it easier to understand the arrangement of output image IDs and input image IDs on the process line, and reduces the workload of the user U1 to reassign IDs. Automatic ID reassignment (resetting) can also be applied even when adding / inserting a processing module.

[0467] Fig.64 The control process flow of the development system 1 related to the above-mentioned must-set unset module display function is shown. In step S641, the development system 1 detects when the user U1 performs an editing operation on the process line in the GUI screen 30. In step S642, the development system 1 determines whether it is a new configuration of the processing module or a setting change / deletion of the processing module. In the case of a new configuration, the process proceeds to step S643 (B), and in the case of a setting change / deletion, the process proceeds to step S646 (C).

[0468] The process shown in the box (B) is an event process when a new process module is configured in the flow line. In step S643, the development system 1 displays an icon indicating that the setting must be completed, especially the setting is the initial value, in the newly configured process module. This icon is the same as the above Fig.57 Similarly, for example, it is a ! mark, but it is not limited to this, and it may be set to other predetermined display indicating that the new configuration and the setting are initial values.

[0469] In step S644, the development system 1 determines whether it is necessary to reset the required setting items in the post-modules when there are post-modules (including modules connected remotely) connected to the rear side (downstream) relative to the newly configured module. If reset is required (yes), the process proceeds to step S645, and if not (no), the process proceeds to step S648. In step S645, the development system 1 displays a predetermined display indicating that the required setting items need to be reset, such as the same icon (! mark) as described above, on the corresponding post-modules.

[0470] The processing shown in the box (C) is an event processing for a case where the setting of a processing module already configured in the process line is changed or deleted. In step S646, the development system 1 determines whether it is necessary to reset the required setting items in the subsequent module when there is a subsequent module (including a module connected remotely) connected to the rear side (downstream) relative to the processing module whose setting is changed or the part where the processing module is deleted. If reset is required (yes), the process proceeds to step S647, and if not required (no), the process proceeds to step S648. In step S647, the development system 1 displays a predetermined display indicating that the required setting items need to be reset, such as the same icon (! mark) as described above, in the corresponding subsequent module.

[0471] The processing shown in the box (A) is an event processing when the processing module is set. In step S648, the development system 1 determines whether the required setting is completed for each processing module of the flow line, and proceeds to step S649 if it is completed, and ends the process if it is not completed. In step S649, the development system 1 removes the icon (! mark) for the processing module for which the required setting is completed, and ends the process.

[0472] [2-6. Module display function during process line processing]

[0473] Fig.65 This is an explanatory diagram related to the module display function in the process line processing. As a problem, when executing the process line, if it is not possible to understand which processing module is currently being executed, it is sometimes difficult for the user U1 such as the developer to understand whether the processing module configured in the process line is suitable. Fig.57 The function of displaying the execution status of the flow line can confirm whether each processing module of the flow line is executed normally or ended with an error, etc. However, it is sometimes difficult to identify which processing module is currently being executed using only this function.

[0474] Therefore, this function is a function of performing a predetermined display indicating which processing module is currently being executed when executing a flow line. In this function, only the processing module currently being processed on the flow line is put into the first display state and is normally displayed, for example, brightly, and the other processing modules are put into the second display state and are displayed, for example, darkly.

[0475] exist Fig.65In the example of FIG. 1 , in the flow line column 32, the "camera capture" module, the "processing A" module, the "processing B" module, and the "processing C" module are connected as the created flow line FL. According to the flow line execution operation performed by the user U1, the development system 1 executes the process of the flow line FL. Although this execution process is different from the above-mentioned preview display process, it can also be set as the execution process for preview display.

[0476] exist Fig.65 In the example, the processing of the "Processing A" module in the flow line FL is being executed. The "Processing A" module portion is in the first display state 6501 and is set to be relatively brightly displayed. The other portions, the "Camera Intake" module, the "Processing B" module, and the "Processing C" module portion, are in the second display state 6502 and are set to be relatively darkly displayed. In this example, the first display state 6501 is a display of normal brightness in the GUI screen 30, and the second display state 6502 is a shaded display in the upper layer of the screen, but it is not limited to this, and various display methods can be used. The first display state 6501 can also be displayed in a predetermined color, icon, frame, etc. in a more eye-catching manner than the normal display.

[0477] In addition, this function can also be combined with the above Fig.57 The flow line execution status display 5702 functions and displays them side by side. Fig.65 In the example of , the display is set to be side by side, as shown in the "Execution" row, a display 5702 of the flow line execution status is set.

[0478] Fig.66 yes Fig.65 The detailed description of FIG. 3 shows the transition of the state of the flow line FL in the process line column 32. The state (A) is the state of the process of the "camera capture" module at the beginning. The state (B) is the same as Fig.65 Similarly, this is the state in which the processing of the "Processing A" module is being executed. Similarly, for each processing module, the first display state 6501 is transferred, and the state (C) is the state in which the processing of the last "Processing C" module is being executed. When the last processing of the flow line FL is completed, it returns to the original display. In addition, in the case of an error end as described above, it returns to the original display at that time.

[0479] According to this function, the user U1 can easily recognize the processing module being executed as the execution status of the flow line. When the above-mentioned flow line is executed while the preview display is executed, for example, Fig.65 In the flowchart column 32 , the processing module being executed is displayed (first display state 6501 ), and at the same time, a preview display image of the processing module being executed is displayed in the preview column 33 .

[0480] [Display example of the internal setting column for each processing module]

[0481] Fig.67 A display example of the internal setting column of each processing module associated with each function of the second embodiment is shown. Fig.67 The example is about the internal setting column of the "color conversion" module. The "inspection process display" item 6701 is in the above Fig.37 The GUI can be used to set whether to display the information of the processing module on the screen during such inspection. If it is set to off, the displayed information on the screen can be reduced. The "Preview display" item 6702 is related to the above-mentioned preview display switching function ( Fig.39 The "format conversion" item 6703 is an example of the above-mentioned internal setting corresponding to the case of the "color conversion" module.

[0482] The "Input image selection" item 6704 is a column that can set the input image of the processing module. Regarding the "Processing image" check item 6705, when the check is set to the on setting, the processing of the processing module is applied to the set input image. When the check is set to the off setting, the processing of the processing module is applied to the image / image set by the default setting. Even if the input image is not set for the processing module, the processing can be applied tentatively. The "Input image ID" item 6706 is an item that can select / specify the ID of the above-mentioned input image. In the input image preview item 6707, a preview of the set input image is displayed. In the "Pixel value" item 6708, the position (x, y) and pixel value (R, G, B) of the pixel specified for the input image preview by hovering the mouse, etc. are displayed.

[0483] It is possible to appropriately transfer between the GUI of the internal setting column of each processing module as in the above example and the GUI of the entire flow line. Fig.67 In such an internal setting column display, a predetermined display is performed for each parameter item in a manner that allows the user U1 to easily understand which is the required setting item and which is the non-required setting item (for example, displaying a mark or highlighting the required setting item).

[0484] The present invention has been specifically described above based on the embodiments, but the present invention is not limited to the above embodiments, and various changes can be made without departing from the gist. In addition to the necessary components, each embodiment can add, delete, replace, etc. the components. In the absence of special limitations, each component can be one or more. It can also be a method of combining the various embodiments.

[0485] Explanation of symbols

[0486] 1: Development system; 2: Image inspection system; 30: GUI screen; 31: Module selection section; 32: Module configuration section; 33: Preview display section; BL: Processing module; FL: Flow line.

Claims

1. A development system that supports the development of an image inspection system. in, The development system comprises a computer system having a processor and a memory. The computer system generates and provides a graphical user interface (GUI) screen for supporting the development to a user who performs the development. The GUI screen has: A module selection unit configured with a plurality of processing modules, the processing modules being components for constituting a flow line of the image inspection system; a module configuration unit that configures the processing modules selected from the module selection unit according to the GUI operation of the user and connects the configured processing modules with lines to form and display the process line being produced; as well as The preview display unit displays a preview image of an operation state of the flow line arranged in the process of being created by the module arrangement unit according to a GUI operation by the user.

2. The development system according to claim 1, in, By dragging and dropping the processing module selected from the module selection unit to the module arrangement unit, the processing module is added to a designated position of the process line. For the process line of the module configuration unit, according to the GUI operation of the user, the processing module of the designated part is deleted, With respect to the process line of the module configuration unit, the processing module at a designated location is replaced with another processing module according to a GUI operation by the user.

3. The development system according to claim 1, in, Regarding the process line configured in the module configuration unit, according to the user's GUI operation, the entire process line or the range from the beginning of the process line to the selected processing module is taken as an object, and a preview of the processing action of the object is displayed in the preview display unit.

4. The development system according to claim 1, in, Regarding the flow line arranged in the module arrangement unit, based on the GUI operation of the user, the process module selected in the flow line is targeted, and a preview of the target processing action is displayed in the preview display unit.

5. The development system according to claim 1, in, Regarding the process line configured in the module configuration unit, according to the user's GUI operation, a range starting from the selected processing module and ending at the very end of the process line is taken as an object, and a preview of the processing action of the object is displayed in the preview display unit, or a range starting from the selected processing module and ending at another selected processing module is taken as an object, and a preview of the processing action of the object is displayed in the preview display unit.

6. The development system according to claim 1, in, Regarding the process line configured in the module configuration unit, based on the GUI operation of the user, a plurality of the processing modules within a selected range in the process line are taken as objects, and a preview of the processing actions of the objects is displayed in the preview display unit, The preview display unit sequentially displays the processing result images of the processing modules in the range on a time axis.

7. The development system according to claim 1, in, Regarding the processing module configured in the module selection unit, according to the GUI operation of the user, the processing module selected is taken as an object, and a preview image of the operation state of the object is displayed on the preview display unit using a sample input image.

8. The development system according to claim 1, in, In the case where the processing module configured in the module selection unit has a plurality of internal settings, The processing module arranged in the flow line of the module arrangement unit can be switched to an internal setting selected from the plurality of internal settings according to a GUI operation of the user.

9. The development system according to claim 1, in, Regarding the processing module arranged in the module selection unit, according to the GUI operation of the user, the operation status of a case where the selected processing module is hypothetically applied to the flow line arranged in the module arrangement unit is displayed in the preview display unit.

10. The development system according to claim 1, in, Regarding the process line configured in the module configuration unit, according to the user's GUI operation, the multiple processing modules in the selected range are taken as objects, and the action states of the respective processing modules in the range are displayed side by side in multiple divided areas in the preview display unit.

11. The development system according to claim 1, in, In the case where the processing module configured in the module selection unit has a plurality of internal settings, Regarding the process line configured in the module configuration section, based on the GUI operation of the user, the operation status of each of the internal settings of the processing module is displayed side by side in a plurality of divided areas in the preview display section with a plurality of internal settings in the selected processing module as the object, The preview display unit sets the selected internal setting for the processing module of the flow line of the module configuration unit according to the GUI operation of the user.

12. The development system according to claim 1, in, As one of the processing modules arranged in the module selection unit, there is a determination processing module for making a determination related to an object from an image, Regarding the flow line configured in the module configuration unit, according to the GUI operation of the user, the determination processing module is taken as the selected processing module, and the operation state of the determination processing module is displayed on the preview display unit, The preview display unit displays the determination processing result image of the determination processing module, and displays the outer frame line of the determination processing result image in different forms according to the determination processing result information.

13. The development system according to claim 1, in, As one of the processing modules arranged in the module selection unit, there is a determination processing module for making a determination related to an object from an image, Regarding the flow line configured in the module configuration unit, according to the GUI operation of the user, the determination processing module is taken as the selected processing module, and the operation state of the determination processing module is displayed on the preview display unit, The preview display unit displays the determination processing result image of the determination processing module, and extracts and displays the color information of the pixel position selected according to the GUI operation of the user.

14. The development system according to claim 1, in, In the case where the processing module configured in the module selection unit has a plurality of internal settings, Regarding the process line configured in the module configuration section, when an internal setting of the processing module is selected, the operation state of each of the internal settings is displayed side by side in a plurality of divided areas in the preview display section with a plurality of internal settings in the selected processing module as objects according to the GUI operation of the user, The preview display unit sequentially extracts internal settings that serve as comparison candidates from the multiple internal settings of the processing module in a comparative manner and displays them for comparison in the multiple divided areas, selects a winning candidate based on the user's GUI operation on the divided areas, and finally determines the internal setting to be adopted in the process line of the module configuration unit.

15. The development system according to claim 1, in, With respect to the process line arranged in the module arrangement section, when the processing module to be arranged next in a designated portion is selected from the plurality of processing modules arranged in the module selection section, the operation status of each of the processing modules is displayed side by side in a plurality of divided areas in the preview display section with the selected plurality of processing modules as objects according to the GUI operation of the user, The preview display unit extracts the processing modules that become candidates for comparison from the multiple processing modules in a comparison manner and displays them for comparison in the multiple segmented areas. Based on the user's GUI operation on the segmented areas, the winning candidate is selected, and the processing module adopted in the process line of the module configuration unit is finally determined.

16. The development system according to claim 1, in, The module configuration unit displays a plurality of candidate flow lines being created according to the user's GUI operation, displays a preview of the operation status of each candidate flow line on the preview display unit, and saves the selected flow line according to the user's GUI operation.

17. The development system according to claim 1, in, As one of the processing modules configured in the module selection unit, there is a camera input processing module, and the camera input processing module has a plurality of internal settings. Regarding the process line configured in the module configuration unit, when the internal settings of the camera input module are selected, based on the user's GUI operation, the multiple internal settings of the selected camera input module are taken as objects, and the action status of each of the internal settings is displayed in the preview display unit in a manner compared with a standard image.

18. The development system according to claim 1, in, As one of the processing modules configured in the module selection unit, there is a camera input processing module, displaying the camera-captured image of the selected camera-capture module on the preview display unit according to the GUI operation of the user, selecting a region as a candidate for a template image based on a GUI operation of the user for the camera captured image displayed on the preview display unit, From the candidate region of the template image, other candidate regions can be generated by changing the size. A plurality of regions as candidates for the template image are compared and displayed on the preview display unit according to a GUI operation of the user, and a candidate selected according to the GUI operation of the user on the preview display unit is registered as the template image.

19. The development system according to claim 1, in, In the module configuration unit, for the entire configured process line or each configured process module, on / off of displaying the preview image in the preview display unit is set according to a GUI operation by the user.

20. The development system according to claim 19, in, In the module arrangement unit, a predetermined display indicating a setting state of on / off of display of the preview image is performed for the entire arranged process line or for each arranged process module.

21. The development system according to claim 1, in, In the module configuration unit, a resolution of a preview image displayed in the preview display unit is set according to a GUI operation of the user for the entire configured process line or for each configured process module.

22. The development system according to claim 1, in, The preview display unit displays the preview image in a rotated or reversed state according to a GUI operation by the user.

23. The development system according to claim 1, in, Regarding the processing module of the object selected from the module selection unit according to the GUI operation of the user, the module configuration unit determines the location in the configured process line where the processing module of the object can be configured and the location where the processing module of the object cannot be configured, and performs a predetermined display indicating that it can be configured at the configurable location and / or performs a predetermined display indicating that it cannot be configured at the non-configurable location.

24. The development system according to claim 23, in, In the relationship between the pre-processing module arranged at the front side and the post-processing module arranged at the rear side of the pre-processing module in the process line, whether the pre-processing module and the post-processing module can be connected is set as a condition, Regarding the processing module of the object, whether it is configurable or not is determined according to the condition.

25. The development system of claim 1, in, In the module configuration section, a predetermined display is performed for each of the processing modules constituting the configured flow line, indicating whether the required setting items of the processing module are set or not set.

26. The development system of claim 1, in, In the module configuration unit, when the processing of the configured flow line is executed, a predetermined display indicating normal, error-completed, or not executed is performed as the execution status for each configured processing module.

27. The development system of claim 1, in, In the module configuration unit, regarding the configured flow line, for each configured processing module, an output image ID of an output image and an input image ID of an input image are set and displayed according to a GUI operation of the user.

28. The development system according to claim 27, in, The output image ID of the processing module is automatically numbered and displayed. Regarding the input image ID of the processing module, the output image ID selected is set according to the GUI operation of the user from among the automatically numbered output image IDs.

29. The development system according to claim 27, in, The setting of the input image ID of the processing module is one of the necessary setting items of the processing module. In the module configuration section, a predetermined display is performed for each of the processing modules constituting the configured flow line, indicating whether the required setting items of the processing module are set or not set.

30. The development system of claim 1, in, In the module arrangement unit, when processing of the arranged flow line is executed, a first display state indicating a processing module being processed and a second display state indicating a processing module not being processed are switched in time series.

31. A development method, in a development system for supporting development of an image inspection system, in, The development system comprises a computer system having a processor and a memory. The computer system generates and provides a graphical user interface (GUI) screen for supporting the development to a user who performs the development. The steps executed by the computer system include: A step of displaying a module selection section configured with a plurality of processing modules as components for constituting a flow line of the image inspection system on the GUI screen, and selecting the processing module from the module selection section according to a GUI operation by the user; A module configuration section in which the process line being produced is displayed is displayed on the GUI screen, and a processing module selected from the module selection section is configured in the module configuration section according to the GUI operation of the user, and the configured processing modules are connected to each other with lines, thereby forming a step of the process line being produced; as well as A preview display unit is displayed in the GUI screen for displaying a preview image of the action status of the process line, and a preview image of the action status of the process line is displayed on the preview display unit according to the GUI operation of the user with respect to the process line configured in the production of the module configuration unit.

32. A development program for causing a development system supporting development of an image inspection system to execute processing, in, The development system comprises a computer system having a processor and a memory. The computer system generates and provides a graphical user interface (GUI) screen for supporting the development to a user who performs the development. The processing to be executed by the computer system includes: A module selection section is displayed on the GUI screen, in which a plurality of processing modules as components constituting a flow line of the image inspection system are configured, and a process of the processing module is selected from the module selection section according to a GUI operation by the user; A module configuration section in which the process line being created is displayed is displayed on the GUI screen, and a processing module selected from the module selection section is configured in the module configuration section according to the GUI operation of the user, and the configured processing modules are connected to each other with lines, thereby forming the processing of the process line being created; as well as A preview display unit is displayed in the GUI screen for displaying a preview image of the action status of the process line. Regarding the process line configured in the module configuration unit, a preview image of the action status of the process line is displayed on the preview display unit according to the user's GUI operation.

Images