Computer vision process processing methods and systems

Abstract

This invention discloses a computer vision workflow processing method and system. The computer vision workflow processing method includes: acquiring a first image information; and determining one or more corresponding computer vision workflow events based on the first image information and one or more computer vision automation steps; wherein the one or more computer vision automation steps are respectively completed using one or more computer vision automation step simulation elements.

Description

[Technical Field]

[0001] This invention relates to a computer vision workflow processing method and system, and in particular to a low-cost computer vision workflow processing method and system. [Previous Technology]

[0002] Improving work efficiency through automated processes is a crucial need for most manufacturers. However, implementing automated processes requires significant R&D and time costs, which cannot meet users' time and cost requirements. Currently, automated processes using computer vision to complete monitoring tasks are not widespread. Therefore, computer vision process handling methods and systems are indeed an important topic in the industry. [Summary of the Invention]

[0003] The technical problem to be solved by the present invention is to provide a computer vision process processing method in view of the shortcomings of the prior art, including: acquiring a first image information; and determining a corresponding one or more computer vision process events based on the first image information and one or more computer vision automation steps; wherein the one or more computer vision automation steps are respectively completed by one or more computer vision automation step simulation elements.

[0004] The present invention also discloses a computer vision workflow processing system, comprising: a control circuit; and a storage circuit, wherein the storage circuit is configured with a computer vision workflow processing program; wherein the computer vision workflow processing program acquires a first image information through the control circuit; wherein the control circuit determines one or more corresponding computer vision workflow events based on the first image information and one or more computer vision automation steps; wherein the one or more computer vision automation steps are respectively completed using one or more computer vision automation step simulation elements.

[0005] One of the beneficial effects of the present invention is that the process automation editing method and process automation editing system provided by the present invention can quickly and effectively assist users in simulating and editing various types of computer vision process events that are complex and require a lot of time and effort. They can accurately execute various user operation processes, effectively save manpower and time costs, and save a lot of process automation setup costs and development time.

[0006] To further understand the features and technical content of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are for reference and illustration only and are not intended to limit the present invention.

Implementation Method

[0011] The following specific embodiments illustrate the implementation of the "automatic document adjustment method and automatic document adjustment system" disclosed in this invention. Those skilled in the art can understand the advantages and effects of this invention from the content disclosed in this specification. This invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of this invention. Furthermore, the accompanying drawings of this invention are for simple illustration only and are not depictions of actual dimensions, as stated in advance. The following embodiments will further describe the relevant technical content of this invention in detail, but the disclosed content is not intended to limit the scope of protection of this invention. In addition, the term "or" used herein should, depending on the actual situation, may include any combination of one or more of the associated listed items.

[0012] [First Embodiment]

[0013] Please refer to Figure 1, which is a flowchart of the computer vision process processing method according to the first embodiment of the present invention.

[0014] This embodiment provides a computer vision workflow processing method. The computer vision workflow processing method includes the following steps.

[0015] Step S110: Obtain a first image information.

[0016] Step S120: Based on the first image information and one or more computer vision automation steps, determine one or more corresponding computer vision process events.

[0017] Step S130: Perform one or more computer vision automation steps.

[0018] In step S110, the first step is to obtain first image information (such as the image information obtained in the first computer vision automation step AFL1). The first image information can be obtained through different acquisition channels, such as the image capturing module of a camera, monitor, mobile phone or tablet, or the screen image acquisition circuit on a computer device.

[0019] Next, in step S120, the first image information M1 is linked to one or more computer vision automation steps SB1-SBN. The one or more computer vision automation steps AFL1-AFLN are each completed using one or more computer vision automation step simulation elements SB1-SBN. In this embodiment, one or more computer vision process events can be achieved by capturing the operation trajectory of a specific user action, or by linking multiple corresponding computer vision automation steps AFL1-AFLN.

[0020] The corresponding computer vision automation steps AFL1-AFLN are determined by using one or more computer vision automation step simulation elements SB1-SBN.

[0021] After obtaining the corresponding computer vision automation steps AFL1-AFLN, the corresponding computer vision automation steps AFL1-AFLN are executed to confirm whether the results are the same as those of the user's operation process recorded or captured.

[0022] Wherein, one or more computer vision automation step simulation elements SB1-SBN execute a mouse click operation, a keyboard operation, or multiple automated operation processes of software executed by a mouse or keyboard on an electronic device (not shown) through a computer vision recognition program. The computer vision automation step simulation element SB1-SBN can be adjusted according to actual needs and is not limited in this invention. Furthermore, the multiple process simulation elements SB1-SBN include an automatic input element, a mouse movement element, multiple mouse click elements, multiple keyboard signal providing elements, or a software click element.

[0023] That is, the computer vision process processing method of this embodiment simulates various operation processes of a user operating an electronic device (not shown). In this embodiment, the simulation content is performed by recording or learning various operation processes of the user operating the electronic device (not shown). In this embodiment, multiple automated operation processes are performed based on multiple peripheral devices, multiple software, multiple firmware, or multiple hardware already set in the electronic device (not shown) without changing the software or code links of the electronic device (not shown). The multiple peripheral devices connected to the electronic device (not shown) include a mouse or a keyboard. The peripheral devices may also include a stylus, a drawing tablet, or a wearable electronic device.

[0024] In this embodiment, the multiple computer vision automation steps AFL1-AFLN are performed without altering the hardware and software environment of the electronic device ED1 (including software settings or code links), based on multiple peripheral devices, software, firmware, or hardware already installed in the electronic device ED1. The various operation processes of the multiple computer vision automation steps AFL1-AFLN are performed from the perspective of a user operating various engineering software, communication software, conducting online queries, filling out forms, etc., on the electronic device ED1. That is, in this embodiment, the multiple computer vision automation steps AFL1-AFLN simulate the process of multiple users modifying documents and operating the electronic device ED1 through a computer vision recognition program.

[0025] In other embodiments, the user can directly use one or more computer vision automation step simulation elements SB1-SBN to directly edit simple computer vision process events.

[0026] [Second Embodiment]

[0027] Please refer to Figures 2, 3, and 4. Figure 2 is a schematic diagram of the computer vision process processing system according to the second embodiment of the present invention. Figure 3 is another schematic diagram of the computer vision process processing system according to the second embodiment of the present invention. Figure 4 is a schematic diagram of the computer vision process processing system according to the second embodiment of the present invention establishing multiple computer vision process events corresponding to operation trajectories.

[0028] In this embodiment, a computer vision workflow processing system SYS1 is provided. The computer vision workflow processing system SYS1 includes at least an electronic device ED1. The electronic device ED1 includes a control circuit CR1, a storage circuit SR1, and a communication circuit CM1. The electronic device ED1 is connected to a mouse ME1 and a keyboard KY1. The electronic device ED1 is also connected to a display device DP1.

[0029] The storage circuit SR1 of the electronic device ED1 is configured with a computer vision workflow processing program AMAPP1. The storage circuit SR1 of the electronic device ED1 also includes a database DB1. The database DB1 includes multiple computer vision automation steps AFL1-AFLN.

[0030] Each computer vision automation step does not require any user intervention; instead, it is processed by the computer vision process processing program AMAPP1, which is installed in the electronic device ED1. The storage circuit SR1 stores the computer vision process processing program AMAPP1. AMAPP1 can edit multiple automated operation processes through computer vision recognition programs. As shown in Figure 3, the linked content of computer vision automation steps AFL1-AFL6 means that when a pointer change (first computer vision automation step AFL1) or a light change (fourth computer vision automation step AFL4) is detected, computer vision automation steps AFL2-AFL3 and AFL5-AFL6 will be performed respectively.

[0031] The computer vision workflow processing program AMAPP1, through the control circuit CR1, determines each computer vision automation step AFL11-AFL17 based on each computer vision automation step AFL11-AFL17 and the first image information M1, using one or more computer vision automation step simulation elements SB1-SBN (e.g., the first computer vision automation step simulation element SB1). The control circuit CR1 assists in executing multiple computer vision automation steps AFL11-AFL17. Among the multiple computer vision automation steps AFL11-AFL17, the first computer vision automation step AFL11, the second computer vision automation step AFL12, the third computer vision automation step AFL13, the fourth computer vision automation step AFL14, and the fifth computer vision automation step AFL15 are... Automation steps AFL14, AFL15, AFL16, and AFL17 can be achieved using one of the computer vision automation step simulation elements SB1-SBN, respectively, and stored in the storage circuit SR1. Alternatively, AFL11, AFL12, AFL13, AFL14, AFL15, AFL16, and AFL17 can also be achieved using the same computer vision automation step simulation element SB1-SBN.

[0032] One or more computer vision automation step simulation elements SB1-SBN execute a mouse click operation, a keyboard operation, or a software automation operation process executed by a mouse or keyboard on the electronic device ED1 through a computer vision recognition program. In this embodiment, the computer vision recognition program is set in the process automation editing program AMAPP1. The computer vision process processing program AMAPP1 uses the computer vision recognition program to complete various computer vision automation steps AFL11-AFL17. In addition, the computer vision process processing program AMAPP1 can also be set in a server (not shown), and after multiple computer vision automation steps AFL11-AFL17 are established, the multiple computer vision automation steps AFL11-AFL17 are stored in a database (not shown) so that multiple users can download them.

[0033] Multiple computer vision automation step simulation elements SB1-SBN include an automatic input element, a mouse movement element, or multiple keyboard signal providing elements.

[0034] The computer vision workflow processing program AMAPP1, as shown in the previous embodiment, can utilize a computer vision recognition program to simulate and edit corresponding computer vision automation steps AFL1-AFLN by using the user's operation process of operating the electronic device ED1, operating the mouse ME1, operating the keyboard KY1, operating the touch panel (not shown), communicating using communication software, and using various engineering software and commercial software. The edited computer vision automation steps AFL1-AFLN are then stored in the storage circuit SR1.

[0035] In this embodiment, the multiple computer vision automation steps AFL11-AFL17 operate without altering the hardware and software environment of the electronic device ED1 (including software settings or code links), based on multiple peripheral devices, software, firmware, or hardware already installed in the electronic device ED1. The various operation processes of the multiple computer vision automation steps AFL11-AFL17 are performed from a user's perspective, operating various engineering software, communication software, conducting online queries, filling out forms, etc., on the electronic device ED1. That is, in this embodiment, the multiple computer vision automation steps AFL11-AFL17 simulate the process of multiple users modifying documents and operating the electronic device ED1 through a computer vision recognition program. Furthermore, in this embodiment, the computer vision automation steps AFL11-AFL17 are multiple processes derived from the first image information M1. That is, the computer vision process processing system SYS1 in this embodiment is not limited to single-line programs, but can accurately complete various computer vision process events based on conditional judgments.

[0036] In addition, the computer vision process processing system of this embodiment can simulate and edit operation processes that require users to repeat a large number of identical actions, can simulate and edit user actions that detect various devices for a long time, can simulate and edit how users create and select components or objects in engineering software, or can execute corresponding operation processes according to an instruction file.

[0037] [Beneficial Effects of the Embodiments]

[0038] One of the beneficial effects of the present invention is that the computer vision process processing method and computer vision process processing system provided by the present invention can quickly and effectively use image information to assist users in completing the simulation and editing of various types of computer vision automation steps that are complex and require a lot of time and effort. They can accurately execute various user operation processes, effectively save manpower and time costs, and save a lot of computer vision automation step construction costs and development time.

[0039] The content disclosed above is only a preferred and feasible embodiment of the present invention, and is not intended to limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the scope of the patent application of the present invention. [Simplified Explanation of the Diagram]

[0007] Figure 1 is a flowchart of the computer vision process processing method according to the first embodiment of the present invention.

[0008] Figure 2 is a schematic diagram of the computer vision process processing system according to the second embodiment of the present invention.

[0009] Figure 3 is another schematic diagram of the computer vision process processing system according to the second embodiment of the present invention.

[0010] Figure 4 is a schematic diagram of the computer vision process processing system of the second embodiment of the present invention establishing multiple automated operation processes corresponding to operation trajectories.

Claims

1. A computer vision workflow processing method, applicable to multiple computer vision workflow events corresponding to a device, the computer vision workflow processing method comprising: A computer vision workflow processing system is used to acquire first image information of a device. The computer vision process processing system determines one or more computer vision process events corresponding to an electronic device based on the first image information of the device and one or more computer vision automation steps. The first image information is different from the one or more computer vision process events corresponding to the electronic device. The one or more computer vision automation steps are completed using one or more computer vision automation step simulation elements. The one or more computer vision process events are established by the computer vision process processing system and transmitted to the electronic device for execution. The order and content of the one or more computer vision process events corresponding to the first image information are adjustable. The one or more computer vision process events corresponding to the first image information can be established in multiple lines. The computer vision process processing system establishes multiple sets of the one or more computer vision automation steps based on multiple conditions to complete different tasks. The electronic device executing the computer vision process processing system is a different electronic device from the device. The electronic device executing the computer vision process processing system can independently execute multiple sets of the one or more computer vision automation steps.

2. The computer vision workflow processing method as described in claim 1 further includes: Perform one or more computer vision automation steps.

3. The computer vision workflow processing method as described in claim 2, wherein, The one or more computer vision automation step simulation elements are executed by a computer vision recognition program, which executes a mouse click operation, a keyboard operation, or a computer vision automation step of software through a mouse or keyboard on the electronic device.

4. The computer vision workflow processing method as described in claim 3, wherein, The multiple computer vision automation step elements include an automatic input element, a mouse movement element, multiple mouse click elements, multiple keyboard signal providing elements, or a software click element.

5. The computer vision workflow processing method as described in claim 4, wherein, The multiple computer vision automation steps are performed without changing the software or code connections of the electronic device, based on multiple peripheral devices, multiple software, multiple firmware, or multiple hardware already installed in the electronic device, wherein the multiple peripheral devices include a mouse or a keyboard.

6. A computer vision workflow processing system, applicable to multiple computer vision workflow events corresponding to a device, the computer vision workflow processing system comprising: One control circuit; The system includes a storage circuit with a computer vision workflow processing program. The computer vision workflow processing program acquires first image information from the device via the control circuit. The control circuit determines one or more computer vision workflow events corresponding to an electronic device based on the first image information and one or more computer vision automation steps. The one or more computer vision automation steps are completed using one or more computer vision automation step simulation elements. The one or more computer vision workflow events are established by the computer vision workflow processing system and transmitted to the electronic device for execution. The order and content of the one or more computer vision workflow events corresponding to the first image information are adjustable. The one or more computer vision workflow events corresponding to the first image information can be established in multiple lines. The computer vision workflow processing system establishes multiple sets of the one or more computer vision automation steps based on multiple conditions to complete different tasks. The electronic device executing the computer vision workflow processing system is a different electronic device from the device, and the electronic device executing the computer vision workflow processing system can independently execute multiple sets of the one or more computer vision automation steps.

7. The computer vision workflow processing system as described in claim 6, wherein, The control circuit assists in executing the corresponding one or more computer vision automation steps, which are stored in the storage circuit.

8. The computer vision workflow processing system as described in claim 7, wherein, The one or more computer vision automation steps are executed by a computer vision recognition program, which executes a mouse click operation, a keyboard operation, or a computer vision automation step of software via a mouse or keyboard on the electronic device.

9. The computer vision workflow processing system as described in claim 8, wherein, The multiple computer vision automation step simulation elements include an automatic input element, a mouse movement element, or multiple keyboard signal providing elements. The multiple computer vision automation steps are performed based on multiple peripheral devices, multiple software, multiple firmware, or multiple hardware that are already set in the electronic device without changing the software or code connections of the electronic device. The multiple peripheral devices include a mouse or a keyboard.

Images