Computer vision processing method and computer vision processing system

Abstract

The present invention provides a computer vision processing method and system.SOLUTION: The computer vision process processing method includes: acquiring first image data; and determining one or more corresponding computer vision process events according to the first image data and one or more computer vision automation processes, wherein the one or more computer vision automation processes are respectively implemented by one or more computer vision automation process simulation components.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a computer vision processing method and a computer vision processing system, and more particularly to a low-cost computer vision processing method and a computer vision processing system. [Background technology]

[0002] Improving work efficiency through automation processes is currently an important need for most manufacturers. However, building an automation process requires significant research and time costs, which cannot meet the time and cost requirements of users. At present, automation processes that can achieve monitoring tasks using computer vision are not widely available. Therefore, developing computer vision processing methods and systems is a significant challenge for the industry. Summary of the Invention [Problem to be solved by the invention]

[0003] The technical problem that the present invention aims to solve is to provide a computer vision processing method and a computer vision processing system in response to the shortcomings of the prior art. [Means for solving the problem]

[0004] The computer vision processing method includes acquiring first image data and determining corresponding one or more computer vision process events based on the first image data and one or more computer vision automation processes, wherein the one or more computer vision automation processes are respectively achieved by one or more computer vision automation process simulation elements.

[0005] The present invention further provides a computer vision processing system, the computer vision processing system including a control circuit and a memory circuit, the memory circuit having a computer vision processing program installed therein, the computer vision processing program being configured to acquire first image data through the control circuit, the control circuit determining one or more corresponding computer vision processing events according to the first image data and one or more computer vision automation processes, each of which is achieved by one or more computer vision automation process simulation elements. [Effects of the Invention]

[0006] One of the beneficial effects of the present invention is that the computer vision processing method and computer vision processing system provided by the present invention can quickly and effectively help users complete the simulation and editing of various computer vision process events, which are complex and require a lot of time and effort, and can accurately execute various user operation processes, effectively reducing human and time costs, and also significantly reducing the cost and development time of building process automation. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a flowchart of a computer vision processing method according to a first embodiment of the present invention. [Figure 2] FIG. 10 is a schematic diagram of a computer vision processing system according to a second embodiment of the present invention. [Figure 3] FIG. 10 is another schematic diagram of the computer vision processing system of the second embodiment of the present invention. [Figure 4] FIG. 10 is a schematic diagram illustrating a computer vision processing system according to a second embodiment of the present invention constructing multiple automatic operation processes corresponding to operation traces. DETAILED DESCRIPTION OF THE INVENTION

[0008] In order to better understand the features and technical contents of the present invention, reference is made to the following detailed description and drawings of the present invention, but the drawings provided are for reference and explanation only and are not intended to limit the present invention.

[0009] The following describes the implementation of the computer vision processing method and computer vision processing system disclosed in the present invention through specific examples, and those skilled in the art can understand the advantages and benefits of the present invention from the content disclosed herein. The present invention can be implemented or applied through other different specific examples, and various modifications and changes can be made to the details herein based on different perspectives and applications without departing from the concept of the present invention. It should be noted in advance that the accompanying drawings are for simple schematic illustrations only and are not drawn to scale. The technical content of the present invention will be described in more detail based on the following examples, but the disclosed content does not limit the scope of protection of the present invention. Furthermore, the term "or" used in this specification may include any one or more combinations of the related listed items depending on the actual situation. [Example]

[0010] [First Example] Please refer to FIG. 1, which is a flowchart of a computer vision processing method in a first embodiment of the present invention.

[0011] In this embodiment, a computer vision processing method is provided, which includes the following steps: Step S110: First image data is acquired. Step S120: Determine one or more corresponding computer vision process events based on the first video data and one or more computer vision automation processes. Step S130: Execute one or more computer vision automation steps.

[0012] In step S110, first, first image data (e.g., image data acquired in the first computer vision automation step AFL1) is acquired. The first image data can be acquired through different acquisition routes (e.g., a camera, a monitor, an image acquisition module of a mobile phone or tablet device, or a screen capture circuit on a computer device).

[0013] Next, in step S120, the first image data M1 is linked with one or more computer vision automation processes AFL1-AFLN, where each of the one or more computer vision automation processes AFL1-AFLN is implemented by one or more computer vision automation process simulation elements SB1-SBN. In this embodiment, the one or more computer vision process events may be implemented by obtaining an operation trace of a user operating a specific action, or by linking multiple corresponding computer vision automation processes AFL1-AFLN.

[0014] The corresponding computer vision automation processes AFL1 to AFLN are determined by one or more computer vision automation process simulation elements SB1 to SBN.

[0015] After the corresponding computer vision automation steps AFL1 to AFLN are obtained, the corresponding computer vision automation steps AFL1 to AFLN are executed to check whether the result is the same as the recorded or acquired user operation process.

[0016] Here, one or more computer vision automation process simulation elements SB1-SBN execute a mouse click operation process, a keyboard operation process, or a plurality of automatic operation processes of software executed by a mouse or keyboard on an electronic device (not shown) through a computer vision identification program. The computer vision automation process simulation elements SB1-SBN may be adjusted according to actual needs and are not particularly limited in the present invention. Furthermore, the plurality of computer vision automation process simulation elements SB1-SBN may include an automatic input element, a mouse movement element, a plurality of mouse click elements, a plurality of keyboard signal providing elements, or a software click element.

[0017] That is, the computer vision processing method of this embodiment simulates various operation processes of a user operating an electronic device (not shown). In this embodiment, various operation processes of a user operating an electronic device (not shown) are recorded or learned to simulate the corresponding content. In this embodiment, the multiple automatic operation processes operate based on multiple peripheral devices, multiple software, multiple firmware, or multiple hardware installed in the electronic device (not shown) under the condition that the software link or program code link of the electronic device (not shown) is not modified. The multiple peripheral devices connected to the electronic device (not shown) include a mouse or a keyboard. The peripheral devices may also include a stylus, a pen tablet, or a wearable electronic device.

[0018] In this embodiment, the multiple computer vision automation processes AFL1-AFLN operate based on multiple peripheral devices, software, firmware, or hardware installed in the electronic device ED1 without modifying the software / hardware environment of the electronic device ED1 (including software configuration or program code linking). The various operation processes of the multiple computer vision automation processes AFL1-AFLN represent the operation of various engineering software and communication software, network searches, form filling, etc. on the electronic device ED1 from a user's perspective. That is, in this embodiment, the multiple computer vision automation processes AFL1-AFLN simulate the process of multiple users modifying documents and operating the electronic device ED1 using a computer vision recognition program.

[0019] In another embodiment, a user can directly use one or more computer vision automation process simulation elements SB1-SBN to directly edit simple computer vision process events.

[0020] [Second Example] Referring to Figures 2, 3 and 4, Figure 2 is a schematic diagram of a computer vision processing system according to a second embodiment of the present invention. Figure 3 is another schematic diagram of a computer vision processing system according to the second embodiment of the present invention. Figure 4 is a schematic diagram of the computer vision processing system according to the second embodiment of the present invention constructing a plurality of computer vision process events corresponding to an operation trace.

[0021] In this embodiment, a computer vision processing system SYS1 is provided. The computer vision processing system SYS1 includes at least an electronic device ED1. The electronic device ED1 includes a control circuit CR1, a memory circuit SR1, and a communication circuit CM1. The electronic device ED1 is connected to a mouse ME1 and a keyboard KY1. The electronic device ED1 is further connected to a display device DP1.

[0022] The storage circuit SR1 of the electronic device ED1 is provided with a computer vision process processing program AMAPP1. The storage circuit SR1 of the electronic device ED1 further includes a database DB1. The database DB1 includes a plurality of computer vision automation steps AFL1 to AFLN.

[0023] Each computer vision automation process is performed by a computer vision process processing program AMAPP1 installed in the electronic device ED1 without requiring any manual operation by the user. The computer vision process processing program AMAPP1 is stored in the storage circuit SR1. The computer vision process processing program AMAPP1 can edit multiple automatic operation processes using a computer vision identification program, and the content links of the computer vision automation processes AFL1 to AFL6 shown in Figure 3 perform the computer vision automation processes AFL2 to AFL3 and the computer vision automation processes AFL5 to AFL6, respectively, when a change in the pointer (first computer vision automation process AFL1) or a change in the optical signal (fourth computer vision automation process AFL4) is detected.

[0024] The computer vision processing program AMAPP1 determines, through the control circuit CR1, each of the computer vision automation processes AFL11-AFL17 based on the first image data M1 and one or more computer vision automation process simulation elements SB1-SBN (e.g., the first computer vision automation process simulation element SB1). The control circuit CR1 supports the execution of the plurality of computer vision automation processes AFL11-AFL17. The first computer vision automation process AFL11, the second computer vision automation process AFL12, the third computer vision automation process AFL13, the fourth computer vision automation process AFL14, the fifth computer vision automation process AFL15, the sixth computer vision automation process AFL16, and the seventh computer vision automation process AFL17 of the plurality of computer vision automation processes AFL11-AFL17 can be achieved by one of the computer vision automation process simulation elements SB1-SBN, respectively, and are stored in the memory circuit SR1. The first computer vision automation process AFL11, the second computer vision automation process AFL12, the third computer vision automation process AFL13, the fourth computer vision automation process AFL14, the fifth computer vision automation process AFL15, the sixth computer vision automation process AFL16, and the seventh computer vision automation process AFL17 may be achieved by the same computer vision automation process simulation elements SB1 to SBN.

[0025] One or more computer vision automation process simulation elements SB1-SBN use a computer vision identification program to execute multiple automatic operation processes of the electronic device ED1, such as mouse click operation processes, keyboard operation processes, or software executed by a mouse or keyboard. In this embodiment, the computer vision identification program is provided in the computer vision process processing program AMAPP1. The computer vision process processing program AMAPP1 uses the computer vision identification program to complete various computer vision automation processes AFL11-AFL17. In addition, the computer vision process processing program AMAPP1 may also be provided in a server (not shown). After the multiple computer vision automation processes AFL11-AFL17 are constructed, the multiple computer vision automation processes AFL11-AFL17 can be stored in a database (not shown) and downloaded by multiple users.

[0026] The plurality of computer vision automation process simulation elements SB1 to SBN include an automatic input element, a mouse movement element, or a plurality of keyboard signal providing elements.

[0027] As shown in the previous embodiment, the computer vision processing program AMAPP1 uses the computer vision identification program to simulate and edit the corresponding computer vision automation steps AFL1-AFLN through the user's operation procedures of the electronic device ED1, the mouse ME1, the keyboard KY1, the touch panel (not shown), the communication software, various engineering software, and various commercial software, and then stores the edited computer vision automation steps AFL1-AFLN in the memory circuit SR1.

[0028] In this embodiment, the computer vision automation processes AFL11-AFL17 are operated by peripheral devices, software, firmware, or hardware installed in the electronic device ED1 in an environment where the software and hardware of the electronic device ED1 are not modified (including the software deployment environment and related content or program code link). The various operation processes of the computer vision automation processes AFL11-AFL17 are, from a user's perspective, the operation of various engineering software and communication software on the electronic device ED1, network searches, form filling, etc. That is, in this embodiment, the computer vision automation processes AFL11-AFL17 simulate the process of multiple users modifying documents or operating the electronic device ED1 using a computer vision recognition program. Furthermore, in this embodiment, the computer vision automation processes AFL11-AFL17 are multiple processes derived from the first image data M1. That is, the computer vision processing system SYS1 in this embodiment not only implements a single thread, but also makes conditional decisions and accurately completes various computer vision processing events.

[0029] In addition, the computer vision processing system of this embodiment can be used to simulate and edit operation processes in which a user must repeat a large number of identical actions, to simulate and edit user actions in various devices that are tracked over a long period of time, to simulate and edit how a user constructs and selects elements or objects in engineering software, or to perform corresponding operation processes based on document instructions. [Industrial Applicability]

[0030] [Beneficial Effects of Examples] One of the beneficial effects of the present invention is that the computer vision processing method and computer vision processing system provided by the present invention can quickly and effectively use video data to help users complete the simulation and editing of various computer vision automation processes, which are complex and require a lot of time and effort, and can accurately execute various user operation processes, effectively reducing human and time costs, and significantly reducing the construction costs and development time of computer vision automation processes.

[0031] The above disclosure is merely a preferred embodiment of the present invention, and the scope of the claims of the present invention is not limited thereto. Therefore, all equivalent technical modifications made by utilizing the contents of the specification and drawings of the present invention are included in the scope of the claims of the present invention. [Explanation of symbols]

[0032] S110-S130 process CR1 control circuit ED1 Electronic equipment SR1 memory circuit DB1 database CM1 communication circuit ME1 mice KY1 Keyboard DP1 display device M1 First video data AFL1~AFLN, AFL11~AFL17 Computer vision automation process AMAPP1 Computer Vision Processing Program AFL1, AFL11 First computer vision automation process AFL4, AFL14 4th computer vision automation process AFL2, AFL12 Secondary computer vision automation process AFL3, AFL13 Third computer vision automation process AFL5, AFL15 5th Computer Vision Automation Process AFL6, AFL16 6th computer vision automation process AFL17 7th Computer Vision Automation Process SB1~SBN Computer Vision Automation Process Simulation Elements

Claims

1. acquiring first video data; determining one or more corresponding computer vision process events based on the first video data and one or more computer vision automation processes; The computer vision processing method, wherein the one or more computer vision automation steps are each accomplished by one or more computer vision automation step simulation elements.

2. The computer vision processing method of claim 1 , further comprising performing the one or more computer vision automation steps.

3. The computer vision process processing method of claim 2, wherein the one or more computer vision automation process simulation elements use a computer vision identification program to execute a mouse click operation process, a keyboard operation process, or a plurality of automatic operation processes of software executed by a mouse or keyboard in an electronic device.

4. 4. The computer vision process processing method of claim 3, wherein the one or more computer vision automation process simulation elements include an automatic input element, a mouse movement element, a plurality of mouse click elements, a plurality of keyboard signal providing elements, or a software click element.

5. The computer vision processing method of claim 4, wherein the one or more computer vision process events are operated by multiple peripheral devices, multiple software, multiple firmware or multiple hardware installed in the electronic device when the software link or program code link of the electronic device is not modified, and the multiple peripheral devices include a mouse or a keyboard.

6. a control circuit; a memory circuit having a computer vision processing program stored therein; the computer vision processing program causes the control circuit to acquire first image data; the control circuitry determines one or more corresponding computer vision process events according to the first image data and one or more computer vision automation processes; A computer vision processing system, wherein the one or more computer vision automation processes are each accomplished by one or more computer vision automation process simulation elements.

7. 7. The computer vision processing system of claim 6, wherein the control circuitry supports execution of the corresponding one or more computer vision process events, and the one or more computer vision process events are stored in the memory circuitry.

8. The computer vision process processing system of claim 7, wherein the one or more computer vision automation process simulation elements in the one or more computer vision automation processes execute a mouse click operation process, a keyboard operation process, or a plurality of automatic operation processes of software executed by a mouse or keyboard of an electronic device through a computer vision identification program.

9. The computer vision process processing system of claim 8, wherein the one or more computer vision automation process simulation elements include an automatic input element, a mouse movement element, or multiple keyboard signal providing elements, and the one or more computer vision process events are operated by multiple peripheral devices, multiple software, multiple firmware, or multiple hardware installed in the electronic device when the software link or program code link of the electronic device is not modified, and the multiple peripheral devices include a mouse or a keyboard.

Images