Image analyzer based on OpenCV

Abstract

The invention discloses an OpenCV-based image analyzer, and relates to the technical field of image analysis, and the OpenCV-based image analyzer comprises an image acquisition and storage module which is used for acquiring and storing image data; the algorithm loading and recognition module is used for loading an image processing algorithm by using an OpenCV database according to task requirements to obtain algorithm execution parameters; the multi-thread processing module is used for performing multi-thread distribution on the image data to obtain an image processing task distribution queue; the algorithm execution and resource management module is used for executing an image processing algorithm according to the algorithm execution parameters and the image processing task allocation queue to obtain an initial image processing result; the processing result output module is used for acquiring an image processing result according to the initial image processing result; the result output module is used for outputting an image processing result. According to the method, image data can be efficiently processed, and the problems of blockage, jamming and time delay when a traditional image analyzer processes multiple images are avoided by utilizing a multi-thread parallel execution algorithm.

Description

Technical Field

[0001] The invention belongs to the technical field of image analysis, and in particular relates to an image analyzer based on OpenCV. Background Art

[0002] With the continuous development of science and technology, image analysis plays an increasingly important role in many fields. Traditional image analyzers have certain limitations in use. After the image is put into the traditional image analyzer, the result is directly obtained based on the calculation. However, when multiple images are put into the analyzer at the same time, blocking may occur. This is because when the analyzer processes multiple images, its processing capacity may not meet the needs of simultaneous processing, resulting in a delay in the image processing process.

[0003] At the same time, when traditional image analyzers process multiple images simultaneously, the output results may be delayed. This is a serious problem for some application scenarios with high real-time requirements. For example, in the fields of industrial automation detection, intelligent security, etc., the results of image analysis need to be fed back in a timely manner for subsequent decision-making and processing.

[0004] In addition, traditional image analyzers also have problems with loading and coordinating multiple algorithms at the same time. Different algorithms may require different resources and processing time. When multiple algorithms are loaded at the same time, resource conflicts and inefficiencies may occur. Moreover, the efficiency of different algorithms running simultaneously is difficult to guarantee, which affects the overall performance of image analysis.

[0005] Therefore, there is an urgent need for an image analyzer based on OpenCV to improve the efficiency and real-time performance of image analysis, realize multi-platform applications, and complete the unified design of the software to meet the needs of image analysis in different fields. Summary of the invention

[0006] The purpose of the present invention is to propose an image analyzer based on OpenCV to solve the problem of system resource scheduling and how to effectively manage and allocate limited resources in the system to ensure that multiple processes or threads can run concurrently and the system can work efficiently.

[0007] To achieve the above object, the present invention provides an image analyzer based on OpenCV, comprising: an image acquisition and storage module, an algorithm loading and recognition module, a multi-thread processing module, an algorithm execution and resource management module, a processing result output module and a result output module;

[0008] The image acquisition and storage module is used to acquire and save image data;

[0009] The algorithm loading and identification module is used to load the image processing algorithm using the OpenCV database to obtain algorithm execution parameters according to task requirements;

[0010] The multi-thread processing module is used to allocate and obtain an image processing task allocation queue using a target thread according to the image data;

[0011] The algorithm execution and resource management module is used to execute the image processing algorithm according to the algorithm execution parameters and the image processing task allocation queue to obtain the initial image processing result;

[0012] The processing result output module is used to obtain an image processing result according to the initial image processing result;

[0013] The result output module is used to output the image processing result.

[0014] Optionally, the image acquisition and storage module includes an image acquisition unit and an image storage unit;

[0015] The image acquisition unit is used to acquire the image data;

[0016] The image storage unit is used to save the image data to a storage device.

[0017] Optionally, using the OpenCV database to load the image processing algorithm and obtain the algorithm execution parameters according to the task requirements includes:

[0018] using the image data to obtain a processing requirement of the image data as the task requirement;

[0019] Acquire the image processing algorithm using the OpenCV database according to the task requirements;

[0020] The algorithm execution parameters are obtained using the image processing algorithm according to the image data.

[0021] Optionally, the algorithm loading and identification module includes an algorithm loading unit and an algorithm identification unit;

[0022] The algorithm loading unit is used to obtain the image processing algorithm using the OpenCV database according to the task requirements;

[0023] The algorithm identification unit is used to obtain algorithm execution parameters according to the image processing algorithm.

[0024] Optionally, allocating and acquiring an image processing task allocation queue using a target thread according to the image data includes:

[0025] obtaining workload characteristics of the image analyzer using the image analyzer;

[0026] Evaluating the scheduling algorithm according to the workload characteristics of the image analyzer to obtain a target scheduling algorithm;

[0027] Constructing a target thread using a multithread management mechanism according to the target scheduling algorithm;

[0028] Obtaining the priority of the target thread by performing priority allocation according to the target thread;

[0029] The image data is allocated using the target thread priority to obtain the image processing task allocation queue.

[0030] Optionally, the multi-thread processing module includes a thread creation unit and a task allocation unit;

[0031] The thread creation unit is used to create the target thread;

[0032] The task allocation unit is used for allocating the image data according to the target thread to obtain an image processing task allocation queue;

[0033] The target thread independently processes one or more image processing tasks.

[0034] Optionally, the algorithm execution and resource management module includes a resource acquisition unit and a parallel execution unit;

[0035] The resource acquisition unit is used to acquire algorithm resources by using a scheduler to perform resource scheduling according to the algorithm execution parameters and the image processing task allocation queue;

[0036] The parallel execution unit is used to allocate queues to the image processing tasks according to the algorithm resources to execute the image processing algorithm in parallel to obtain the initial image processing result.

[0037] Optionally, the processing result output module includes a processing result unit and a result integration unit;

[0038] The processing result unit is used to receive the initial image processing result;

[0039] The result integration unit is used to integrate the initial image processing result to obtain the image processing result.

[0040] Optionally, the image analyzer further includes seamlessly connecting the target thread by defining an interface specification.

[0041] Optionally, the image analyzer further includes a management service module and an external interaction module;

[0042] The management service module is used to manage the synchronously opened target threads and perform multi-threaded parallel processing to obtain the image processing results;

[0043] The external interaction module is used to realize the information interaction of the image analyzer. Compared with the closest prior art, the present invention has the following beneficial effects:

[0044] The present invention can process image data efficiently, utilize multi-threaded parallel execution algorithms, and ultimately obtain the image processing results required by users, thereby avoiding the blocking, freezing and delay problems of traditional image analyzers when processing multiple images; it realizes unified design of software and multi-platform application, and improves the flexibility, maintainability and scalability of the system through modular design and unified interface specifications; the present invention avoids deadlock and starvation problems through reasonable thread scheduling and resource allocation strategies, thereby improving the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 This is a structural diagram of an OpenCV-based image analyzer according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention and are not intended to limit the present invention.

[0049] like Figure 1 As shown, an embodiment of the present invention provides an image analyzer based on OpenCV, including: an image acquisition and storage module, an algorithm loading and recognition module, a multi-thread processing module, an algorithm execution and resource management module, a processing result output module and a result output module;

[0050] The image acquisition and storage module is used to acquire and save image data, wherein the image acquisition and storage module includes an image acquisition unit and an image storage unit. The image acquisition unit is used by the image analyzer to acquire image data from devices such as network cameras and MiPi, and the image storage unit is used to save the acquired image data to storage devices such as memory or disk for subsequent processing.

[0051] The algorithm loading and identification module is used to load the image processing algorithm using the OpenCV database to obtain the algorithm execution parameters according to the task requirements. Specifically: use the image data to obtain the image data processing requirements as the task requirements; use the OpenCV database to obtain the image processing algorithm according to the task requirements; use the image processing algorithm according to the image data to obtain the algorithm execution parameters, for example:

[0052] Example 1: In belt inspection for foreign objects, it is necessary to detect the presence of an object or identify its shape;

[0053] Algorithm: Binarization algorithm;

[0054] Requirement description: Check whether objects on the transport production line meet shape standards;

[0055] Corresponding algorithm: Otsu binarization algorithm;

[0056] How to use: The optimal threshold is automatically calculated through the Otsu binarization algorithm, and the grayscale image is converted into a binary image to highlight the outline of the target object, which is convenient for subsequent shape detection and analysis.

[0057] Example 2: Check the relative position of equipment while it is running. It is necessary to identify and extract the edges of the area of ​​interest (such as a belt conveyor, etc.);

[0058] Algorithm: Edge detection algorithm

[0059] Requirement description: Extract the edge of the belt in the monitoring belt conveyor image so that the staff can distinguish whether the belt is running abnormally;

[0060] Corresponding algorithm: Canny edge detection algorithm;

[0061] How to use: Use the Canny algorithm to detect the edge of the image and obtain accurate and continuous edge information, which is convenient for workers to determine the edge of the belt and check whether it is deviating.

[0062] Among them, the algorithm loading and identification module includes an algorithm loading unit and an algorithm identification unit. The algorithm loading unit is used to load the image processing algorithm obtained through the OpenCV database according to task requirements, that is, to obtain the image processing algorithm such as binarization and edge detection by using the OpenCV database according to task requirements. The algorithm identification unit is used to determine the type of algorithm to be applied and prepare the corresponding algorithm execution parameters, that is, to obtain the algorithm execution parameters according to the image processing algorithm, such as threshold setting, edge detection method, etc.

[0063] The multi-thread processing module is used to allocate and obtain an image processing task allocation queue using a target thread according to image data. The specific steps are: allocating and obtaining an image processing task allocation queue using a target thread according to image data includes: using an image analyzer to obtain the workload characteristics of the image analyzer; evaluating the scheduling algorithm according to the workload characteristics of the image analyzer to obtain a target scheduling algorithm; constructing a target thread using a multi-thread management mechanism according to the target scheduling algorithm; allocating a priority according to the target thread to obtain a target thread priority; and allocating image data using the target thread priority to obtain an image processing task allocation queue.

[0064] Among them, the multi-thread processing module includes a thread creation unit and a task allocation unit. The thread creation unit is used to create multiple threads in the image processing analyzer, and each thread independently processes one or more image processing tasks. The task allocation module is used to allocate the acquired image data to the image processing task queue of each thread to obtain the image processing task allocation queue.

[0065] The algorithm execution and resource management module is used to execute the image processing algorithm according to the algorithm execution parameters and the image processing task allocation queue to obtain the initial image processing result, wherein the algorithm execution and resource management module includes a resource acquisition unit and a parallel execution unit. The algorithm execution unit is used for each thread to obtain the required algorithm resources from the scheduler according to the task requirements, that is, to use the scheduler to perform resource scheduling to obtain the algorithm resources according to the algorithm execution parameters and the image processing task allocation queue. The algorithm resources include mutex locks or other synchronization mechanisms, and the exclusive access to the algorithm is controlled by mutex locks or other synchronization mechanisms. The parallel execution unit is used to execute the image processing algorithm in parallel on each thread, that is, to execute the image processing algorithm in parallel on the image processing task allocation queue according to the algorithm resources to obtain the initial image processing result, thereby ensuring mutual exclusive access to the resources corresponding to the algorithm to avoid conflicts.

[0066] The processing result output module is used to obtain the image processing result based on the initial image processing result, wherein the processing result output module includes a processing result unit and a result integration unit. The processing result unit is used to obtain processed image data or specific image processing results after each thread completes the algorithm execution, that is, to receive the image processing results after the algorithm execution and the resource management module are processed. The result integration unit is used to integrate or summarize the image processing results obtained by each thread to form the final image processing result.

[0067] The result output module is used to output the final processing results to the display, save them to files or transmit them to other systems. That is, the final processing results can be notified to other systems through the API, completing the entire image processing process and outputting the final image processing results.

[0068] This embodiment divides the image analysis task into multiple independent threads, each thread focuses on a specific image processing task, and improves the flexibility and maintainability of the image analyzer. According to the specific image processing requirements, different threads can be divided, such as binarization processing, Hough transform, edge detection, image classification module, etc. Each thread has its own interface and algorithm implementation, and can run independently and work with other threads.

[0069] Define a unified interface specification so that different threads can be seamlessly connected, reduce coupling, and improve the scalability and maintainability of the image analyzer. Define unified interface specifications for input parameters, output formats, function calls, etc. for each thread, such as unified function signatures, data formats, and error handling methods. This allows different threads to be interchanged and replaced without affecting other parts of the image analyzer.

[0070] Set up a management service module. The main purpose of the management service module is to manage synchronously opened threads and obtain image processing results at the same time. The management service module implements multi-threaded parallel processing, improves the processing speed and efficiency of the image analyzer, and ensures the synchronous operation of different threads. Adopt a multi-threaded management mechanism to dynamically create and destroy threads according to the load of the image analyzer, and then schedule them according to priority and resource allocation. You can use a thread pool to manage the life cycle of threads and control the number of threads and resource usage. Specifically:

[0071] (1) Select an appropriate scheduling algorithm;

[0072] Understand workload: Analyze the workload characteristics of the image analyzer, such as the operation of the CPU and NPU, the number of processes, the process running time, the I / O operation frequency, etc.

[0073] Evaluate scheduling algorithms: Consider the characteristics and applicable scenarios of different scheduling algorithms, such as First Come First Served (FCFS), Shortest Job First (SJF), Round Robin (RR), priority scheduling, etc.

[0074] Select algorithm: Select a suitable scheduling algorithm based on the image analyzer requirements, and then use the multi-thread management mechanism to create multiple threads in the image processing analyzer. For example, a real-time system may require priority scheduling, while a multi-tasking operating system may choose time slice rotation.

[0075] (2) Setting priorities;

[0076] Determine the priority strategy: Determine whether to use static priority or dynamic priority. Static priority is determined when the process is created and remains unchanged, while dynamic priority can be adjusted according to specific rules at runtime;

[0077] Assign priorities: Assign appropriate priorities to each process or thread, taking into account the importance and urgency of the process;

[0078] Priority adjustment: implement priority inheritance or priority promotion mechanism to avoid priority inversion problem;

[0079] (3) Monitoring and adjustment: Through reasonable resource scheduling, the overall performance of the system can be improved, such as reducing process waiting time and improving system throughput and response time;

[0080] Set up monitoring tools: Configure system monitoring tools to collect scheduling-related data, such as CPU utilization, process waiting time, context switching times, etc.

[0081] Analyze data: Regularly monitor the data to evaluate the effectiveness of the current scheduling strategy and identify performance bottlenecks and potential problems;

[0082] Adjustment strategy: According to the analysis results, timely adjust the scheduling strategy, such as changing the time slice size, adjusting the priority allocation strategy, switching the scheduling algorithm, etc.

[0083] (4) Avoid deadlock and starvation;

[0084] Deadlock prevention: Implement deadlock prevention strategies, such as resource ordering or banker's algorithm, to ensure that the system does not enter a deadlock state;

[0085] Deadlock detection and recovery: Deadlock detection algorithms are run periodically and once a deadlock is detected, recovery measures are taken, such as terminating the process or rolling back the operation;

[0086] Prevent starvation: Implement an aging strategy to gradually increase the priority of long-waiting processes to ensure that all processes can eventually obtain resources.

[0087] The external interaction module is used to realize the information interaction of the image analyzer, that is, to realize the communication with the external system or application, receive external requests and distribute the requests to the corresponding image processing module for processing, and return the processing results. Design a configuration management system that can dynamically load and configure different image processing modules from external configuration files or databases. You can use the configuration file to specify the module path, parameter configuration and other information, and dynamically load and initialize these modules at runtime.

[0088] The image analyzer based on OpenCV is mainly used for embedded platforms. Different embedded platforms can be associated with different internal interfaces. The system uses a unified API interface to complete the scheduling of different computing platforms through a unified interface. After defining the unified framework of the system, it is necessary to explain the embedded hardware resources and the software resources of the analyzer at the same time:

[0089] Design each algorithm that uses a mutex lock to associate with the hardware. If you want to call binarization or a certain algorithm, if you need multiple algorithms, bind the lock to multiple algorithms.

[0090] Then create an image processing task class. Each instance of the class represents an image processing task. Define the methods for executing image processing tasks in the class, such as combining binarization and edge calculation to find the process of straight line edges in an image.

[0091] Create a scheduler class to manage and schedule the execution of image processing tasks. At the same time, in the scheduler, maintain a list of mutexes, each mutex corresponds to an algorithm, and then the scheduler implements the scheduling logic of image processing tasks, including obtaining different algorithms, releasing algorithms that have completed calculations, and loading algorithms that are waiting to be executed.

[0092] When entering the analyzer, multiple threads are created, each thread represents a corresponding image processing task. When each thread is executed, the method of the scheduler class is called to obtain and release the algorithm loading computing resources and manage the execution of the image processing task.

[0093] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An image analyzer based on OpenCV, characterized in that: include: Image acquisition and storage module, algorithm loading and recognition module, multi-threaded processing module, algorithm execution and resource management module, processing result output module and result output module; The image acquisition and storage module is used to acquire and save image data; The algorithm loading and identification module is used to load the image processing algorithm using the OpenCV database to obtain algorithm execution parameters according to task requirements; The multi-thread processing module is used to allocate and obtain an image processing task allocation queue using a target thread according to the image data; The algorithm execution and resource management module is used to execute the image processing algorithm according to the algorithm execution parameters and the image processing task allocation queue to obtain the initial image processing result; The processing result output module is used to obtain an image processing result according to the initial image processing result; The result output module is used to output the image processing result.

2. The OpenCV-based image analyzer according to claim 1, characterized in that: The image acquisition and storage module includes an image acquisition unit and an image storage unit; The image acquisition unit is used to acquire the image data; The image storage unit is used to save the image data to a storage device.

3. The OpenCV-based image analyzer according to claim 1, characterized in that: According to the task requirements, using the OpenCV database to load the image processing algorithm to obtain the algorithm execution parameters includes: using the image data to obtain a processing requirement of the image data as the task requirement; Acquire the image processing algorithm using the OpenCV database according to the task requirements; The algorithm execution parameters are obtained using the image processing algorithm according to the image data.

4. The OpenCV-based image analyzer according to claim 3, characterized in that: The algorithm loading and identification module includes an algorithm loading unit and an algorithm identification unit; The algorithm loading unit is used to obtain the image processing algorithm using the OpenCV database according to the task requirements; The algorithm identification unit is used to obtain algorithm execution parameters according to the image processing algorithm.

5. The OpenCV-based image analyzer according to claim 1, characterized in that: The image processing task allocation queue is obtained by using the target thread to allocate the image data, including: obtaining workload characteristics of the image analyzer using the image analyzer; Evaluating the scheduling algorithm according to the workload characteristics of the image analyzer to obtain a target scheduling algorithm; Constructing a target thread using a multithread management mechanism according to the target scheduling algorithm; Obtaining the priority of the target thread by performing priority allocation according to the target thread; The image data is allocated using the target thread priority to obtain the image processing task allocation queue.

6. The OpenCV-based image analyzer according to claim 5, characterized in that: The multi-thread processing module includes a thread creation unit and a task allocation unit; The thread creation unit is used to create the target thread; The task allocation unit is used for allocating the image data according to the target thread to obtain an image processing task allocation queue; The target thread independently processes one or more image processing tasks.

7. The OpenCV-based image analyzer according to claim 1, characterized in that: The algorithm execution and resource management module includes a resource acquisition unit and a parallel execution unit; The resource acquisition unit is used to acquire algorithm resources by using a scheduler to perform resource scheduling according to the algorithm execution parameters and the image processing task allocation queue; The parallel execution unit is used to allocate queues to the image processing tasks according to the algorithm resources to execute the image processing algorithm in parallel to obtain the initial image processing result.

8. The OpenCV-based image analyzer according to claim 1, characterized in that: The processing result output module includes a processing result unit and a result integration unit; The processing result unit is used to receive the initial image processing result; The result integration unit is used to integrate the initial image processing result to obtain the image processing result.

9. The OpenCV-based image analyzer according to claim 6, characterized in that: The image analyzer also includes seamlessly connecting the target thread by defining an interface specification.

10. The OpenCV-based image analyzer according to claim 9, characterized in that: The image analyzer also includes a management service module and an external interaction module; The management service module is used to manage the synchronously opened target threads and perform multi-threaded parallel processing to obtain the image processing results; The external interaction module is used to realize the information interaction of the image analyzer.

Images