Image processing method and related equipment

Abstract

The present invention provides an image processing method and related equipment applicable to an integrated chip. The method comprises: obtaining an image to be processed, the image including at least one connected domain; detecting attribute information of the image, the attribute information including the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; assigning a search processing engine to each connected domain based on the image attribute information; and scheduling the search processing engines corresponding to each connected domain in parallel to perform contour search processing on each connected domain. The present invention can improve the efficiency of image contour search.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, specifically to an image processing method and related equipment, and in particular to an image processing method, an image processing device, an integrated chip, an image processing device, a computer-readable storage medium and a computer program product. Background Art

[0002] With the development of image processing technology, image contour search has been introduced in fields such as computer vision and image recognition. For example, in the field of license plate recognition, it is necessary to perform contour search on the image containing the license plate to obtain the contour of the license plate, and then recognize the license plate contour.

[0003] Currently, image contour search is mainly implemented based on raster scanning technology, such as Figure 1a A schematic diagram of image contour search provided by the prior art. Figure 1a In the image Y, there are four contours. The contour search process of the image Y includes: scanning from left to right and from top to bottom by raster scanning technology, and first scanning the first contour in the image (i.e. Figure 1a The contour shown by number 1 in the image), and then scan to obtain the second contour in the image (i.e. Figure 1a The contour shown by number 2 in the figure), and then continue scanning to obtain the third contour (i.e. Figure 1a The contour shown by number 3 in the figure is obtained by scanning the fourth contour (i.e. Figure 1a As shown in the contour of the image by the number 4 in the figure). As can be seen from the above, the process of searching for image contours by raster scanning technology is a serial and interdependent process, and the efficiency of contour search is relatively low. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method and related devices, which can improve the efficiency of image contour search.

[0005] In one aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire an image to be processed, wherein the image includes at least one connected domain;

[0007] Detecting attribute information of the image, including the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image;

[0008] Based on the attribute information of the image, a search processing engine is assigned to each connected domain;

[0009] The search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain.

[0010] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0011] An acquisition unit, configured to acquire an image to be processed, wherein the image includes at least one connected domain;

[0012] a processing unit configured to detect attribute information of an image, wherein the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image;

[0013] The processing unit is further used to assign a search processing engine to each connected domain based on attribute information of the image;

[0014] The processing unit is further used to schedule the search processing engines corresponding to each connected domain in parallel, and perform contour search processing on each connected domain.

[0015] The device is provided on an integrated chip, which includes a marking processing engine; the processing unit is further used to:

[0016] The marking processing engine is scheduled to perform connected domain marking processing on the image to obtain a marking result;

[0017] The labeling result includes a label for each connected domain in the image and an index created for each label; the label of each connected domain is used to uniquely identify the corresponding connected domain.

[0018] The integrated chip also includes a data handling engine and a scan processing engine; a processing unit, specifically used to:

[0019] Calling the data handling engine to write each connected domain and the marking results into a storage area, which includes a memory and / or an on-chip cache;

[0020] During the writing process, the scanning processing engine is called to detect the attribute information of the image in real time.

[0021] The processing unit is specifically used for:

[0022] During the writing process, the scanning processing engine is called to perform statistical analysis on the number of labels in the labeling results to obtain the number of connected domains in the image;

[0023] Calling the scanning processing engine to sequentially record the physical addresses of the tags corresponding to the respective indexes in the storage area according to the indexes of the tags, and obtaining the physical locations of the respective connected domains; and

[0024] The scanning processing engine is called to sequentially detect the coordinates of the starting points of each connected domain in the image according to the index of each label.

[0025] The device is provided on an integrated chip, which includes a task scheduling management engine and M search processing engines; the number of connected domains in the image is N; M and N are both integers greater than 1;

[0026] The processing unit is specifically used to:

[0027] Construct image processing tasks for images;

[0028] Call the task scheduling management engine to split the image processing task into N subtasks based on the image attribute information, and the N subtasks correspond one-to-one to the N connected domains;

[0029] The task scheduling management engine is called to allocate N subtasks to M search processing engines.

[0030] The task scheduling management engine allocates the N subtasks in at least one of the following ways:

[0031] Randomly assign N subtasks to M search processing engines;

[0032] According to the load balancing principle, N subtasks are assigned to M search processing engines;

[0033] According to the principle of minimizing the processing time of the image processing task, N subtasks are assigned to M search processing engines;

[0034] Each connected domain is marked with a label, and each label corresponds to its own index; N subtasks are assigned to M search processing engines in ascending order of their indexes;

[0035] Distribute N subtasks evenly to M search processing engines.

[0036] The processing unit is specifically used for:

[0037] Calling the task scheduling management engine to send parallel execution instructions to the M search processing engines;

[0038] By executing the instructions in parallel, the M search processing engines are controlled to execute their assigned subtasks in parallel.

[0039] Among them, N subtasks include subtask i, subtask i corresponds to connected domain I; M search processing engines include search processing engine j; subtask i is assigned to search processing engine j;

[0040] The process of performing contour search processing on the connected region I by the search processing engine j includes:

[0041] The search processing engine j obtains the connected domain I from the storage area according to the physical location of the connected domain I;

[0042] The search processing engine j performs contour search starting from the starting point according to the coordinates of the starting point of the connected region I in the image to obtain the boundary points of the connected region I;

[0043] Based on the starting point and boundary points of the connected region I, the outline of the connected region I is determined.

[0044] The processing unit is further configured to:

[0045] Based on the results of the contour search processing of each connected domain, the contour corresponding to each connected domain is extracted from the image;

[0046] Perform recognition processing on each extracted contour to obtain the target object;

[0047] The target object includes at least one of the following: a license plate, text, subtitle, moving object, and region of interest.

[0048] On the one hand, an embodiment of the present application provides an integrated chip, which includes a storage area and at least one engine; the storage area includes memory and / or on-chip cache; wherein the integrated chip schedules each engine to execute the above-mentioned image processing method.

[0049] On the one hand, an embodiment of the present application provides an image processing device, which includes a processor, an input device, an output device and a memory; a computer program is stored in the memory; when the computer program is executed by the processor, the above-mentioned image processing method is executed.

[0050] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image processing method is performed.

[0051] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned image processing method is performed.

[0052] In an embodiment of the present application, an image to be processed is obtained, which includes at least one connected domain; by marking at least one connected domain in the image, this can serve as the basis and prerequisite for subsequent contour search processing. The attribute information of the image is detected, and the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; by analyzing and detecting the attribute information of the image in advance before performing contour search processing on the image, the necessary data required for processing can be provided for the subsequent contour search processing, thereby facilitating the improvement of the efficiency of the subsequent contour search processing. Based on the attribute information of the image, a search processing engine is assigned to each connected domain, and the search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain. By flexibly assigning a search processing engine to each connected domain and scheduling each search processing engine in parallel, each search processing engine can be controlled to quickly perform contour search processing on each connected domain in parallel based on the attribute information of the image, thereby greatly improving the efficiency of image contour search. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1a A schematic diagram of image contour search provided by the prior art;

[0055] Figure 1b A schematic structural diagram of an integrated chip provided as an exemplary embodiment of the present application;

[0056] Figure 2 A schematic diagram of the interaction between engines in an integrated chip provided by an exemplary embodiment of the present application;

[0057] Figure 3 A schematic diagram of an image processing flow provided for an exemplary embodiment of the present application;

[0058] Figure 4 A schematic diagram of a connected domain in an image provided as an exemplary embodiment of the present application;

[0059] Figure 5 A flowchart of an image processing method provided as an exemplary embodiment of the present application;

[0060] Figure 6 A schematic diagram of contour search comparison provided for an exemplary embodiment of the present application;

[0061] Figure 7 A schematic structural diagram of an image processing device provided as an exemplary embodiment of the present application;

[0062] Figure 8 A schematic structural diagram of an image processing device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0064] 1. Connected Domain

[0065] A connected domain, as the name implies, refers to a connected area. The so-called connectivity means that if two identical pixels (such as pixel A and pixel B) are adjacent, then the two pixels are considered to be connected. At the same time, if pixel A and pixel B are connected, and pixel B and pixel C are connected, then pixel A and pixel C are also considered to be connected. In an embodiment of the present application, a connected domain refers to an image area composed of pixels with the same pixel value and adjacent positions in the image. The connected domain in an image can be obtained by performing a connected domain analysis on the image. The so-called connected domain analysis refers to finding a connected domain in an image and marking the connected domain found, wherein the pixels belonging to the same connected domain are marked with the same label, and the pixels of different connected domains are marked with different labels, that is, each connected domain corresponds to a different label. Any image can include at least one connected domain.

[0066] 2. Outline of Connected Domain

[0067] The contour of the connected domain is composed of the boundary points of the connected domain. In the process of contour search for the connected domain, the boundary points of the connected domain are actually obtained, and the contour of the connected domain is obtained based on the obtained boundary points.

[0068] 3. Artificial Intelligence (AI)

[0069] AI is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI domains. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0070] Computer vision (CV), a field within artificial intelligence software technology, is the study of how machines can "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying and measuring objects, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the field of vision, such as the Swin Transformer, ViT, V-MOE, and MAE, can be fine-tuned to quickly and widely apply to specific downstream tasks. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0071] The embodiments of the present application relate to CV in AI. Schematically, after performing contour search on a connected domain of an image, the contour of the connected domain can be extracted through CV, and objects included in the contour can be identified (such as OCR recognition).

[0072] The embodiment of the present application provides an image processing solution, the general principle of which is as follows: obtaining an image to be processed, the image including at least one connected domain; marking at least one connected domain in the image, which can serve as the basis and prerequisite for subsequent contour search processing. Detecting the attribute information of the image, the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; by analyzing and detecting the attribute information of the image in advance before performing contour search processing on the image, the necessary data required for the subsequent contour search processing can be provided, thereby facilitating the improvement of the efficiency of the subsequent contour search processing. Based on the attribute information of the image, a search processing engine is assigned to each connected domain, and the search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain. By flexibly assigning search processing engines to each connected domain and scheduling each search processing engine in parallel, each search processing engine can be controlled to quickly perform contour search processing on each connected domain in parallel based on the attribute information of the image, thereby greatly improving the efficiency of image contour search.

[0073] Next, the integrated chip provided in the embodiments of the present application is described.

[0074] See Figure 1b , is a schematic diagram of the structure of an integrated chip provided by an exemplary embodiment of the present application. The integrated chip may include a storage area 101 and at least one engine. The storage area 101 and each engine may be connected via a bus 102. Among them:

[0075] The storage area 101 may include memory and / or on-chip cache. Among them, the on-chip cache can also be called Cache, which is an intermediate storage layer introduced to make up for the speed mismatch between high-speed devices (devices with relatively fast processing speeds) and low-speed devices (devices with relatively slow processing speeds), and plays a role in accelerating access speed; schematically, the on-chip cache can be a high-speed, small-capacity temporary memory. Schematically, the on-chip cache can be set between the CPU (high-speed device) and the memory (low-speed device). Its processing speed is faster than that of the memory, and it can exchange data before the memory, thereby improving the access rate of the data.

[0076] At least one engine in the integrated chip is used to accelerate various tasks in the integrated chip, and the tasks here may include but are not limited to: connected domain labeling tasks, data handling tasks, contour search processing tasks, etc. Figure 2As shown, the engines in the integrated chip of the embodiment of the present application may include but are not limited to: a tag processing engine 103, a data handling engine 104, a scan processing engine 105, a search processing engine 106, and a task scheduling management engine 107. In the embodiment of the present application, there is no limitation on the number of tag processing engines 103, data handling engines 104, scan processing engines 105, task scheduling management engines 107, and search processing engines 106. Illustratively, the number of search processing engines 106 can be M, where M is a positive integer. The tag processing engine, data handling engine, scan processing engine, task scheduling management engine, and search processing engine are described below:

[0077] See Figure 2 , is a schematic diagram of the interaction between various engines in an integrated chip provided by an exemplary embodiment of the present application. Figure 2 In the example, the labeling processing engine 103 is used to perform the connected domain labeling task, and performs connected domain labeling processing on the image to be processed to obtain a labeling result. The data handling engine 104 is used to write each connected domain in the image and the labeling result into the storage area 101.

[0078] The scanning and processing engine 105 is used to perform a rapid scanning task while the data handling engine 104 writes the connected domains and labeling results to the storage area 101, detect and analyze the labeling results, and obtain image attribute information. This attribute information may include: the number of connected domains in the image to be processed, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image. The physical location of each connected domain refers to the physical address at which each connected domain is stored in the storage area 101.

[0079] The search processing engine 106 is used to execute image processing tasks and perform contour search processing on each connected domain in the image. Illustratively, the number of search processing engines may be M, and each search processing engine can perform contour search on different connected domains in the image in parallel, thereby achieving parallel contour search of the image. It should be noted that the value of M can be flexibly set according to actual needs. For example, the value of M can be set according to the processing time requirement of the image processing task. For example, if the image processing task is required to be completed within a shorter time, the value of M can be larger. Conversely, if the image processing task is required to be completed within a longer time, the value of M can be relatively smaller. For another example, the value of M can be set according to the performance (such as throughput) index requirements of the integrated chip. For example, if the throughput requirement is higher, the value of M can be relatively larger; conversely, if the throughput requirement is lower, the value of M can be relatively smaller.

[0080] The task scheduling management engine 107 can be used to schedule the above-mentioned engines (e.g., the search processing engines 106) to perform corresponding tasks. In the embodiment of the present application, the task scheduling management engine 107 is used to assign a search processing engine to each connected domain in the image, schedule the search processing engines of each connected domain in parallel, perform contour searches on each connected domain in the image, obtain contour search results for each connected domain, and implement parallel searches for the connected domain contours.

[0081] In one embodiment, Figure 2 The interaction flow of each engine shown can be combined with Figure 3 Provide a detailed description. Figure 3 This is a schematic diagram of an image processing process provided by an exemplary embodiment of the present application. The image processing process can be executed by an image processing device. Schematically, it can be called by the image processing device. Figure 1b The integrated chip shown is implemented; see Figure 3 , the image processing process may include the following steps S11-S17:

[0082] S11 , obtaining an image to be processed, and calling the marking processing engine 103 in the integrated chip to perform connected component marking processing on the image to be processed.

[0083] The images may include, but are not limited to, static images, dynamic images, or any video frame images. The number of images may be one or more, and this embodiment of the application does not impose any restrictions on this. As an implementation method, the image to be processed can be obtained by real-time image acquisition using a camera component (such as a camera, an external camera device, etc.); as another implementation method, the image to be processed can be obtained from a pre-saved image resource or video resource.

[0084] As an implementation, performing connected component labeling on the image to be processed to obtain the labeling result may include: ① binarizing the image to be processed to obtain a binary image; and ② using a connected component labeling algorithm to perform connected component labeling on the binary image. In a specific implementation, the connected component labeling algorithm is used to determine pixels in the image that have the same pixel points and are adjacent in position, and based on the pixels that have the same pixel points and are adjacent in position, a connected component in the image is obtained.

[0085] The connected component labeling algorithms herein may include, but are not limited to, the 4-neighbor algorithm and the 8-neighbor algorithm. The 4 and 8 refer to directions; taking the 4-neighbor algorithm as an example, the 4-neighbor algorithm states that if white pixels exist in the four directions (a total of four directions) immediately above, below, to the right, or to the left of a target white pixel X in a binary image, then the corresponding white pixels in the four directions above, below, to the right, or to the left are determined to be adjacent to the target white pixel X, and the corresponding white pixels in the four directions above, below, to the right, or to the left are labeled with the same label as the target white pixel X. The white pixels adjacent to the white pixel corresponding to the adjacent white pixel above are then determined, and the white pixels adjacent to the white pixel corresponding to the adjacent white pixel above are labeled with the same label as the target white pixel X. This is repeated to form a connected component in the image based on pixels with the same label. Similar to the 4-neighbor algorithm, the 8-neighbor algorithm determines a connected component by searching for white pixels in the eight directions (a total of eight directions) diagonally above, below, left, or right of the target white pixel X in the binary image. This will not be further described here.

[0086] S12: Determine whether all connected domains in the image have been marked. If all connected domains in the image have been marked, execute step S13. If all connected domains in the image have not been marked, continue to execute step S11 until a marking result is obtained.

[0087] S13. If all connected domains in the image are marked, a marking result is obtained.

[0088] The labeling results include the labels for each connected domain in the image and the index created for each label; the label of each connected domain is used to uniquely identify the corresponding connected domain. For example, the number of connected domains included in the image to be processed is N, where N is an integer greater than 1. The labels marked for each connected domain can be expressed as label0, label1, ...label(N-1). The index created for each label can be recorded as Label Value, and the corresponding pseudo code is:

[0089] for(i=0;i <N;i=i+1)

[0090] label_value(i)=get_label_val(i).

[0091] Among them, i represents the i-th label, get_label_val(i) means creating an index for the i-th label by obtaining the number value of the i-th label, and label_value(i) represents the index created for the i-th label.

[0092] Indicative, such as Figure 4FIG. 1 is a schematic diagram of a connected domain in an image provided by an exemplary embodiment of the present application. Figure 4 In , the image includes 4 connected domains (ie, N=4), and the indexes created for the labels marked by these 4 connected domains are 1, 2, 3, and 4 respectively.

[0093] S14: The data handling engine 104 is called to write the connected domains and the labeling results in the image to the storage area 101. During the writing process, the scanning processing engine 105 is called to detect the image's attribute information in real time. This attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image.

[0094] As an implementation method, the scanning processing engine 105 can detect the attribute information of the image in real time by the following steps: ① During the writing process, the scanning processing engine 105 performs a statistical analysis on the number of labels in the labeling results to obtain the number of connected domains in the image. It should be understood that the label of each connected domain is used to uniquely identify the corresponding connected domain, and the number of labels (Label Number) is the number of connected domains in the image. ② The scanning processing engine 105 records the physical address of the label corresponding to each index in the storage area in sequence according to the index of each label to obtain the physical position of each connected domain. Assuming that the number of connected domains is N, the physical position of each connected domain can be recorded as Addr0, Addr1, ...Addr(N-1) respectively. The physical address of the label corresponding to each index in the storage area can be understood as the physical position of each connected domain. ③ The scanning processing engine 105 detects the coordinates of the starting point (Starting Point) of each connected domain in the image in sequence according to the index of each label. Schematically, the number of connected domains is N. According to the index of each label, the coordinates of the starting points of the N connected domains in the image are detected sequentially, and the coordinates of the detected N connected starting points are recorded in sequence as S0, S1, ...S(N-1).

[0095] S15 , calling the task scheduling management engine 107 to allocate a search processing engine 106 to each connected domain according to the attribute information of the image.

[0096] In one implementation, the task scheduling management engine 107 can generate a scheduling strategy based on the attribute information of the image. According to the scheduling strategy, a search processing engine is assigned to each connected domain based on the attribute information of the image. For example, the scheduling strategy includes a random assignment strategy. The task scheduling management engine can randomly assign a search processing engine to each connected domain based on the attribute information of the image. For example, if the attribute information of the image includes the number N of connected domains in the image, a search processing engine can be randomly assigned to each connected domain. For another example, the scheduling strategy includes a specified assignment strategy. The specified assignment strategy includes: assigning search processing engine A to connected domain 1 and assigning search processing engine B to connected domain 2. The task scheduling management engine can assign search processing engine A to connected domain 1 and assign search processing engine B to connected domain 2 based on the attribute information of the image according to the scheduling strategy; and so on.

[0097] S16 , the task scheduling management engine 107 schedules the search processing engines 106 corresponding to the connected domains in parallel, performs contour search processing on each connected domain, and obtains contour search processing results of the connected domains.

[0098] Assume that the search processing engine j is assigned to the connected domain I. After the search processing engine j is started, the process of the search processing engine j performing contour search processing on the connected domain I includes: ① According to the index of the connected domain label, the physical position of the connected domain I and the coordinates of the starting point of the connected domain I in the image are obtained. ② According to the physical position of the connected domain I, the connected domain I is obtained from the storage area. ③ According to the coordinates of the starting point of the connected domain I in the image, the search processing engine j performs contour search from the starting point to obtain the boundary points of the connected domain I. ④ Based on the starting point and boundary points of the connected domain I, the contour of the connected domain I is determined. In this embodiment, by assigning the search processing engine j to the connected domain i, the search processing engine j can directly jump to the physical address in the storage area according to the physical position of the connected domain I, and perform contour search from the coordinates of the starting point of the connected domain I in the image, saving a lot of invalid data handling and contour search time, thereby improving the efficiency and performance of the contour search.

[0099] It should be understood that the image processing process provided in the embodiments of the present application can be applied to computer vision and pattern recognition and image analysis processing. Specifically, the image processing process provided in the embodiments of the present application can be widely used in character segmentation and extraction in OCR recognition (such as license plate recognition, text recognition, subtitle recognition, etc.), motion foreground target segmentation and extraction in visual following (motion object detection, left-behind object detection, vision-based vehicle detection and following, etc.), medical image processing (region of interest extraction), etc. Correspondingly, the image can come from the images in the above-mentioned application scenarios, and the image can include one or more objects. In the character segmentation and extraction scenario, the objects included in the image can be license plates, text, subtitles, etc.; in the motion target segmentation and extraction scenario in visual tracking, the objects included in the image include moving objects (such as animals, vehicles, etc.); in the medical image processing scenario, the objects included in the image can include: regions of interest (such as hands, feet, etc.).

[0100] Furthermore, the above image processing process may also include:

[0101] S17 . Based on the results of the contour search processing of each connected domain, the contours corresponding to each connected domain are extracted from the image, and recognition processing is performed on each extracted contour to obtain the target object.

[0102] The contour of the connected domain may include a target object. The target object may include at least one of the following: license plate, text, subtitle, moving object, region of interest, etc. Accordingly, the recognition processing includes but is not limited to: license plate recognition, text recognition, subtitle recognition, moving object recognition, region of interest recognition, etc. The following two examples are used to illustrate:

[0103] Example 1: In a character segmentation and extraction scenario, the image to be processed contains a license plate, and the recognition process includes license plate recognition. The image includes at least one connected domain, each of which may contain a license plate. After scheduling the search processing engines corresponding to each connected domain in parallel, contour search processing is performed on each connected domain. After obtaining the contour search results for each connected domain, the contours corresponding to each connected domain can be extracted from the image based on the contour search results. License plate recognition is then performed on the contours corresponding to each connected domain to obtain the license plate in the image. The recognized license plate can be used to process a specific vehicle.

[0104] Example 2: In a medical image processing scenario (such as extraction of regions of interest); the image to be processed contains multiple regions, the recognition processing includes identifying the regions of interest, the image includes at least one connected domain, and each connected domain may include a region; after the search processing engines corresponding to each connected domain are scheduled in parallel and contour search processing is performed on each connected domain, and the results of the contour search processing of each connected domain are obtained, the contours corresponding to each connected domain can be extracted from the image based on the results of the contour search processing of each connected domain, and the contours corresponding to each connected domain can be identified to obtain the region of interest.

[0105] Next, the image processing method provided in the embodiments of the present application is described.

[0106] See Figure 5 , which is a flow chart of an image processing method provided by an exemplary embodiment of the present application. The image processing method can be executed by an image processing device, and illustratively, it can be executed by calling an integrated chip by the image processing device. The integrated chip can be set in the image processing device, and the image processing device can be a terminal or a server. Among them, the terminal can be a smart phone, tablet computer, laptop computer, desktop computer, car computer, smart home, wearable electronic device, VR / AR device, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms and other basic cloud computing services, etc. The image processing method provided in the embodiment of the present application may include the following steps S501-S504:

[0107] S501: Acquire an image to be processed, where the image includes at least one connected domain.

[0108] The integrated chip includes a labeling processing engine. After obtaining the image to be processed, the labeling processing engine can be scheduled to perform connected domain labeling on the image to obtain a labeling result. The labeling result includes a label for each connected domain in the image and an index created for each label. The label of each connected domain is used to uniquely identify the corresponding connected domain.

[0109] S502: Detect attribute information of the image, where the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image.

[0110] As an implementation method, the integrated chip also includes a data handling engine and a scanning processing engine. Detecting the attribute information of the image may include: calling the data handling engine to write each connected domain and the marking results to a storage area, which includes memory and / or on-chip cache; and during the writing process, calling the scanning processing engine to detect the attribute information of the image in real time.

[0111] During the writing process, calling the scanning processing engine to detect the image's attribute information in real time can include the following steps: during the writing process, calling the scanning processing engine to perform statistical analysis on the number of tags in the labeling results to obtain the number of connected domains in the image; calling the scanning processing engine to sequentially record the physical addresses of the tags corresponding to each index in the storage area according to the index of each tag to obtain the physical location of each connected domain; and calling the scanning processing engine to sequentially detect the coordinates of the starting point of each connected domain in the image according to the index of each tag. The scanning processing engine can quickly scan and detect the image's attribute information.

[0112] S503: Allocate a search processing engine to each connected domain based on the attribute information of the image.

[0113] As an implementation method, the attribute information includes the number of each connected domain, and a search processing engine is assigned to each connected domain based on the number of each connected domain, wherein one connected domain is assigned one search processing engine, and one search processing engine can be assigned to one or more connected domains; schematically, when the number of each connected domain is 3, one search processing engine can be assigned to each of the 3 connected domains; it can be understood that if the number of search processing engines is greater than or equal to 3, the 3 connected domains can be assigned to different search processing engines respectively; if the number of search processing engines is less than 3, there may be multiple (2 or more) connected domains in the 3 connected domains that are assigned to the same search processing engine.

[0114] S504 : Parallel scheduling of the search processing engines corresponding to the connected domains to perform contour search processing on the connected domains.

[0115] As an implementation, the integrated chip includes a task scheduling management engine. Step S504 may include: invoking the task scheduling management engine to send a parallel execution instruction to the search processing engine corresponding to each connected domain; and controlling the search processing engine corresponding to each connected domain via the parallel execution instruction to perform contour search processing on each connected domain in parallel. By performing contour search processing on each connected domain in parallel, the computational effort of scanning each contour can be reduced, thereby improving contour search efficiency.

[0116] In an embodiment of the present application, an image to be processed is obtained, and the image includes at least one connected domain; by marking at least one connected domain in the image, this can serve as the basis and prerequisite for subsequent contour search processing. The attribute information of the image is detected, and the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; by analyzing and detecting the attribute information of the image in advance before performing contour search processing on the image, the necessary data required for processing can be provided for the subsequent contour search processing, thereby facilitating the improvement of the efficiency of the subsequent contour search processing. Based on the attribute information of the image, a search processing engine is assigned to each connected domain, and the search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain. By flexibly assigning a search processing engine to each connected domain and scheduling each search processing engine in parallel, each search processing engine can be controlled to quickly perform contour search processing on each connected domain in parallel based on the attribute information of the image, thereby greatly improving the efficiency of image contour search.

[0117] In some feasible embodiments, the integrated chip may include a task scheduling management engine and M search processing engines, where M is an integer greater than 1. In this embodiment, the specific implementation of step S403 may include the following steps:

[0118] s21. Construct an image processing task for the image. The image processing task is used to perform contour search processing on N connected components. The image processing task is a serial contour search task.

[0119] s22. Call the task scheduling management engine to split the image processing task into N subtasks based on the attribute information of the image.

[0120] Among them, the N subtasks correspond one-to-one to the N connected domains. Each subtask can be used to perform contour search processing on a connected domain. Each subtask can include the physical location of the connected domain and the coordinates of the connected domain starting point in the image. For example, N = 3. Based on the attribute information of the image, the image processing task can be split into three subtasks: subtask 1, subtask 2, and subtask 3. Subtask 1 can be used to perform contour search processing on connected domain 1, subtask 2 can be used to perform contour search processing on connected domain 2, and subtask 3 can be used to perform contour search processing on connected domain 3. Here, by splitting the image processing task into N subtasks, it is possible to split the serial image processing task into N subtasks that are executed in parallel.

[0121] s23. Call the task scheduling management engine to assign N subtasks to M search processing engines.

[0122] The task scheduling management engine may allocate the N subtasks in at least one of the following ways:

[0123] ① Random assignment. The task scheduling management engine can be called to randomly assign N subtasks to M search processing engines. For example, if N = 3 and M = 2, the task scheduling engine can be called to randomly assign 3 subtasks to 2 search processing engines. For example, subtasks 1 and 3 can be assigned to search processing engine 1, and subtask 2 can be assigned to search processing engine 2.

[0124] ② According to the principle of load balancing, N subtasks are assigned to M search processing engines. As an implementation method, the current load of each search processing engine can be obtained, and the task scheduling management engine can be called to evenly distribute the N subtasks to the M search processing engines based on the current load of each search processing engine. Specifically, the N subtasks can be assigned to the search processing engine with a smaller load. The current load of the search processing engine can include any of the following: the number of subtasks waiting to be processed, the time required for the execution of the subtasks waiting to be processed, etc. For example, N = 2, M = 3, among the three search processing engines, the current load of search processing engine 1 is greater than the current load of search processing engine 2, and the current load of search processing engine 2 is greater than the current load of search processing engine 3. By calling the task scheduling management engine, one of the two subtasks can be assigned to search processing engine 3, and the other subtask can be assigned to search processing engine 2. For another example, N=3, M=3, among the three search processing engines, the current load of search processing engine 1 is greater than the current load of search processing engine 2, and the current load of search processing engine 2 is greater than the current load of search processing engine 3. The task scheduling management engine can be called to assign subtask 1 and subtask 2 of the three subtasks to search processing engine 3, and assign subtask 3 to search processing engine 2.

[0125] ③ Call the task scheduling management engine to evenly distribute N subtasks to M search processing engines. For example, N = 4, M = 4; then call the task scheduling management engine to evenly distribute N subtasks to M search processing engines, with each search processing engine handling one subtask.

[0126] ④ Based on the principle of minimizing the processing time of the image processing task, N subtasks are assigned to M search processing engines. It should be understood that the time required for a subtask to be executed by different search processing engines may vary. For example, if the time required for subtask 1 to be executed by search processing engine 1 is greater than the time required for subtask 2 to be executed by search processing engine 1, then subtask 2 can be assigned to search processing engine 1. Based on this, step s23 may include: calculating the time required to execute N subtasks under M search processing engines, determining the time required to execute N subtasks as the processing time of the image processing task, and calling the task scheduling management engine to assign N subtasks to M search processing engines based on minimizing the processing time of the image processing task.

[0127] The processing time of an image processing task is calculated as follows: T = ceil(N / M)*max(1, 2...M), where N is an arbitrary integer and M is a positive integer greater than 1. T represents the processing time of the image processing task, and ceil() is a ceiling function.

[0128] ⑤ Each connected domain is marked with a label, and each label corresponds to its own index. The task scheduling management engine may allocate N subtasks in the following manner: the task scheduling management engine allocates N subtasks to M search processing engines in the order of the index of the label marked for each connected domain from small to large. Schematically, N=2, M=2, subtask A of the two subtasks corresponds to connected domain 1, and subtask B corresponds to connected domain 2. The index of the label marked for connected domain 1 is 1, and the index of the label marked for connected domain 2 is 2. The task scheduling management engine is called to allocate subtask A to search processing engine 1 and allocate subtask B to search processing engine 2 in the order of the index of the label marked for each connected domain from small to large.

[0129] It should be understood that when N is greater than M, this means that M search processing engines are scheduled in parallel at a time, making it impossible to perform contour search processing on all N connected regions. In this case, the remaining subtasks can be assigned to the M search processing engines using at least one of the above allocation methods. Alternatively, the remaining subtasks can be preferentially assigned to the search processing engines that have completed their subtasks.

[0130] Furthermore, parallel scheduling of the search processing engines corresponding to each connected domain to perform contour search processing on each connected domain may include: calling the task scheduling management engine to send parallel execution instructions to the M search processing engines, and controlling the M search processing engines to execute their assigned subtasks in parallel through the parallel execution instructions.

[0131] In which, it is assumed that N subtasks include subtask i, i∈[1,N], and subtask i corresponds to a connected domain I; M search processing engines include search processing engine j, j∈[1,M]; subtask i is assigned to search processing engine j; the process of search processing engine j performing contour search processing on the connected domain I includes: the search processing engine j obtains the connected domain I from the storage area according to the physical position of the connected domain I, the search processing engine j performs contour search from the starting point according to the coordinates of the starting point of the connected domain I in the image, obtains the boundary points of the connected domain I, and determines the contour of the connected domain I based on the starting point and boundary points of the connected domain I.

[0132] In some optional embodiments, after scheduling the search processing engines corresponding to each connected domain in parallel and performing contour search processing on each connected domain, the contours corresponding to each connected domain can be extracted from the image based on the contour search results of each connected domain, and recognition processing can be performed on each extracted contour to obtain target objects. The target objects include at least one of the following: license plates, text, subtitles, moving objects, and regions of interest.

[0133] To summarize, by splitting a serial image processing task into N subtasks, calling the task scheduling management engine to assign the N subtasks to M search processing engines, and scheduling the M search processing engines in parallel to execute their assigned subtasks, the computational complexity of the final contour search can be effectively reduced to 1 / M of the original serial image processing task, and the effective throughput becomes at least M times that of the serial image processing task.

[0134] Through the image processing method provided in the embodiment of the present application, the contour search of an image including multiple connected domains can be effectively optimized, the contour search time can be shortened, and the contour search efficiency can be improved. Schematically, it is assumed that an image includes 4 connected domains, and the integrated chip contains 4 search processing engines. Through the image processing method provided in the embodiment of the present application, the image processing task consisting of an image including 4 connected domains can be split into 4 subtasks, and the 4 subtasks can be assigned to 4 search processing engines, with one subtask corresponding to one search processing engine. The task scheduling management engine schedules the 4 search processing engines in parallel to directly execute their assigned subtasks to obtain the contour search results of each connected domain. Through the image processing method provided in the embodiment of the present application, a large amount of invalid data handling and contour search time can be saved, and the effective throughput can be at least 4 times the original. In order to facilitate the intuitive display of the effects of the image processing method provided in the embodiment of the present application, as shown below: Figure 6 As shown in FIG, a schematic diagram of a contour search comparison provided by an embodiment of the present application is provided. Figure 6 In the embodiment of the present application, the contour search time of the image processing method can be as follows: Figure 6As shown in Q, the contour search time for serial execution of an image processing task can be as follows: Figure 6 As shown in P in , the unit of contour search time can be microseconds (us). It can be seen that the contour search time using the image processing method provided by the embodiment of the present application is shorter than the contour search time of serially executing an image processing task. In other words, the contour search efficiency of the image processing method provided by the embodiment of the present application is improved by 2.3 times compared with the contour search efficiency of serially executing an image processing task.

[0135] Next, the image processing device provided in the embodiment of the present application is described.

[0136] See Figure 7 , Figure 7 is a structural diagram of an image processing apparatus provided by an exemplary embodiment of the present application. The image processing apparatus is a computer program (including program code) in an image processing device. For example, the image processing apparatus may be an application software in the image processing device. The image processing apparatus may be used to execute Figure 3 and Figure 5 Some or all of the steps in the method embodiment shown. Figure 7 , the image processing device includes the following units:

[0137] An acquisition unit 701 is configured to acquire an image to be processed, wherein the image includes at least one connected domain;

[0138] The processing unit 702 is configured to detect attribute information of the image, where the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image.

[0139] The processing unit 702 is further configured to assign a search processing engine to each connected component based on the attribute information of the image;

[0140] The processing unit 702 is further configured to schedule the search processing engines corresponding to the connected components in parallel, and perform contour search processing on the connected components.

[0141] The device is provided on an integrated chip, which includes a tag processing engine; the processing unit 702 is further configured to:

[0142] The marking processing engine is scheduled to perform connected domain marking processing on the image to obtain a marking result;

[0143] The labeling result includes a label for each connected domain in the image and an index created for each label; the label of each connected domain is used to uniquely identify the corresponding connected domain.

[0144] The integrated chip also includes a data handling engine and a scan processing engine; the processing unit 702 is specifically used to:

[0145] Calling the data handling engine to write each connected domain and the marking results into a storage area, which includes a memory and / or an on-chip cache;

[0146] During the writing process, the scanning processing engine is called to detect the attribute information of the image in real time.

[0147] The processing unit 702 is specifically configured to:

[0148] During the writing process, the scanning processing engine is called to perform statistical analysis on the number of labels in the labeling results to obtain the number of connected domains in the image;

[0149] Calling the scanning processing engine to sequentially record the physical addresses of the tags corresponding to the respective indexes in the storage area according to the indexes of the tags, and obtaining the physical locations of the respective connected domains; and

[0150] The scanning processing engine is called to sequentially detect the coordinates of the starting points of each connected domain in the image according to the index of each label.

[0151] The device is provided on an integrated chip, which includes a task scheduling management engine and M search processing engines; the number of connected domains in the image is N; M and N are both integers greater than 1;

[0152] The processing unit 702 is specifically configured to:

[0153] Construct image processing tasks for images;

[0154] Call the task scheduling management engine to split the image processing task into N subtasks based on the image attribute information, and the N subtasks correspond one-to-one to the N connected domains;

[0155] The task scheduling management engine is called to allocate N subtasks to M search processing engines.

[0156] The task scheduling management engine allocates the N subtasks in at least one of the following ways:

[0157] Randomly assign N subtasks to M search processing engines;

[0158] According to the load balancing principle, N subtasks are assigned to M search processing engines;

[0159] According to the principle of minimizing the processing time of the image processing task, N subtasks are assigned to M search processing engines;

[0160] Each connected domain is marked with a label, and each label corresponds to its own index; N subtasks are assigned to M search processing engines in ascending order of their indexes;

[0161] Distribute N subtasks evenly to M search processing engines.

[0162] The processing unit 702 is specifically configured to:

[0163] Calling the task scheduling management engine to send parallel execution instructions to the M search processing engines;

[0164] By executing the instructions in parallel, the M search processing engines are controlled to execute their assigned subtasks in parallel.

[0165] Among them, N subtasks include subtask i, subtask i corresponds to connected domain I; M search processing engines include search processing engine j; subtask i is assigned to search processing engine j;

[0166] The process of performing contour search processing on the connected region I by the search processing engine j includes:

[0167] The search processing engine j obtains the connected domain I from the storage area according to the physical location of the connected domain I;

[0168] The search processing engine j performs contour search starting from the starting point according to the coordinates of the starting point of the connected region I in the image to obtain the boundary points of the connected region I;

[0169] Based on the starting point and boundary points of the connected region I, the outline of the connected region I is determined.

[0170] The processing unit 702 is further configured to:

[0171] Based on the results of the contour search processing of each connected domain, the contour corresponding to each connected domain is extracted from the image;

[0172] Perform recognition processing on each extracted contour to obtain the target object;

[0173] The target object includes at least one of the following: a license plate, text, subtitle, moving object, and region of interest.

[0174] In an embodiment of the present application, an image to be processed is obtained, and the image includes at least one connected domain; by marking at least one connected domain in the image, this can serve as the basis and prerequisite for subsequent contour search processing. The attribute information of the image is detected, and the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; by analyzing and detecting the attribute information of the image in advance before performing contour search processing on the image, the necessary data required for processing can be provided for the subsequent contour search processing, thereby facilitating the improvement of the efficiency of the subsequent contour search processing. Based on the attribute information of the image, a search processing engine is assigned to each connected domain, and the search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain. By flexibly assigning a search processing engine to each connected domain and scheduling each search processing engine in parallel, each search processing engine can be controlled to quickly perform contour search processing on each connected domain in parallel based on the attribute information of the image, thereby greatly improving the efficiency of image contour search.

[0175] Next, the image processing device provided in the embodiments of the present application is described.

[0176] Furthermore, the present invention also provides a schematic diagram of the structure of an image processing device. Figure 8 The computer device may include a processor 801, an input device 802, an output device 803, and a memory 804. The processor 801, input device 802, output device 803, and memory 804 are connected via a bus. The memory 804 is used to store a computer program, which includes program instructions. The processor 801 is used to execute the program instructions stored in the memory 804.

[0177] In one embodiment, the processor 801 executes the following operations by executing program instructions in the memory 804:

[0178] Acquire an image to be processed, wherein the image includes at least one connected domain;

[0179] Detecting attribute information of the image, including the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image;

[0180] Based on the attribute information of the image, a search processing engine is assigned to each connected domain;

[0181] The search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain.

[0182] The processor executes by calling an integrated chip, which includes a tag processing engine; the processor 801 may also execute the following steps:

[0183] The marking processing engine is scheduled to perform connected domain marking processing on the image to obtain a marking result;

[0184] The labeling result includes a label for each connected domain in the image and an index created for each label; the label of each connected domain is used to uniquely identify the corresponding connected domain.

[0185] The integrated chip also includes a data handling engine and a scanning processing engine. When detecting attribute information of an image, the processor 801 may specifically perform the following steps:

[0186] Calling the data handling engine to write each connected domain and the marking results into a storage area, which includes a memory and / or an on-chip cache;

[0187] During the writing process, the scanning processing engine is called to detect the attribute information of the image in real time.

[0188] When the processor 801 calls the scanning processing engine to detect the attribute information of the image in real time during the writing process, the following steps may be specifically performed:

[0189] During the writing process, the scanning processing engine is called to perform statistical analysis on the number of labels in the labeling results to obtain the number of connected domains in the image;

[0190] Calling the scanning processing engine to sequentially record the physical addresses of the tags corresponding to the respective indexes in the storage area according to the indexes of the tags, and obtaining the physical locations of the respective connected domains; and

[0191] The scanning processing engine is called to sequentially detect the coordinates of the starting points of each connected domain in the image according to the index of each label.

[0192] The processor 801 executes by calling an integrated chip, which includes a task scheduling management engine and M search processing engines; the number of connected domains in the image is N; M and N are both integers greater than 1;

[0193] When allocating a search processing engine to each connected domain based on the attribute information of the image, the processor 801 may specifically perform the following steps:

[0194] Construct image processing tasks for images;

[0195] Call the task scheduling management engine to split the image processing task into N subtasks based on the image attribute information, and the N subtasks correspond one-to-one to the N connected domains;

[0196] The task scheduling management engine is called to allocate N subtasks to M search processing engines.

[0197] The task scheduling management engine allocates the N subtasks in at least one of the following ways:

[0198] Randomly assign N subtasks to M search processing engines;

[0199] According to the load balancing principle, N subtasks are assigned to M search processing engines;

[0200] According to the principle of minimizing the processing time of the image processing task, N subtasks are assigned to M search processing engines;

[0201] Each connected domain is marked with a label, and each label corresponds to its own index; N subtasks are assigned to M search processing engines in ascending order of their indexes;

[0202] Distribute N subtasks evenly to M search processing engines.

[0203] The processor 801 may specifically perform the following steps when scheduling the search processing engines corresponding to the connected domains in parallel and performing contour search processing on the connected domains:

[0204] Calling the task scheduling management engine to send parallel execution instructions to the M search processing engines;

[0205] By executing the instructions in parallel, the M search processing engines are controlled to execute their assigned subtasks in parallel.

[0206] Among them, N subtasks include subtask i, subtask i corresponds to connected domain I; M search processing engines include search processing engine j; subtask i is assigned to search processing engine j;

[0207] The process of performing contour search processing on the connected region I by the search processing engine j includes:

[0208] The search processing engine j obtains the connected domain I from the storage area according to the physical location of the connected domain I;

[0209] The search processing engine j performs contour search starting from the starting point according to the coordinates of the starting point of the connected region I in the image to obtain the boundary points of the connected region I;

[0210] Based on the starting point and boundary points of the connected region I, the outline of the connected region I is determined.

[0211] The processor 801 further performs the following steps:

[0212] Based on the results of the contour search processing of each connected domain, the contour corresponding to each connected domain is extracted from the image;

[0213] Perform recognition processing on each extracted contour to obtain the target object;

[0214] The target object includes at least one of the following: a license plate, text, subtitle, moving object, and region of interest.

[0215] In an embodiment of the present application, an image to be processed is obtained, and the image includes at least one connected domain; by marking at least one connected domain in the image, this can serve as the basis and prerequisite for subsequent contour search processing. The attribute information of the image is detected, and the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; by analyzing and detecting the attribute information of the image in advance before performing contour search processing on the image, the necessary data required for processing can be provided for the subsequent contour search processing, thereby facilitating the improvement of the efficiency of the subsequent contour search processing. Based on the attribute information of the image, a search processing engine is assigned to each connected domain, and the search processing engines corresponding to each connected domain are scheduled in parallel to perform contour search processing on each connected domain. By flexibly assigning a search processing engine to each connected domain and scheduling each search processing engine in parallel, each search processing engine can be controlled to quickly perform contour search processing on each connected domain in parallel based on the attribute information of the image, thereby greatly improving the efficiency of image contour search.

[0216] In addition, it should be noted that the present invention also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 3 and Figure 5 The method in the corresponding embodiment will therefore not be described in detail here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on an image processing device, or executed on multiple image processing devices located in one location, or executed on multiple image processing devices distributed in multiple locations and interconnected by a communication network.

[0217] According to one aspect of the present application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of an image processing device reads the computer program from the computer-readable storage medium and executes the computer program, so that the image processing device can perform the above-mentioned Figure 3 and Figure 5 The method in the corresponding embodiment will therefore not be described in detail here.

[0218] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0219] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method is executed by calling an integrated chip, wherein the integrated chip includes a task scheduling management engine and M search processing engines; the method includes: Acquire an image to be processed, wherein the image includes at least one connected domain; Detecting attribute information of the image, the attribute information including the number of connected domains in the image, the physical location of each connected domain, and the coordinates of the starting point of each connected domain in the image; Based on the attribute information of the image, a search processing engine is assigned to each connected domain, one connected domain is assigned to one search processing engine, or multiple connected domains are assigned to one search processing engine; the number of connected domains in the image is N, and the N connected domains correspond one-to-one to the N subtasks of the image processing task; M and N are both integers greater than 1; Parallel scheduling of search processing engines corresponding to the connected domains to perform contour search processing on the connected domains; The task scheduling management engine allocates the N subtasks in the following manner: Randomly assigning the N subtasks to the M search processing engines; Allocating the N subtasks to the M search processing engines according to a load balancing principle; Allocating the N subtasks to the M search processing engines according to the principle of minimizing the processing time of the image processing task; Each of the connected domains is marked with a label, and each of the labels corresponds to a respective index, and the N subtasks are assigned to the M search processing engines in ascending order of the indexes; The N subtasks are evenly distributed to the M search processing engines.

2. The method according to claim 1, wherein The integrated chip includes a tag processing engine; the method further includes: Scheduling the marking processing engine to perform connected component marking processing on the image to obtain a marking result; The marking result includes a label marked for each connected domain in the image and an index created for each label; the label of each connected domain is used to uniquely identify the corresponding connected domain.

3. The method according to claim 2, wherein The integrated chip further includes a data handling engine and a scanning processing engine; the detecting of attribute information of the image includes: Invoking the data handling engine to write each of the connected domains and the marking results into a storage area, wherein the storage area includes a memory and / or an on-chip cache; During the writing process, the scanning processing engine is called to detect the attribute information of the image in real time.

4. The method according to claim 3, wherein During the writing process, calling the scanning processing engine to detect the attribute information of the image in real time includes: During the writing process, the scanning processing engine is called to perform statistical analysis on the number of labels in the marking result to obtain the number of connected domains in the image; Calling the scanning processing engine to sequentially record the physical addresses of the tags corresponding to the respective indexes in the storage area according to the indexes of the tags, and obtaining the physical locations of the respective connected domains; and The scanning processing engine is called to sequentially detect the coordinates of the starting points of the connected regions in the image according to the indexes of the tags.

5. The method according to any one of claims 1 to 4, characterized in that Allocating a search processing engine to each of the connected domains based on the attribute information of the image includes: constructing the image processing task for the image; Invoking the task scheduling management engine to split the image processing task into the N subtasks based on the number of connected domains in the image included in the attribute information of the image; The task scheduling management engine is called to allocate the N subtasks to the M search processing engines.

6. The method according to claim 5, wherein The parallel scheduling of the search processing engines corresponding to the connected domains to perform contour search processing on the connected domains includes: Invoke the task scheduling management engine to send parallel execution instructions to the M search processing engines; The M search processing engines are controlled to execute the assigned subtasks in parallel through the parallel execution instructions.

7. The method according to claim 5, wherein The N subtasks include subtask i, and the subtask i corresponds to a connected domain I; the M search processing engines include search processing engine j; The subtask i is assigned to the search processing engine j; The process of the search processing engine j performing contour search processing on the connected region I includes: The search processing engine j obtains the connected domain I from a storage area according to the physical location of the connected domain I; The search processing engine j performs contour search starting from the starting point of the connected region I according to the coordinates of the starting point in the image to obtain the boundary points of the connected region I; Based on the starting point and boundary points of the connected region I, the outline of the connected region I is determined.

8. The method according to claim 1, wherein The method further comprises: extracting contours corresponding to the connected domains from the image based on the results of the contour search process of the connected domains; Performing recognition processing on each of the extracted contours to obtain a target object; The target object includes at least one of the following: a license plate, text, subtitle, a moving object, and a region of interest.

9. An image processing device, characterized in that: The device is provided on an integrated chip, wherein the integrated chip includes a task scheduling management engine and M search processing engines; the device includes: an acquisition unit, configured to acquire an image to be processed, wherein the image includes at least one connected domain; a processing unit, configured to detect attribute information of the image, wherein the attribute information includes the number of connected domains in the image, the physical location of each connected domain, and the coordinates of a starting point of each connected domain in the image; The processing unit is further configured to assign a search processing engine to each connected domain based on attribute information of the image, wherein one connected domain is assigned one search processing engine, or one search processing engine is assigned to multiple connected domains; the number of connected domains in the image is N, and the N connected domains correspond one-to-one to the N subtasks of the image processing task; and M and N are both integers greater than 1; The processing unit is further configured to schedule the search processing engines corresponding to the connected domains in parallel, and perform contour search processing on the connected domains; The task scheduling management engine allocates the N subtasks in the following manner: Randomly assigning the N subtasks to the M search processing engines; Allocating the N subtasks to the M search processing engines according to a load balancing principle; Allocating the N subtasks to the M search processing engines according to the principle of minimizing the processing time of the image processing task; Each of the connected domains is marked with a label, and each of the labels corresponds to a respective index, and the N subtasks are assigned to the M search processing engines in ascending order of the indexes; The N subtasks are evenly distributed to the M search processing engines.

10. An integrated chip, characterized in that: The integrated chip includes a storage area and at least one engine; the storage area includes a memory and / or an on-chip cache; wherein, The integrated chip schedules each of the engines to execute the method according to any one of claims 1-8.

11. An image processing device, characterized in that: Includes processor, input device, output device and memory; The memory stores a computer program; When the computer program is executed by the processor, the image processing method according to any one of claims 1 to 8 is performed.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is executed.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is executed.

Images