A connected domain search method, device and connected domain searcher

Abstract

The embodiment of the present invention provides a connected domain retrieval method, device and connected domain retriever, the method comprising: obtaining a valid target pixel point in the target image data to be retrieved, and obtaining valid pixel points around the target pixel point; retrieving the connectivity between the valid pixel point and the target pixel point; when a valid pixel point and the target pixel point have connectivity, determining that the valid pixel point and the target pixel point belong to the same connected domain; at least one valid pixel point can be used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point is re-executed based on the new target pixel point; when there are no valid pixels around the unretrieved pixel points in the target image data, outputting the connected domain retrieval result for the target image data based on the connected domain to which each pixel point belongs. By simultaneously reading and retrieving the valid pixel points around the target pixel point, the efficiency of the connected domain retrieval is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a connected domain retrieval method, device and a connected domain retriever. Background Art

[0002] SGM (Semi-Global Matching) is a matching method based on binocular stereo vision. As a matching method between local matching and global matching, it is faster and more efficient than the global matching method. It combines SGM with post-processing to improve support for weak textures and repeated textures, and its accuracy is comparable to that of the global matching method. At present, the depth estimation method based on SGM is widely used in applications such as drones and autonomous driving.

[0003] The SGM method processing includes two parts: pre-processing and post-processing. The pre-processing includes center transformation, cost calculation, cost aggregation and disparity calculation. The common post-processing operations include filtering, connected domain retrieval, hole filling and other optimization methods, which are used to improve disparity accuracy and eliminate abnormal points.

[0004] In the process of implementing the present invention, the inventors found that connected domain retrieval is the most computationally complex part in SGM post-processing. Therefore, improving the implementation efficiency of the connected domain retrieval algorithm is the key to improving the system throughput. Summary of the invention

[0005] Based on this, on one hand, a connected domain retrieval method is provided to improve the efficiency of connected domain retrieval.

[0006] On the other hand, a connected domain retrieval device and a connected domain retriever are also provided to ensure the implementation and application of the above method in practice.

[0007] A connected domain retrieval method is provided, comprising:

[0008] Acquire a valid target pixel point in the target image data to be retrieved, and acquire at least one valid pixel point around the target pixel point;

[0009] Retrieving connectivity between the at least one valid pixel and the target pixel;

[0010] When a valid pixel point has connectivity with the target pixel point, determining that the valid pixel point and the target pixel point belong to the same connected domain;

[0011] The at least one valid pixel point is used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point is re-executed based on the new target pixel point;

[0012] When there are no valid pixels around the unsearched pixels in the target image data, a connected domain search result for the target image data is output based on the connected domain to which each pixel belongs.

[0013] Optionally, the method further comprises:

[0014] Storing a plurality of pixels in the target image data in a cache module;

[0015] The step of obtaining a valid target pixel point in the target image data to be retrieved includes:

[0016] The target pixel is obtained from the cache module.

[0017] Optionally, acquiring at least one valid pixel point around the target pixel point includes:

[0018] Determine at least one pixel point around the target pixel point, and determine a first layer mask corresponding to the at least one pixel point; the first layer mask is used to indicate the validity of the corresponding pixel point;

[0019] Determine the at least one valid pixel point according to the first layer mask, and determine the target address information of each valid pixel point in the cache module;

[0020] According to the target address information, the at least one valid pixel is obtained from the cache module.

[0021] Optionally, the cache module includes a first-level cache submodule and a second-level cache submodule, and acquiring the at least one valid pixel from the cache module according to the target address information includes:

[0022] Acquire the at least one valid pixel from the first cache submodule according to the target address information;

[0023] Cache the at least one valid pixel point in the second-level cache submodule;

[0024] The at least one valid pixel is obtained from the second-level cache submodule.

[0025] Optionally, the method further comprises:

[0026] For each pixel point cached in the cache module, corresponding record information is generated; the record information is used to indicate whether the corresponding pixel point has been retrieved;

[0027] The obtaining the at least one valid pixel from the cache module comprises:

[0028] According to the record information, valid pixels that have not been retrieved are obtained from the cache module.

[0029] Optionally, the method further comprises:

[0030] Based on the connectivity between the valid pixel point and the target pixel point, a corresponding second layer mask is generated; the second layer mask is used to determine whether the corresponding valid pixel point belongs to the connected domain of the target pixel point.

[0031] A connected domain retrieval device is provided, comprising:

[0032] A retrieval module, used to obtain a valid target pixel point in the target image data to be retrieved, and obtain at least one valid pixel point around the target pixel point; retrieve the connectivity between the at least one valid pixel point and the target pixel point; when a valid pixel point and the target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain; use the at least one valid pixel point as a new target pixel point, and re-execute the step of obtaining at least one valid pixel point around the target pixel point based on the new target pixel point;

[0033] The output module is used to output the connected domain search result for the target image data based on the connected domain to which each pixel belongs when there are no valid pixels around the unsearched pixel in the target image data.

[0034] Optionally, the device further comprises:

[0035] A pixel point cache module, used for storing a plurality of pixel points in the target image data into the cache module;

[0036] The retrieval module is used to obtain the target pixel point from the cache module.

[0037] Optionally, the retrieval module is used to determine at least one pixel point around the target pixel point, and determine a first layer mask corresponding to the at least one pixel point; the first layer mask is used to indicate the validity of the corresponding pixel point; based on the first layer mask, the at least one valid pixel point is determined, and the target address information of each valid pixel point in the cache module is determined; based on the target address information, the at least one valid pixel point is obtained from the cache module.

[0038] Optionally, the cache module includes a first-level cache sub-module and a second-level cache sub-module, and the retrieval module is used to obtain the at least one valid pixel point from the first cache sub-module according to the target address information; cache the at least one valid pixel point to the second-level cache sub-module; and obtain the at least one valid pixel point from the second-level cache sub-module.

[0039] Optionally, the device further comprises:

[0040] A marking module, used for generating corresponding record information for each pixel point cached in the cache module; the record information is used to indicate whether the corresponding pixel point has been retrieved;

[0041] The retrieval module is used to obtain valid pixels that have not been retrieved from the cache module according to the record information.

[0042] Optionally, the device further comprises:

[0043] The second mask generation module is used to generate a corresponding second layer mask based on the connectivity between the valid pixel point and the target pixel point; the second layer mask is used to determine whether the corresponding valid pixel point belongs to the connected domain of the target pixel point.

[0044] A connected domain searcher is provided, comprising: a cache module and a connected domain search module;

[0045] The cache module is used to cache the pixels in the image data to be retrieved;

[0046] The connected domain retrieval module is used to obtain a valid target pixel point in the target image data to be retrieved from the cache module, and obtain at least one valid pixel point around the target pixel point; retrieve the connectivity between the at least one valid pixel point and the target pixel point; when a valid pixel point and the target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain; use the at least one valid pixel point as a new target pixel point, and re-execute the step of obtaining at least one valid pixel point around the target pixel point based on the new target pixel point; when there are no valid pixels around the pixels that have not been retrieved in the target image data, output the connected domain retrieval result for the target image data based on the connected domain to which each pixel point belongs.

[0047] Optionally, the cache module includes a first-level cache submodule and a second-level cache submodule;

[0048] The first-level cache submodule is used to cache pixel points in the target image data to be retrieved;

[0049] The second-level cache submodule is used to obtain the at least one valid pixel from the first cache submodule according to the target address information; and cache the at least one valid pixel.

[0050] Optionally, the second level cache submodule is composed of a register file

[0051] In addition, an electronic device is provided, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the above connected domain retrieval method when executed by the processor.

[0052] Furthermore, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the connected domain retrieval method described above is implemented.

[0053] The above technical solution has at least the following beneficial effects:

[0054] In an embodiment of the present invention, a valid target pixel point in the target image data to be retrieved can be first obtained, and at least one valid pixel point around the target pixel point can be obtained; then the connectivity between the at least one valid pixel point and the target pixel point can be retrieved; when a valid pixel point and the target pixel point have connectivity, it is determined that the valid pixel point and the target pixel point belong to the same connected domain; then, at least one valid pixel point can be used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point can be re-executed based on the new target pixel point; when there are no valid pixels around the unretrieved pixel points in the target image data, the connected domain retrieval result for the target image data is output based on the connected domain to which each pixel point belongs. By simultaneously reading and retrieving at least one valid pixel point around the target pixel point, the efficiency of the connected domain retrieval is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0056] Figure 1 is a flowchart of a method for searching a connected domain according to an embodiment of the present invention;

[0057] Figure 2 is a flowchart of another method for searching a connected domain according to an embodiment of the present invention;

[0058] Figure 3is a schematic diagram of a pixel point according to an embodiment of the present invention;

[0059] Figure 4 is a schematic structural diagram of a connected domain searcher according to an embodiment of the present invention;

[0060] Figure 5 is a schematic structural diagram of another connected domain searcher according to an embodiment of the present invention;

[0061] Figure 6 It is a partial structural diagram of a connected domain searcher according to an embodiment of the present invention;

[0062] Figure 7 is a schematic diagram of another pixel point according to an embodiment of the present invention;

[0063] Figure 8 is a flow chart of the steps of state changes of a state machine according to an embodiment of the present invention;

[0064] Fig. 9 It is a partial structural diagram of another connected domain searcher according to an embodiment of the present invention;

[0065] Fig.10 A structural block diagram of a connected domain retrieval device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Reference Figure 1 , shows a flowchart of a method for searching a connected domain according to an embodiment of the present invention, which may include the following steps:

[0068] Step 101: Acquire a valid target pixel point in the target image data to be retrieved, and acquire at least one valid pixel point around the target pixel point.

[0069] Among them, the target image data can be image data that will be searched for a connected domain. The target image data can be applied to image data of drones, automatic driving, etc., and can also be applied to other image data that need to be searched for a connected domain. The embodiment of the present invention does not limit this.

[0070] The target pixel point can be a valid pixel point on the target image data; that is, a pixel point that meets the preset conditions, for example, a pixel point that belongs to a preset area. The target pixel point can be a valid pixel point randomly obtained from the target image data, or a valid pixel point that has been previously searched for a connected domain.

[0071] When it is necessary to search for connectivity of pixels around a target pixel to determine whether the pixels around the target pixel are connected to the target pixel, at least one valid pixel around the target pixel and adjacent to the target pixel may be first obtained.

[0072] Step 102: Retrieve the connectivity between at least one valid pixel and a target pixel.

[0073] After obtaining at least one valid pixel point for the target pixel point, the connectivity between each valid pixel point and the target pixel point may be retrieved based on a preset connectivity algorithm.

[0074] The preset connectivity algorithm may be set according to actual needs; specifically, the connectivity algorithm may be set according to the definition of required connectivity; for example, the corresponding connectivity algorithm may be set based on the definition of character connectivity.

[0075] Step 103: When a valid pixel point and a target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain.

[0076] In practical applications, if it is determined that any valid pixel point among at least one valid pixel point has connectivity with the target pixel point, it can be determined that the any valid pixel point and the target pixel point belong to the same connected domain.

[0077] At this time, any valid pixel point can be marked, so that the connected domain retrieval result for the target image data can be output based on the mark later; that is, the retrieval result of which pixels in the target image data belong to the same connected domain.

[0078] Step 104: taking at least one valid pixel point as a new target pixel point, and re-performing the step of acquiring at least one valid pixel point around the target pixel point based on the new target pixel point.

[0079] After performing connectivity search on at least one valid pixel point around the target pixel point, the at least one valid pixel point can be used as a new target pixel point.

[0080] Then, steps 102 to 103 may be re-executed based on these new target pixels respectively until there are no valid pixels around the pixels that have not been retrieved in the target image data.

[0081] Step 105 : When there are no valid pixels around the unretrieved pixels in the target image data, output the connected domain retrieval result for the target image data based on the connected domain to which each pixel belongs.

[0082] In practical applications, when there are no valid pixels around the unretrieved pixels in the target image data, it can be said that the target image data has been retrieved; at this time, based on the connected domain to which each pixel determined by the retrieval belongs, a connected domain retrieval result for the target image data can be generated; wherein the connected domain retrieval result can be used to indicate which pixels in the target image data belong to the same connected domain.

[0083] After generating the connected domain retrieval result for the target image data, the connected domain retrieval result for the target image data may be output so that a downstream application / device may identify / process the target image data based on the connected domain retrieval result.

[0084] In an embodiment of the present invention, a valid target pixel point in the target image data to be retrieved can be first obtained, and at least one valid pixel point around the target pixel point can be obtained; then the connectivity between the at least one valid pixel point and the target pixel point can be retrieved; when a valid pixel point and the target pixel point have connectivity, it is determined that the valid pixel point and the target pixel point belong to the same connected domain; then, at least one valid pixel point can be used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point can be re-executed based on the new target pixel point; when there are no valid pixels around the unretrieved pixel points in the target image data, the connected domain retrieval result for the target image data is output based on the connected domain to which each pixel point belongs. By simultaneously reading and retrieving at least one valid pixel point around the target pixel point, the efficiency of the connected domain retrieval is effectively improved.

[0085] Reference Figure 2 , shows a flowchart of another method for searching a connected domain according to an embodiment of the present invention, which may include the following steps:

[0086] Step 201: store multiple pixels in the target image data into a cache module.

[0087] In practical applications, after the target image data is acquired, a plurality of pixels in the target image data may be stored in a cache module.

[0088] Step 202: Obtain target pixel points from the cache module.

[0089] When the connected domain of the target image data needs to be retrieved, a valid pixel point can be randomly obtained from the cache module as the target pixel point; or, a valid pixel point that has been retrieved can be used as the target pixel point, which is not limited in the embodiment of the present invention.

[0090] Step 203: determine at least one pixel point around the target pixel point, and determine a first layer mask corresponding to the at least one pixel point; the first layer mask is used to indicate the validity of the corresponding pixel point.

[0091] After obtaining the target pixel point, at least one pixel point adjacent to the target pixel point can be determined; Figure 3 , the eight pixels around the target pixel can be regarded as the pixels around the target pixel.

[0092] After determining at least one pixel point around the target pixel point, a first layer mask for indicating the validity of the corresponding pixel point may be generated based on the validity of the surrounding pixel points respectively.

[0093] For example, 0 represents invalid and 1 represents valid, so "10-1" can be used to represent that pixel 10 is valid, and "11-0" can be used to represent that pixel 10 is not valid. The embodiment of the present invention is not limited to this.

[0094] Step 204: Determine at least one valid pixel point according to the first layer mask, and determine the target address information of each valid pixel point in the cache module.

[0095] After determining the first layer mask corresponding to at least one pixel point around the target pixel point, at least one valid pixel point can be determined from the at least one pixel point around the target pixel point according to the first layer mask.

[0096] For example: if the first layer mask corresponding to at least one pixel point around the target pixel point is "0-0", "1-0", "2-0", "3-0", "5-1", "6-0", "7-1", "8-1", then pixels 5, 7 and 8 can be used as valid pixels for the target pixel point.

[0097] After determining at least one valid pixel for the target pixel, target address information of each valid pixel in the cache module may be determined respectively; the target address information may be used to indicate the location where the valid pixel is stored in the cache module.

[0098] Step 205: Obtain at least one valid pixel from the cache module according to the target address information.

[0099] After the target address information of each valid pixel is obtained, at least one valid pixel for the target pixel may be obtained from the cache module based on the target address information.

[0100] In practical applications, in order to improve the efficiency of random access connected domain retrieval, a multi-level cache can be set; specifically, the cache module can include a first-level cache submodule and a second-level cache submodule. Therefore, when obtaining at least one valid pixel point, it can be achieved through the following sub-steps:

[0101] Sub-step 11: acquiring at least one valid pixel from the first cache sub-module according to the target address information.

[0102] The target address information may be used to indicate the location where the pixel point is stored in the first cache submodule.

[0103] After the target address information is obtained, at least one valid pixel point for the target pixel point may be acquired from the first buffer submodule based on the target address information.

[0104] Sub-step 12: Cache at least one valid pixel point in the second-level cache submodule.

[0105] After obtaining at least one valid pixel point for the target pixel point from the first cache submodule, it may be cached in the second-level cache submodule.

[0106] As an example, the second-level cache submodule may be composed of a register file to further improve the efficiency of the connected domain retrieval.

[0107] Sub-step 13: Acquire at least one valid pixel from the second-level cache submodule.

[0108] After the valid pixel points are cached in the second-level cache submodule, when a connected domain search is required, at least one valid pixel point for the target pixel point can be simultaneously obtained from the second-level cache submodule.

[0109] Step 206: Retrieve the connectivity between at least one valid pixel and the target pixel.

[0110] After obtaining at least one valid pixel point for the target pixel point, the connectivity between each valid pixel point and the target pixel point may be retrieved based on a preset connectivity algorithm.

[0111] In one embodiment of the present invention, in order to avoid repeated retrieval of the same pixel point, each pixel point can also be marked based on the retrieval process; specifically, corresponding record information can be generated for each pixel point cached in the cache module; the record information is used to indicate whether the corresponding pixel point has been retrieved.

[0112] The record information may be stored in the first cache submodule to avoid repeatedly extracting the same pixel point to the second cache submodule, thereby avoiding repeatedly obtaining the same pixel point from the second cache submodule for retrieval and reducing the efficiency of the connected domain retrieval.

[0113] After determining whether a valid pixel point has connectivity with a target pixel point / not, the record information of the valid pixel point in the first-level cache submodule can be modified; thus, when the valid pixel point is subsequently cached from the first-level cache submodule to the second-level cache submodule, it can be determined based on the record information whether the valid pixel point currently to be processed has been retrieved.

[0114] In practical applications, before searching for at least one valid pixel, it is possible to determine whether the valid pixel has been searched based on the following steps, thereby avoiding repeated operations on the same valid pixel:

[0115] According to the recorded information, valid pixels that have not been retrieved are obtained from the cache module.

[0116] When it is necessary to read valid pixels from the first buffer submodule to the second buffer submodule, the record information of the corresponding valid pixels may be read first; and based on the record information, it is determined whether the corresponding valid pixels have been retrieved.

[0117] If it has been searched, the valid pixel is skipped and other valid pixels are processed, thereby avoiding reducing the efficiency of connected domain retrieval due to repeated retrieval of the same valid pixel.

[0118] Step 207: When a valid pixel point and a target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain.

[0119] In practical applications, if it is determined that any valid pixel point among at least one valid pixel point has connectivity with the target pixel point, it can be determined that the any valid pixel point and the target pixel point belong to the same connected domain.

[0120] At this time, any valid pixel point can be marked, so that the connected domain retrieval result for the target image data can be output based on the mark later; that is, the retrieval result of which pixels in the target image data belong to the same connected domain.

[0121] In one embodiment of the present invention, any valid pixel point can be marked by the following steps:

[0122] Based on the connectivity between the valid pixel points and the target pixel points, a corresponding second-layer mask is generated; the second-layer mask is used to determine whether the corresponding valid pixel points belong to the connected domain of the target pixel points.

[0123] In practical applications, after determining the connectivity between each valid pixel point and the target pixel point, a corresponding second-layer mask can be generated for each valid pixel point based on whether the connectivity exists; the second-layer mask can be used to indicate whether the corresponding valid pixel point belongs to the connected domain of the target pixel point.

[0124] Step 208: taking at least one valid pixel point as a new target pixel point, and re-performing the step of acquiring at least one valid pixel point around the target pixel point based on the new target pixel point.

[0125] After performing connectivity search on at least one valid pixel point around the target pixel point, the at least one valid pixel point can be used as a new target pixel point.

[0126] Then, the above steps may be re-executed based on these new target pixel points respectively until there are no valid pixel points around the pixel points that have not been retrieved in the target image data.

[0127] Step 209: When there are no valid pixels around the unretrieved pixels in the target image data, output the connected domain retrieval result for the target image data based on the connected domain to which each pixel belongs.

[0128] In practical applications, when there are no valid pixels around the unretrieved pixels in the target image data, it can be said that the target image data has been retrieved; at this time, based on the connected domain to which each pixel determined by the retrieval belongs, a connected domain retrieval result for the target image data can be generated; wherein the connected domain retrieval result can be used to indicate which pixels in the target image data belong to the same connected domain.

[0129] As an example, the connected domain retrieval result for the target image data may be performed based on the second layer mask of each pixel.

[0130] After generating the connected domain retrieval result for the target image data, the connected domain retrieval result for the target image data may be output so that a downstream application / device may identify / process the target image data based on the connected domain retrieval result.

[0131] As an example, the above-mentioned connected domain retrieval method can be applied to FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit) to meet the needs of FPGA or ASIC for high-efficiency connected domain retrieval, which is not limited in the embodiment of the present invention.

[0132] In an embodiment of the present invention, a plurality of pixels in the target image data can be first stored in a cache module; then the target pixel is obtained from the cache module; then at least one pixel around the target pixel is determined, and a first-layer mask corresponding to the at least one pixel is determined; the first-layer mask is used to indicate the validity of the corresponding pixel; according to the first-layer mask, at least one valid pixel is determined, and the target address information of each valid pixel in the cache module is determined; according to the target address information, at least one valid pixel is obtained from the cache module; the connectivity of at least one valid pixel with the target pixel is retrieved; when a valid pixel has connectivity with the target pixel, it is determined that the valid pixel and the target pixel belong to the same connected domain; at least one valid pixel is used as a new target pixel, and the step of obtaining at least one valid pixel around the target pixel is re-executed based on the new target pixel; when there are no valid pixels around the pixels that have not been retrieved in the target image data, the connected domain retrieval result for the target image data is output based on the connected domain to which each pixel belongs. The connected domain retrieval efficiency of random access is improved by using a two-level cache to retrieve the connected domain.

[0133] The two-layer mask processing retrieval can make the retrieval process more in line with pipeline processing. Based on the content of the two-layer mask, invalid data can be skipped during the retrieval process, thereby minimizing the access time and improving system performance.

[0134] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0135] Reference Figure 4 , shows a schematic diagram of the structure of a connected domain searcher according to an embodiment of the present invention, which may include the following modules: a cache module and a connected domain search module;

[0136] A cache module, used for caching pixels in the image data to be retrieved;

[0137] A connected domain retrieval module is used to obtain a valid target pixel point in the target image data to be retrieved from the cache module, and obtain at least one valid pixel point around the target pixel point; retrieve the connectivity between at least one valid pixel point and the target pixel point; when a valid pixel point and the target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain; use at least one valid pixel point as a new target pixel point, and re-execute the step of obtaining at least one valid pixel point around the target pixel point based on the new target pixel point; when there are no valid pixels around the pixels that have not been retrieved in the target image data, output the connected domain retrieval result for the target image data based on the connected domain to which each pixel point belongs.

[0138] In practical applications, after the target image data is acquired, a plurality of pixels in the target image data may be stored in a cache module.

[0139] When it is necessary to search for a connected domain of the target image data, the connected domain search module may randomly obtain a valid pixel point from the cache module as the target pixel point; or, use a valid pixel point that has been searched as the target pixel point, which is not limited in this embodiment of the present invention.

[0140] When it is necessary to perform a connectivity search on the pixels around a target pixel to determine whether the pixels around the target pixel are connected to the target pixel, the connected domain search module can first obtain at least one valid pixel around the target pixel and adjacent to the target pixel.

[0141] After acquiring at least one valid pixel point for the target pixel point, the connected domain retrieval module may retrieve the connectivity between each valid pixel point and the target pixel point based on a preset connectivity algorithm.

[0142] In practical applications, if it is determined that any valid pixel point among at least one valid pixel point has connectivity with the target pixel point, it can be determined that the any valid pixel point and the target pixel point belong to the same connected domain.

[0143] At this time, the connected domain retrieval module can mark any valid pixel point, so that the connected domain retrieval result for the target image data can be output based on the mark later; that is, the retrieval result of which pixels in the target image data belong to the same connected domain.

[0144] After performing connectivity search on at least one valid pixel point around the target pixel point, the connected domain search module may use the at least one valid pixel point as a new target pixel point.

[0145] Then, the connected domain retrieval module can reacquire surrounding valid pixels based on these new target pixels, and perform connected domain retrieval on the valid pixels.

[0146] When there are no valid pixels around the unretrieved pixels in the target image data, it can be said that the target image data has been retrieved; at this time, the connected domain retrieval module can generate a connected domain retrieval result for the target image data based on the connected domain to which each pixel determined by the retrieval belongs.

[0147] After generating the connected domain retrieval result for the target image data, the connected domain retrieval module may output the connected domain retrieval result for the target image data so that the downstream application / device may identify / process the target image data based on the connected domain retrieval result.

[0148] In one embodiment of the present invention, the cache module includes a first-level cache submodule and a second-level cache submodule; the first-level cache submodule is used to cache pixel points in the target image data to be retrieved; the second-level cache submodule is used to obtain at least one valid pixel point from the first cache submodule according to the target address information; and cache at least one valid pixel point.

[0149] In practical applications, in order to improve the efficiency of connected domain retrieval in random access, a multi-level cache can be set; specifically, the cache module can include a first-level cache submodule and a second-level cache submodule. Thus, when obtaining at least one valid pixel point, at least one valid pixel point for the target pixel point can be obtained from the first cache submodule based on the target address information.

[0150] After obtaining at least one valid pixel point for the target pixel point from the first cache submodule, it may be cached in the second-level cache submodule.

[0151] After the valid pixel points are cached in the second-level cache submodule, when a connected domain search is required, at least one valid pixel point for the target pixel point can be simultaneously obtained from the second-level cache submodule.

[0152] As an example, the second-level cache submodule may be composed of a register file to further improve the efficiency of the connected domain retrieval.

[0153] The following is an example of a connected domain searcher:

[0154] like Figure 5 and Figure 6 , the connected domain retriever may include a data aggregation and transmission control engine module (i.e., a first-level cache submodule) related to data reading and writing and data caching ( Figure 5② in ), speckle SearchBuf (connected domain search cache) (i.e., the second-level cache submodule), and the breadth-first connected domain search module ( Figure 5 ①); The interaction between the data aggregation and transmission control engine module and the connected domain retrieval module is carried out through the connected domain retrieval cache.

[0155] For the connected domain retrieval module:

[0156] The connected domain retrieval module can be composed of addr Calc (address generation module), get One+getMask (single monitoring confirmation module) and FSM (finite-state machine).

[0157] The speckle SearchBuf provides input for the connected domain search module. The connected domain search results can be output to the Speckle Table (connected domain storage table) and recorded through the WriteBack operation.

[0158] The speckle Search Buf consists of two separate parts: scanTable and visitedTable; visitedTable records the status information of the entire connected domain search process from a global perspective, for example: the connected domain to which each pixel corresponds; scanTable is used to record the results of the current connected domain search.

[0159] The speckle SearchBuf can be composed of register stacks to improve the efficiency of connected domain retrieval.

[0160] get one is used to obtain the target pixel and the first-layer mask corresponding to the pixels around the target pixel. After generating the first-layer mask, getone can pass it to addrCalc so that addrCalc can obtain the address information of the valid pixels around the target pixel based on the first-layer mask. The first-layer mask can be used to instruct the FSM to control the retrieval of the next pixel.

[0161] addrCalc is used to generate the address of valid pixels based on the first-layer mask and pass it to get Mask. getMask is used to complete the connected domain algorithm and generate the corresponding second-layer mask. The second-layer mask can be used to indicate whether the pixels around the target pixel are retained for the next retrieval.

[0162] like Figure 7 As shown:

[0163] For the target pixel point (c1, r1), its surrounding valid pixel points (c0, r2), (c2, r0) and (c2, r1) are reserved for the next retrieval.

[0164] After the valid pixel point (c2, r0) is taken as the new target pixel point, its surrounding pixels may include (c2, r-1), (c3, r-1), (c4, r-1), (c2, r-1), (c2, r0), (c4, r0), (c2, r1), (c3, r1) and (c4, r1); among them, (c4, r0) can be taken as the next new target pixel point and the retrieval continues.

[0165] FSM is used to control the cooperation between modules, such as Figure 8 As shown:

[0166] FSM is a nested state machine. The main state machine completes the entire algorithm process including data acquisition, result storage and connected domain detection, and the slave state machine completes the given connected domain detection algorithm calculation.

[0167] The states of the main state machine are: "Idle - idle", "ReqData - request data", "ScanDomain - calculate connected domain", "RspData - send calculated connected graph", and "Finish - connected domain detection process completed".

[0168] The states of the state machine are: "Idle - idle", "HasOne - preliminary validity Mask", "ScanSurround - scan the eight points around the current retrieval point", "HasMask - calculate whether the point with valid value is in the connected domain according to the definition rules of the connected domain", "Finish - the calculation of the given connected domain detection algorithm is completed", "qaddrmask_valid - if at least one point in the connected domain to be detected is detected, the state machine jumps and retrieves again whether the surrounding points also belong to the connected domain".

[0169] Using breadth-first search, the number of iterations of the search state depends on the shape of the connected domain. For any search, it needs to go through the HasOne and HasMask states, which represent:

[0170] 1. For a given pixel, the coordinate distribution of surrounding valid pixels;

[0171] 2. Retrieve and select one of the valid pixel points, and pass it to the next retrieval in cascade; if it is found that there is no valid pixel point after the retrieval, restore the previous status information and return to the last retrieval position to continue detection.

[0172] For data aggregation and transmission control engine modules:

[0173] The data aggregation and transmission control engine module can consist of two parts, such as Fig. 9 :

[0174] ①Data cache module composed of buf_data (data buffer unit) and checkertable (data checking unit).

[0175] ②Data reading and writing module composed of addr_convert (address conversion), Reader (cache reader) and Writer (cache writer).

[0176] The data cache module can be composed of 9 sub-Blocks; among them, the 1st, 2nd, 4th and 5th sub-Blocks can be coupled into a coupled Block block, the 2nd, 3rd, 5th and 6th sub-Blocks can be coupled into a coupled Block block, the 4th, 5th, 7th and 8th sub-Blocks can be coupled into a coupled Block block, and the 5th, 6th, 8th and 9th sub-Blocks can be coupled into a coupled Block block.

[0177] Each sub-block is composed of buf_data and checker table. The data cache of the 9 sub-blocks can come from DDR (Double Data Rate SDRAM, double rate SDRAM (synchronous dynamic random-access memory, synchronous dynamic random access memory)) memory and is updated in blocks. When the connected domain retrieval module finishes and starts reading adjacent sub-blocks, the used blocks will be updated. The update rules are as follows:

[0178] From right to left and from bottom to top. The buf_data and checkertable in the sub-block are separate storage blocks. Since the writing rules of buf_data and checker table may be different in some applications, buf_data is a data block implemented by on-chip RAM storage, and checkertable is an identification block implemented by on-chip register stack. In order to improve performance, they will be updated separately, so there will be two update units corresponding to buf_data and checkertable.

[0179] The data reading and writing module can be composed of addr_convert, Reader and Writer; for the connected domain retrieval module, it only retrieves one sub-Block each time, and the connected domain retrieval module uses relative addresses, so read and write conversion operations are required when performing read and write operations. The conversion rule is to convert the relative address of the retrieval module in the sub-Block into the absolute address of 9 data caches.

[0180] The search sub-block (i.e., coupled block) is generated by the data read-write module, and the generation rule is the first position that has not been searched. The data read-write module will slide on the 9 sub-blocks to find the starting address of the first cache block that has not been searched, and update the sub-block starting with the starting address to the speckleSearch buf cache. This completes a round of data transmission, that is, the data is taken out from the DDR in blocks.

[0181] The data of the retrieved sub-block is filled into the sub-block that needs to be updated among the 9 sub-blocks, and the data of the retrieved sub-block is updated into specklSearchbuf.

[0182] The data read / write module will also return the information generated by the retrieval process to the DDR or subsequent stage in the form of stream data out.

[0183] The data update jump number of DDR is in blocks, while the update of output speckelSearchbuf is in data units. The specific value is determined by checkertable.

[0184] The design of the data cache module adopts the sub-block reading and writing method, which fully utilizes the DDR bandwidth, and because of the design of the secondary cache, the time overhead of the retrieval algorithm can be fully covered. Compared with the software algorithm, it can effectively improve the efficiency of the connected domain retrieval.

[0185] The on-board test experiment shows that for each retrieval of a connected domain point (i.e., a pixel point), the average delay for one point is 10 clock cycles; for each retrieval of two connected domain points, the average delay for one point is 6 clock cycles; for each retrieval of three connected domain points, the average delay for one point is 4 clock cycles; and for each retrieval of four connected domain points, the average delay for one point is 3 clock cycles.

[0186] Since each connected domain accounts for a small proportion, the delay is significantly reduced compared to the CPU (Central Processing Unit / Processor) software to implement connected domain retrieval.

[0187] Reference Fig.10 , shows a schematic diagram of the structure of a connected domain search device according to an embodiment of the present invention, which may include the following modules:

[0188] The retrieval module 1001 is used to obtain a valid target pixel point in the target image data to be retrieved, and obtain at least one valid pixel point around the target pixel point; retrieve the connectivity between the at least one valid pixel point and the target pixel point; when the valid pixel point and the target pixel point have connectivity, determine that the valid pixel point and the target pixel point belong to the same connected domain; use the at least one valid pixel point as a new target pixel point, and re-execute the step of obtaining at least one valid pixel point around the target pixel point based on the new target pixel point;

[0189] The output module 1002 is used to output the connected domain search result for the target image data based on the connected domain to which each pixel belongs when there are no valid pixels around the unsearched pixel in the target image data.

[0190] In an optional embodiment of the present invention, the device further comprises:

[0191] A pixel cache module, used for storing multiple pixel points in the target image data into the cache module;

[0192] The retrieval module is used to obtain the target pixel point from the cache module.

[0193] In an optional embodiment of the present invention, the retrieval module 1001 is used to determine at least one pixel point around the target pixel point, and determine a first layer mask corresponding to the at least one pixel point; the first layer mask is used to indicate the validity of the corresponding pixel point; based on the first layer mask, at least one valid pixel point is determined, and the target address information of each valid pixel point in the cache module is determined; based on the target address information, at least one valid pixel point is obtained from the cache module.

[0194] In an optional embodiment of the present invention, the cache module includes a first-level cache sub-module and a second-level cache sub-module, and the retrieval module 1001 is used to obtain at least one valid pixel point from the first cache sub-module according to the target address information; cache at least one valid pixel point to the second-level cache sub-module; and obtain at least one valid pixel point from the second-level cache sub-module.

[0195] In an optional embodiment of the present invention, the device further comprises:

[0196] The marking module is used to generate corresponding record information for each pixel point cached in the cache module; the record information is used to indicate whether the corresponding pixel point has been retrieved;

[0197] The retrieval module 1001 is used to obtain valid pixels that have not been retrieved from the cache module according to the record information.

[0198] In an optional embodiment of the present invention, the device further comprises:

[0199] The second mask generation module is used to generate a corresponding second-layer mask based on the connectivity between the valid pixel points and the target pixel points; the second-layer mask is used to determine whether the corresponding valid pixel points belong to the connected domain of the target pixel points.

[0200] In an embodiment of the present invention, a valid target pixel point in the target image data to be retrieved can be first obtained, and at least one valid pixel point around the target pixel point can be obtained; then the connectivity between the at least one valid pixel point and the target pixel point can be retrieved; when a valid pixel point and the target pixel point have connectivity, it is determined that the valid pixel point and the target pixel point belong to the same connected domain; then, at least one valid pixel point can be used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point can be re-executed based on the new target pixel point; when there are no valid pixels around the unretrieved pixel points in the target image data, the connected domain retrieval result for the target image data is output based on the connected domain to which each pixel point belongs. By simultaneously reading and retrieving at least one valid pixel point around the target pixel point, the efficiency of the connected domain retrieval is effectively improved.

[0201] An embodiment of the present invention further provides an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the above connected domain retrieval method is implemented.

[0202] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned connected domain retrieval method is implemented.

[0203] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0204] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0205] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0206] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0207] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0209] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0210] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0211] The above provides a detailed introduction to a connected domain retrieval method, device and a connected domain retriever. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for searching a connected domain, characterized in that: The method comprises: Acquire a valid target pixel point in the target image data to be retrieved, determine a first layer mask corresponding to at least one pixel point around the target pixel point; acquire at least one valid pixel point around the target pixel point according to the first layer mask; Retrieving connectivity between the at least one valid pixel and the target pixel; generating a corresponding second-layer mask based on the connectivity between the valid pixel and the target pixel; the second-layer mask is used to determine whether the corresponding valid pixel belongs to a connected domain of the target pixel; When a valid pixel point has connectivity with the target pixel point, determining that the valid pixel point and the target pixel point belong to the same connected domain; The at least one valid pixel point is used as a new target pixel point, and the step of obtaining at least one valid pixel point around the target pixel point is re-executed based on the new target pixel point; When there are no valid pixels around the unretrieved pixels in the target image data, a connected domain retrieval result for the target image data is output based on the connected domain to which each pixel belongs determined according to the second layer mask retrieval.

2. The method according to claim 1, characterized in that: The method further comprises: Storing a plurality of pixels in the target image data in a cache module; The step of obtaining a valid target pixel point in the target image data to be retrieved includes: The target pixel is obtained from the cache module.

3. The method according to claim 2, characterized in that The acquiring at least one valid pixel point around the target pixel point comprises: Determine at least one pixel point around the target pixel point, and determine a first layer mask corresponding to the at least one pixel point; the first layer mask is used to indicate the validity of the corresponding pixel point; Determine the at least one valid pixel point according to the first layer mask, and determine the target address information of each valid pixel point in the cache module; According to the target address information, the at least one valid pixel is obtained from the cache module.

4. The method according to claim 3, characterized in that The cache module includes a first-level cache submodule and a second-level cache submodule, and acquiring the at least one valid pixel from the cache module according to the target address information includes: According to the target address information, acquiring the at least one valid pixel from the first-level cache submodule; Cache the at least one valid pixel point in the second-level cache submodule; The at least one valid pixel is obtained from the second-level cache submodule.

5. The method according to claim 3, characterized in that: The method further comprises: For each pixel point cached in the cache module, corresponding record information is generated; the record information is used to indicate whether the corresponding pixel point has been retrieved; The obtaining the at least one valid pixel from the cache module comprises: According to the record information, valid pixels that have not been retrieved are obtained from the cache module.

6. The method according to claim 1, characterized in that The method further comprises: Based on the connectivity between the valid pixel point and the target pixel point, a corresponding second layer mask is generated; the second layer mask is used to determine whether the corresponding valid pixel point belongs to the connected domain of the target pixel point.

7. A connected domain searcher, characterized in that: It includes a cache module and a connected domain retrieval module; The cache module is used to cache the pixels in the image data to be retrieved; The connected domain retrieval module is used to obtain a valid target pixel point in the target image data to be retrieved from the cache module, determine a first layer mask corresponding to at least one pixel point around the target pixel point; obtain at least one valid pixel point around the target pixel point according to the first layer mask; and retrieve the connectivity between the at least one valid pixel point and the target pixel point; Based on the connectivity between the valid pixel point and the target pixel point, generating a corresponding second layer mask; The second layer mask is used to determine whether the corresponding valid pixel point belongs to the connected domain of the target pixel point; When a valid pixel point has connectivity with the target pixel point, determining that the valid pixel point and the target pixel point belong to the same connected domain; taking the at least one valid pixel point as a new target pixel point, and re-performing the step of acquiring at least one valid pixel point around the target pixel point based on the new target pixel point; When there are no valid pixels around the unretrieved pixels in the target image data, a connected domain retrieval result for the target image data is output based on the connected domain to which each pixel belongs determined according to the second layer mask retrieval.

8. The connected domain searcher according to claim 7, characterized in that: The cache module includes a first-level cache submodule and a second-level cache submodule; The first-level cache submodule is used to cache pixel points in the target image data to be retrieved; The second-level cache submodule is used to obtain the at least one valid pixel point from the first-level cache submodule according to the target address information; and cache the at least one valid pixel point.

9. The connected domain searcher according to claim 8, characterized in that: The second-level cache submodule is composed of a register file.

10. A connected domain search device, characterized in that: The device comprises: A retrieval module, used to obtain a valid target pixel point in the target image data to be retrieved, determine a first layer mask corresponding to at least one pixel point around the target pixel point; obtain at least one valid pixel point around the target pixel point based on the first layer mask; retrieve the connectivity between the at least one valid pixel point and the target pixel point; generate a corresponding second layer mask based on the connectivity between the valid pixel point and the target pixel point; the second layer mask is used to determine whether the corresponding valid pixel point belongs to a connected domain of the target pixel point; when a valid pixel point has connectivity with the target pixel point, determine that the valid pixel point and the target pixel point belong to the same connected domain; use the at least one valid pixel point as a new target pixel point, and re-execute the step of obtaining at least one valid pixel point around the target pixel point based on the new target pixel point; The output module is used to output the connected domain search result for the target image data based on the connected domain to which each pixel point belongs determined according to the second layer mask search when there are no valid pixels around the unsearched pixel points in the target image data.

11. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for searching a connected domain according to any one of claims 1 to 6 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for searching a connected domain according to any one of claims 1 to 6 is implemented.

Images