Image processing method and device, equipment, chip and medium

Abstract

The invention provides an image processing method and device, equipment, a chip and a medium, and the method comprises the steps: determining a first label value of a first pixel in an image obtained through superpixel segmentation, the first pixel being a pixel shared by a plurality of search sub-windows, the search sub-windows being obtained through dividing a search window, and the first label value being a label value of the first pixel; the first label value is used for indicating the superpixel to which the first pixel belongs; a second label value of a second pixel of the image is determined, the second pixel is located in any search sub-window, and the second label value is used for indicating a superpixel to which the second pixel belongs; and according to the first label value and the second label value, determining whether the second pixel is connected with the first pixel, and obtaining a target result. The technical problems that in the prior art, communication searching consumes more power consumption, and the communication searching efficiency is not high are solved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular, to an image processing method, apparatus, device, chip, and medium. Background Art

[0002] Image processing technology and computer vision technology are widely applied to products such as terminals, automobiles, security, and aerial photography, and can be used for image acquisition, image processing, vision computing, etc. When applied to hardware such as chips and field programmable gate arrays (FPGAs), the algorithm processing performance can be improved, so as to better and more real-time serve various related product applications and improve the user experience. Summary of the Invention

[0003] The present disclosure aims to at least solve one of the technical problems in the related art to some extent.

[0004] To this end, the present disclosure provides an image processing method, apparatus, electronic device, chip, and storage medium, which can save the power consumption of connectivity search and improve the connectivity search efficiency.

[0005] The first aspect of the embodiments of the present disclosure provides an image processing method, including: in an image obtained by superpixel segmentation, determining a first label value of a first pixel, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs; determining a second label value of a second pixel of the image, where the second pixel is located in any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs; and determining whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain a target result.

[0006] The second aspect of the embodiments of the present disclosure provides an image processing apparatus, including: a first determination module, configured to determine a first label value of a first pixel in an image obtained by superpixel segmentation, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs; a second determination module, configured to determine a second label value of a second pixel of the image, where the second pixel is located in any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs; and a third determination module, configured to determine whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain a target result.

[0007] A third aspect embodiment of the present disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the image processing method proposed in the first aspect embodiment of the present disclosure.

[0008] A fourth aspect embodiment of the present disclosure provides a chip, which includes a processing circuit and an interface circuit; wherein, the interface circuit is used to read instructions, and the interface circuit sends the instructions to the processing circuit so that the processing circuit executes the image processing method proposed in the first aspect embodiment of the present disclosure.

[0009] A fifth aspect embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above-mentioned image processing method.

[0010] The image processing method, device, electronic device, chip and storage medium provided by the present disclosure determine a first label value of a first pixel in an image obtained by superpixel segmentation, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, the first label value is used to indicate the superpixel to which the first pixel belongs, and determine a second label value of a second pixel in the image, where the second pixel is located in any one of the search sub-windows, the second label value is used to indicate the superpixel to which the second pixel belongs, and according to the first label value and the second label value, determine whether the second pixel is connected to the first pixel to obtain a target result. Since the search window is divided, it is possible to perform a connectivity search in parallel based on the pixels in multiple search sub-windows, and in the process of performing a connectivity search for the pixels in any one of the search sub-windows, it is possible to compare the label values of the pixels in any one of the search sub-windows with the pixels shared by multiple search sub-windows to determine connectivity. In this process, the search can be performed based on a specific direction. Thus, it is possible to save the power consumption of the connectivity search and improve the efficiency of the connectivity search.

[0011] Additional aspects and advantages of the present disclosure will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present disclosure. Description of the Drawings

[0012] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0013] Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure;

[0014] Figure 2Schematic flowchart of another image processing method provided by an embodiment of the present disclosure;

[0015] Figure 3 Schematic diagram of a search window in an embodiment of the present disclosure;

[0016] Figure 4 Schematic flowchart of another image processing method provided by an embodiment of the present disclosure;

[0017] Figure 5 Schematic diagram of an application scenario in an embodiment of the present disclosure;

[0018] Figure 6 Schematic diagram of another application scenario in an embodiment of the present disclosure;

[0019] Figure 7 Schematic diagram of the forward search path direction in an embodiment of the present disclosure;

[0020] Figure 8 Schematic diagram of the reverse search path direction in an embodiment of the present disclosure;

[0021] Figure 9 Schematic diagram of the corresponding relationship between the hardware pipeline implementation process and steps in an embodiment of the present disclosure;

[0022] Figure 10 Schematic diagram of hardware implementation in an embodiment of the present disclosure;

[0023] Figure 11 Schematic diagram of the structure of an image processing apparatus provided by an embodiment of the present disclosure;

[0024] Figure 12 Block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure;

[0025] Figure 13 Schematic diagram of the structure of a chip proposed in an embodiment of the present disclosure;

[0026] Figure 14 Schematic diagram of the structure of another chip proposed in an embodiment of the present disclosure. Detailed implementation manners

[0027] Some embodiments of the present disclosure will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those set forth herein, but may be changed as will be apparent after understanding the present disclosure, except for operations that must be performed in a specific order. Additionally, descriptions of features known in the art may be omitted for the sake of clarity and conciseness.

[0028] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in the present disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards in relevant countries and regions.

[0030] In the related art, the solution of image connected components is mainly implemented through a recursive method. For example, it is possible to compare whether the current label value of a superpixel is equal to the label values of its surrounding 8 neighbors. If they are equal, a connectivity flag is set, and then an arbitrary connected position that has not been defined as a search center is selected as a new recursive center and a new recursion is entered. In this way, searching is performed in any recursive direction, resulting in relatively high power consumption for connected search and low efficiency of connected search.

[0031] In the embodiments of the present disclosure, in order to solve the above technical problems, an image processing method is provided. In the image obtained by superpixel segmentation, a first label value of a first pixel is determined, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs. And a second label value of a second pixel in the image is determined, where the second pixel is located in any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs. And according to the first label value and the second label value, it is determined whether the second pixel is connected to the first pixel to obtain a target result. Since the search window is divided, parallel connectivity search can be implemented based on the pixels in multiple search sub-windows. And in the process of performing connectivity search on the pixels in any one of the search sub-windows, the pixels in any one of the search sub-windows can be compared with the pixels shared by multiple search sub-windows for label values to determine connectivity. In this process, search can be performed based on a specific direction. Thus, the power consumption of connectivity search can be saved and the efficiency of connectivity search can be improved.

[0032] The description of "superpixel segmentation" is as follows: In computer vision processing, in order to facilitate image understanding and analysis, an image can be further divided into multiple sub-regions, and the shapes of these sub-regions can be irregular. This process can be called superpixel segmentation. The division of superpixels can be classified by feature values such as color, brightness, texture, and position distance. For example, a superpixel segmentation method based on a clustering algorithm (Simple Linear Iterative Clustering, SLIC) can be used for superpixel segmentation. That is, adjacent pixels with similar characteristics in the image are "aggregated" to form more representative "elements", and this representative "element" can be called a superpixel. Superpixel segmentation can be used to assist in calculating missing or incorrect motion vectors in sparse optical flow or sparse disparity maps, thereby improving the accuracy of the optical flow or disparity map to meet various image alignment or depth calculation requirements. Optionally, in some embodiments, determining whether pixels are connected can be referred to as performing connectivity processing on the pixels or determining whether there is connectivity between the pixels. In addition, connectivity processing can also be performed on the segmented superpixels, that is, adjacent superpixels with the same label are connected, and the adjacent boundaries are modified to form a larger and connected superpixel, thereby improving the quality of superpixel segmentation to assist texture or edge recognition during optical flow or disparity calculation.

[0033] Figure 1 It is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure.

[0034] In the embodiments of the present disclosure, the image processing method is exemplified by being configured in an image processing apparatus, which can be applied to any electronic device or chip, so that the electronic device or chip can perform image processing functions.

[0035] As Figure 1 shown, the image processing method may include the following steps:

[0036] Step S101, in the image obtained by superpixel segmentation, determine the first label value of the first pixel, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs.

[0037] Optionally, in some embodiments, the image to be processed may be subjected to superpixel segmentation, and the image obtained by superpixel segmentation may include one or more superpixels. Then, connected component search is performed based on the image obtained by superpixel segmentation.

[0038] Optionally, in some embodiments, connected component search may be performed on the image obtained by superpixel segmentation based on a search window. The search window may be divided to obtain multiple search sub-windows, and the pixel shared by the multiple search sub-windows is used as the reference pixel for connected component search, and this pixel may be referred to as the first pixel. The first label value is used to indicate the superpixel to which the first pixel belongs. The first label value of the first pixel may be determined during the process of superpixel segmentation, and the first label value may be used for connected component search.

[0039] Optionally, in some embodiments, during the process of dividing the search window, the search window may be divided based on the horizontal direction and the vertical direction respectively to obtain four search sub-windows. Of course, the search window may also be divided based on any other possible method to obtain several search sub-windows, and the pixel shared by the several search sub-windows is determined as the first pixel.

[0040] Step S102, determine the second label value of the second pixel of the image, where the second pixel is located in any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs.

[0041] Optionally, in some embodiments, the pixel located in any one of the search sub-windows may be referred to as the second pixel, and the second label value is used to indicate the superpixel to which the second pixel belongs. The second label value of the second pixel may be determined during the process of superpixel segmentation, and the second label value may be used for connected component search.

[0042] Optionally, in some embodiments, for the second pixel in any one of the search sub-windows, the second label value of the second pixel may be determined.

[0043] Optionally, in some embodiments, the first tag value of the first pixel and the second tag value of the second pixel may be the same or different. If the first tag value and the second tag value are the same, it indicates that the first pixel and the second pixel belong to the same superpixel. If the first tag value and the second tag value are different, it indicates that the first pixel and the second pixel belong to different superpixels. There is no limitation on this.

[0044] Step S103: Determine whether the second pixel is connected to the first pixel according to the first tag value and the second tag value, and obtain the target result.

[0045] Optionally, in some embodiments, after determining the first tag value of the first pixel and the second tag value of the second pixel, a connectivity search may be performed based on the first tag value and the second tag value, that is, to detect whether there is connectivity between the second pixel and the first pixel. If the second pixel is connected to the first pixel, it indicates that the second pixel and the first pixel can be divided into the same connected domain. If the second pixel is not connected to the first pixel, it indicates that the second pixel and the first pixel cannot be divided into the same connected domain.

[0046] Optionally, in some embodiments, the first tag value and the second tag value may be compared, and whether the second pixel is connected to the first pixel may be determined based on the comparison result. For example, based on the first tag value and the second tag value, combined with any connectivity detection method, it can be realized to determine whether the second pixel is connected to the first pixel; or the first tag value and the second tag value may be processed in an artificial intelligence manner to determine whether the second pixel is connected to the first pixel; or any other possible method may also be used to realize determining whether the second pixel is connected to the first pixel according to the first tag value and the second tag value. There is no limitation on this.

[0047] Optionally, in some embodiments, the target result may correspond to the second pixel. For example, for each second pixel, there is a corresponding target result. The target result is used to indicate that the corresponding second pixel is connected to the first pixel, or is used to indicate that the corresponding second pixel is not connected to the first pixel. There is no limitation on this.

[0048] In this embodiment, in the image obtained by superpixel segmentation, the first label value of the first pixel is determined, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, the first label value is used to indicate the superpixel to which the first pixel belongs, and the second label value of the second pixel of the image is determined, where the second pixel is located in any one of the search sub-windows, the second label value is used to indicate the superpixel to which the second pixel belongs, and based on the first label value and the second label value, it is determined whether the second pixel is connected to the first pixel to obtain a target result. Since the search window is divided, parallel connectivity search can be implemented based on the pixels in multiple search sub-windows, and in the process of performing connectivity search on the pixels in any one of the search sub-windows, the pixels in any one of the search sub-windows can be compared with the pixels shared by multiple search sub-windows for label values to determine connectivity. In this process, search can be performed in a specific direction. Thus, the power consumption of the connectivity search can be saved, and the efficiency of the connectivity search can be improved.

[0049] Optionally, in some embodiments of the present disclosure, in the process of implementing the determination of whether the second pixel is connected to the first pixel based on the first label value and the second label value to obtain a target result, in addition to referring to the first label value and the second label value for connectivity search, it can also be determined whether the third pixel of the image is connected to the first pixel to obtain a reference result, where the third pixel is a pixel adjacent to the second pixel, and based on the first label value, the second label value, and the reference result, it is determined whether the second pixel is connected to the first pixel to obtain a target result. Thus, the accuracy of the connectivity search can be effectively improved.

[0050] Optionally, in some embodiments, the number of pixels adjacent to the second pixel can be multiple, and some pixels can be selected from the pixels adjacent to the second pixel as the third pixel. Optionally, in some embodiments, the pixels adjacent to the second pixel and that have been detected and scanned can be selected, and then the selected pixels are determined as the third pixel. The third pixel can be located in the same search sub-window as the second pixel, or the third pixel can also be located in different search sub-windows from the second pixel. If the number of third pixels is multiple, the multiple third pixels can simultaneously include the third pixels located in the same search sub-window as the second pixel and the third pixels located in different search sub-windows from the second pixel, and there is no limitation on this.

[0051] Optionally, in some embodiments, in the process of implementing the determination of whether the second pixel is connected to the first pixel based on the first label value, the second label value, and the reference result to obtain a target result, operations can be performed on the first label value, the second label value, and the reference result, and the target result is determined based on the result of the operation.

[0052] Optionally, in some embodiments, at least two third pixels may be selected to participate in the connectivity search. In the process of determining whether the second pixel is connected to the first pixel according to the first tag value, the second tag value, and the reference result to obtain the target result, it may be to determine whether the first tag value and the second tag value are the same to obtain the first operation result, perform a logical OR operation on at least two reference results to obtain the second operation result, and perform a logical AND operation on the first operation result and the second operation result to obtain the target result. Thus, in the connectivity search process, the connectivity search can be achieved by comparing whether the tag values of the second pixel and the first pixel are the same, and whether the second pixel can be connected to the first pixel through a certain connectivity path, thereby greatly improving the accuracy of the connectivity search, and at the same time supporting the improvement of the efficiency of the connectivity search.

[0053] Figure 2 It is a schematic flowchart of another image processing method provided by an embodiment of the present disclosure.

[0054] As Figure 2 shown, the image processing method may include the following steps:

[0055] Step S201, in the image obtained by superpixel segmentation, determine the first tag value of the first pixel, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing the search window, and the first tag value is used to indicate the superpixel to which the first pixel belongs.

[0056] Step S202, determine the second tag value of the second pixel in the image, where the second pixel is located in any one of the search sub-windows, and the second tag value is used to indicate the superpixel to which the second pixel belongs.

[0057] For the description of S201 - S202, specific reference may be made to the above embodiments, which will not be elaborated here.

[0058] Step S203, determine the scanning direction corresponding to the search sub-window.

[0059] Wherein, the scanning direction is used to describe the direction of performing connectivity search on several second pixels in the search sub-window. Different search sub-windows are located at different positions in the search window, and the second pixels in different search sub-windows are all searched based on the first pixel as the center. Therefore, the scanning directions of different search sub-windows may be different.

[0060] Exemplarily, as Figure 3 shown, Figure 3 is a schematic diagram of the search window in an embodiment of the present disclosure. Figure 3A way of dividing the search window is shown. The search window can be divided into four search sub-windows (block0, block1, block2, block3). Adjacent search sub-windows can include overlapping window parts. Each point represents a pixel. The pixel located at the center can be, for example, an optional example of the first pixel. The pixels within any block can be, for example, an optional example of the second pixel. For the scanning direction of block0, it can be based on the central pixel and scan column by column. In each column of pixels, scan each pixel one by one, starting from the central pixel and scanning towards the second pixel at the top left corner, or starting from the pixel at the top left corner and scanning towards the central pixel. For the scanning direction of block1, it can be based on the central pixel and scan column by column. In each column of pixels, scan each pixel one by one, starting from the central pixel and scanning towards the second pixel at the top right corner, or starting from the pixel at the top right corner and scanning towards the central pixel. For the scanning direction of block2, it can be based on the central pixel and scan column by column. In each column of pixels, scan each pixel one by one, starting from the central pixel and scanning towards the second pixel at the bottom left corner, or starting from the pixel at the bottom left corner and scanning towards the central pixel. For the scanning direction of block2, it can be based on the central pixel and scan column by column. In each column of pixels, scan each pixel one by one, starting from the central pixel and scanning towards the second pixel at the bottom right corner, or starting from the pixel at the bottom right corner and scanning towards the central pixel.

[0061] Optionally, in some embodiments, in the process of implementing the determination of the scanning direction corresponding to the search sub-window, it can be based on the position information of the search sub-window within the search window to determine the scanning direction corresponding to the search sub-window. Thus, it is possible to quickly and conveniently determine the scanning direction suitable for the search sub-window, and when performing the connectivity search for the second pixels within the corresponding search sub-window based on the scanning direction, the connectivity search efficiency can be greatly improved.

[0062] Step S204, determine the search order of the second pixels according to the scanning direction.

[0063] Optionally, in some embodiments, after determining the scanning direction, the search order of the second pixels can be determined according to the scanning direction. The search order is used to represent the order of performing the connectivity search for each second pixel within the search sub-window. The search order can be, for example, to search column by column of pixels (that is to say, several second pixels within the search sub-window can be arranged in several columns and several rows), and within the same column, each pixel can be searched one by one.

[0064] Optionally, in some embodiments, as described above Figure 3An example of the block is given. For the scanning direction of block0, it can be centered on the central pixel and scanned column by column. In each column of pixels, each pixel is scanned one by one, starting from the central pixel and scanning towards the second pixel in the top left corner, or starting from the top left pixel and scanning towards the central pixel. When the scanning direction is: starting from the central pixel and scanning towards the second pixel in the top left corner, the search order can be, for example, starting from the rightmost column of block0, searching column by column to the left, and within the same column, searching from bottom to top. When the scanning direction is: starting from the top left pixel and scanning towards the central pixel, the search order can be, for example, starting from the leftmost column of block0, searching column by column to the right, and within the same column, searching from top to bottom. The description of the search order for block1, block2, and block3 can be extended by analogy.

[0065] Step S205: Based on the search order, determine whether each second pixel is connected to the first pixel in sequence according to the second tag value and the first tag value of each second pixel, to obtain the target result.

[0066] Optionally, in some embodiments, after determining the search order, it is possible to determine whether each second pixel is connected to the first pixel in sequence according to the second tag value and the first tag value of each second pixel, to obtain the target result. That is to say, based on the search order, each second pixel is searched one by one within each search sub-window to determine whether each second pixel is connected to the first pixel, to obtain the target result.

[0067] Optionally, in some embodiments, the search order includes: a first search order and a second search order, and the first search order and the second search order are opposite search orders. By way of example, as described above Figure 3Examples of the block are given. For the scanning direction of block0, the search order can be, for example, starting from the rightmost column of block0, searching column by column to the left, and within the same column, searching from bottom to top. When the scanning direction is: starting from the central pixel and scanning to the second pixel in the upper left corner, the search order can be, for example, starting from the rightmost column of block0, searching column by column to the left, and within the same column, searching from bottom to top. When the scanning direction is: starting from the pixel in the upper left corner and scanning to the central pixel, the search order can be, for example, starting from the leftmost column of block0, searching column by column to the right, and within the same column, searching from top to bottom. Among them, "the search order can be, for example, starting from the rightmost column of block0, searching column by column to the left, and within the same column, searching from bottom to top", at this time, this search order can be called the first search order, "the search order can be, for example, starting from the leftmost column of block0, searching column by column to the right, and within the same column, searching from top to bottom", at this time, this search order can be called the second search order. And the description of the search order for block1, block2, and block3 can be extended by analogy. Thus, it can be seen that the first search order and the second search order are opposite search orders to each other.

[0068] Optionally, in some embodiments, based on the first search order, the connectivity between each second pixel and the first pixel can be determined in sequence according to the second label value and the first label value of each second pixel to obtain a first result, and based on the second search order, the connectivity between each second pixel and the first pixel can be determined in sequence according to the second label value and the first label value of each second pixel to obtain a second result, and the target result can be determined according to the first result and the second result. Thus, the accuracy of the search can be maximally improved, the omission of searching for connected pixels can be avoided, and the accuracy of the connectivity search can be improved.

[0069] Optionally, in some embodiments, first, based on the first search order, the connectivity between each second pixel and the first pixel can be determined in sequence according to the second label value and the first label value of each second pixel to obtain a first result, and then, based on the second search order again, the connectivity between each second pixel and the first pixel can be determined in sequence according to the second label value and the first label value of each second pixel to obtain a second result. For each second pixel, a first result and a second result will be calculated. If the first result and the second result are the same, either result can be determined as the target result of the corresponding second pixel. If the first result and the second result are different, the second result can be determined as the target result of the corresponding second pixel, and there is no limitation on this.

[0070] Optionally, in some embodiments, in the process of determining whether each second pixel is connected to the first pixel in sequence according to the second tag value and the first tag value of each second pixel based on the search order, for each second pixel, the reference result of the third pixel adjacent to the second pixel can be used to assist in the connectivity search.

[0071] Optionally, in some embodiments, two adjacent search sub-windows include an overlapping window part; when the second pixel is located in the overlapping window part, determine whether the second pixel is connected to the first pixel according to the first tag value and the second tag value, and obtain a first result corresponding to each of the two adjacent search sub-windows; merge the two first results, and determine the merged result as the target result (for example, the two first results can be logically ORed, and the result of the operation can be used as the target result). Thus, for the second pixel located in the overlapping window part of two adjacent search sub-windows, the accuracy of the connectivity search can be improved.

[0072] In this embodiment, since the search window is divided, it is possible to perform the connectivity search in parallel based on the pixels in multiple search sub-windows, and in the process of performing the connectivity search for the pixels in any one search sub-window, it can be to compare the tag values of the pixels in any one search sub-window with the pixels shared by multiple search sub-windows to determine the connectivity. In this process, the search can be performed based on a specific direction. Thus, the power consumption of the connectivity search can be saved, and the efficiency of the connectivity search can be improved. And it is possible to quickly and conveniently determine the scanning direction suitable for the search sub-window. When performing the connectivity search for the second pixels in the corresponding search sub-window based on the scanning direction, the efficiency of the connectivity search can be greatly improved. And for the second pixel located in the overlapping window part of two adjacent search sub-windows, the accuracy of the connectivity search can also be improved.

[0073] Figure 4 It is a schematic flowchart of another image processing method provided by the embodiments of the present disclosure.

[0074] As Figure 4 shown, the image processing method may include the following steps:

[0075] Step S401, in the image obtained by superpixel segmentation, determine the first tag value of the first pixel, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing the search window, and the first tag value is used to indicate the superpixel to which the first pixel belongs.

[0076] Step S402, determine the second tag value of the second pixel of the image, where the second pixel is located in any one search sub-window, and the second tag value is used to indicate the superpixel to which the second pixel belongs.

[0077] Step S403: Determine whether the second pixel is connected to the first pixel based on the first tag value and the second tag value, and obtain the target result.

[0078] For the descriptions of S401 - S403, please refer to the above embodiments for details, which will not be elaborated here.

[0079] Step S404: Select at least one second pixel connected to the first pixel from multiple second pixels of the image.

[0080] Step S405: Form a connected region based on the selected second pixels.

[0081] Optionally, in some embodiments, after determining whether the second pixel in any search sub - window is connected to the first pixel and obtaining the target result, at least one second pixel connected to the first pixel can be selected from multiple second pixels of the image, and a connected region can be formed based on the selected second pixels. Thus, the connected region can be quickly searched out, and when the connected region is used to assist subsequent image processing, the efficiency and effect of image processing can be improved.

[0082] Exemplarily, after super - pixel segmentation of the image, the first pixel can belong to one super - pixel, and the second pixel can belong to one super - pixel. The two super - pixels can be the same super - pixel. If it is determined that the second pixel is connected to the first pixel, the adjacent boundary of the super - pixel can be modified to obtain a larger and connected super - pixel, which can contain the mutually - connected first pixel and second pixel. The image region covered by this super - pixel can be referred to as the connected region. Subsequently, this connected region can be used to assist image processing.

[0083] Step S406: Determine the connected size of the connected region.

[0084] Optionally, in some embodiments, after determining the connected region, the connected size of the connected region can be determined. The connected size can represent the size of the connected region. The larger the connected size, the more pixel numbers the connected region contains; the smaller the connected size, the fewer pixel numbers the connected region contains.

[0085] Step S407: When the connected size is less than the size threshold, determine the target tag value, where the target tag value is the tag value corresponding to the largest boundary size among multiple boundary sizes, and the boundary size is the size of a section of the boundary of the connected region, and the tag values of different sections of the boundary are different.

[0086] Among them, the size threshold is the threshold value of the connected size for determining whether the first label value of the first pixel needs to be updated. If the connected size is less than the size threshold, it can be determined that the first label value of the first pixel needs to be updated, and if the connected size is greater than or equal to the size threshold, it can be determined that there is no need to update the first label value of the first pixel.

[0087] Optionally, in some embodiments, when the connected size is less than the size threshold, the target label value can be used to update the first label value of the first pixel. Among them, the connected component has a boundary, and the label values corresponding to different segments on the boundary can be the same or different. The boundary of the connected component can be segmented based on the label value, and the label values of different segment boundaries are different, and then the label value corresponding to the maximum boundary size is determined as the target label value.

[0088] Step S408: Update the first label value of the first pixel according to the target label value.

[0089] After determining the target label value when the connected size is less than the size threshold as described above, the target label value can be used as the first label value of the first pixel, and then the next round of iterative search is triggered. Thus, the reference value of the first label value of the first pixel used as a reference in each round of iterative search can be improved, thereby maximizing the accuracy of the connected search.

[0090] In this embodiment, since the search window is divided, parallel connected search can be realized based on the pixels in multiple search sub-windows. And in the process of performing connected search on the pixels in any one search sub-window, the pixels in any one search sub-window can be compared with the pixels shared by multiple search sub-windows for label values to determine connectivity. In this process, the search can be performed based on a specific direction. Thus, the power consumption of the connected search can be saved and the efficiency of the connected search can be improved. After obtaining the target result by determining whether the second pixel in any one search sub-window is connected to the first pixel as described above, at least one second pixel connected to the first pixel can be selected from the multiple second pixels of the image, and a connected component can be formed according to the selected second pixel. Thus, the connected component can be quickly searched out, and when the subsequent image processing is assisted based on this connected component, the efficiency and effect of the image processing can be improved. After determining the target label value when the connected size is less than the size threshold as described above, the target label value can be used as the first label value of the first pixel, and then the next round of iterative search is triggered. Thus, the reference value of the first label value of the first pixel used as a reference in each round of iterative search can be improved, thereby maximizing the accuracy of the connected search.

[0091] Optionally, in some embodiments, in the process of implementing the use of the connected component to assist in image processing, it may be to determine the feature information of the pixels in the connected component, and based on the feature information, determine the disparity depth and / or optical flow information of the image. Thereby, it supports improving the calculation accuracy of the disparity depth of the image, or can also improve the optical flow calculation accuracy.

[0092] As Figure 5 shown, Figure 5 is a schematic diagram of an application scenario in an embodiment of the present disclosure. The image can be iteratively segmented and connected component processing can be performed. In the process of obtaining the image, the image sensor (main camera and secondary camera) in the electronic device can be used to collect the original image (Raw Data) respectively, and then the original image is processed into an RGB image (or referred to as RGB Data, an RGB image is an image generated by combining different intensities of three primary colors, namely Red, Green, and Blue) and / or a YUV image (or referred to as YUV Data, a YUV image is an encoding method that uses three components, Y (luminance information), U, and V (U and V represent chrominance information)) by the image processor. Then, the image of the main camera is deformed or aligned, and then iterative segmentation and connected component processing are performed, and the disparity depth is calculated based on the obtained superpixel map and the image of the secondary camera, so as to obtain the depth map.

[0093] As Figure 6 shown, Figure 6 is another schematic diagram of an application scenario in an embodiment of the present disclosure. The image can be iteratively segmented and connected component processing can be performed. In the process of obtaining the image, the image sensor (main camera) in the electronic device can be used to collect two frames of original images (Raw Data) respectively, and then the original image is processed into an RGB image and / or a YUV image by the image processor. Then, the current frame of the image is deformed or aligned, and then iterative segmentation and connected component processing are performed, and the optical flow is calculated based on the obtained superpixel map and the previous frame of the image, so as to obtain the vector map.

[0094] An example of the connected component search process is described as follows. Taking the number of search sub-windows as 4 as an example, the following description can be referred to the above Figure 3 .

[0095] Step 1 (step1): The search window can be divided into 4 blocks (search sub-windows), and each block has two repeated central edges. The connected search is to compare whether the label value (an optional example of the first label value) of the search node (an optional example of the second pixel) is the same as that of the central node (an optional example of the first pixel), and whether it can be connected to the position of the central node through a certain connected path.

[0096] Step 2: Forward search. Rule: Block0 searches from the center of the window towards the upper left direction, advancing in a column-scanning manner. It starts from the rightmost column of block0 and scans column by column to the left. For each column, it scans from bottom to top. In this way, it scans from the central node (an optional example of the first pixel) to the upper left node (an optional example of the second pixel of block0) and ends. The scanning methods of block1, block2, and block3 are similar, except for the scanning directions. Block1 searches from the center towards the upper right, block2 searches from the center towards the lower left, and block3 searches from the center towards the lower right. Whether each node (an optional example of the second pixel) is connected to the central node is represented by 1 indicator bit (indicator bit being 1 means connected, indicator bit being 0 means not connected). The solution of the indicator bit depends on the 4 directions and whether its label value (an optional example of the second label value) is equal to the label value of the central node (an optional example of the first label value). Taking block0 as an example, the solution of the indicator bit of the current node (an optional example of the second pixel) in block0 is the logical OR result of the 4 adjacent indicator bits of down, lower right, right, and top right (an optional example of the reference result of the third pixel), and the label value of the current node is consistent with the label value of the central node. The calculation formula is as follows:

[0097]

[0098] Among them, connect_valid[curr] represents the indicator bit of the current node (an optional example of the second pixel) in block0, connect_valid[down], connect_valid[lower_right], connect_valid[right], connect_valid[top_right] respectively represent the 4 adjacent indicator bits of down, lower right, right, and top right of the current node. "||" represents logical OR operation, "&&" represents logical AND operation, Label[current] represents the label value of the current node (an optional example of the second pixel), and Label[central] represents the label value of the central node (an optional example of the first pixel). For the solution of the indicator bits in other blocks, it can be deduced by analogy, except that the adjacent directions relied on are different.

[0099] As Figure 7 shown, Figure 7 is the schematic diagram of the forward search path direction in the embodiment of the present disclosure.

[0100] Step 3: Reverse search. Rule: After the forward search is completed, relying on the results of the forward search, block0 searches from the top-leftmost point towards the center of the window, advancing in a column-scanning manner. It starts from the leftmost column of block0 and scans column by column from left to right. For each column, it scans from top to bottom. In this way, the scan ends when reaching the center node from the top-left node. The scanning methods of block1, block2, and block3 are similar, except for the scanning directions. Block1 searches from the top-right towards the center, block2 searches from the bottom-left towards the center, and block3 searches from the bottom-right towards the center. Whether each node is connected to the center node is represented by 1 indicator bit (indicator bit = 1: indicates connection, indicator bit = 0: indicates disconnection). The solution of the indicator bit depends on 4 directions and whether its label value (an optional example of the second label value) is equal to the center label value (an optional example of the first label value). Taking block0 as an example, the indicator bit solution for the nodes (an optional example of the second pixel) in this block0 is the logical OR result of the 4 indicator bits of its adjacent upper, upper-left, left, and bottom-left positions, and the label value (an optional example of the second label value) of the current node is consistent with the label value (an optional example of the first label value) of the center node. The calculation formula is as follows:

[0101]

[0102] Among them, connect_valid[curr] represents the indicator bit of the current node (an optional example of the second pixel) in block0, connect_valid[upper], connect_valid[top_left], connect_valid[left], and connect_valid[bottom_left] respectively represent the 4 indicator bits of the upper, upper-left, left, and bottom-left positions adjacent to the current node. "||" represents the logical OR operation, "&&" represents the logical AND operation, Label[current] represents the label value of the current node (an optional example of the second pixel), and Label[central] represents the label value of the center node (an optional example of the first pixel). For the solution of the indicator bits in other blocks, it can be analogized in the same way, except that the adjacent directions relied on are different.

[0103] As Figure 8 shown, Figure 8 is the schematic diagram of the reverse search path direction in the embodiment of the present disclosure.

[0104] Step 4: Search and merge. After the reverse search ends, the indication bit results of the four central edges are merged (that is to say, the regions of the four central edges can be an optional example of the above overlapping window part). That is, if a certain node is within the overlapping edge, the indication bit results obtained for this node based on different scanning directions (an optional example of the above first result) can be logically ORed, and the obtained result can be an optional example of the above target result.

[0105] Step 5: Re-perform a forward search again. The rules are the same as in Step 2. By re-searching, the missing connected nodes can be compensated for to approach the theoretical result of the original algorithm.

[0106] Step 6: Determine whether to update the label value (an optional example of the first label value) of the central node with the label value corresponding to the maximum boundary size (an optional example of the above target label value) by comparing the connected size counted in Step 5 with the filtering threshold (an optional example of the above size threshold).

[0107] Optionally, each of the above steps can be implemented in a hardware pipelining manner. For example, Steps 2, 3, 4, and 5 respectively complete the logical calculations in one clock cycle. As Figure 9 shown, Figure 9 is a schematic diagram of the corresponding relationship between the hardware pipeline implementation process and the steps in the embodiments of the present disclosure. As Figure 10 shown, Figure 10 is a schematic diagram of the hardware implementation in the embodiments of the present disclosure. The above image processing method in the embodiments can be implemented through a dedicated hardened accelerator (Static Random Access Memory (SRAM)), and the obtained target result can be stored in a Double Data Rate (DDR) memory.

[0108] The method provided in the embodiments of the present disclosure can be effectively applied to the pure-hardening implementation of a chip. Because its search path is relatively determined, it is convenient to be designed into a pipelined pure-hardware accelerator, and the connected search of one pixel can be completed with only several clock cycles. In addition, it can meet the application requirements of low latency and low power consumption, and thus is effectively applicable to mobile devices. There is no need to repeatedly schedule operations through instructions, and there is no need to repeatedly schedule the intermediate data obtained by calculation between the processor and the memory, thereby effectively saving memory power consumption and bandwidth, and being able to meet the requirements for real-time performance and power consumption in mobile vision computing.

[0109] Figure 11 is a schematic structural diagram of an image processing device provided by the embodiments of the present disclosure.

[0110] As Figure 11As shown, the image processing apparatus 110 includes:

[0111] A first determination module 1101, configured to determine a first label value of a first pixel in an image obtained by superpixel segmentation, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs.

[0112] A second determination module 1102, configured to determine a second label value of a second pixel of the image, where the second pixel is located within any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs.

[0113] A third determination module 1103, configured to determine whether the second pixel is connected to the first pixel according to the first label value and the second label value, to obtain a target result.

[0114] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is configured to:

[0115] Determine a scanning direction corresponding to the search sub-window;

[0116] Determine the search order of the second pixel according to the scanning direction;

[0117] Based on the search order, sequentially determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel, to obtain a target result.

[0118] Optionally, in some embodiments of the present disclosure, the search order includes: a first search order and a second search order, and the first search order and the second search order are opposite search orders;

[0119] Wherein, the third determination module 1103 is configured to:

[0120] Based on the first search order, sequentially determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel, to obtain a first result;

[0121] Based on the second search order, sequentially determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel, to obtain a second result;

[0122] Determine the target result according to the first result and the second result.

[0123] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is configured to:

[0124] Determine a scanning direction corresponding to the search sub-window based on the position information of the search sub-window within the search window.

[0125] Optionally, in some embodiments of the present disclosure, two adjacent search sub-windows include an overlapping window portion;

[0126] Wherein, the third determination module 1103 is configured to:

[0127] When the second pixel is located in the overlapping window portion, determine whether the second pixel is connected to the first pixel according to the first tag value and the second tag value, so as to obtain a first result corresponding to each of the two adjacent search sub-windows;

[0128] Merge the two first results, and determine the merged result as the target result.

[0129] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is configured to:

[0130] Determine whether a third pixel of the image is connected to the first pixel, so as to obtain a reference result, where the third pixel is a pixel adjacent to the second pixel;

[0131] Determine whether the second pixel is connected to the first pixel according to the first tag value, the second tag value and the reference result, so as to obtain the target result.

[0132] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is configured to:

[0133] Determine whether the first tag value and the second tag value are the same, so as to obtain a first operation result;

[0134] Perform a logical OR operation on at least two reference results to obtain a second operation result;

[0135] Perform a logical AND operation on the first operation result and the second operation result to obtain the target result.

[0136] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is further configured to:

[0137] Select at least one second pixel connected to the first pixel from multiple second pixels of the image;

[0138] Form a connected domain according to the selected second pixels.

[0139] Optionally, in some embodiments of the present disclosure, the third determination module 1103 is further configured to:

[0140] Determine the feature information of the pixels in the connected domain;

[0141] Determine the disparity depth and / or optical flow information of the image according to the feature information.

[0142] Optionally, in some embodiments of the present disclosure, the first determination module 1101 is further configured to:

[0143] Determine the connectivity size of the connected component;

[0144] When the connectivity size is smaller than the size threshold, determine a target label value, where the target label value is the label value corresponding to the largest boundary size among multiple boundary sizes, the boundary size is the size of a section of the boundary of the connected component, and the label values of different sections of the boundary are different;

[0145] Update the first label value of the first pixel according to the target label value.

[0146] It should be noted that the foregoing explanation of the embodiments of the image processing method also applies to the image processing apparatus of this embodiment, and will not be repeated here.

[0147] In this embodiment, in the image obtained by superpixel segmentation, the first label value of the first pixel is determined, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, the first label value is used to indicate the superpixel to which the first pixel belongs, and the second label value of the second pixel of the image is determined, where the second pixel is located in any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs, and according to the first label value and the second label value, it is determined whether the second pixel is connected to the first pixel to obtain a target result. Since the search window is divided, it is possible to perform connectivity search in parallel based on the pixels in multiple search sub-windows, and in the process of performing connectivity search for the pixels in any one of the search sub-windows, it is possible to compare the label values of the pixels in any one of the search sub-windows with the pixels shared by multiple search sub-windows to determine connectivity. In this process, search can be performed based on a specific direction. Thus, the power consumption of the connectivity search can be saved, and the efficiency of the connectivity search can be improved.

[0148] To implement the above embodiments, the present disclosure also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0149] Figure 12 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure is shown. Figure 12 The illustrated electronic device 12 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure. The electronic device may be, for example, a terminal, and this is not limited. As Figure 12As shown, the electronic device 12 is presented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a memory 28, and a bus 18 that connects different system components (including the memory 28 and the processing unit 16).

[0150] The bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the multiple bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0151] The electronic device 12 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0152] The memory 28 may include computer system-readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or a cache 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 can be used for reading and writing non-removable, non-volatile magnetic media ( Figure 12 not shown, commonly referred to as a "hard disk drive").

[0153] Although Figure 12Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM) or other optical media) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data medium interfaces. The memory 28 can include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0154] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in the memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present disclosure.

[0155] The electronic device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a human body to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Also, the electronic device 12 can communicate with one or more networks (e.g., a Local Area Network (hereinafter referred to as: LAN), a Wide Area Network (hereinafter referred to as: WAN) and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the electronic device 12 through the bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0156] The processing unit 16 executes various functional applications and data processing by running the programs stored in the memory 28, such as implementing the methods mentioned in the foregoing embodiments.

[0157] To implement the above embodiments, the present disclosure also provides a chip, including: The chip includes a processing circuit configured to execute the method provided in the foregoing embodiments.

[0158] Figure 13 is a schematic structural diagram of a chip proposed in an embodiment of the present disclosure. It can be referred to Figure 13 the schematic structural diagram of the chip 1300 shown, but not limited thereto.

[0159] The chip 1300 includes a processing circuit 1301 and an interface circuit 1302. The interface circuit 1302 is used to read instructions and send the instructions to the processing circuit 1301 so that the processing circuit 1301 executes the above method.

[0160] Optionally, as Figure 14 shown, Figure 14 is a schematic structural diagram of another chip proposed in an embodiment of the present disclosure. The chip 1300 may further include: a memory 1303 for storing instructions, and the interface circuit 1302 may be used to read the instructions stored in the memory 1303.

[0161] Optionally, the interface circuit 1302 is connected to the memory 1303. The interface circuit 1302 can be used to receive signals from the memory 1303 or other devices, and the interface circuit 1302 can be used to send signals to the memory 1303 or other devices. For example, the interface circuit 1302 can read the instructions stored in the memory 1303 and send the instructions to the processing circuit 1301.

[0162] Optionally, the number of memories 1303 can be one or more. The number of interface circuits 1302 can also be one or more. In some embodiments, the interface circuit 1302 executes at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 1301 executes other steps.

[0163] In some embodiments, terms such as interface circuit, interface, transceiver pin, transceiver, etc. can be replaced with each other.

[0164] Optionally, all or part of the memory 1303 can also be outside the chip 1300.

[0165] To implement the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing a computer program, which when executed by a processor, implements the method proposed in the foregoing embodiments of the present disclosure.

[0166] To implement the above embodiments, the present disclosure also provides a computer program product, which when the instructions in the computer program product are executed by a processor, executes the method proposed in the foregoing embodiments of the present disclosure.

[0167] The collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0168] It should be noted that personal information from users should be collected for legal and reasonable purposes and not shared or sold outside of these legal uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization including authorizing relevant user information before the user uses the function. In addition, any necessary steps should be taken to safeguard and protect access to such personal information data and ensure that others with access to the personal information data comply with their privacy policies and procedures.

[0169] This disclosure anticipates providing embodiments where users can selectively block the use or access of personal information data. That is, this disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of the user.

[0170] In the description of the foregoing embodiments, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0171] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of such features. In the description of this disclosure, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0172] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, where functions may be executed in a manner that is not shown or discussed, including in a substantially simultaneous manner according to the involved functions or in a reverse order, which should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0173] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing a logical function and can be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. As used in this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0174] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0175] Those of ordinary skill in the art can understand that all or part of the steps carried out in implementing the above-described embodiment methods can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0176] In addition, in each of the various embodiments of the present disclosure, the functional units can be integrated in a processing module, or each unit can exist physically alone, or two or more units can be integrated in a module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0177] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that, Comprising: In the image obtained by superpixel segmentation, determine the first label value of the first pixel, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first label value is used to indicate the superpixel to which the first pixel belongs; Determine the second label value of the second pixel of the image, where the second pixel is located within any one of the search sub-windows, and the second label value is used to indicate the superpixel to which the second pixel belongs; According to the first label value and the second label value, determine whether the second pixel is connected to the first pixel to obtain a target result.

2. The method according to claim 1, wherein The step of determining whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain a target result includes: Determine the scanning direction corresponding to the search sub-window; According to the scanning direction, determine the search order of the second pixel; Based on the search order, successively determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel to obtain the target result.

3. The method according to claim 2, wherein The search order includes: a first search order and a second search order, and the first search order and the second search order are opposite search orders to each other; Wherein, the step of determining whether each second pixel is connected to the first pixel according to the second label value and the first label value based on the search order to obtain the target result includes: Based on the first search order, successively determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel to obtain a first result; Based on the second search order, successively determine whether each second pixel is connected to the first pixel according to the second label value and the first label value of each second pixel to obtain a second result; According to the first result and the second result, determine the target result.

4. The method according to claim 2, wherein The step of determining the scanning direction corresponding to the search sub-window includes: Based on the position information of the search sub-window within the search window, determine the scanning direction corresponding to the search sub-window.

5. The method according to claim 1, wherein There is an overlapping window part between two adjacent search sub-windows; Wherein, the step of determining whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain a target result includes: When the second pixel is located in the overlapping window part, determine whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain first results corresponding to two adjacent search sub-windows respectively; Merge the two first results, and determine the merged result as the target result.

6. The method according to claim 1, wherein The step of determining whether the second pixel is connected to the first pixel according to the first label value and the second label value to obtain a target result includes: Determine whether a third pixel of the image is connected to the first pixel to obtain a reference result, where the third pixel is a pixel adjacent to the second pixel; Determine whether the second pixel is connected to the first pixel based on the first tag value, the second tag value, and the reference result, to obtain a target result.

7. The method according to claim 6, characterized in that The determining whether the second pixel is connected to the first pixel based on the first tag value, the second tag value, and the reference result, to obtain a target result includes: Determine whether the first tag value and the second tag value are the same, to obtain a first operation result; Perform a logical OR operation on at least two of the reference results, to obtain a second operation result; Perform a logical AND operation on the first operation result and the second operation result, to obtain the target result.

8. The method according to claim 1, wherein The method further includes: Select at least one second pixel connected to the first pixel from multiple second pixels of the image; Form a connected region according to the selected second pixels.

9. The method according to claim 8, wherein The method further includes: Determine the feature information of the pixels within the connected region; Determine the disparity depth and / or optical flow information of the image according to the feature information.

10. The method according to claim 8, wherein The method further includes: Determine the connected size of the connected region; When the connected size is less than a size threshold, determine a target tag value, where the target tag value is a tag value corresponding to the largest boundary size among multiple boundary sizes, the boundary size is the size of a section of the boundary of the connected region, and the tag values of different sections of the boundary are different; Update the first tag value of the first pixel according to the target tag value.

11. An image processing apparatus, characterized in that, Includes: A first determination module, configured to determine a first tag value of a first pixel in an image obtained by superpixel segmentation, where the first pixel is a pixel shared by multiple search sub-windows, the search sub-windows are obtained by dividing a search window, and the first tag value is used to indicate the superpixel to which the first pixel belongs; A second determination module, configured to determine a second tag value of a second pixel of the image, where the second pixel is located in any one of the search sub-windows, and the second tag value is used to indicate the superpixel to which the second pixel belongs; A third determination module, configured to determine whether the second pixel is connected to the first pixel according to the first tag value and the second tag value, to obtain a target result.

12. An electronic device, characterized in that, Includes: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1-10.

14. A chip, characterized in that, The chip includes a processing circuit and an interface circuit; wherein, the interface circuit is configured to read instructions, and the interface circuit sends the instructions to the processing circuit, so that the processing circuit executes the method according to any one of claims 1-10.

Images