Redundant pixel processing method, device and readable storage medium of road image

Abstract

The present specification provides a method, device and readable storage medium for processing redundant pixels of road images. The method includes: firstly, binarizing the road image to ensure that the matrix data of the image matrix and its feature matrix are 0 or 1; secondly, before using the state machine to determine the redundant elements in the feature matrix, the image matrix and its feature matrix are corrected to avoid the disappearance or interruption of a specific pattern of roads. Therefore, the present case can effectively reduce the resources consumed in eliminating redundant pixels and improve the efficiency of eliminating redundant pixels while ensuring the connectivity of the image road after the redundant pixel elimination process.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a method, device and readable storage medium for processing redundant pixels of a road image. Background Art

[0002] By analyzing specific images containing geographic information and road information, some features of interest can be obtained. However, the width, length, and shape of each road in the image are different and intertwined. If the image is directly used as the input of machine learning (especially deep learning) without processing, a lot of computing resources will be consumed. At the same time, due to the overlapping errors of the signal pattern, further extraction of certain features will be completely impossible. Therefore, before further discovering features, the image will be preprocessed to obtain a feature vector. It generally includes the following three angles of processing: one is normalized and quantized image information; the second is information enhancement; and the third is denoising. Among them, the second and third are mainly to eliminate redundant pixels.

[0003] Since the existing methods need to use experience to determine the depth of a corrosion algorithm when eliminating redundant pixels, some wider roads will still be wide after redundant pixels are eliminated, while narrower roads or narrower parts in the middle of the road will disappear after unified depth redundant elimination, resulting in the disappearance or interruption of the road, causing information loss or errors, such as Figure 1a For the road image shown in FIG. 1 , if redundancy elimination is performed with a depth of 2 pixels, the resulting road image is as follows: Figure 1b As shown, we can see that the first road on the left is directly eliminated because it is too narrow, and the road on the right is also disconnected from the middle, while the widest road in the middle is too wide and does not achieve the effect of information enhancement.

[0004] Therefore, a more reliable redundant pixel processing solution needs to be provided. Summary of the invention

[0005] The embodiments of the present specification provide a method, device and readable storage medium for processing redundant pixels of a road image, which are used to effectively reduce the resources consumed in searching for redundant pixels and improve the efficiency and accuracy of eliminating redundant pixels while ensuring the connectivity of image roads after redundant processing.

[0006] The embodiment of this specification also provides a method for processing redundant pixels of a road image, including:

[0007] Performing a first convolution filtering process on the image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on the binarized road image;

[0008] Performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix;

[0009] Based on the first characteristic matrix and the second characteristic matrix, generate a third characteristic matrix;

[0010] Based on the third characteristic matrix, redundant pixels corresponding to each target element in the road image are eliminated, and the target element is an element with a first eigenvalue in the third characteristic matrix.

[0011] The embodiment of this specification also provides a redundant pixel processing device for a road image, comprising:

[0012] A first processing module, used for performing a first convolution filtering process on an image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on a binary road image;

[0013] A second processing module, used for performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix;

[0014] A third processing module, configured to generate a third characteristic matrix based on the first characteristic matrix and the second characteristic matrix;

[0015] The fourth processing module is used to perform redundancy processing on pixels corresponding to each target element in the road image based on the third characteristic matrix, and the target element is an element of the first eigenvalue in the third characteristic matrix.

[0016] The embodiment of this specification also provides an electronic device, including: a communication interface, a processor and a memory;

[0017] The processor calls the program instructions in the memory to execute the steps of the above method.

[0018] The embodiments of the present specification also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0019] In any of the above embodiments of the specification, on the one hand, the road image is binarized to ensure that the matrix data of the image matrix and its feature matrix are 0 or 1, thereby reducing the complexity of data processing; on the other hand, before determining the redundant elements in the feature matrix, the image matrix and its feature matrix are corrected to avoid the disappearance or interruption of a specific pattern of roads. Thus, while ensuring the connectivity of the image road after the redundant pixel elimination process, the resources consumed in eliminating redundant pixels can be effectively reduced, thereby improving the efficiency of eliminating redundant pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0021] Figure 1a and Figure 1b A schematic diagram of a road image before and after processing based on the prior art provided in an embodiment of this specification;

[0022] Figure 2 A schematic flow chart of a redundant pixel processing method for a road image provided in an embodiment of the present specification;

[0023] Figure 3 A flowchart of an implementation method of step 208 provided in an embodiment of this specification;

[0024] Figure 4 A schematic diagram of a flow chart of two convolution filtering and one state machine processing provided in an embodiment of this specification;

[0025] Figure 5 A logic diagram of a state machine provided in one embodiment of this specification;

[0026] Figure 6 A schematic diagram of a road image after redundant pixels are eliminated according to an embodiment of this specification;

[0027] Figure 7 A schematic diagram of the structure of a redundant pixel processing device for a road image provided by an embodiment of this specification;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this document.

[0030] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0031] Figure 2 A schematic diagram of a redundant pixel processing method for a road image provided in an embodiment of this specification is shown in FIG. Figure 2 , the method may specifically include the following steps:

[0032] Step 202: performing a first convolution filtering process on the image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on the binary road image;

[0033] Specifically: collect road images of a specified area, such as Figure 1a As shown, the road image is binarized, and the pixel value of the road part is 1, and the pixel value of the non-road part is 0; the binarized road image is matrixed to obtain an image matrix; then, a specific 3*3 matrix convolution kernel is used to perform the first convolution filtering process on the image matrix. Among them, the convolution step size of the first convolution filtering process is preferably 1; the convolution kernel used can ensure that the matrix obtained after the first convolution filtering process is still a binarized matrix, and its specific function is not limited here.

[0034] In addition, to ensure that the matrix sizes before and after the convolution are consistent, before the first convolution filtering process is performed, the image matrix is ​​padded with zeros in the outer circle, and the number of rows and columns of the matrix after the outer circle is padded with zeros is increased by 2.

[0035] Step 204: performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix;

[0036] Specifically: the elements of the first feature matrix are inverted; a specific 3*3 matrix convolution kernel is used to perform a second convolution filtering process on the inner product of the first feature matrix after the elements are inverted and the image matrix. The convolution step size of the second convolution filtering process is preferably 1.

[0037] Similarly, to ensure that the matrix sizes before and after convolution are consistent, before performing the second convolution filtering process, the matrix obtained by the inner product of the first feature matrix and the image matrix after element inversion is padded with zeros in the outer circle, and the number of rows and columns of the matrix after the outer circle zero padding is increased by 2.

[0038] Based on this, this embodiment can effectively reduce the amount of data processing by adopting the matrix inner product method; moreover, this embodiment ensures that the final feature matrix is ​​still a binary matrix by using a specific convolution kernel and processing logic, providing support for the subsequent reduction of computing resources.

[0039] Step 206: Generate a third characteristic matrix based on the first characteristic matrix and the second characteristic matrix;

[0040] Specifically: performing element-by-element logical AND on the first characteristic matrix and the second characteristic matrix to obtain a third characteristic matrix.

[0041] Based on this, this embodiment corrects the first feature matrix through steps 204 and 206 to avoid the disappearance or interruption of roads in certain specific modes (such as 2×10) due to direct execution of the state machine to delete pixels due to lack of correction, thereby affecting the accuracy of subsequent hazard point positioning.

[0042] Step 208: Eliminate redundant pixels corresponding to target elements in the road image based on the third characteristic matrix, where the target elements are elements with the first eigenvalue in the third characteristic matrix.

[0043] Among them, the target element generally refers to the element corresponding to the pixel with a pixel value of 1; the state machine is a directed graph consisting of a set of nodes and a set of corresponding transition functions. The state machine "runs" by responding to a series of events. Each event is within the control scope of the transition function belonging to the "current" node, where the scope of the function is a subset of the nodes. The function returns the "next" (perhaps the same) node. At least one of these nodes must be a terminal state. When the terminal state is reached, the state machine stops. A computational model that includes a set of states, a start state, a set of input symbols (alphabet), and a transition function (transition function) that maps the input symbols and the current state to the next state. When a string of symbols is input, the model immediately enters the start state. It depends on the transition function to change to a new state. In a finite state machine, there will be many variables, for example, the state machine has many actions associated with action transitions (Mealy machine) or states (Moore machine), multiple start states, transitions based on no input symbols, or multiple transitions of specified symbols and states (indefinite finite state machines), one or more states assigned to the receiving state (recognizer), and so on.

[0044] See also Figure 3 , one implementation of step 208 may be:

[0045] Step 302: determine a vector corresponding to each target element, where the vector corresponding to the target element is obtained by vectorizing the element corresponding to the target element and its surrounding elements in the image matrix;

[0046] Among them, the element corresponding to the target element in the image matrix refers to the element at a position corresponding to the position of the target element in the third characteristic matrix in the image matrix; the surrounding elements of the target element refer to the elements in a 3×3 matrix with the target element as the center element.

[0047] Specifically: after obtaining the image matrix, all elements in the image matrix can be vectorized, and then the elements corresponding to the target element and the vectors corresponding to its surrounding elements can be obtained therefrom; alternatively, the elements corresponding to the target element and its surrounding elements in the image matrix can be determined first, and then only the elements corresponding to the target element and its surrounding elements can be vectorized.

[0048] Step 304: input the vector corresponding to each target element into a pre-built state machine to obtain the element state output by the state machine, and the conversion function of the state machine performs state transfer according to the distribution of elements around the target element;

[0049] Step 306: Based on the element status of each target element, determine the redundant pixels corresponding to each target element in the road image and perform elimination processing.

[0050] Specifically: the pixel corresponding to the target element may be a redundant pixel or a non-redundant pixel, which can be determined by the element state of the target element. If the element state of the target element is a redundant state, the pixel corresponding to the target element is a redundant pixel and is eliminated; if the element state of the target element is a non-redundant state, the pixel corresponding to the target element is a non-redundant pixel and is not eliminated. Thus, an image after redundant pixel elimination and its corresponding enhanced feature matrix can be obtained.

[0051] Combination Figure 4 , the implementation of the secondary convolution filtering process and the state machine redundancy elimination process is described below:

[0052] See also Figure 4 , taking Imn as input, after two filtering and state machine FSM processing, the redundant pixels are eliminated while ensuring the existence, connectivity and directionality of the road image remain unchanged, thus obtaining the feature matrix after information enhancement, namely Omn.

[0053] Assume that the input is a two-dimensional road image containing M*N pixels, the value of the part with road is 1, and the value of the part without road is 0, which is represented by an m*n matrix to obtain the image matrix:

[0054]

[0055] Step 1: First Filter

[0056] This step of feature extraction combined with subsequent operations can automatically eliminate all redundant pixels without manually setting the depth of redundant pixels, avoiding overlapping errors in signal patterns caused by insufficient removal of redundant pixels on some overly wide roads, and also avoiding the subsequent steps from being unable to proceed correctly due to such overlapping errors. Specifically:

[0057] Use a specific 3*3 matrix (f1) to perform the first convolution filter on the above original matrix (image matrix) to obtain the first feature matrix T1 mn .

[0058] Among them, the calculation formula for the first convolution filter is as follows:

[0059]

[0060] In addition, in order to ensure that the size of the matrix after convolution is consistent with the original matrix, in the specific implementation, you can also add a circle of 0s in the outer circle before performing the convolution, and the convolution step size is 1.

[0061] Combined with the calculation formula, it is not difficult to understand that the first convolution filtering process is used to extract the features of the element based on the adjacent elements of the element to obtain the first feature matrix. The implementation principle can be to consider the adjacent elements of the element of the current convolution filtering, calculate the features of the element based on the distribution of the adjacent elements of the element, and so on, to obtain the features of each element, thereby obtaining the first feature matrix. Taking a single element in the original matrix as an example, it can be specifically exemplified as:

[0062] Example 1:

[0063] Assume that the current convolution filter object is I mn The first element in, the 3*3 submatrix centered on the first element is After I mn *The calculation of f1 is 3, and then after ÷2 and mod2, the calculation is 1, and 1 is recorded as the eigenvalue of the first element.

[0064] In this example, among the four adjacent elements of the first element, the adjacent element on its left is 0, indicating that the first element may correspond to a pixel on the edge of the road, while the other adjacent elements all have a eigenvalue of 1, indicating that the other adjacent elements all correspond to road pixels. Therefore, the first element is a suspected redundant element and can be marked as 1.

[0065] Example 2:

[0066] Assume that the current convolution filter object is I mn The second element in, the 3*3 submatrix centered on the second element is After I mn *The calculation of f1 is 4, and then after ÷2 and mod2, the calculation is 0, and 0 is recorded as the eigenvalue of the second element.

[0067] In this example, the four adjacent elements of the second element all have the eigenvalue 1, indicating that its adjacent elements all correspond to road pixels, and thus the second element is not an element corresponding to a road edge pixel. Therefore, the second element is not a suspected redundant element and can be marked as 0.

[0068] Based on this, this embodiment can accurately determine the suspected redundant elements by performing convolution filtering processing according to the distribution of the adjacent elements of the element, thereby providing sufficient data support for accurately eliminating redundant pixels.

[0069] Step 2: Second filtering

[0070] This step is mainly used to correct the feature matrix obtained in the first step. If the feature matrix is ​​not corrected and the state machine is directly executed to delete it, for some specific modes of roads, the road will disappear or be interrupted, that is, the complete and effective retention of the features cannot be guaranteed, resulting in deviation in the subsequent positioning of hidden danger points. Specifically:

[0071] First, use filter 2 (f2) to perform convolution filtering on T1 and the image matrix to obtain the second feature matrix T2, the formula is as follows:

[0072] in, For T1 mn The matrix after inverting each element.

[0073] Similarly, in order to ensure that the size of the matrix after convolution is consistent with the original matrix, you can also fill a circle of 0s in the outer circle before performing the convolution, and the convolution step size is 1.

[0074] Secondly, perform element-by-element logical AND on T1 and T2 to obtain the third characteristic matrix T3, the formula is as follows:

[0075] T3 mn =T2 mn ∧T1 mn , where t3 mn ={0,1}.

[0076] Combined with the calculation formula of the second convolution filter, it is not difficult to understand that the second convolution filter processing is used to correct the first feature matrix T1 based on the surrounding elements of the element, including correcting the eigenvalues ​​of some suspected redundant elements in the first feature matrix to 0 to remove the rows and columns of suspected redundant elements. Its implementation principle can be to further consider other elements around the suspected redundant elements on the basis of the adjacent elements of the suspected redundant elements currently undergoing the second convolution filter, and accurately judge whether the element is a suspected redundant element based on the distribution of all elements in the 3*3 submatrix centered on the suspected redundant element. If not, correct the eigenvalues ​​of the elements marked as suspected redundant elements. Specific examples can be:

[0077] Assuming that the road pixels are 2*N, that is, two rows or two columns of road pixels, its image matrix is The eigenvalues ​​of the elements represented by the ellipsis in the first and fourth rows are 0, and the eigenvalues ​​of the elements represented by the ellipsis in the second and third rows are 1. The elements with eigenvalues ​​of 1 correspond to road pixels, and the elements with eigenvalues ​​of 0 correspond to non-road pixels. For the elements with eigenvalues ​​of 1, such as the third element i in the second row (denoted as the third element), the 3*3 submatrix centered on the third element is Based on the calculation of 1 by the first convolution filter processing, the third element is marked as a suspected redundant element. If it is directly input into the state machine for redundancy elimination, the third element may be misjudged as a redundant element and eliminated. Similarly, other elements or some other elements with a feature value of 1 in the second and third rows may also be marked as redundant elements and misjudged as redundant elements by the state machine and eliminated, thereby causing the 2*N road to disappear or be interrupted.

[0078] Based on this, this embodiment combines the processing results of the first convolution filter with a second convolution filter processing, which can correct some suspected redundant elements into non-suspected redundant elements, thereby achieving the purpose of avoiding elements from being misjudged as redundant elements and ensuring the connectivity of the road.

[0079] Combining the above two convolution filtering operations, it is not difficult to understand that, since only the suspected redundant elements in the outer circle of the matrix can be marked each time, for a road with a large width (such as 6*N road pixels), after eliminating redundant pixels once, wider pixels may still remain. Therefore, in order to eliminate all redundant pixels as much as possible, this embodiment provides a redundant pixel cyclic elimination step, which may be specifically:

[0080] If there are redundant pixels in the road image after the elimination process, the steps of correcting the regenerated first feature matrix through the first convolution filtering process and the second convolution filtering process are cyclically executed, and the redundant pixels corresponding to each target element in the regenerated third feature matrix are eliminated until a road image without redundant pixels is obtained. Specifically, an example is as follows:

[0081] Assuming that the road image includes 6*N pixels, through the above two convolution filters and the redundant processing of the state machine, some of the outermost pixels in the 6*N pixels can be eliminated to obtain 4*N pixels, and there are still redundant pixels; reconstructing the image matrix of 4*N pixels, and re-performing two convolution filters and the redundant processing of the state machine, 2*N pixels can be obtained, see Figure 1a On the left, there are still redundant pixels; continue to reconstruct the image matrix of 2*N pixels, and perform convolution filtering twice and redundant processing of the state machine. Figure 6 The final road image is shown on the left.

[0082] Step 3: Traverse the target elements with a feature value of 1 in T3 and use the state machine to delete the redundant pixels.

[0083] The main function of this step is to eliminate redundant pixels based on the third characteristic matrix obtained in the above two steps. If the state machine method is not used, it is necessary to traverse each pixel, combine the characteristic matrix to determine the status and information of each pixel around the current pixel, and then calculate the distance and path of all related points to determine whether deleting the current pixel will affect connectivity or information loss. Multiple loops and iterations are required. The time complexity of this mode is usually at least O(n^2). Although this time complexity is acceptable in laboratory environments or in advanced operating system environments, it is almost impossible to achieve it with simple hardware in practical applications, such as line inspection scenarios, edge computing, and the Internet of Things.

[0084] This step is implemented in a state machine mode, avoiding multiple loops and iterations for each element. Since the matrix elements are either 0 or 1, the operation process can be directly implemented by hardware, and the time complexity is only O(n). It can be seen that this solution is suitable for both high-level language implementation based on computer operating systems and hardware implementation. It can be applied to the manufacturing scenarios of single-chip microcomputers or application chips, and used in scenarios such as the Internet of Things. Specifically:

[0085] First, let the current traversal target pixel be i xy , where x∈{1,2...m}, y∈{1,2...n}, take the elements around the current element and vectorize them, that is:

[0086]

[0087] Secondly, run the state machine. The state machine logic is as follows Figure 5 As shown:

[0088] State description: State 1 is the initial state; State 9 and State 10 are the final states. When entering State 9, the current i xy is a redundant element, so delete it; enter state 10, then the current i xy It is a non-redundant element and cannot be deleted.

[0089] Conditions:

[0090] A:a2+a4+a6+a8=1

[0091] B: a2+a4+a6+a8=2

[0092] C: a2+a4+a6+a8=3

[0093] D:a2=1

[0094] E:a2=0

[0095] F:a2+a8=2

[0096] G:a2+a8=1

[0097] H: (a1+a4=2)∨(a3+a6=2)

[0098]

[0099] J:a8=1

[0100] K:a8=0

[0101] L:(a1+a4+a7=3)∨(a3+a6+a9=3)

[0102]

[0103]

[0104] O:(a1+a2+a3=3)∨(a7+a8+a9=3)

[0105] P:(a7+a4=2)∨(a9+a6=2)

[0106]

[0107] The following is a description of the above conditions and Figure 5 Detailed description of the state machine logic:

[0108] S1, determining the sum of adjacent elements of the first target element;

[0109] Among them, the first target element is t xy Corresponding to a5, the sum of its adjacent elements is the sum of a2, a4, a6 and a8.

[0110] S2, transferring the first state of the state machine to a second target state corresponding to a transition function matching the sum of the adjacent elements, wherein the first state is the initial state of the state machine ( Figure 5 State 1); specifically:

[0111] If the sum of the adjacent elements of the first target element is 2, the first state of the state machine is transferred to the second state corresponding to the transition function matching the sum of the adjacent elements ( Figure 5 State 2 in the state); that is, the sum of a2, a4, a6 and a8 conforms to the transition function B, then the state is transferred to state 2.

[0112] or,

[0113] If the sum of the adjacent elements of the first target element is 3, the first state of the state machine is transferred to the third state corresponding to the transition function matching the sum of the adjacent elements ( Figure 5 State 3 in the example); that is, the sum of a2, a4, a6, and a8 satisfies the transition function C, and the state is transferred to state 3.

[0114] or,

[0115] If the sum of the adjacent elements of the first target element is 1, the first state of the state machine is transferred to the first final state corresponding to the transition function matching the sum of the adjacent elements ( Figure 5 State 9 in the figure). That is, the sum of a2, a4, a6 and a8 meets the transition function A, and the state is transferred to state 9.

[0116] S3. If the second target state is a final state, use the second target state as the element state of the first target element.

[0117] For example, if the second target state is state 9, the element state of the first target element is determined to be a redundant element, and elimination is performed.

[0118] S3', if the second target state is a non-final state, determine the distribution of adjacent elements in the same column as the first target element; transfer the second target state to a third target state corresponding to a conversion function matching the distribution of adjacent elements in the same column; specifically:

[0119] When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 1 (ie, a2=1), the second state is transitioned to the fourth state ( Figure 5 in state 4); or,

[0120] When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 0 (ie, a2=0), the second state is transitioned to the fifth state ( Figure 5 5 in the state), and determine the second adjacent element in the next column of the first target element (i.e., a8); if the second adjacent element is 1 (i.e., a8=1), the fifth state is transferred to the eighth state ( Figure 5 8 in state); if the second adjacent element is 0 (i.e. a8 = 0), the state is transferred to the second final state ( Figure 5 in state 10); or,

[0121] When transferring to the third state, if the sum of the adjacent elements in the same column of the first target element is 2 (ie, a2+a8=2), the third state is transferred to the sixth state ( Figure 5or, if the sum of the adjacent elements in the same column of the first target element is 1 (i.e., a2+a8=1), the third state is transferred to the seventh state ( Figure 5 State 7).

[0122] S4, based on the distribution of elements other than the adjacent elements in the same column of the first target element or the distribution of elements in other rows / columns other than the row / column where the first target element is located, transfer the third target state to a final state corresponding to a conversion function matching the distribution to obtain the element state of the first target element. Specifically:

[0123] When transferring to the fourth state, based on the distribution of other elements in the same line as the first target element and the first adjacent element, the fourth state is transferred to the final state corresponding to the conversion function matching the distribution; that is, if the distribution of a1, a4, a3 and a6 in the same line as a5 and a2 meets the conversion function H, the state is transferred to state 9; if it meets the conversion function I, the state is transferred to state 10.

[0124] or,

[0125] When transferring to the eighth state, based on the distribution of other elements in the same line as the first target element and the second adjacent element, the eighth state is transferred to the final state corresponding to the conversion function matching the distribution; that is, if the distribution of a4, a6, a7 and a9 in the same line as a5 and a8 meets the conversion function P, the state is transferred to state 9; if it meets the conversion function Q, the state is transferred to state 10.

[0126] or,

[0127] When transferring to the sixth state, based on the distribution of the elements in the column before and the elements in the column after the first target element, the sixth state is transferred to the final state corresponding to the conversion function matching the distribution; that is, the distribution of the elements a1, a4 and a7 in the column before a5 and the elements a3, a6 and a9 in the column after, if it meets the conversion function L, the state is transferred to state 9; if it meets the conversion function M, the state is transferred to state 10.

[0128] or,

[0129] When transferring to the seventh state, based on the distribution of the elements in the previous row and the next row of the first target element, the seventh state is transferred to the final state corresponding to the conversion function matching the distribution. That is, if the distribution of the elements a1, a2 and a3 in the previous row of a5 and the elements a7, a8 and a9 in the next row of a5 meet the conversion function N, the state is transferred to state 10; if it meets the conversion function O, the state is transferred to state 9.

[0130] Based on this, this embodiment can perform state transfer according to the matching relationship between the element distribution and the state machine's conversion function through a reasonably constructed state machine logic, and obtain redundant pixels in the road image, thereby avoiding the situation in the prior art where multiple loops and iterations are required for a single element, thereby effectively improving the efficiency of searching for redundant pixels; and, since the matrix elements are either 0 or 1, the complexity of the redundant pixel search scheme of the state machine is very low, which can effectively reduce the restrictions on the processing platform during the application process.

[0131] Furthermore, through this embodiment, redundant pixels can be eliminated while ensuring the connectivity of the road image. The road width in the deleted image will not exceed 2 pixels, and only key information is retained, achieving the effect of denoising and signal enhancement, providing a feature vector containing only key information for subsequent hidden danger point positioning.

[0132] Moreover, for the input two-dimensional road image, after deleting redundant pixels through the above method, the actual area corresponding to the diagonal distance of one pixel is about 2.8 meters. This minimizes the road error, accurately locates the road, and avoids overlapping errors of signal patterns. Of course, it is also suitable for preprocessing of other application scenarios or road information enhancement.

[0133] In addition, the detailed process shows that the entire calculation process is a simple calculation of 0 or 1, without complex multiplication or multiple loops and judgments. The subsequent deletion of redundant pixels uses the state machine mode. If this series of processing processes is implemented based on an advanced operating system, the complexity is O(n). Experiments show that the efficiency of processing a series of pictures of 100 megabytes is in seconds. If the overall project implementation process is extremely suitable for hardware implementation, the entire processing process can be deployed in application scenarios such as the Internet of Things to realize the discovery of hidden dangers anytime and anywhere.

[0134] Combination Figure 6 , the solution provided in this embodiment is adopted to eliminate redundant pixels in the road image. It can be seen that no matter it is a wide road or a narrow road image, as many redundant pixels as possible are eliminated while retaining existence, connectivity and directionality, thereby achieving the retention of key features and signal enhancement to the greatest extent.

[0135] In summary, this embodiment, on the one hand, performs binarization processing on the road image to ensure that the matrix data of the image matrix and its feature matrix are 0 or 1; on the other hand, before using the state machine to determine the redundant elements in the feature matrix, the image matrix and its feature matrix are corrected to avoid the disappearance or interruption of the road of a specific pattern. In this way, while ensuring the connectivity of the image road after the redundant pixel elimination process, the resources consumed in eliminating redundant pixels can be effectively reduced, and the efficiency of eliminating redundant pixels can be improved.

[0136] Figure 7 This is a schematic diagram of the structure of a redundant pixel processing device for a road image provided by an embodiment of this specification, see Figure 7 , the device may specifically include:

[0137] A first processing module 701 is used to perform a first convolution filtering process on an image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on a binary road image;

[0138] A second processing module 702 is used to perform a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix;

[0139] A third processing module 703 is used to generate a third characteristic matrix based on the first characteristic matrix and the second characteristic matrix;

[0140] The fourth processing module 704 is used to perform redundancy processing on pixels corresponding to each target element in the road image based on the third characteristic matrix, where the target element is an element with a first eigenvalue in the third characteristic matrix.

[0141] Optionally, the first convolution filtering process is used to extract the features of the element based on the adjacent elements of the element to obtain a first feature matrix; the second convolution filtering process is used to correct the first feature matrix based on the surrounding elements of the element;

[0142] The device also includes:

[0143] The loop module is used for, if redundant pixels exist in the road image after the elimination process, to control the first processing module 701, the second processing module 702, the third processing module 703 and the fourth processing module 704 to loop through the first convolution filtering process and the second convolution filtering process, correct the regenerated first feature matrix, and eliminate the redundant pixels corresponding to each target element in the regenerated third feature matrix until a road image without redundant pixels is obtained.

[0144] Optionally, the second processing module 702 is specifically configured to:

[0145] Inverting the elements of the first feature matrix; performing a second convolution filtering process on the inner product of the first feature matrix after the elements are inverted and the image matrix;

[0146] The third processing module 703 is specifically used for:

[0147] Performing element-by-element logical AND on the first characteristic matrix and the second characteristic matrix to obtain a third characteristic matrix.

[0148] Optionally, the device further comprises:

[0149] A first zero-padding module is used to perform outer circle zero-padding processing on the image matrix before performing the first convolution filtering processing, and the number of rows and columns of the matrix after the outer circle zero-padding is increased by 1;

[0150] The second zero-padding module performs outer circle zero-padding on the matrix obtained by the inner product of the first feature matrix and the image matrix after the elements are inverted before the second convolution filtering process, and the number of rows and columns of the matrix after the outer circle zero-padding is increased by 1.

[0151] Optionally, the fourth processing module 704 is specifically configured to:

[0152] Determine a vector corresponding to each target element, wherein the vector corresponding to the target element is obtained by vectorizing the element corresponding to the target element and its surrounding elements in the image matrix; input the vector corresponding to each target element into a pre-built state machine to obtain the element state output by the state machine, wherein the conversion function of the state machine performs state transfer according to the distribution of elements surrounding the target element; and determine and eliminate redundant pixels corresponding to each target element in the road image based on the element state of each target element.

[0153] Optionally, the fourth processing module 704 is specifically configured to:

[0154] Determine the sum of adjacent elements of a first target element; transfer the first state of the state machine to a second target state corresponding to a transition function matching the sum of the adjacent elements, wherein the first state is the initial state of the state machine; if the second target state is a final state, use the second target state as the element state of the first target element.

[0155] Optionally, the fourth processing module 704 is specifically configured to:

[0156] If the second target state is a non-final state, determine the distribution of adjacent elements in the same column as the first target element; transfer the second target state to a third target state corresponding to a conversion function that matches the distribution of adjacent elements in the same column; based on the distribution of elements other than the adjacent elements in the same column of the first target element or the distribution of elements in other rows / columns other than the row / column where the first target element is located, transfer the third target state to a final state corresponding to a conversion function that matches the distribution, and obtain the element state of the first target element.

[0157] Optionally, the fourth processing module 704 is specifically configured to:

[0158] If the sum of the adjacent elements of the first target element is 2, the first state of the state machine is transferred to the second state corresponding to the transition function matching the sum of the adjacent elements; or,

[0159] If the sum of the adjacent elements of the first target element is 3, the first state of the state machine is transferred to a third state corresponding to a transition function matching the sum of the adjacent elements; or,

[0160] If the sum of the adjacent elements of the first target element is 1, the first state of the state machine is transferred to a first final state corresponding to a transition function matching the sum of the adjacent elements.

[0161] Optionally, the fourth processing module 704 is specifically configured to:

[0162] When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 1, the second state is transitioned to the fourth state; or,

[0163] When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 0, the second state is transitioned to the fifth state, and the second adjacent element in the next column of the first target element is determined; if the second adjacent element is 1, the fifth state is transitioned to the eighth state; if the second adjacent element is 0, the state is transitioned to the second final state; or,

[0164] When transferring to the third state, if the sum of the adjacent elements in the same column of the first target element is 2, the third state is transferred to the sixth state; or, if the sum of the adjacent elements in the same column of the first target element is 1, the third state is transferred to the seventh state.

[0165] Optionally, the fourth processing module 704 is specifically configured to:

[0166] When transferring to the fourth state, based on the distribution of other elements in the same row as the first target element and the first adjacent element, transferring the fourth state to a final state corresponding to a conversion function matching the distribution; or,

[0167] When transferring to the eighth state, based on the distribution of other elements in the same row as the first target element and the second adjacent element, transferring the eighth state to a final state corresponding to a conversion function matching the distribution; or,

[0168] When transferring to the sixth state, based on the distribution of the elements in the column before and after the first target element, transferring the sixth state to a final state corresponding to the conversion function matching the distribution; or,

[0169] When transferring to the seventh state, based on the distribution of the elements in the previous row and the elements in the next row of the first target element, the seventh state is transferred to a final state corresponding to the conversion function matching the distribution.

[0170] In summary, this embodiment, on the one hand, performs binarization processing on the road image to ensure that the matrix data of the image matrix and its feature matrix are 0 or 1, thereby reducing the complexity of data processing; on the other hand, before using the state machine to determine the redundant elements in the feature matrix, the image matrix and its feature matrix are corrected to avoid the disappearance or interruption of a specific pattern of roads. In this way, while ensuring the connectivity of the image road after the redundant pixel elimination process, the resources consumed in eliminating redundant pixels can be effectively reduced, thereby improving the efficiency of eliminating redundant pixels.

[0171] In addition, for the above-mentioned device implementation, since it is basically similar to the method implementation, the description is relatively simple, and the relevant parts can refer to the partial description of the method implementation. It should be noted that among the various components of the device in this specification, the components are logically divided according to the functions to be implemented, but this specification is not limited to this, and the various components can be re-divided or combined as needed.

[0172] Figure 8 A schematic diagram of a network device provided in an embodiment of this specification is shown in FIG. Figure 8 , the network device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a redundant pixel processing device for the road image at the logical level. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0173] The network interface, processor and memory can be connected to each other through a bus system. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0174] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may include high-speed random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.

[0175] The processor is used to execute the program stored in the memory, and specifically perform:

[0176] Performing a first convolution filtering process on the image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on the binarized road image;

[0177] Performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix;

[0178] Based on the first characteristic matrix and the second characteristic matrix, generate a third characteristic matrix;

[0179] Based on the third characteristic matrix, redundant pixels corresponding to each target element in the road image are eliminated, and the target element is an element with a first eigenvalue in the third characteristic matrix.

[0180] The above is as in this manual Figure 7The method performed by the redundant pixel processing device or the manager (Master) node of the road image disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this specification can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0181] The redundant pixel processing device of the road image can also perform Figure 2-5 and implement the method executed by the manager node.

[0182] Based on the same invention, the embodiment of this specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a network device including multiple application programs, the network device executes Figure 2-5 The corresponding embodiment provides a method for processing redundant pixels of a road image.

[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0184] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0185] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0186] This specification is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0187] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0188] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0189] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0190] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0191] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0192] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0193] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for processing redundant pixels of a road image, characterized in that: include: Performing a first convolution filtering process on the image matrix to obtain a first feature matrix, wherein the image matrix is ​​constructed based on the binarized road image; Performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix; Based on the first characteristic matrix and the second characteristic matrix, generate a third characteristic matrix; Eliminating redundant pixels corresponding to target elements in the road image based on the third characteristic matrix, wherein the target elements are elements with the first eigenvalue in the third characteristic matrix; The performing a second convolution filtering process on the first feature matrix and the image matrix to obtain a second feature matrix includes: Inverting the elements of the first feature matrix; using a specific 3*3 matrix convolution kernel, performing a second convolution filtering process on the inner product of the first feature matrix after the elements are inverted and the image matrix to obtain a second feature matrix; The step of generating a third characteristic matrix based on the first characteristic matrix and the second characteristic matrix includes: Performing element-by-element logical AND on the first characteristic matrix and the second characteristic matrix to obtain a third characteristic matrix; The eliminating of redundant pixels corresponding to each target element in the road image based on the third characteristic matrix includes: Determine a vector corresponding to each target element, where the vector corresponding to the target element is obtained by vectorizing the element corresponding to the target element and its surrounding elements in the image matrix; Input the vectors corresponding to the target elements into a pre-built state machine to obtain the element states output by the state machine; Based on the element states of the target elements, determining redundant pixels corresponding to the target elements in the road image and performing elimination processing; Among them, the specific 3*3 matrix is .

2. The method according to claim 1, characterized in that The first convolution filtering process is used to extract the features of the element according to the adjacent elements of the element to obtain a first feature matrix; The second convolution filtering process is used to modify the first feature matrix according to the surrounding elements of the element; The method also includes: If there are redundant pixels in the road image after the elimination process, the steps of correcting the regenerated first feature matrix through the first convolution filtering process and the second convolution filtering process are cyclically executed, and eliminating the redundant pixels corresponding to each target element in the regenerated third feature matrix until a road image without redundant pixels is obtained.

3. The method according to claim 1 or 2, characterized in that: Also includes: Before performing the first convolution filtering process, the image matrix is ​​subjected to outer circle zero padding, and the number of rows and columns of the matrix after the outer circle zero padding is increased by 2; Before performing the second convolution filtering process, a matrix obtained by the inner product of the first feature matrix and the image matrix after the elements are inverted is padded with zeros in the outer circle, and the number of rows and columns of the matrix after the outer circle zero padding is increased by 2.

4. The method according to claim 1, characterized in that: The transition function of the state machine performs state transfer according to the distribution of elements around the target element.

5. The method according to claim 4, characterized in that The step of inputting the vectors corresponding to the target elements into a pre-built state machine to obtain the element state output by the state machine includes: Determine the sum of adjacent elements of the first target element; Transferring a first state of the state machine to a second target state corresponding to a transition function matching the sum of the adjacent elements, the first state being an initial state of the state machine; If the second target state is a final state, taking the second target state as the element state of the first target element; If the second target state is a non-final state, determine the distribution of adjacent elements in the same column as the first target element; transfer the second target state to a third target state corresponding to a conversion function that matches the distribution of adjacent elements in the same column; based on the distribution of elements other than the adjacent elements in the same column of the first target element or the distribution of elements in other rows / columns other than the row / column where the first target element is located, transfer the third target state to a final state corresponding to a conversion function that matches the distribution, and obtain the element state of the first target element.

6. The method according to claim 5, characterized in that The transferring the first state of the state machine to a second target state corresponding to a transition function matching the sum of the adjacent elements comprises: If the sum of the adjacent elements of the first target element is 2, the first state of the state machine is transferred to the second state corresponding to the transition function matching the sum of the adjacent elements; or, If the sum of the adjacent elements of the first target element is 3, the first state of the state machine is transferred to a third state corresponding to a transition function matching the sum of the adjacent elements; or, If the sum of the adjacent elements of the first target element is 1, the first state of the state machine is transferred to a first final state corresponding to a transition function matching the sum of the adjacent elements.

7. The method according to claim 6, characterized in that The transferring the second target state to a third target state corresponding to a conversion function matching the distribution of adjacent elements in the same column includes: When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 1, the second state is transitioned to the fourth state; or, When transitioning to the second state, if the first adjacent element in the previous column of the first target element is 0, the second state is transitioned to the fifth state, and the second adjacent element in the next column of the first target element is determined; if the second adjacent element is 1, the fifth state is transitioned to the eighth state; if the second adjacent element is 0, the state is transitioned to the second final state; or, When transferring to the third state, if the sum of the adjacent elements in the same column of the first target element is 2, the third state is transferred to the sixth state; or, if the sum of the adjacent elements in the same column of the first target element is 1, the third state is transferred to the seventh state.

8. The method according to claim 7, characterized in that The transferring the third target state to a final state corresponding to a conversion function matching the distribution based on the distribution of elements other than adjacent elements in the same column of the first target element or the distribution of elements in other rows / columns other than the row / column where the first target element is located comprises: When transferring to the fourth state, based on the distribution of other elements in the same row as the first target element and the first adjacent element, transferring the fourth state to a final state corresponding to a conversion function matching the distribution; or, When transferring to the eighth state, based on the distribution of other elements in the same row as the first target element and the second adjacent element, transferring the eighth state to a final state corresponding to a conversion function matching the distribution; or, When transferring to the sixth state, based on the distribution of the elements in the column before and after the first target element, transferring the sixth state to a final state corresponding to the conversion function matching the distribution; or, When transferring to the seventh state, based on the distribution of the elements in the previous row and the elements in the next row of the first target element, the seventh state is transferred to a final state corresponding to the conversion function matching the distribution.

9. An electronic device, characterized in that: include: Communications interface, processor and memory; The processor calls the program instructions in the memory to execute the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images