Image processing method and device, equipment and storage medium
By extracting line features in the image and matching intersection points, the problem of poor accuracy of line segment matching in the prior art is solved, and the robustness and accuracy of image processing are improved.
Patent Information
- Application Number
- CN202310667205.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-06
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, the image processing method LBD has poor accuracy in matching line segments in the image under low definition conditions and is insufficiently robust.
Line features are extracted separately from the matching images, the intersection points corresponding to the line segments are determined, and the point feature matching target line segments are determined based on the matching point pairs.
Improves the accuracy and robustness of line segment matching and enhances the processing capability of low-definition images.
Smart Images

Figure CN119091164A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular, to an image processing method, apparatus, device and storage medium. Background Art
[0002] With the development of image processing technology, technicians have developed more and more image processing scenarios, such as image recognition scenarios, augmented reality scenarios, etc. In many image processing scenarios, it is necessary to match the line segments in two images and identify the same line segments in the two images.
[0003] In the prior art, image processing can use LBD (Line Binary Discriptor) to achieve line segment matching. This method forms a strip area with a piece of pixels around the line segment, and calculates the gradient relationship of the pixels in the strip area. The line segments in the two images are matched based on the gradient relationship of the pixels in the strip area.
[0004] However, when LBD is used to match line segments in two images, the image clarity is required to be high. When the image clarity is low, the accuracy of line segment matching is poor. Therefore, the above method has poor robustness. Summary of the invention
[0005] The embodiments of the present disclosure provide an image processing method, apparatus, device and storage medium to overcome the problems of low accuracy and poor robustness in matching line segments in two images.
[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, including:
[0007] Performing line feature extraction on the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments;
[0008] Determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments;
[0009] Matching the plurality of first intersection points with the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point;
[0010] Based on the multiple matching point pairs, multiple matching target line segments in the first image and the second image are determined.
[0011] In a second aspect, an embodiment of the present disclosure provides an image processing device, including:
[0012] An extraction module is used to extract line features from the first image and the second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments;
[0013] A first determining module, configured to determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments;
[0014] A matching module, used for matching the plurality of first intersection points with the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point;
[0015] The second determination module is used to determine a plurality of target line segments that match the first image and the second image based on the plurality of matching point pairs.
[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
[0017] A processor, and a memory communicatively connected to the processor;
[0018] The memory stores computer-executable instructions;
[0019] The processor executes the computer-executable instructions stored in the memory to implement the image processing method as described in the first aspect and various possible designs of the first aspect.
[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method described in the first aspect and various possible designs of the first aspect is implemented.
[0021] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method as described in the first aspect and various possible designs of the first aspect.
[0022] The image processing method, apparatus, device and storage medium provided in this embodiment respectively extract line features from the first image and the second image to be matched to obtain a plurality of first line segments and a plurality of second line segments; determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments; match the plurality of first intersection points and the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point; and determine a plurality of matching target line segments in the first image and the second image based on the plurality of matching point pairs. Since the line features in the first image and the second image are extracted, the line segments in the images are extracted, and then the intersection points corresponding to the line segments are determined, the line segments are matched through the correlation relationship between points and lines, and the stability and accuracy of the point feature matching are relatively high, thus improving the accuracy and robustness of the image processing method for matching line segments. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0024] Figure 1 An application scenario diagram of the image processing method provided by the embodiment of the present disclosure;
[0025] Figure 2 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 1 ;
[0026] Figure 3 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 2 ;
[0027] Figure 4 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 3 ;
[0028] Figure 5 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 4 ;
[0029] Figure 6 A structural block diagram of an image processing device provided by an embodiment of the present disclosure;
[0030] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0033] The application scenarios of the embodiments of the present disclosure are explained below:
[0034] The image processing method provided by the embodiments of the present disclosure can be applied to multiple scenarios, for example, it can be applied to image comparison scenarios, and for example, it can be applied to AR (Augmented Reality) navigation scenarios. Figure 1 This is an application scenario diagram of the image processing method provided by the embodiment of the present disclosure. Specifically, Figure 1 As shown, the terminal takes real-time pictures to obtain images ( Figure 1 The first image in the image is used as a query frame to retrieve the closest key frame in the constructed visual map, and then the first image and the key frame are matched with local features by the image processing method in the present application, and the pose information of the first image is determined on this basis, and the corresponding AR navigation special effects are added to the first image based on the pose information to obtain a special effect navigation image (such as Figure 1 The special effects navigation image in the image is finally displayed in the terminal.
[0035] In the prior art, image processing can use LBD (Line Binary Discriptor) to achieve line segment matching. This method forms a strip area with a piece of pixels around the line segment, and calculates the gradient relationship of the pixels in the strip area. The line segments in the two images are matched based on the gradient relationship of the pixels in the strip area.
[0036] In the above application scenario, since LBD is used to match line segments in two images, the clarity requirements for the first image and the second image are relatively high. If the image acquisition device is shaken or the pixel of the captured image is low, the accuracy of the pixel gradient in the strip area will be affected, resulting in poor accuracy of line segment matching.
[0037] Therefore, the robustness of line segment matching using LBD in image processing is poor.
[0038] In order to solve the above problems, the present embodiment provides an image processing method, apparatus, device and storage medium. First, line features are extracted from the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments; then, a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments are determined; finally, the plurality of first intersection points and the plurality of second intersection points are matched to obtain a plurality of matching point pairs, and based on the plurality of matching point pairs, a plurality of matching target line segments in the first image and the second image are determined. Since the line features in the first image and the second image are extracted, the line segments in the images are extracted, and then the intersection points corresponding to the line segments are determined, the line segments are matched through the correlation relationship between points and lines, and the stability and accuracy of the point feature matching are relatively high, thus improving the accuracy and robustness of the image processing method for matching line segments.
[0039] refer to Figure 2 , Figure 2 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied to electronic devices such as terminal devices and servers. The image processing method includes:
[0040] S201: extracting line features from the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments.
[0041] In this embodiment, the first image may be image data collected in real time by a camera, and the second image may be image data corresponding to the first image confirmed by a map based on visual data. Line feature extraction is performed on the first image to obtain a plurality of first line segments, and line feature extraction is performed on the second image to obtain a plurality of second line segments. Exemplarily, the first image may be street image data collected in real time by a camera, wherein the image data may be a picture or a video frame in a video. Exemplarily, the second image may be image data corresponding to the first image (street image data collected in real time by a camera) determined in a constructed AR navigation map.
[0042] It should be noted that the first line segment (or the second line segment) may be all line segments of the scene (such as the vehicle in the figure) in the combined image, or part of the line segments. Exemplarily, the first line segment (or the second line segment) may be a boundary line segment of traffic indication lines such as a roadway, a driving direction, a road edge, a crosswalk, etc. in the first image, or a boundary line segment of a building outline such as an office building, a residential building, a gas station, etc., or a boundary line segment of a public facility such as a street lamp and a trash can.
[0043] S202: Determine a plurality of first intersection points corresponding to a plurality of first line segments and a plurality of second intersection points corresponding to a plurality of second line segments.
[0044] In this embodiment, the extension directions of the plurality of first line segments are the same or different; wherein, some of the first line segments can intersect (or the straight lines where the first line segments are located can intersect), and the intersection point where the first line segments (or the straight lines where the first line segments are located) intersect is the first intersection point. Similarly, the extension directions of the plurality of second line segments are the same or different; wherein, some of the second line segments can intersect (or the straight lines where the second line segments are located can intersect), and the intersection point where the second line segments (or the straight lines where the second line segments are located) intersect is the second intersection point.
[0045] Optionally, the specific steps of determining multiple first intersections and multiple first intersections may include: determining multiple first straight lines where multiple first line segments are located, determining the intersection of any two of the multiple first straight lines, and obtaining multiple first intersections; determining multiple second straight lines where multiple second line segments are located, determining the intersection of any two of the multiple second straight lines, and obtaining multiple second intersections.
[0046] Among them, the first intersection point can be located at the endpoint, inside or outside of the first line segment. In some possible cases, the first intersection point is located at the endpoint of the first line segment. For example, the endpoints of two first line segments intersect and coincide at a point (i.e., two first line segments forming an "L" shape). In this case, the first intersection point is located at the endpoints of the two first line segments. For another example, the endpoints of three first line segments intersect and coincide at a point (i.e., three first line segments forming a "Y" shape). In this case, the first intersection point is the endpoints of the three first line segments. It should be noted that the coincidence of the endpoints of two or more first line segments can be that the pixel coordinates corresponding to the endpoints are the same, or that the distance between the pixel coordinates corresponding to the endpoints is less than a preset threshold, which will not be repeated here.
[0047] In some other possible cases, the endpoint of one first line segment intersects with the middle of another first line segment (i.e., two first line segments form a "T" shape). In this case, the first intersection point is located at the endpoint of one first line segment and inside the other first line segment. In some other possible cases, the two first line segments may intersect with the middle of the line segments (i.e., two first line segments form an "X" shape). In this case, the first intersection point is located inside the two first line segments. In some other possible cases, the extension line of the straight line where one first line segment is located intersects with the extension line of the straight line where the other first line segment is located (i.e., two first line segments form an "Eight" shape). In this case, the first intersection point is located outside the two first line segments.
[0048] It should be noted that after the first intersections corresponding to the first line segments, the positional relationship between the first intersections and the first line segments to which they belong can also be associated to obtain the positional relationship information between the first intersections and the first line segments. For example, the positional relationship information is: the first intersection A is located inside the first line segment A and at the endpoint of the first line segment B.
[0049] The second intersection point may be located at the end point, inside or outside of the second line segment. The positional relationship between the second intersection point and the second line segment is the same as the positional relationship between the first intersection point and the first line segment, which will not be repeated here.
[0050] S203: Matching the multiple first intersection points and the multiple second intersection points to obtain multiple matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point.
[0051] In this embodiment, multiple matching point pairs can be obtained by performing point feature matching on the first image and the second image. Optionally, the feature point of the real scene represented by the first intersection point in the matching point pair and the feature point of the real scene represented by the second intersection point in the matching point pair are the same feature point. Exemplarily, the first intersection point can be the intersection point of the street lamp A and the road edge L in the first image, and the second intersection point can be the intersection point of the street lamp A and the road edge L in the second image.
[0052] S204: Determine a plurality of target line segments that match the first image and the second image based on the plurality of matching point pairs.
[0053] In this embodiment, for each matching point pair, at least one first line segment associated with the first intersection point and at least one second line segment associated with the second intersection point in the matching point pair are determined, and two line segments whose line segment attributes satisfy similarity conditions in at least one first line segment and at least one second line segment are determined to be matching target line segments in the first image and the second image, and finally multiple target line segments are obtained.
[0054] Optionally, line segment attributes may include: the angle between the line segment and a reference line in the image, or a feature vector for characterizing the geometry and / or color information of the line segment extracted using a deep learning model, or line segment attributes determined using an existing method for describing / characterizing the line segment. Accordingly, line segment attributes satisfying similarity conditions include: the line segment slopes being the same / similar (i.e., the angle between the line segment and the reference line in the image being the same / similar), or the feature vectors characterizing the geometry and / or color information of the line segment being the same / similar, or the line segment attributes determined using an existing method for describing / characterizing the line segment being the same / similar.
[0055] Exemplarily, the plurality of matching point pairs include: a first intersection point A and a second intersection point A', and a first intersection point B and a second intersection point B'. Among them, at least one first line segment associated with the first intersection point A includes a first line segment A and a first line segment B; at least one first line segment associated with the first intersection point B includes a first line segment C and a first line segment D. Among them, at least one second line segment associated with the second intersection point A' includes a second line segment A' and a second line segment B'; at least one first line segment associated with the second intersection point B' includes a second line segment C' and a second line segment D'.
[0056] In this exemplary embodiment, multiple target line segments that match can be determined by judging whether the first line segment A, B, C, D and the second line segment A', B', C', D' meet the line segment attribute similarity condition. Take judging whether the first line segment A and the second line segment A' are the matching target line segments as an example: if the first line segment A and the second line segment A' meet the line segment attribute similarity condition, then the first line segment A and the second line segment A' are determined to be the matching target line segments; if the first line segment A and the second line segment A' do not meet the line segment attribute similarity condition, then the first line segment A and the second line segment A' are determined not to be the matching target line segments. Exemplarily, it is judged that among the first line segments A, B, C, D and the second line segments A', B', C', D', there are 3 target line segments that meet the line segment attribute similarity condition, namely: the first line segment A and the second line segment A', the first line segment C and the second line segment C', the first line segment D and the second line segment D' (the first line segment B and the second line segment B' do not meet the line segment attribute similarity condition).
[0057] In this embodiment, firstly, line features are extracted from the first image and the second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments; then, a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments are determined; finally, the plurality of first intersection points and the plurality of second intersection points are matched to obtain a plurality of matching point pairs, and based on the plurality of matching point pairs, a plurality of matching target line segments in the first image and the second image are determined. Since the line features in the first image and the second image are extracted, the line segments in the images are extracted, and then the intersection points corresponding to the line segments are determined, the line segments are matched through the correlation relationship between points and lines, and the stability and accuracy of the point feature matching are relatively high, thus improving the accuracy and robustness of the image processing method for matching line segments.
[0058] refer to Figure 3 , Figure 3 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2 Based on the embodiment shown, step S201 is further refined. In this embodiment, LSD (Line Segment Detector) can be used to extract line features from the first image and the second image to be matched, respectively, to obtain multiple first line segments and multiple second line segments. Specifically, the following steps may be included:
[0059] S301. Perform Gaussian downsampling processing on a first image to obtain a plurality of first pixels, and perform Gaussian downsampling processing on a second image to obtain a plurality of second pixels.
[0060] In this embodiment, performing Gaussian downsampling processing on the first image can reduce or even eliminate the jagged effect appearing in the image. The degree of Gaussian downsampling processing can be represented by the scaling factor. The scaling factor is a value between 0 and 1. The smaller the scaling factor, the more obvious the effect of reducing the jagged effect by Gaussian downsampling, but the image clarity will decrease due to the reduction in the number of collected pixels. Exemplarily, the scaling factor is 0.8. At this time, the first image is scaled to 80% of the original in both the x and y directions, that is, the total number of pixels is scaled to 64% of the original.
[0061] S302: Determine gradients of a plurality of first pixels and gradients of a plurality of second pixels.
[0062] In this embodiment, the gradient of a pixel is the degree of change in brightness of the pixel, and the gradient of the pixel can be determined by the grayscale value of the pixel. Exemplarily, the gradient of each first pixel can be determined by the grayscale value of each first pixel and the grayscale values of the four first pixels below and to the right of the pixel. The specific implementation method is the prior art and will not be described in detail. Similarly, the gradient of each second pixel can also be determined by the grayscale value of each second pixel and the grayscale values of the four second pixels below and to the right of the pixel.
[0063] S303 . Perform line feature extraction on the first image based on the gradients of the plurality of first pixels to obtain a plurality of first line segments, and perform line feature extraction on the second image based on the gradients of the plurality of second pixels to obtain a plurality of second line segments.
[0064] In this embodiment, line feature extraction can be achieved by performing region growing through seed pixels. Accordingly, this step is: based on the gradients of multiple first pixels, determine the gradient direction of each first pixel, select a first seed pixel from the multiple first pixels, perform region growing on the first seed pixel based on the gradient direction of each first pixel, obtain a first line support region, and determine a first line segment based on the first line support region; based on the gradients of multiple second pixels, determine the gradient direction of each second pixel, select a second seed pixel from the multiple second pixels, perform region growing on the second seed pixel based on the gradient direction of each second pixel, obtain a second line support region, and determine a second line segment based on the second line support region.
[0065] It should be noted that, based on the gradient direction of each first seed pixel, the first pixel with a small gradient direction change and adjacent to the first seed pixel can be used as a connected domain for region growth, and the obtained line support area is a strip rectangle, that is, the first line segment. In some embodiments, the first seed pixel can be randomly selected, and the number of first seed pixels is a preset number. In other embodiments, the first seed pixels can also be selected in order from small to large according to the grayscale value of the first pixel. For example, the grayscale value of the first pixel is divided into 0-255 categories from small to large, and one or more first pixels are selected as the first seed pixel in each category. Similarly, based on the gradient direction of each second seed pixel, the second pixel with a small gradient direction change and adjacent to the second seed pixel can be used as a connected domain for region growth, and the obtained line support area is a strip rectangle, that is, the second line segment.
[0066] In this embodiment, LSD is used to extract line features from the first image and the second image to be matched respectively, and this technology can weaken or even eliminate the jagged effect in the image, and can reduce the dependence on the gradient and reduce the influence of noise. In this way, line feature extraction for noisy images is more robust.
[0067] refer to Figure 4 , Figure 4 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 3 In this embodiment Figure 2 Based on the embodiment shown, step S203 is further refined. In this embodiment, ASL (Arterial Spin Labeling) can be used to match multiple first intersection points and multiple second intersection points to obtain multiple matching point pairs. Figure 4 As shown, the following steps may be specifically included:
[0068] S401, extracting first eigenvectors of a plurality of first intersections and second eigenvectors of a plurality of second intersections by using arterial spin labeling (ASL).
[0069] In this embodiment, the first feature vector may include the first feature information corresponding to the first intersection. The first feature information is used to represent any attribute information of the first intersection. Optionally, the first feature information includes one or more of position information, domain information, and color information. The dimension of the first feature vector is related to the amount of attribute information represented by the first feature information. Exemplarily, the first feature information includes position information, and the first feature vector of the first intersection A is (x1, y1); wherein x1 represents the abscissa of the first intersection A, and y1 represents the ordinate of the first intersection A. In this case, the first feature vector is a two-dimensional vector. Exemplarily, the first feature information includes position information and color information, and the first feature vector of the first intersection A is (x1, y1, c1), wherein x1 represents the abscissa of the first intersection A, y1 represents the ordinate of the first intersection A, and c1 represents the grayscale of the first intersection A. In this case, the first feature vector is a three-dimensional vector.
[0070] Similarly, the second feature vector may include the second feature information corresponding to the second intersection. The second feature information is used to represent any attribute information of the second intersection. Optionally, the second feature information includes one or more of position information, domain information, and color information. The dimension of the second feature vector is related to the amount of attribute information represented by the second feature information. Exemplarily, the second feature information includes position information, and the second feature vector of the second intersection A' is (x2, y2); wherein x2 represents the abscissa of the second intersection A', and y2 represents the ordinate of the second intersection A'. In this case, the second feature vector is a two-dimensional vector. Exemplarily, the second feature information includes position information and color information, and the second feature vector of the second intersection A' is (x2, y2, c2), wherein x2 represents the abscissa of the second intersection A', y2 represents the ordinate of the second intersection A', and c2 represents the grayscale of the second intersection A'. In this case, the second feature vector is a three-dimensional vector.
[0071] In a possible implementation, deep learning can be performed on the collected ASL data to train an intersection extraction model, and the first feature vectors of multiple first intersections and the second feature vectors of multiple second intersections can be extracted through the model. Exemplarily, the intersection extraction model can be a deep convolutional neural network model.
[0072] S402: Determine a plurality of matching point pairs based on the distance between the first eigenvector and the second eigenvector.
[0073] In this embodiment, the first feature vector includes first feature information corresponding to multiple first intersections, and the second feature vector includes second feature information corresponding to multiple second intersections; accordingly, this step is: for each first intersection, based on the first feature information of the first intersection and the second feature information corresponding to the multiple second intersections, determine the distance between the first intersection and the multiple second intersections; select the second intersection that reaches the preset distance threshold and has the smallest distance from the multiple second intersections, and combine it with the first intersection into a matching point pair to obtain multiple matching point pairs. In the disclosed embodiment, the value of the preset distance threshold is not specifically limited and can be set and modified as needed. Exemplarily, the preset distance threshold can be set to 0.5, 1, or 2.
[0074] Optionally, the distance between the first intersection point and the second intersection point is also the distance between the first eigenvector and the second eigenvector.
[0075] Exemplarily, the first feature information includes position information, and the first feature vector of the first intersection A is (x1, y1), where x1 represents the abscissa of the first intersection A, and y1 represents the ordinate of the first intersection A. The second feature information includes position information. The first feature vector of the second intersection A' is (x2, y2); where x2 represents the abscissa of the second intersection A', and y2 represents the ordinate of the second intersection A'. The distance D between the first intersection A and the second intersection A' is:
[0076] Exemplarily, the first feature information includes position information and color information, and the first feature vector of the first intersection A is (x1, y1, c1); wherein x1 represents the abscissa of the first intersection A, y1 represents the ordinate of the first intersection A, and c1 represents the grayscale of the first intersection A. The second feature information includes position information and color information, and the second feature vector of the second intersection A' is (x2, y2, c2); wherein x2 represents the abscissa of the second intersection A', y2 represents the ordinate of the second intersection A', and c2 represents the grayscale of the second intersection A'. The distance D between the first intersection A and the second intersection A' is:
[0077] In this embodiment, the ASL technology is used to match the multiple first intersection points and the multiple second intersection points, and the ASL technology can improve the efficiency of intersection extraction; and the ASL technology can reduce image noise, further improving the accuracy of intersection extraction.
[0078] refer to Figure 5 , Figure 5 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 4 In this embodiment Figure 2 Based on the embodiment shown, the specific implementation method of determining multiple matching target line segments in the first image and the second image based on multiple matching point pairs in step S104 is described in detail. Figure 5 As shown, the method includes:
[0079] S501. For each matching point pair, determine a first target line segment associated with a first intersection point and a second target line segment associated with a second intersection point.
[0080] In this embodiment, the electronic device stores the association relationship between the first intersection and the first line segment, and the association relationship between the second intersection and the second line segment. Accordingly, this step is: for each matching point pair, determine the first target line segment associated with the first intersection according to the association relationship between the first intersection and the first line segment, and determine the second target line segment associated with the second intersection according to the association relationship between the second intersection and the second line segment.
[0081] Optionally, the association relationship between the first intersection point and the first line segment is the positional relationship between the first intersection point and the first line segment. Exemplarily, the step of determining the first target line segment through the association relationship includes: for the first intersection point A in the matching point pair, if the stored association relationship is: the first intersection point A is located inside the first line segment A and is located at the endpoint of the first line segment B; then the first target line segment associated with the first intersection point A is determined to be: the first line segment A and the first line segment B. Optionally, the association relationship between the second intersection point and the second line segment is the positional relationship between the second intersection point and the second line segment, wherein the specific steps of determining the second target line segment are the same as the specific steps of determining the first target line segment, and will not be repeated here.
[0082] S502: Determine a first slope of a straight line where a first target line segment is located and a second slope of a straight line where a second target line segment is located.
[0083] In this embodiment, the first slope of the straight line where the first target line segment is located is the tangent value of the angle between the first target line segment and the positive semi-axis of the x-axis of the first image. The second slope of the straight line where the second target line segment is located is the tangent value of the angle between the second target line segment and the positive semi-axis of the x-axis of the second image.
[0084] S503: Determine a plurality of target line segments that match the first image and the second image based on the first slope and the second slope.
[0085] In this embodiment, the target line segments with too large a slope difference in the matching point can be screened out by the slope difference between the first target line segment and the second target line segment. Accordingly, the specific steps are: determine the slope difference between the first slope and the second slope; if the slope difference is less than a preset threshold, determine the first target line segment and the second target line segment as matching target line segments, and obtain a plurality of matching target line segments in the first image and the second image; if the slope difference is greater than or equal to the preset threshold, determine the first target line segment and the second target line segment as mismatching. In the disclosed embodiment, the value of the preset threshold is not specifically limited and can be set and modified as needed. Exemplarily, the preset threshold can be set to 0.1, 0.2, or 0.3.
[0086] In this embodiment, multiple matching target line segments are preliminarily determined by matching point pairs, and then matching point pairs with too large a slope difference are screened out by using the slope difference. The target line segments with too large a slope difference are most likely not the line segments at the same position in the first image and the second image (matching target line segments), thereby improving the accuracy of determining multiple matching target line segments.
[0087] Corresponding to the image processing method of the above embodiment, Figure 6 This is a structural block diagram of an image processing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 6The image processing device includes: an extraction module 601, a first determination module 602, a matching module 603 and a second determination module 604.
[0088] The extraction module 601 is used to extract line features from the first image and the second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments;
[0089] A first determining module 602, configured to determine a plurality of first intersection points corresponding to a plurality of first line segments and a plurality of second intersection points corresponding to a plurality of second line segments;
[0090] A matching module 603 is used to match the multiple first intersection points and the multiple second intersection points to obtain multiple matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point;
[0091] The second determination module 604 is configured to determine a plurality of target line segments that match the first image and the second image based on a plurality of matching point pairs.
[0092] In one embodiment of the present disclosure, the extraction module 601 is specifically used to:
[0093] Perform Gaussian downsampling processing on the first image to obtain multiple first pixels, and perform Gaussian downsampling processing on the second image to obtain multiple second pixels; determine the gradients of the multiple first pixels and the gradients of the multiple second pixels; perform line feature extraction on the first image based on the gradients of the multiple first pixels to obtain multiple first line segments, and perform line feature extraction on the second image based on the gradients of the multiple second pixels to obtain multiple second line segments.
[0094] In one embodiment of the present disclosure, the extraction module 601 performs line feature extraction on the first image based on the gradients of multiple first pixels to obtain multiple first line segments, and performs line feature extraction on the second image based on the gradients of multiple second pixels to obtain multiple second line segments. The extraction module 601 is specifically used to: determine the gradient direction of each first pixel based on the gradients of multiple first pixels, select a first seed pixel from the multiple first pixels, perform region growth on the first seed pixel based on the gradient direction of each first pixel to obtain a first line support region, and determine the first line segment based on the first line support region; determine the gradient direction of each second pixel based on the gradients of multiple second pixels, select a second seed pixel from the multiple second pixels, perform region growth on the second seed pixel based on the gradient direction of each second pixel to obtain a second line support region, and determine the second line segment based on the second line support region.
[0095] In one embodiment of the present disclosure, the first determination module 602 is specifically used to: determine multiple first straight lines where multiple first line segments are located, determine the intersection points of any two of the multiple first straight lines, and obtain multiple first intersection points; determine multiple second straight lines where multiple second line segments are located, determine the intersection points of any two of the multiple second straight lines, and obtain multiple second intersection points.
[0096] In one embodiment of the present disclosure, the matching module 603 is specifically used to: extract first feature vectors of multiple first intersection points and second feature vectors of multiple second intersection points using arterial spin labeling (ASL); and determine multiple matching point pairs based on the distance between the first feature vector and the second feature vector.
[0097] In one embodiment of the present disclosure, the first feature vector includes first feature information corresponding to multiple first intersections, and the second feature vector includes second feature information corresponding to multiple second intersections; the matching module 603, when determining multiple matching point pairs based on the distance between the first feature vector and the second feature vector, is specifically used to: for each first intersection, determine the distance between the first intersection and the multiple second intersections based on the first feature information of the first intersection and the second feature information corresponding to the multiple second intersections; select a second intersection that reaches a preset distance threshold and has the smallest distance from the multiple second intersections, and combine it with the first intersection as a matching point pair to obtain multiple matching point pairs.
[0098] In one embodiment of the present disclosure, the second determination module 604 is specifically used to: for each matching point pair, determine the first target line segment associated with the first intersection point and the second target line segment associated with the second intersection point; determine the first slope of the straight line where the first target line segment is located and the second slope of the straight line where the second target line segment is located; based on the first slope and the second slope, determine multiple target line segments that match in the first image and the second image.
[0099] In one embodiment of the present disclosure, when the second determination module 604 determines multiple matching target line segments in the first image and the second image based on the first slope and the second slope, it is specifically used to: determine the slope difference between the first slope and the second slope; if the slope difference is less than a preset threshold, determine the first target line segment and the second target line segment as matching target line segments, and obtain multiple matching target line segments in the first image and the second image.
[0100] The extraction module 601, the first determination module 602, the matching module 603 and the second determination module 604 are connected in sequence. The image processing device provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.
[0101] Figure 7A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, the electronic device 7 includes
[0102] A processor 701, and a memory 702 communicatively connected to the processor 701;
[0103] The memory 702 stores computer-executable instructions;
[0104] The processor 701 executes the computer-executable instructions stored in the memory 702 to implement the following Figure 2-Figure 5 The image processing method in the illustrated embodiment.
[0105] Optionally, the processor 701 and the memory 702 are connected via a bus 703 .
[0106] For related instructions, please refer to Figure 2-Figure 5 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further elaboration is made here.
[0107] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the present invention when executed by a processor. Figure 2-Figure 5 The image processing provided by any one of the corresponding embodiments.
[0108] refer to Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure, and the electronic device 700 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0109] like Figure 7As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0110] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0111] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0112] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0113] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0114] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.
[0115] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0117] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, the first acquisition module may also be described as a "module for acquiring at least two Internet Protocol addresses".
[0118] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0119] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0120] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an image processing method, comprising: extracting line features from a first image and a second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments; determining a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments; matching the plurality of first intersection points and the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point; and determining a plurality of matching target line segments in the first image and the second image based on the plurality of matching point pairs.
[0121] According to one or more embodiments of the present disclosure, the method of performing line feature extraction on the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments includes: performing Gaussian downsampling processing on the first image to obtain a plurality of first pixels, and performing Gaussian downsampling processing on the second image to obtain a plurality of second pixels; determining the gradients of the plurality of first pixels and the gradients of the plurality of second pixels; performing line feature extraction on the first image based on the gradients of the plurality of first pixels to obtain a plurality of first line segments, and performing line feature extraction on the second image based on the gradients of the plurality of second pixels to obtain a plurality of second line segments.
[0122] According to one or more embodiments of the present disclosure, the line feature extraction of the first image based on the gradients of the multiple first pixels to obtain multiple first line segments, and the line feature extraction of the second image based on the gradients of the multiple second pixels to obtain multiple second line segments include: determining the gradient direction of each first pixel based on the gradients of the multiple first pixels, selecting a first seed pixel from the multiple first pixels, performing region growing on the first seed pixel based on the gradient direction of each first pixel to obtain a first line support region, and determining the first line segment based on the first line support region; determining the gradient direction of each second pixel based on the gradients of the multiple second pixels, selecting a second seed pixel from the multiple second pixels, performing region growing on the second seed pixel based on the gradient direction of each second pixel to obtain a second line support region, and determining the second line segment based on the second line support region.
[0123] According to one or more embodiments of the present disclosure, determining a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments includes: determining a plurality of first straight lines where the plurality of first line segments are located, determining the intersection points of any two of the plurality of first straight lines, and obtaining a plurality of first intersection points; determining a plurality of second straight lines where the plurality of second line segments are located, determining the intersection points of any two of the plurality of second straight lines, and obtaining a plurality of second intersection points.
[0124] According to one or more embodiments of the present disclosure, the matching of the multiple first intersection points and the multiple second intersection points to obtain multiple matching point pairs includes: using arterial spin labeling (ASL) to extract first eigenvectors of the multiple first intersection points and second eigenvectors of the multiple second intersection points; and determining the multiple matching point pairs based on the distance between the first eigenvector and the second eigenvector.
[0125] According to one or more embodiments of the present disclosure, the first feature vector includes first feature information corresponding to multiple first intersections, and the second feature vector includes second feature information corresponding to multiple second intersections; determining the multiple matching point pairs based on the distance between the first feature vector and the second feature vector includes: for each first intersection, determining the distance between the first intersection and the multiple second intersections based on the first feature information of the first intersection and the second feature information corresponding to the multiple second intersections; selecting a second intersection that reaches a preset distance threshold and has the smallest distance from the multiple second intersections, and combining it with the first intersection as a matching point pair to obtain multiple matching point pairs.
[0126] According to one or more embodiments of the present disclosure, determining multiple target line segments that match in the first image and the second image based on the multiple matching point pairs includes: for each matching point pair, determining a first target line segment associated with a first intersection point and a second target line segment associated with a second intersection point; determining a first slope of a straight line where the first target line segment is located and a second slope of a straight line where the second target line segment is located; and determining multiple target line segments that match in the first image and the second image based on the first slope and the second slope.
[0127] According to one or more embodiments of the present disclosure, determining multiple matching target line segments in the first image and the second image based on the first slope and the second slope includes: determining the slope difference between the first slope and the second slope; if the slope difference is less than a preset threshold, determining that the first target line segment and the second target line segment are matching target line segments, thereby obtaining multiple matching target line segments in the first image and the second image.
[0128] According to one or more embodiments of the present disclosure, the first image is image data collected in real time by a camera, and the second image is image data corresponding to the first image confirmed by a map based on visual data.
[0129] In a second aspect, according to one or more embodiments of the present disclosure, there is provided an image processing device, comprising: an extraction module, used to extract line features of a first image and a second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments; a first determination module, used to determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments; a matching module, used to match the plurality of first intersection points and the plurality of second intersection points, to obtain a plurality of matching point pairs, wherein each matching point pair comprises a first intersection point and a corresponding second intersection point; and a second determination module, used to determine a plurality of matching target line segments in the first image and the second image based on the plurality of matching point pairs.
[0130] In one embodiment of the present disclosure, the extraction module is specifically used to: perform Gaussian downsampling processing on the first image to obtain multiple first pixels, and perform Gaussian downsampling processing on the second image to obtain multiple second pixels; determine the gradients of the multiple first pixels and the gradients of the multiple second pixels; perform line feature extraction on the first image based on the gradients of the multiple first pixels to obtain multiple first line segments, and perform line feature extraction on the second image based on the gradients of the multiple second pixels to obtain multiple second line segments.
[0131] In one embodiment of the present disclosure, the extraction module performs line feature extraction on the first image based on the gradients of multiple first pixels to obtain multiple first line segments, and performs line feature extraction on the second image based on the gradients of multiple second pixels to obtain multiple second line segments. The extraction module is specifically used to: determine the gradient direction of each first pixel based on the gradients of multiple first pixels, select a first seed pixel from the multiple first pixels, perform region growth on the first seed pixel based on the gradient direction of each first pixel to obtain a first line support area, and determine the first line segment based on the first line support area; determine the gradient direction of each second pixel based on the gradients of multiple second pixels, select a second seed pixel from the multiple second pixels, perform region growth on the second seed pixel based on the gradient direction of each second pixel to obtain a second line support area, and determine the second line segment based on the second line support area.
[0132] In one embodiment of the present disclosure, the first determination module is specifically used to: determine multiple first straight lines where multiple first line segments are located, determine the intersection points of any two of the multiple first straight lines, and obtain multiple first intersection points; determine multiple second straight lines where multiple second line segments are located, determine the intersection points of any two of the multiple second straight lines, and obtain multiple second intersection points.
[0133] In one embodiment of the present disclosure, the matching module is specifically used to: extract first feature vectors of multiple first intersection points and second feature vectors of multiple second intersection points using arterial spin labeling (ASL); and determine multiple matching point pairs based on the distance between the first feature vector and the second feature vector.
[0134] In one embodiment of the present disclosure, the first feature vector includes first feature information corresponding to multiple first intersections, and the second feature vector includes second feature information corresponding to multiple second intersections; the matching module, based on the distance between the first feature vector and the second feature vector, determines multiple matching point pairs, specifically for: for each first intersection, based on the first feature information of the first intersection and the second feature information corresponding to the multiple second intersections, determining the distance between the first intersection and the multiple second intersections; selecting from the multiple second intersections a second intersection that reaches a preset distance threshold and has the smallest distance, and combining it with the first intersection as a matching point pair to obtain multiple matching point pairs.
[0135] In one embodiment of the present disclosure, the second determination module is specifically used to: for each matching point pair, determine the first target line segment associated with the first intersection point and the second target line segment associated with the second intersection point; determine the first slope of the straight line where the first target line segment is located and the second slope of the straight line where the second target line segment is located; based on the first slope and the second slope, determine multiple target line segments that match in the first image and the second image.
[0136] In one embodiment of the present disclosure, when the second determination module determines multiple matching target line segments in the first image and the second image based on the first slope and the second slope, it is specifically used to: determine the slope difference between the first slope and the second slope; if the slope difference is less than a preset threshold, determine the first target line segment and the second target line segment as matching target line segments, and obtain multiple matching target line segments in the first image and the second image.
[0137] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: a processor, and a memory communicatively connected to the processor;
[0138] The memory stores computer-executable instructions;
[0139] The processor executes the computer-executable instructions stored in the memory to implement the image processing method as described in the first aspect and various possible designs of the first aspect.
[0140] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the image processing method described in the first aspect and various possible designs of the first aspect is implemented.
[0141] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method as described in the first aspect and various possible designs of the first aspect.
[0142] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0143] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0144] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.
Claims
1. An image processing method, It is characterized in that include: Performing line feature extraction on the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments; Determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments; Matching the plurality of first intersection points with the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point; Based on the multiple matching point pairs, multiple matching target line segments in the first image and the second image are determined.
2. The method according to claim 1, It is characterized in that The method extracts line features from the first image and the second image to be matched respectively to obtain a plurality of first line segments and a plurality of second line segments, including: Performing Gaussian downsampling processing on the first image to obtain a plurality of first pixels, and performing Gaussian downsampling processing on the second image to obtain a plurality of second pixels; determining a gradient of the plurality of first pixels and a gradient of the plurality of second pixels; Line features are extracted from the first image based on the gradients of the multiple first pixels to obtain multiple first line segments, and line features are extracted from the second image based on the gradients of the multiple second pixels to obtain multiple second line segments.
3. The method according to claim 2, It is characterized in that The extracting line features of the first image based on the gradients of the plurality of first pixels to obtain a plurality of first line segments, and extracting line features of the second image based on the gradients of the plurality of second pixels to obtain a plurality of second line segments, comprises: Based on the gradients of the plurality of first pixels, determining the gradient direction of each first pixel, selecting a first seed pixel from the plurality of first pixels, performing region growing on the first seed pixel based on the gradient direction of each first pixel to obtain a first line support region, and determining a first line segment according to the first line support region; Based on the gradients of the multiple second pixels, the gradient direction of each second pixel is determined, a second seed pixel is selected from the multiple second pixels, and based on the gradient direction of each second pixel, the second seed pixel is subjected to region growth to obtain a second line support area, and a second line segment is determined according to the second line support area.
4. The method according to claim 1, It is characterized in that The determining of a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments comprises: Determine a plurality of first straight lines where the plurality of first line segments are located, determine the intersection point of any two straight lines among the plurality of first straight lines, and obtain a plurality of first intersection points; A plurality of second straight lines on which the plurality of second line segments are located are determined, and an intersection point of any two straight lines among the plurality of second straight lines is determined to obtain a plurality of second intersection points.
5. The method according to claim 1, It is characterized in that The matching of the plurality of first intersection points and the plurality of second intersection points to obtain a plurality of matching point pairs includes: Extracting first feature vectors of the plurality of first intersection points and second feature vectors of the plurality of second intersection points by using arterial spin labeling (ASL); The plurality of matching point pairs are determined based on the distance between the first feature vector and the second feature vector.
6. The method according to claim 5, It is characterized in that The first feature vector includes first feature information corresponding to a plurality of first intersection points, and the second feature vector includes second feature information corresponding to a plurality of second intersection points; The determining the plurality of matching point pairs based on the distance between the first feature vector and the second feature vector comprises: For each first intersection, determining a distance between the first intersection and the plurality of second intersections based on the first feature information of the first intersection and the second feature information corresponding to the plurality of second intersections; A second intersection point that reaches a preset distance threshold and has the smallest distance is selected from the plurality of second intersection points, and is combined with the first intersection point into a matching point pair to obtain a plurality of matching point pairs.
7. The method according to claim 1, It is characterized in that The determining, based on the multiple matching point pairs, multiple target line segments that match in the first image and the second image comprises: For each matching point pair, determining a first target line segment associated with the first intersection point and a second target line segment associated with the second intersection point; Determine a first slope of a straight line where the first target line segment is located and a second slope of a straight line where the second target line segment is located; Based on the first slope and the second slope, a plurality of target line segments matching in the first image and the second image are determined.
8. The method according to claim 7, It is characterized in that The determining, based on the first slope and the second slope, a plurality of target line segments that match in the first image and the second image comprises: determining a slope difference between the first slope and the second slope; If the slope difference is less than a preset threshold, the first target line segment and the second target line segment are determined to be matching target line segments, and a plurality of matching target line segments in the first image and the second image are obtained.
9. The method according to any one of claims 1 to 8, It is characterized in that The first image is image data collected in real time by a camera, and the second image is image data corresponding to the first image confirmed by a map based on visual data.
10. An image processing device, It is characterized in that include: An extraction module is used to extract line features from the first image and the second image to be matched, respectively, to obtain a plurality of first line segments and a plurality of second line segments; A first determining module, configured to determine a plurality of first intersection points corresponding to the plurality of first line segments and a plurality of second intersection points corresponding to the plurality of second line segments; A matching module, used for matching the plurality of first intersection points with the plurality of second intersection points to obtain a plurality of matching point pairs, wherein each matching point pair includes a first intersection point and a corresponding second intersection point; The second determination module is used to determine a plurality of target line segments that match the first image and the second image based on the plurality of matching point pairs.
11. An electronic device, It is characterized in that include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 9 is implemented.
13. A computer program product, It is characterized in that The invention comprises a computer program, which implements the image processing method according to any one of claims 1 to 9 when being executed by a processor.
Citation Information
Patent Citations
Identity recognition device and method thereof
CN113722692A
