Image processing methods, devices, and vehicles

Abstract

This invention discloses an image processing method, apparatus, and vehicle. The method includes: acquiring a grayscale image of the scene in which the vehicle is located; acquiring the pixel mean of multiple pixels in the grayscale image; using the pixel mean to filter out first feature points from the grayscale image; and determining a target image in the grayscale image based on the first feature points. This invention solves the technical problem of low accuracy in image recognition.

Description

Technical Field

[0001] This invention relates to the field of vehicles, and more specifically, to an image processing method, apparatus, and vehicle. Background Technology

[0002] Currently, corner detection algorithms are crucial for the realization of high-precision vehicle positioning technology. In related technologies, the image feature extraction and feature matching methods for fast feature detection and corner detection are usually manually set and screened. Although the extraction speed is fast, it may extract edge points or local non-maximum points for different images. Feature points tend to cluster and are not sensitive to different gray-dark scenes. This results in poor stability of corner recognition by image feature algorithms under different lighting conditions, thus leading to the technical problem of low accuracy in image recognition.

[0003] There is currently no effective solution to the problem of low accuracy in existing image recognition methods. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, and vehicle to at least address the technical problem of low accuracy in image recognition.

[0005] According to one aspect of the present invention, an image processing method is provided. The method may include: acquiring a grayscale image of a scene in which a vehicle is located; acquiring the pixel mean of multiple pixels in the grayscale image; using the pixel mean to filter out first feature points from the grayscale image; and determining a target image in the grayscale image based on the first feature points.

[0006] Optionally, obtaining the pixel mean of multiple pixels in a grayscale image includes: determining the pixel values ​​of multiple pixels in the grayscale image; and determining the pixel mean of the grayscale image as the ratio between the sum of the pixel values ​​of the multiple pixels and the number of pixels in the grayscale image.

[0007] Optionally, the first feature point is selected from the grayscale image using the pixel mean, including: identifying pixels whose pixel values ​​are within the range of the pixel mean as candidate points; determining a first difference between the pixel values ​​of the candidate points and the pixel values ​​of other candidate points; determining a target threshold based on the first difference and the pixel mean; identifying candidate points corresponding to the first difference where the number of targets is greater than the target threshold as second feature points; selecting the second feature points to obtain a third feature point; and selecting the third feature point to obtain the first feature point.

[0008] Optionally, the second feature point is filtered to obtain the third feature point, including: determining a second difference between the pixel value of the second feature point and the pixel values ​​of other second feature points in the first region, wherein the first region is used to represent the geometry centered on the second feature point; determining the difference set of the second feature points based on the second difference; determining the sum of the difference sets as the response value of the second feature point; and determining the second feature point corresponding to the maximum response value in the first region as the third feature point.

[0009] Optionally, the third feature point is filtered to obtain the first feature point, including: determining the Hamming distance of the third feature point in the second region, wherein the second region is used to represent the geometry centered on the third feature point; determining the third feature point whose Hamming distance in the second region is greater than a distance threshold as the fourth feature point; determining the Hamming distance of the fourth feature point in the third region, wherein the third region is used to represent the geometry centered on the fourth feature point, and the third region has a different size than the second region; and determining the fourth feature point whose Hamming distance in the third region is greater than a distance threshold as the first feature point.

[0010] Optionally, the grayscale image can be divided into regions of equal area to obtain multiple sub-images; the angle and position coordinates of feature points in the corresponding sub-images can be determined using a scale pyramid.

[0011] Optionally, determining the target image in the grayscale image based on the first feature point includes: restoring the first feature point based on its angle and position coordinates to obtain the target image.

[0012] Optionally, multiple original images of the scene where the vehicle is located are acquired; the multiple original images are corrected to obtain multiple normal images; the multiple normal images are stitched together to obtain a stitched image; and the stitched image is processed to obtain a grayscale image.

[0013] According to another aspect of the present invention, an image processing apparatus is also provided. The apparatus includes: a first acquisition unit for acquiring a grayscale image of a scene in which a vehicle is located; a second acquisition unit for acquiring the pixel mean of a plurality of pixels in the grayscale image; a filtering unit for filtering out first feature points from the original image using the pixel mean; and a determination unit for determining a target image in the original image based on the first feature points.

[0014] According to another aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the image processing method of the present invention.

[0015] According to another aspect of the present invention, a processor is also provided. The processor is used to run a program, wherein the program executes the image processing method of the present invention during runtime.

[0016] According to another aspect of the present invention, a vehicle is also provided for performing the image processing method of the present invention.

[0017] In this embodiment of the invention, a grayscale image of the scene where the vehicle is located is acquired; the pixel mean of multiple pixels in the grayscale image is acquired; a first feature point is selected from the grayscale image using the pixel mean; and a target image in the grayscale image is determined based on the first feature point. In other words, this embodiment of the invention improves the accuracy of image recognition by using the pixel mean of multiple pixels in the grayscale image of the scene where the vehicle is located to select a first feature point, and then determines the target image in the grayscale image based on the first feature point. This solves the technical problem of low accuracy in image recognition. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of another image processing method according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] According to an embodiment of the present invention, an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps:

[0027] Step S102: Obtain a grayscale image of the scene where the vehicle is located.

[0028] In the technical solution provided in step S102 of the present invention, an image acquisition device can acquire images of the scene and convert the acquired images into grayscale images. The image acquisition device can be a device applied to a vehicle and can be used to acquire images of the scene around the vehicle. For example, it can be a vehicle-mounted camera, a four-way surround-view vehicle-mounted camera, etc.

[0029] For example, an in-vehicle camera can capture images of the scene in which the vehicle is located. An image feature extraction algorithm with fast feature detection (Features from Accelerated Segment Test, or FAST) and corner detection (Oriented Fast and Rotated Brief, or ORB) can be used to convert the captured raw image into a grayscale image, thereby achieving the purpose of obtaining a grayscale image of the scene in which the vehicle is located.

[0030] Step S104: Obtain the pixel mean of multiple pixels in the grayscale image.

[0031] In the technical solution provided in step S104 of the present invention, the acquired grayscale image is processed to obtain the pixel value of each pixel in the grayscale image. Based on the pixel value of each pixel in the grayscale image, the pixel mean of the grayscale image is determined. The pixel value can be determined by X. (i) The pixel mean can be used to characterize the average pixel value of a grayscale image, and the grayscale mean can be represented by μ.

[0032] Optionally, a grayscale image of the scene where the vehicle is located is obtained, the grayscale image is downsampled, and the pixel mean of multiple pixels in the downsampled grayscale image is obtained. Downsampling is a multi-rate digital signal processing technique in the field of digital signal processing. It can be a process of reducing the signal sampling rate. For example, it can be a process of scaling the grayscale image by 1 / 1.2 to obtain a total of 8 images.

[0033] Step S106: Use the pixel mean to select the first feature point from the grayscale image.

[0034] In the technical solution of step S106 of the present invention, the pixels in the grayscale image are filtered based on the obtained pixel mean to obtain the first feature point in the grayscale image, wherein the first feature point is used to determine the target image in the grayscale image.

[0035] Step S108: Determine the target image in the grayscale image based on the first feature point.

[0036] In the above-mentioned step S108 technical solution of the present invention, the first feature points obtained by screening are processed to obtain the target image in the grayscale image.

[0037] Alternatively, the target image in the grayscale image can be determined by restoring the first feature points.

[0038] In the above steps S102 to S108 of this application, the average pixel value of multiple pixels in a grayscale image is obtained; a first feature point is selected from the grayscale image using the average pixel value; and a target image in the grayscale image is determined based on the first feature point. In other words, this embodiment of the invention improves the accuracy of image recognition by using the average pixel value of multiple pixels in a grayscale image of the scene where the vehicle is located to select a first feature point from the grayscale image, and then determines the target image in the grayscale image based on the first feature point. This solves the technical problem of low accuracy in image recognition.

[0039] The method described in this embodiment will be further described below.

[0040] As an optional embodiment, step S104, obtaining the pixel mean of multiple pixels in a grayscale image, includes: determining the pixel values ​​of multiple pixels in the grayscale image; and determining the pixel mean of the grayscale image as the ratio between the sum of the pixel values ​​of multiple pixels and the number of pixels in the grayscale image.

[0041] In this embodiment, the acquired grayscale image is processed to obtain the pixel value of each pixel in the grayscale image. Based on the pixel value of each pixel in the grayscale image, the pixel mean of the grayscale image is determined. The pixel values ​​of multiple pixels in the grayscale image are determined. The pixel values ​​of the multiple pixels are added together. The ratio between the sum of the pixel values ​​of the multiple pixels and the number of pixels in the grayscale image is determined as the pixel mean of the grayscale image.

[0042] As an optional embodiment, step S106, using the pixel mean, filters out the first feature point from the grayscale image, including: determining the pixels whose pixel values ​​are within the range of the pixel mean as candidate points; determining the first difference between the pixel values ​​of the candidate points and the pixel values ​​of other candidate points; determining a target threshold based on the first difference and the pixel mean; determining the candidate points corresponding to the first difference where the number of targets is greater than the target threshold as second feature points; filtering the second feature points to obtain a third feature point; and filtering the third feature point to obtain the first feature point.

[0043] In this embodiment, pixels whose pixel values ​​fall within the range of the pixel mean can be identified as candidate points. One candidate point is randomly selected, and the difference between its pixel value and the pixel values ​​of other candidate points is determined to obtain the first difference of the selected candidate point. A target threshold is then determined based on this first difference and the pixel mean. The pixel mean range can characterize the range of fluctuations in the grayscale mean, and can be a positive or negative range of the grayscale threshold set according to actual conditions, represented by μ±α. It can be used to filter out points with abnormal pixel values, such as those that are abnormally large or small. The target threshold can be represented by t.

[0044] Optionally, a pixel mean range can be set according to the actual situation. Abnormal points in the grayscale image can be filtered out using the pixel mean. Pixels whose pixel values ​​belong to the pixel mean range among multiple pixels are identified as candidate points, resulting in multiple candidate points. The difference between the pixel value of the candidate point and the pixel values ​​of other candidate points is determined to obtain the first difference of the selected candidate points. The target threshold is determined based on the first difference and the pixel mean.

[0045] Optionally, the target threshold can be calculated by determining the square of the difference between the pixel value (I(x)) and the average gray level (μ) of the image, summing all the squared differences, dividing by the number of pixels (n) in the grayscale image, obtaining the average value, dividing the average value by the average gray level (μ), taking the square root of the ratio, and multiplying the square root by the coefficient (k). The product can be calculated using the following formula:

[0046]

[0047] The coefficient can be a value set according to the actual situation, such as the number 5. This is just an example and is not a specific limitation.

[0048] In this embodiment, the difference between the pixel values ​​of a candidate point and other candidate points is determined to obtain multiple first differences. When there is a first difference among the multiple first differences of a candidate point where the number of targets is greater than the target threshold, the candidate point can be determined as a second feature point. Based on the second feature point, the second feature point is filtered to obtain a third feature point. Based on the third feature point, the third feature point is filtered to obtain a first feature point. The number of targets can be a value set according to the actual situation, such as 3 or 9.

[0049] Optionally, when determining whether the pixel value of a pixel among multiple pixels falls within the range of the pixel mean, the grayscale mean of the downsampled image is calculated, pixels whose grayscale mean falls within the range of the pixel mean are removed, and the pixels within the range are determined as candidate points. The set of removed pixels can be represented as a point set A. Taking a certain pixel in point set A as the center and a radius of r, it is determined whether the 16 pixels in this circle are feature points. A certain pixel can be represented by p, and the 16 pixels can be represented by p1, p2, ..., p16 respectively.

[0050] Optionally, a first difference between the pixel values ​​of a candidate point and other candidate points is determined, and it is determined whether there is a first difference between the number of target points and the target threshold. If there is, the selected candidate point is determined as the second feature point. If there is no, the selected candidate point is deleted. The determined second feature point is filtered to obtain the third feature point. The third feature point is filtered to obtain the first feature point.

[0051] For example, when point p is a candidate point, we can determine whether the absolute value of the difference between the pixel values ​​of candidate point p and other candidate points is greater than the target threshold. For instance, we can determine whether the absolute value of the first difference between the pixel values ​​of candidate point p and candidate points p1, p9, p5, and p13 is greater than the target threshold. If there are 3 candidate points whose absolute values ​​of the first difference are greater than the target threshold, we can continue to determine the absolute value of the pixel differences between point p and other candidate points p1-p16. Otherwise, point p is deleted. For instance, we can determine whether the absolute value of the first difference between the pixel values ​​of candidate point p and other candidate points p1-p16 is greater than the target threshold. If there are 9 candidate points whose absolute values ​​of the first difference are greater than the target threshold, then the candidate point is determined to be the second feature point. Otherwise, point p is deleted. Here, the first difference changes with the candidate point and is not a fixed value.

[0052] In related technologies, target thresholds are usually set manually and randomly to filter feature points. However, this method has the technical problem of feature point aggregation and is not sensitive to different grayscale scenes. The larger the threshold, the fewer corner points are detected, and the smaller the threshold, the more corner points are detected. In the embodiments of this invention, candidate points are determined by using the pixel mean and further filtered by using the target threshold. This can take into account the detection of corner points under different grayscale scenes, so that the corner detection threshold can change with the change of image grayscale, thereby improving the accuracy of image recognition.

[0053] As an optional embodiment, filtering the second feature points to obtain the third feature point includes: determining a second difference between the pixel value of the second feature point and the pixel values ​​of other second feature points in the first region, wherein the first region is used to characterize the geometry centered on the second feature point; determining the difference set of the second feature points based on the second difference; determining the sum of the difference sets as the response value of the second feature point; and determining the second feature point corresponding to the maximum response value in the first region as the third feature point.

[0054] In this embodiment, a second difference is determined between the pixel value of the second feature point and the pixel values ​​of other second feature points in the first region, thereby obtaining the difference set of the second feature point objects. The differences in the difference set of the second feature points are summed, and the sum of the difference set is determined as the response value of the second feature point, thus obtaining the response value corresponding to each second feature point. The second feature point corresponding to the maximum response value in the first region is determined, and this second feature point is determined as the third feature point. The first region can be used to represent the geometric shape centered on the second feature point, and can be a rectangle, circle, square, or other geometric shape. For example, it can be a 3x3 or 5x5 rectangle. This is only an example and is not specifically limited.

[0055] Optionally, the sum of the differences is determined as the response value of the second feature point. The response value can be used to characterize the matching rate of the feature point. It can be the sum of the absolute values ​​of the differences between the 16 points centered on a certain second feature point and the center point. For example, the larger the response value, the higher the matching degree of the feature point. It can be represented by N.

[0056] For example, when there is only one second feature point, it is directly identified as the third feature point; when there are multiple second feature points, the N value of each second feature point is determined, and the second feature point with the largest N value is identified as the third feature point. The N value can be calculated by summing the absolute values ​​of the differences between pixel values ​​I(x) and other pixel values ​​I(p) in the first region, which can be expressed by the following formula:

[0057]

[0058] As an optional embodiment, the process of filtering out the third feature point to obtain the first feature point includes: determining the Hamming distance of the third feature point in a second region, wherein the second region is used to characterize the geometry centered on the third feature point; identifying the third feature point whose Hamming distance in the second region is greater than a distance threshold as the fourth feature point; determining the Hamming distance of the fourth feature point in a third region, wherein the third region is used to characterize the geometry centered on the fourth feature point, and the third region has a different size than the second region; and identifying the fourth feature point whose Hamming distance in the third region is greater than a distance threshold as the first feature point.

[0059] In this embodiment, based on the third feature point, the Hamming distance of the third feature point in the second region is determined. When the Hamming distance is greater than a distance threshold, the third feature point is determined as the fourth feature point. Based on the fourth feature point, the Hamming distance of the fourth feature point in the third region is determined. When the Hamming distance is greater than a distance threshold, the fourth feature point is determined as the first feature point. The second region is used to represent the geometric figure centered on the third feature point; the third region is used to represent the geometric figure centered on the fourth feature point. For example, the second region or the third region can be represented as an SxS rectangle, wherein the size of the third region is different from that of the second region.

[0060] Optionally, the Hamming distance of the feature points is determined. A neighborhood window of S x S is taken centered on a certain feature point. A pair of points is randomly selected within the window, and their pixel sizes are compared and binary values ​​are assigned. The binary assignment (τ) can be expressed by the following formula:

[0061]

[0062] Here, s(x) and s(y) can be used to represent the pixel values ​​of random points x = (u1, v1) and y = (u2, v2) respectively. After binary assignment, N pairs of random points are randomly selected in the window, and binary assignment is repeated to form a binary code. After the code is formed, an XOR operation is performed on the point pair to obtain the Hamming distance between the point pairs.

[0063] Optionally, the similarity of the binary codes of point pairs is calculated using Hamming distance. When the similarity of the third feature point in the second region is greater than a distance threshold, the third feature point is determined as the fourth feature point. The binary assignment and encoding are repeated to determine the Hamming distance of the fourth feature point in the third region. When the similarity of the fourth feature point in the third region is greater than a distance threshold, the fourth feature point is determined as the first feature point. The distance threshold can be determined by T. HM To express.

[0064] In related technologies, feature points are determined by sequentially judging the Hamming distance of pixels within any given region. However, in this embodiment of the invention, it is determined whether the Hamming distance of pixels within any given region is greater than a distance threshold. If the Hamming distance of a pixel is greater than the distance threshold, it is further determined whether the Hamming distance of pixels within other regions of different sizes is greater than the distance threshold. When the Hamming distance of a pixel is greater than the distance threshold, the pixel is determined as a feature point, thereby achieving the goal of accurately identifying feature points and improving the accuracy of image recognition.

[0065] As an optional embodiment, the method further includes: dividing the grayscale image into regions of equal area to obtain multiple sub-images; and using a scale pyramid to determine the angle and position coordinates of feature points in the corresponding sub-images.

[0066] In this embodiment, a grayscale image of the scene where the vehicle is located is acquired, and the grayscale image is divided into regions of equal area to obtain multiple sub-images. Based on the multiple sub-images, a scale pyramid is used to determine the angle and position coordinates of feature points in the corresponding sub-images.

[0067] Optionally, the determined third feature point can be partitioned using a quadtree to calculate the grayscale coordinates in any layer of the image pyramid, where any layer of the pyramid can be represented by A; the grayscale coordinates can be obtained by... The representation is given, where M in the grayscale coordinates... 00 M 10 M 01 The calculation method can be expressed by the following formula:

[0068] M 00 =∑ x，y∈A I(x, y) M 10 =∑x，y∈A x*I(x, y) M 01 =∑ x，y∈A y*I(x, y)

[0069] Where x and y can be pixel coordinates, and I(x,y) can be the grayscale value of the pixel coordinates. Based on the grayscale coordinates, the direction of the feature point is obtained. The angle of the feature point can be represented by θ, and the calculation method can be expressed by the following formula:

[0070]

[0071] In this embodiment of the invention, in order to filter image noise, abnormal pixels can be filtered by performing Gaussian filtering on the image. The variance of the Gaussian filter can be represented by x, and the Gaussian window can be represented as M×N.

[0072] As an optional embodiment, determining the target image in a grayscale image based on a first feature point includes: restoring the first feature point based on its angle and position coordinates to obtain the target image.

[0073] In this embodiment, the first feature point is restored based on its angle and position coordinates to determine the target image in the grayscale image.

[0074] Optionally, utilize The first feature point is restored using θ to obtain the target image in the grayscale image.

[0075] As an optional embodiment, the method further includes: acquiring multiple original images of the scene where the vehicle is located; performing correction processing on the multiple original images to obtain multiple normal images; stitching the multiple normal images to obtain a stitched image; and performing grayscale processing on the stitched image to obtain a grayscale image.

[0076] In this embodiment, multiple original images of the scene where the vehicle is located are acquired. Based on the original images, correction processing is performed on them to obtain multiple normal images. Based on the obtained normal images, the multiple normal images are stitched together to obtain a stitched image. Finally, the stitched image is processed to obtain a grayscale image.

[0077] Optionally, converting the original image to a grayscale image can be achieved by performing distortion correction on the original image, stitching the images based on the distortion correction, and then performing grayscale processing on the stitched images to convert them into grayscale images.

[0078] This embodiment utilizes the pixel mean of multiple pixels in a grayscale image of the scene where the vehicle is located to filter out a first feature point from the grayscale image, and determines the target image in the grayscale image based on the first feature point, thereby achieving the technical effect of improving the accuracy of image recognition and solving the technical problem of low accuracy in image recognition.

[0079] Example 2

[0080] The technical solutions of the embodiments of the present invention will be illustrated below with reference to preferred embodiments.

[0081] In autonomous driving, the execution process can be divided into four major modules: perception fusion, high-precision positioning, planning and decision-making, and control execution. Among them, high-precision positioning technology is mainly achieved through the Global Navigation Satellite System (GNSS), matching positioning, and inertial measurement unit (IMU). However, due to the lack of GNSS signal scenarios, positioning technology is crucial for the research of the positioning industry, taking into account mass production costs.

[0082] In one related technology, the existing fast feature detection method for corner detection (FAST) and image feature extraction (ORB) involves converting the original image to grayscale, initializing parameters, calculating the pyramid image for each image, calculating the feature points of each pyramid layer, dividing all extracted keypoints into quadtrees, calculating the principal direction for the feature points in all the saved nodes, calculating the feature descriptor algorithm (BRIEF) for each keypoint in the image, and finally calculating Hamming distance matching. However, the threshold in this method needs to be set manually, and it may extract edge points or local non-maximum points for different images. Although the extraction speed is fast, there is a problem of feature point clustering, and it is not sensitive to different grayscale scenes. The larger the threshold, the fewer corner points are detected; the smaller the threshold, the more corner points are detected, resulting in poor adaptability.

[0083] In another related technology, an ORB feature extraction and matching method is proposed. This method converts the original image into a grayscale image, performs FAST corner detection, corner extraction (HARRIS), centroid calculation, and Gaussian blurring, outputting key feature point information and a Gaussian blurred image. The image is downsampled to obtain multiple sets of key feature point information and Gaussian blurred images. The descriptor information corresponding to the feature points is calculated, and the descriptor information of two frames of images is matched to obtain the ORB feature matching result. The original image and the ORB feature matching result synchronized with the original image are output, which improves the speed of recognizing the same target image. While ensuring the accuracy of feature extraction and matching, it also ensures real-time performance. However, this method has a technical problem that the corner detection threshold cannot be reasonably changed with the change of image grayscale.

[0084] In another related technique, an improved ORB feature matching method is proposed. This method uses an improved FAST14-24 method for initial corner extraction. Based on the obtained corners, a corner detection algorithm (Shi-Tomasi) is used for feature point optimization. The gray-scale centroid method is used to process the feature point set to determine the orientation of the feature points. Based on the feature point set, a retina-like descriptor extraction algorithm is used to obtain feature descriptors. Based on the obtained feature descriptors, a learning method is used to extract the positions of low-correlation sampling point pairs to obtain optimized feature descriptors. Hamming distance is used for feature matching. Compared with the existing descriptor (rBRIEF), it has better robustness and higher accuracy. However, the corner detection threshold in this method cannot be reasonably changed with the change of image grayscale.

[0085] In another related technology, an adaptive ORB feature extraction method based on image segmentation was proposed. This method includes image segmentation, pyramid construction, and improved FAST feature extraction. It uses the Harris corner score to select better feature points and displays the feature point positions in the original image, so that the feature points are more evenly distributed in the image. However, this method has the technical problem that the corner detection threshold cannot be reasonably changed with the change of image grayscale.

[0086] To address the aforementioned issues, this invention proposes an adaptive algorithm for FAST corner detection pixel thresholds in vehicle-mounted scenarios. This algorithm enhances the stability of corner recognition under varying lighting conditions, thereby achieving the technical effect that the FAST algorithm can still identify sufficient key points even under extreme lighting conditions.

[0087] The embodiments of the present invention will be further described below.

[0088] Figure 2 This is a flowchart of another image processing method according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps.

[0089] Step S201: Obtain a grayscale image of the scene where the vehicle is located.

[0090] In this embodiment of the invention, the original image of the scene where the vehicle is located is captured by the vehicle-mounted camera. The original image is then converted into a grayscale image. This can be achieved by an image acquisition device that captures images of the scene and converts the captured images into grayscale images. The image acquisition device can be a device applied to the vehicle and can be used to capture images of the scene around the vehicle. For example, it can be a vehicle-mounted camera, a 4-way surround-view vehicle-mounted camera, etc.

[0091] Optionally, the vehicle-mounted camera can capture images of the scene in which the vehicle is located, and the original image can be converted into a grayscale image using an ORB feature detection algorithm with FAST corner detection, thereby achieving the purpose of obtaining a grayscale image of the scene in which the vehicle is located.

[0092] Optionally, multiple original images of the scene where the vehicle is located are acquired. Based on the original images, the multiple original images are corrected to obtain multiple normal images. Based on the obtained normal images, the multiple normal images are stitched together to obtain a stitched image. Based on the stitched image, the stitched image is processed to obtain a grayscale image.

[0093] Step S202: Calculate the image pyramid.

[0094] In this embodiment of the invention, a grayscale image of the scene where the vehicle is located is obtained, and an image pyramid is calculated based on the grayscale image. The image pyramid can be used to characterize the multi-scale expression of the image and can be used to interpret the structure of the image at multiple resolutions.

[0095] Optionally, a grayscale image of the scene where the vehicle is located can be obtained, and the grayscale image can be downsampled. Downsampling is a multi-rate digital signal processing technique in the field of digital signal processing or a process of reducing the signal sampling rate. This can be achieved by scaling the grayscale image by 1 / 1.2 in sequence to obtain a total of 8 images.

[0096] Optionally, the grayscale image is downsampled, and based on the downsampled grayscale image, the pixel mean of multiple pixels in the grayscale image is obtained. This can be to obtain the grayscale mean of the downsampled image, where the grayscale mean can be represented by μ.

[0097] Optionally, based on the grayscale mean of the downsampled image, it is determined whether the pixels of a pixel among multiple pixels belong to the pixel mean range. If they belong to the mean range, pixels within the grayscale mean range of the downsampled image are removed. The pixel mean range can be used to characterize the range of grayscale mean fluctuations, and can be a positive or negative range of a grayscale threshold set according to actual conditions. It can be represented by μ±α and can be used to filter out points with abnormal pixel values, such as filtering out points with abnormally large or small pixel values. The set of removed pixels can be represented as point set A, and α can be any set value.

[0098] Step S203: Determine image feature points.

[0099] In this embodiment of the invention, image feature points are determined based on the pixel differences obtained from calculating the image pyramid.

[0100] Optionally, taking a certain pixel in the point set A as the center and a radius of r, calculate 16 pixels in this circle. A certain pixel can be represented by p, and the 16 pixels can be represented by p1, p2, ..., p16 respectively.

[0101] Optionally, the pixel differences between p1, p9 and the center p are calculated. If the absolute value of the difference is less than the target threshold, then point p cannot be a feature point; otherwise, it is used as a candidate point for further screening. The target threshold can be calculated by determining the square of the difference between the pixel value (I(x)) and the average gray value (μ) of the image, summing all the squared differences, dividing by the number of pixels (n) in the grayscale image, calculating the average value, dividing the average value by the average gray value (μ), taking the square root of the ratio, and multiplying the square root by the coefficient (k). The product can be calculated using the following formula:

[0102]

[0103] The coefficient can be a value set according to the actual situation, such as the number 5. This is just an example and is not a specific limitation.

[0104] Optionally, when p is a candidate point, the pixel differences between p1, p9, p5, p13 and the center p are calculated. If at least three of the obtained absolute values ​​of the differences exceed the target threshold, they can be used as candidate points for further screening.

[0105] Optionally, based on the filtered results, when p is a candidate point, the absolute value of the pixel difference between point p and other candidate points p1-p16 is calculated. If at least 9 of the absolute values ​​of the differences exceed the target threshold, they can be determined as feature points.

[0106] Optionally, based on a given feature point, when there are multiple feature points, the sum of the absolute values ​​of the differences between each feature point and the center within the neighborhood of feature point p is determined. The neighborhood can be any geometric shape, such as a 3x3 or 5x5 rectangle; this is merely an example and not a specific limitation. The sum of the absolute values ​​of the differences between each feature point and the center can be represented by N. The value of N can be calculated by summing the absolute values ​​of the differences between pixel value I(x) and other pixel values ​​I(p) within the geometric region, which can be expressed by the following formula:

[0107]

[0108] When N is at its maximum value, the corresponding feature point is retained; otherwise, the feature point is suppressed. When there is only one feature point in the neighborhood, it is directly retained.

[0109] Step S204: Divide the key points and determine the direction of the feature points.

[0110] In this embodiment of the invention, the determined feature points are used as key points, and all extracted key points are divided in the form of a quadtree to determine the direction of the feature points.

[0111] Optionally, a quadtree can be used for partitioning to calculate the grayscale coordinates of any layer in the image pyramid, where any layer of the pyramid can be represented by A; the grayscale coordinates can be obtained by... The representation is given, where M in the grayscale coordinates... 00 M 10 M 01 The calculation method can be expressed by the following formula:

[0112] M 00 =∑ x，y∈A I(x, y) M 10 =∑ x，y∈A x*I(x, y) M 01 =∑ x，y∈A y*I(x, y)

[0113] Where x and y can be pixel coordinates, and I(x,y) can be the grayscale value of the pixel coordinates. The direction of the feature point is obtained based on the grayscale coordinates. The angle of the feature point can be represented by θ, and the calculation method can be expressed by the following formula:

[0114]

[0115] In this embodiment of the invention, in order to filter image noise, abnormal pixels can be filtered by performing Gaussian filtering on the image. The variance of the Gaussian filter can be represented by x, and the Gaussian window can be represented as MxN.

[0116] Step S205: Calculate the key point BRIEF.

[0117] In this embodiment of the invention, after dividing all extracted key points and determining the direction of feature points, the descriptor BRIEF is calculated for each key point in the image.

[0118] Optionally, taking a neighborhood window of SxS centered on a certain feature point, randomly selecting a pair of points within the window, comparing the pixel sizes, and assigning binary values, wherein the binary assignment can be expressed by the following formula:

[0119]

[0120] Where s(x) and s(y) can be used to represent the pixel values ​​of random points x = (u1, v1) and y = (u2, v2) respectively. Based on the pixel points after binary assignment, N pairs of random points are randomly selected in the window, and binary assignment is repeated to form a binary code. After the code is formed, the point pairs are XORed to obtain the Hamming distance between the point pairs.

[0121] Step S206: Clarify the Hamming distance threshold and the Hamming distance matching.

[0122] In this embodiment of the invention, based on calculating the descriptor BRIEF for each key point in the image, the Hamming distance threshold is determined and the Hamming distance matching is calculated.

[0123] Optionally, the similarity of binary codes for point pairs can be calculated using Hamming distance. If the similarity of feature points is greater than a distance threshold, binary assignment and encoding are repeated; otherwise, a mismatch is directly determined. The distance threshold can be determined by T. HM To express.

[0124] Optionally, based on the determined feature point, calculate the Hamming distance matching similarity of other points in the region centered on feature point p with a range of NxN, and determine whether it is greater than the distance threshold.

[0125] Optionally, based on the above judgment results, the matching rate of other point sets in the NxN region is calculated. If the matching rate is greater than the threshold, the two points are considered to match; otherwise, they are not matched.

[0126] This embodiment utilizes the pixel mean of multiple pixels in a grayscale image of the scene where the vehicle is located to filter out a first feature point from the grayscale image, and determines the target image in the grayscale image based on the first feature point, thereby achieving the technical effect of improving the accuracy of image recognition and solving the technical problem of low accuracy in image recognition.

[0127] Example 3

[0128] According to an embodiment of the present invention, an image processing apparatus is also provided. It should be noted that this image processing apparatus can be used to execute the image processing method in Embodiment 1.

[0129] Figure 3 This is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. Figure 3 As shown, the image processing device 300 may include: a first acquisition unit 302, a second acquisition unit 304, a filtering unit 306, and a determination unit 308.

[0130] The first acquisition unit 302 is used to acquire a grayscale image of the scene where the vehicle is located;

[0131] The second acquisition unit 304 is used to acquire the pixel mean of multiple pixels in a grayscale image;

[0132] The filtering unit 306 is used to filter out the first feature point from the grayscale image using the pixel mean.

[0133] The determining unit 308 is used to determine the target image in the grayscale image based on the first feature point.

[0134] Optionally, the second acquisition unit 304 includes: a first determining module, used to determine the pixel values ​​of multiple pixels in the grayscale image; and to determine the pixel mean of the grayscale image as the ratio between the sum of the pixel values ​​of the multiple pixels and the number of pixels in the grayscale image.

[0135] Optionally, the filtering unit 306 includes: a second determining module, configured to determine pixels whose pixel values ​​are within the range of the pixel mean among a plurality of pixels as candidate points; determine a first difference between the pixel values ​​of the candidate points and the pixel values ​​of other candidate points; determine a target threshold based on the first difference and the pixel mean; determine the candidate points corresponding to the first difference where the number of targets is greater than the target threshold as second feature points; and a first filtering module, configured to filter the second feature points to obtain a third feature point; and filter the third feature point to obtain a first feature point.

[0136] Optionally, the first filtering module includes: a first determining submodule, configured to determine a second difference between the pixel value of the second feature point and the pixel values ​​of other second feature points in the first region, wherein the first region is used to characterize a geometric shape centered on the second feature point; configured to determine a difference set of the second feature points based on the second difference; configured to determine the sum of the difference sets as the response value of the second feature point; and configured to determine the second feature point corresponding to the maximum response value in the first region as the third feature point.

[0137] Optionally, the second filtering module includes: a second determining submodule, configured to determine the Hamming distance of the third feature point in the second region, wherein the second region is used to characterize the geometry centered on the third feature point; configured to determine the third feature point whose Hamming distance in the second region is greater than a distance threshold as the fourth feature point; configured to determine the Hamming distance of the fourth feature point in the third region, wherein the third region is used to characterize the geometry centered on the fourth feature point, and the third region has a different size from the second region; and configured to determine the fourth feature point whose Hamming distance in the third region is greater than a distance threshold as the first feature point.

[0138] Optionally, the apparatus further includes: a first processing unit, configured to divide the grayscale image into regions of equal area to obtain multiple sub-images; and configured to use a scale pyramid to determine the angle and position coordinates of feature points in the corresponding sub-images.

[0139] Optionally, the determining unit 308 includes: a second processing module, used to reconstruct the first feature point based on the angle and position coordinates of the first feature point to obtain the target image.

[0140] Optionally, the device further includes: a first acquisition unit, configured to acquire multiple original images of the scene where the vehicle is located; to perform correction processing on the multiple original images to obtain multiple normal images; to stitch the multiple normal images together to obtain a stitched image; and to perform grayscale processing on the stitched image to obtain a grayscale image.

[0141] In this embodiment of the invention, a first acquisition unit acquires a grayscale image of the scene in which the vehicle is located; a second acquisition unit acquires the average pixel value of multiple pixels in the grayscale image; a filtering unit uses the average pixel value to filter out a first feature point from the grayscale image; and a determination unit determines a target image in the grayscale image based on the first feature point. In other words, this invention improves the accuracy of image recognition by using the average pixel value of multiple pixels in a grayscale image of the scene in which the vehicle is located to filter out a first feature point, and then determines a target image in the grayscale image based on the first feature point. This solves the technical problem of low accuracy in image recognition.

[0142] Example 4

[0143] According to an embodiment of the present invention, a computer-readable storage medium is also provided, the storage medium including a stored program, wherein the program executes the image processing method of Embodiment 1.

[0144] Example 5

[0145] According to an embodiment of the present invention, a processor is also provided for running a program, wherein the program executes the image processing method of Embodiment 1 during runtime.

[0146] Example 6

[0147] According to an embodiment of the present invention, a vehicle is also provided for performing the image processing method of Embodiment 1.

[0148] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0149] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0153] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0154] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of processing an image, characterized by, Applied to a vehicle, the vehicle is deployed with an image acquisition device, comprising: Obtaining a gray image of a scene where the vehicle is located; Obtaining a pixel mean value of a plurality of pixels in the gray image; Using the pixel mean value, screening out a first feature point from the gray image; Determining a target image in the gray image based on the first feature point; Wherein, the first feature point is screened out from the gray image by using the pixel mean value, including: determining the pixel points in the pixel mean value range of the gray image as candidate points; determining the first difference value between the pixel value of the candidate point and the pixel value of other candidate points; determining the target threshold value based on the first difference value and the pixel mean value; determining the candidate points corresponding to the first difference value whose target number is greater than the target threshold value as second feature points; screening the second feature points to obtain third feature points; screening the third feature points to obtain the first feature points.

2. The method of claim 1, wherein, Obtaining a pixel mean value of a plurality of pixels in the gray image, including: Determining the pixel value of a plurality of pixel points in the gray image; The ratio between the sum of the pixel values of the plurality of pixel points and the number of pixel points in the gray image is determined as the pixel mean value of the gray image.

3. The method of claim 1, wherein, Screening the second feature points to obtain the third feature points, including: Determining the second difference value between the pixel value of the second feature point and the pixel value of other second feature points in the first area, wherein the first area is used to represent a geometric figure centered on the second feature point; Based on the second difference value, determine the difference set of the second feature point; The sum of the difference set is determined as the response value of the second feature point; The second feature point corresponding to the maximum response value in the first area is determined as the third feature point.

4. The method of claim 2, wherein, Screening the third feature points to obtain the first feature points, including: Determining the Hamming distance of the third feature point in the second area, wherein the second area is used to represent a geometric figure centered on the third feature point; The third feature point whose Hamming distance in the second area is greater than the distance threshold is determined as the fourth feature point; Respectively determine the Hamming distance of the fourth feature point in the third area, wherein the third area is used to represent a geometric figure centered on the fourth feature point, and the size of the third area is different from that of the second area; The fourth feature point whose Hamming distance in the third area is greater than the distance threshold is determined as the first feature point.

5. The method of any of claim 1, wherein, The method further comprises: Divide the gray image into areas of the same area to obtain a plurality of sub-images; Using the scale pyramid, determine the angle and position coordinates of the feature points in the corresponding sub-images.

6. The method of claim 5, wherein, Based on the first feature point, determining a target image in the gray image, including: Restoring the first feature point based on the angle and position coordinates of the first feature point to obtain the target image.

7. The method of claim 1, wherein, The method further comprises: Obtaining a plurality of original images of a scene where the vehicle is located; Correcting the plurality of original images to obtain a plurality of normal images; The plurality of normal images are spliced to obtain a spliced image; The spliced image is subjected to grayscale processing to obtain the grayscale image.

8. An image processing apparatus characterized by comprising: The method comprises the following steps: A first acquisition unit is configured to acquire a grayscale image of a scene in which a vehicle is located; A second acquisition unit is configured to acquire a pixel mean value of a plurality of pixels in the grayscale image; A screening unit is configured to screen a first feature point from the grayscale image by using the pixel mean value; A determination unit is configured to determine a target image in the grayscale image based on the first feature point; The screening unit is configured to screen the first feature point from the grayscale image by using the pixel mean value through the following steps: determining a candidate point from a plurality of pixel points of the grayscale image, wherein the candidate point is within the pixel mean value range; determining a first difference value between a pixel value of the candidate point and a pixel value of another candidate point; determining a target threshold value based on the first difference value and the pixel mean value; determining a second feature point from the candidate point corresponding to the first difference value, wherein the target number of the second feature point is greater than the target threshold value; screening the second feature point to obtain a third feature point; and screening the third feature point to obtain the first feature point.

Images