Image processing method based on radar information

Abstract

The embodiment of the invention provides an image processing method based on radar information, and belongs to the technical field of computers. The image processing method comprises the following steps: acquiring a real-time image of a laser radar; performing ground point segmentation according to the real-time image; performing feature point extraction on the real-time image; the feature points are classified; and filtering the feature points to obtain the road boundary in the real-time image. According to the radar information-based image processing method provided by the invention, the real-time image of the laser radar is acquired, ground point segmentation is firstly performed on the real-time image, then the feature points in the real-time image are extracted and classified, and finally the feature points are filtered, so that clear and accurate road boundaries in the real-time image are obtained, and the real-time image processing accuracy is improved. Therefore, accurate movement and inspection of the fire-fighting robot can be facilitated, and the movement precision and accuracy of the fire-fighting robot during inspection are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method based on radar information. Background Art

[0002] The fire safety of substations is an important factor in maintaining the safe and stable operation of the power grid system. Therefore, it is necessary to conduct regular inspections of the fire protection conditions of substations.

[0003] In the prior art, firefighting robots are usually used in conjunction with radar positioning to conduct automatic firefighting inspections. During radar positioning, the image sent by the radar needs to be analyzed and positioned. However, the existing recognition method has a lot of recognition noise for the roads in the substation, resulting in large positioning errors, which in turn affects the inspection process of the firefighting robot.

[0004] The inventors of the present application discovered during the process of implementing the present invention that the above-mentioned solution in the prior art has the defect of large positioning error. Summary of the invention

[0005] The purpose of the embodiment of the present invention is to provide an image processing method based on radar information, which has a function of good positioning effect.

[0006] In order to achieve the above object, an embodiment of the present invention provides an image processing method based on radar information, comprising:

[0007] Get real-time images from LiDAR;

[0008] Perform ground point segmentation according to the real-time image;

[0009] Extracting feature points from the real-time image;

[0010] classifying the feature points;

[0011] The feature points are filtered to obtain the road boundary in the real-time image.

[0012] Optionally, performing ground point segmentation according to the real-time image includes:

[0013] constructing a road boundary detection area in the real-time image;

[0014] Preset distance threshold;

[0015] A ground point is acquired in the real-time image according to the distance threshold.

[0016] Optionally, extracting feature points from the real-time image includes:

[0017] According to formula (1) and formula (2), vertical threshold filtering is performed on the point cloud points in the image.

[0018] A1≤Z max -Z min ≤A2, (1)

[0019]

[0020] Among them, A1 is the first height threshold, A2 is the second height threshold, A3 is the third height threshold, Z max is the maximum z value of the points around the point cloud point, Z min is the minimum z value of the points around the point cloud point, z li is the z value of each neighboring point of the ith point cloud point of the lth radar beam, α is the mean parameter, α=∑z li / n h , n h is the number of neighboring points, l and i are integer numbers.

[0021] Optionally, extracting feature points from the real-time image further includes:

[0022] Obtain a set of neighboring points of the point cloud point;

[0023] The smoothness of the point cloud is calculated according to formula (3):

[0024]

[0025] Wherein, s is the smoothness of the ith point cloud point of the lth radar beam, S is the set of neighboring points of the point cloud point, and j is an integer number.

[0026] Optionally, extracting feature points from the real-time image further includes:

[0027] The horizontal distance of the point cloud points is calculated according to formula (4):

[0028]

[0029] Among them, X li is the horizontal distance of the ith point cloud point of the lth radar beam, H s is the absolute value of the height of the i-th point cloud point of the l-th radar beam, θ l is the vertical azimuth of the lth radar beam, θ a is the horizontal resolution of the laser radar.

[0030] Optionally, classifying the feature points includes:

[0031] Obtaining the longest light beams in front of and behind the laser radar;

[0032] The angle at which the longest light beam is located is regarded as a road dividing line;

[0033] The feature points of the image are divided into left road feature points and right road feature points according to the road segmentation line.

[0034] Optionally, filtering the feature points includes:

[0035] Acquire a feature image according to the feature points;

[0036] filtering the feature image;

[0037] The model was fitted using a random sampling consensus algorithm;

[0038] Get the points in the fitted model whose distance is less than the threshold and use them as seed points.

[0039] Optionally, filtering the feature points further includes:

[0040] Setting the seed point to an initial value;

[0041] Construct Gaussian process regression algorithm GPR calculation model;

[0042] The initial value is input into a Gaussian process regression algorithm (GPR) calculation model to obtain the road boundary in the real-time image.

[0043] Through the above technical scheme, the image processing method based on radar information provided by the present invention obtains the real-time image of the laser radar, first segments the ground points of the real-time image, then extracts and classifies the feature points in the real-time image, and finally filters the feature points to obtain clear and accurate road boundaries in the real-time image, thereby facilitating the accurate movement and patrol of the fire-fighting robot and improving the movement accuracy and precision of the fire-fighting robot during patrol.

[0044] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0046] Figure 1 is a flowchart of an image processing method based on radar information according to an embodiment of the present invention;

[0047] Figure 2is a flowchart of obtaining ground points in an image processing method based on radar information according to an embodiment of the present invention;

[0048] Figure 3 is a flowchart of obtaining the horizontal distance of a point cloud point in an image processing method based on radar information according to an embodiment of the present invention;

[0049] Figure 4 is a flowchart of dividing feature points in an image processing method based on radar information according to an embodiment of the present invention;

[0050] Figure 5 The present invention is a flowchart of obtaining road boundaries in an image processing method based on radar information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0052] Figure 1 FIG. 1 is a flowchart of an image processing method based on radar information according to an embodiment of the present invention. Figure 1 In the method, the image processing may include:

[0053] In step S10, a real-time image of the laser radar is acquired.

[0054] In step S11, ground point segmentation is performed based on the real-time image.

[0055] In step S12, feature points are extracted from the real-time image.

[0056] In step S13, the feature points are classified.

[0057] In step S14, the feature points are filtered to obtain the road boundary in the real-time image.

[0058] In step S10 to step S14, the real-time image of the laser radar is first obtained, and then the ground points in the real-time image are segmented. After the segmentation is completed, the feature points of the real-time image are extracted and classified. Finally, the feature points are filtered to obtain a more accurate road boundary in the real-time image.

[0059] Traditional substations generally use firefighting robots in conjunction with radar positioning to perform automatic firefighting inspections. During the radar positioning process, the image sent by the radar needs to be analyzed and positioned. However, the existing recognition method has a lot of recognition noise for the roads in the substation, resulting in large positioning errors, which in turn affects the inspection process of the firefighting robot. In this embodiment of the present invention, the method of extracting, classifying and filtering real-time image features can facilitate the accurate movement and inspection of the firefighting robot, and improve the movement accuracy and precision of the firefighting robot during inspection.

[0060] In this embodiment of the present invention, in the process of processing the real-time image, it is also necessary to segment the ground points in the real-time image. The specific segmentation step can be as follows: Figure 2 Specifically, Figure 2 In the method for image processing, the image processing method may further include:

[0061] In step S20, a road boundary detection area is constructed in the real-time image.

[0062] In step S21, a distance threshold is preset.

[0063] In step S22, ground points are acquired in the real-time image according to the distance threshold.

[0064] In this embodiment of the present invention, when extracting feature points in a real-time image, it is necessary to first filter the points in the image. The specific steps can be as follows: Figure 3 Specifically, Figure 3 In the above, the image processing method may further include:

[0065] In step S30, vertical threshold filtering is performed on the point cloud points in the image according to formula (1) and formula (2).

[0066] A1≤Z max -Z min ≤A2, (1)

[0067]

[0068] Among them, A1 is the first height threshold, A2 is the second height threshold, A3 is the third height threshold, Z max is the maximum z value of the points around the point cloud point, Z min is the minimum z value of the points around the point cloud point, z li is the z value of each neighboring point of the ith point cloud point of the lth radar beam, α is the mean parameter, α=∑z li / n h , n h is the number of neighboring points, l and i are integer numbers.

[0069] In step S31, a set of neighboring points of a point cloud point is obtained.

[0070] In step S32, the smoothness of the point cloud points is calculated according to formula (3):

[0071]

[0072] Among them, s is the smoothness of the i-th point cloud point of the l-th radar beam, S is the set of neighboring points of the point cloud point, and j is an integer number.

[0073] In step S33, the horizontal distance of the point cloud points is calculated according to formula (4):

[0074]

[0075] Among them, X li is the horizontal distance of the ith point cloud point of the lth radar beam, H s is the absolute value of the height of the i-th point cloud point of the l-th radar beam, θ l is the vertical azimuth of the lth radar beam, θ a is the horizontal resolution of the lidar.

[0076] In step S30 to step S33, the point cloud points in the image are firstly filtered by vertical threshold, then their smoothness is calculated, and finally the horizontal distance of the point cloud points is calculated to complete the extraction of the features of the point cloud points. This extraction method has high extraction accuracy.

[0077] In this embodiment of the present invention, after the feature points are acquired, the feature points need to be classified. The specific classification steps can be as follows: Figure 4 Specifically, Figure 4 In the method for image processing, the image processing method may further include:

[0078] In step S40, the longest light beams in front of and behind the laser radar are obtained.

[0079] In step S41, the angle at which the longest light beam is located is regarded as the road dividing line.

[0080] In step S42, the feature points of the image are divided into road left feature points and road right feature points according to the road dividing line.

[0081] In step S40 to step S42, the road dividing line is first determined based on the longest light beams in front and behind the laser radar, and then the left feature points and the right feature points on both sides of the road are obtained.

[0082] In this embodiment of the present invention, after the feature points are classified, the feature points need to be filtered. Specifically, the filtering step can be as follows: Figure 5 Specifically, Figure 5 In the method for image processing, the image processing method may further include:

[0083] In step S50, a feature image is acquired according to the feature points.

[0084] In step S51, the feature image is filtered.

[0085] In step S52, a random sampling consensus algorithm is used to fit the model.

[0086] In step S53, points in the fitting model whose distance is less than the threshold are obtained and used as seed points.

[0087] In step S54, the seed point is set as an initial value.

[0088] In step S55, a Gaussian process regression algorithm GPR calculation model is constructed.

[0089] In step S56, the initial value is input into the Gaussian process regression algorithm GPR calculation model to obtain the road boundary in the real-time image.

[0090] In step S50 to step S56, a feature image is first obtained based on the feature points, and then the feature image is filtered. The model is fitted using a random sampling consistency algorithm, and a seed point is obtained, which is input into the Gaussian process regression algorithm model to obtain a converged road boundary in the real-time image.

[0091] Through the above technical scheme, the image processing method based on radar information provided by the present invention obtains the real-time image of the laser radar, first segments the ground points of the real-time image, then extracts and classifies the feature points in the real-time image, and finally filters the feature points to obtain clear and accurate road boundaries in the real-time image, thereby facilitating the accurate movement and patrol of the fire-fighting robot and improving the movement accuracy and precision of the fire-fighting robot during patrol.

[0092] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0097] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0098] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0099] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0100] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image processing method based on radar information, characterized in that: include: Get real-time images from LiDAR; Perform ground point segmentation according to the real-time image; Extracting feature points from the real-time image; classifying the feature points; The feature points are filtered to obtain the road boundary in the real-time image.

2. The image processing method according to claim 1, characterized in that: Segmenting ground points according to the real-time image includes: constructing a road boundary detection area in the real-time image; Preset distance threshold; A ground point is acquired in the real-time image according to the distance threshold.

3. The image processing method according to claim 1, characterized in that: Extracting feature points from the real-time image includes: According to formula (1) and formula (2), vertical threshold filtering is performed on the point cloud points in the image. A1≤Z max -WITH min ≤A2, (1) Among them, A1 is the first height threshold, A2 is the second height threshold, A3 is the third height threshold, Z max is the maximum z value of the points around the point cloud point, Z min is the minimum z value of the points around the point cloud point, z li is the z value of each neighboring point of the ith point cloud point of the lth radar beam, α is the mean parameter, α=∑z li / n h , n h is the number of neighboring points, l and i are integer numbers.

4. The image processing method according to claim 3, characterized in that: Extracting feature points from the real-time image also includes: Obtain a set of neighboring points of the point cloud point; The smoothness of the point cloud is calculated according to formula (3): Wherein, s is the smoothness of the ith point cloud point of the lth radar beam, S is the set of neighboring points of the point cloud point, and j is an integer number.

5. The image processing method according to claim 4, characterized in that: Extracting feature points from the real-time image also includes: The horizontal distance of the point cloud points is calculated according to formula (4): Among them, X li is the horizontal distance of the ith point cloud point of the lth radar beam, H s is the absolute value of the height of the i-th point cloud point of the l-th radar beam, θ l is the vertical azimuth of the lth radar beam, θ a is the horizontal resolution of the laser radar.

6. The image processing method according to claim 1, characterized in that: Classifying the feature points includes: Obtaining the longest light beams in front of and behind the laser radar; The angle at which the longest light beam is located is regarded as a road dividing line; The feature points of the image are divided into left road feature points and right road feature points according to the road segmentation line.

7. The image processing method according to claim 1, characterized in that: Filtering the feature points includes: Acquire a feature image according to the feature points; filtering the feature image; The model was fitted using a random sampling consensus algorithm; Get the points in the fitted model whose distance is less than the threshold and use them as seed points.

8. The image processing method according to claim 7, characterized in that: Filtering the feature points further includes: Setting the seed point to an initial value; Construct Gaussian process regression algorithm GPR calculation model; The initial value is input into a Gaussian process regression algorithm (GPR) calculation model to obtain the road boundary in the real-time image.

Images