Radar and vision fusion matching method and device based on instance segmentation, and medium

Abstract

The application discloses a radar and vision fusion matching method and device based on instance segmentation, and a medium. The method comprises the following steps: acquiring visual image data of a target object and performing target instance segmentation to obtain a visual instance segmentation region; a sampling function with a probability distribution having a center probability lower than that of the periphery is used to perform random non-uniform sampling in the visual instance segmentation region, so that the sampling density of the center region is lower than that of the periphery region, and the sampling points are inside the segmentation mask; each sampling point obtained by sampling is used as a visual matching point; a plurality of radar matching points generated according to detection data of the target object by a radar are acquired; each visual matching point is matched with each radar matching point, and parameter information of a target point is obtained. The application can improve the accuracy and success rate of matching in the radar target and visual target detection process, and reduce the risk of missed detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a radar and vision fusion matching method based on instance segmentation, a device and a medium. BACKGROUND

[0002] In application fields such as automatic driving, intelligent monitoring and robot navigation, radar (such as millimeter wave radar) and vision sensors (such as cameras) are often used in combination to detect targets by using radar and vision sensors respectively, so as to make up for the shortcomings of a single sensor, and thus to achieve more accurate and comprehensive environmental perception and target identification. Among them, radar can provide stable distance and speed information, while a camera can provide rich texture and target appearance information. However, due to the different perception mechanisms of radar and camera, how to match the detection results of the two, and construct a high-precision multi-sensor fusion system, is the key to realizing high-precision target detection.

[0003] For matching and fusion between radar targets and targets detected by vision sensors (radar-vision fusion), the existing technology usually adopts a uniform sampling method based on target detection boxes, that is, a series of uniformly distributed sampling points are generated by uniformly sampling in the target detection box of the vision sensor according to a fixed form, and then the sampling points of the vision sensor are matched with the radar point cloud data to determine the matching relationship between the vision sampling points and the radar points. However, since the target detection of radar is based on electromagnetic wave reflection, the detected target point positions are usually randomly distributed, and can correspond to any point on the target surface, so if the target detection box of the vision sensor is sampled by using the uniform sampling method, when the radar detection points are mapped to the image, the target points can not be at the center position of the actual target, and the reflected target position can come from any position of the object, causing a large mapping error of the radar target points, and thus leading to matching deviation and reducing the matching accuracy. Especially when the appearance of the detected target is irregular, the traditional uniform sampling method based on target detection boxes can not sample the key part of the target, resulting in target mismatch, and thus causing missed detection and false alarm. If a low-resolution millimeter wave radar is used, it cannot provide complete target size information, and can also cause deviation in the matching method based on the target center point. For example, when the edge of the target enters the warning area but the center does not enter, it can cause a missed detection problem. SUMMARY

[0004] The technical problem to be solved by the present application is that in view of the technical problems existing in the prior art, the present application provides a radar and vision fusion matching method based on instance segmentation, a device and a medium, which can improve the accuracy and success rate of matching in the radar target and vision target detection process, and reduce the risk of missed detection.

[0005] To solve the above technical problems, the technical scheme provided by the present application is:

[0006] A radar and vision fusion matching method based on instance segmentation, comprising the following steps:

[0007] Obtaining visual image data of a target object and performing target instance segmentation to obtain a visual instance segmentation region;

[0008] In the visual instance segmentation region, a sampling function with a probability distribution having a center probability lower than that of the periphery is used to perform random non-uniform sampling, so that the sampling density of the center region is lower than that of the periphery region and the sampling points are inside the segmentation mask, and the multiple sampling points obtained by sampling are used as visual matching points;

[0009] Obtaining multiple radar matching points generated according to the detection data of the target object by radar, matching each visual matching point with each radar matching point, and obtaining the parameter information of the target point.

[0010] Further, in the visual instance segmentation region, random non-uniform sampling is performed according to a sampling function of any one of an inverse Gaussian distribution function, a uniform ring distribution, and a radial power distribution, so that the sampling density of the center region is lower than that of the periphery region.

[0011] Further, the inverse Gaussian distribution function is an inverse Gaussian function is constructed, and the probability function of the inverse Gaussian distribution function is is:

[0012]

[0013]

[0014] wherein, is a correction term, is a sample mean position, is a sample standard deviation.

[0015] Further, in the random non-uniform sampling in the visual instance segmentation region, the sample mean position is also set according to the center point position of the instance segmentation region.

[0016] Further, the number of visual matching points is determined according to radar performance parameters, target size parameters, and target type parameters.

[0017] Further, in the random non-uniform sampling in the visual instance segmentation area, a rotation number method is used to determine whether the sampling point is in the segmentation mask, wherein, by traversing all edges of a polygon formed by the outer contour of the segmentation mask, a rotation number is calculated according to the positional relationship between the current sampling point and the edge, and whether the current sampling point is inside the segmentation mask is determined according to the calculated rotation number.

[0018] Further, the step of using the rotation number method to determine whether the sampling point is in the segmentation mask comprises:

[0019] initializing the rotation number;

[0020] traversing each edge of the polygon formed by the outer contour of the segmentation mask, if a horizontal line passing through the sampling point is on the edge, determining whether the current sampling point is on the left side of the edge, and if so, increasing the rotation number; if a horizontal line passing through the sampling point is below the edge, determining whether the current sampling point is on the left side of the edge, and if so, decreasing the rotation number;

[0021] after traversing all edges, if the final rotation number is not 0, it is determined that the current sampling point is inside the segmentation mask, otherwise it is determined that the current sampling point is outside the segmentation mask.

[0022] Further, the parameter information of the target point includes first type parameter information obtained according to a visual image and second type parameter information obtained according to radar data, the first type parameter information includes classification category and segmentation mask information, and the second type parameter information includes any one or more of distance information, angle information and speed information.

[0023] A computer device comprises a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above method.

[0024] A computer readable storage medium storing a computer program, the computer program is executed by a processor to implement the above method.

[0025] Compared with the prior art, the present application has the beneficial effects that: the present application realizes radar and vision fusion matching based on instance segmentation, obtains visual image data of the target object and performs target instance segmentation to obtain a visual instance segmentation area, adopts a random non-uniform sampling manner to perform random non-uniform sampling in the visual instance segmentation area, at the same time, the sampling density of the central area is lower than that of the surrounding area and the sampling points are inside the segmentation mask, and the plurality of sampling points obtained by sampling are matched with radar matching points as visual matching points, since the random non-uniform sampling manner is adopted, the distribution of the matching points can be more in line with the reflection characteristics of the radar, the radar target and the visual target are matched with the target points outside as much as possible, that is, the radar target tends to be matched to the edge points of the vision, the risk of missing detection of the irregular target can be avoided, and the matching success rate is improved. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 It is a sampling point distribution diagram in a specific application embodiment using a random uniform sampling manner.

[0027] Figure 2 It is an effect diagram on an image in a specific application embodiment using a random uniform sampling manner.

[0028] Figure 3 It is an implementation flow diagram of the radar and vision fusion matching method based on instance segmentation of the present embodiment.

[0029] Figure 4 It is an effect diagram of random sampling inside the instance segmentation area obtained in a specific application embodiment.

[0030] Figure 5 It is an inverse Gaussian function distribution diagram obtained in a specific application embodiment.

[0031] Figure 6 It is a sampling point distribution effect diagram obtained in a specific application embodiment using an inverse Gaussian function distribution.

[0032] Figure 7 It is a principle diagram for judging whether the generated points are inside the segmentation mask in a specific application implementation, wherein (a) corresponds to the points outside the polygon, and (b) corresponds to the points inside the polygon.

[0033] Figure 8 It is a complete flow diagram of the present application for realizing radar and visual target matching using an inverse Gaussian function and a number of rotations in a specific application embodiment.

[0034] Figure 9 It is a sampling result diagram obtained in a specific application embodiment of the present application. DETAILED DESCRIPTION

[0035] The application will be further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the application is not limited thereby.

[0036] For the convenience of understanding, first, the related technical background involved in the application is exemplarily introduced.

[0037] The radar and vision matching method based on instance segmentation is to combine instance segmentation and radar and vision sensor data, which can enrich target size information and realize accurate matching and recognition of the target. The steps of the radar and vision matching method based on instance segmentation can be divided into:

[0038] 1. Obtain the distance, speed and angle information of the target through the millimeter wave radar sensor, convert it into point cloud data in the radar coordinate system, and use the camera to collect image data, and perform instance segmentation on the target in the image through the instance segmentation algorithm to generate a segmentation mask for each target, and each mask identifies all the pixels of a target in the image;

[0039] 2. Extract the visual features of the target from the segmentation mask, and extract the radar features such as distance, speed, angle, radial velocity, etc. of the target from the radar point cloud data;

[0040] 3. Fuse the features extracted by vision and radar, and determine the matching relationship between the visual segmentation mask and the radar points according to the similarity measurement between the fused features.

[0041] The current instance segmentation-based matching method is usually based on random uniform sampling to select matching points within the target area, and the matching weight of the center point is often configured to be higher than that of the edge area to highlight the target semantic information of the center area. However, this type of method may not be able to sample the key parts of the target, resulting in target mismatch, and thus still has the risk of missing detection, especially in the case of irregular target or discrete radar point distribution.

[0042] Taking the random uniform sampling method for sampling the visual image target detection data of a target in a specific application embodiment as an example, the sampling point distribution diagram of the random uniform sampling method is as shown in Figure 1 , the sampling points are uniformly distributed in the entire target detection frame area, and the target point is at the center position. The sampling effect of the sampling method on the image is as shown in Figure 2 . As can be clearly seen from Figure 2 , since the sampling points are uniformly distributed, and the appearance of the target is irregular, there may be a lack of sampling points in some key parts of the target (for example, in the forearm of the target in Figure 2 ), which makes it impossible to sample the key parts and causes the target to mismatch, thereby causing missing detection.

[0043] In the process of matching the radar target with the visual image target, the position of the target point detected by the radar is randomly distributed, which can correspond to any point on the surface of the target, and the appearance shape of the target can be irregular. For such irregular target, the edge contour region needs to be detected as completely as possible to ensure that each key part can be detected. Meanwhile, compared with the missed detection, the false alarm has less influence on the target detection, that is, the acceptance of the false alarm is usually higher than that of the missed detection. The radar and visual fusion matching is realized based on instance segmentation in the application. First, the visual image data of the target object is acquired, and the target instance segmentation is performed to obtain a visual instance segmentation region. Random non-uniform sampling is performed in the visual instance segmentation region. Meanwhile, the sampling density of the central region is lower than that of the surrounding region, and the sampling points are inside the segmentation mask. The plurality of sampling points obtained by sampling are matched with the radar matching points. Since the random non-uniform sampling is adopted, the distribution of the matching points can be more consistent with the reflection characteristics of the radar. The radar target and the visual target are matched with the target points on the outside (the surrounding region of the target detection frame) as much as possible. Therefore, the radar target tends to be matched to the edge points of the visual target, which can avoid the missed detection of the irregular target, greatly reduces the risk of missed detection, and improves the matching success rate.

[0044] The application will be further described below in combination with specific embodiments.

[0045] As shown in the figure, the steps of the radar and visual fusion matching method based on instance segmentation in the embodiment include: Figure 3

[0046] Step S01. Acquire the visual image data of the target object and perform target instance segmentation to obtain a visual instance segmentation region.

[0047] In the embodiment, the image data of the target object can be acquired based on a high-resolution optical camera to ensure that the target is clear and complete. Then, the collected data is preprocessed, for example, including picture resolution setting, data labeling and segmentation model building.

[0048] Preferably, the YOLO framework can be used as a segmentation model for target instance segmentation. Based on the existing pre-trained parameters on the COCO dataset, the network model of real-time instance segmentation with high recognition rate is realized by fine-tuning on the local dataset. Compared with the traditional target detection frame method, the YOLO framework is used for target instance segmentation, which can more accurately acquire the boundary information of the target. The efficiency and accuracy of the segmentation are higher, and the segmentation result can provide the complete contour of the target, not just a rectangular frame, thereby further improving the matching accuracy and reducing the matching error.

[0049] ​Taking the target instance segmentation using the YOLO framework as an example, the generated segmentation data is in multiple lines of text, and each line of data corresponds to an instance, which can be organized in the following format, for example:

[0050]

[0051] wherein class-index is the index of the class, The boundary coordinates of the segmentation mask are n coordinates.

[0052] For convenience of expression, let the vertex sequence be wherein , is connected to , is connected to . Connecting the coordinates in order is the outer contour of the instance segmentation mask, and the target category is output at the same time.

[0053] Then the pixel coordinates are established in the image, with the top left corner of the picture as (0, 0) and the bottom right corner as , which is equal in value to the pixel size of the picture , and the horizontal minimum value, horizontal maximum value, vertical minimum value and vertical maximum value of the instance segmentation mask characterize the outermost region of the target, that is:

[0054] (1)

[0055] In this embodiment, the mask center point is set as the center point position, that is:

[0056] (2)

[0057] In a specific application embodiment, the above method is used for instance segmentation of a collected visual image, and the model can directly output the region of the target in the picture and provide the target classification category and classification confidence.

[0058] It can be understood that other types of instance segmentation deep learning networks can also be used to implement target instance segmentation according to actual needs, such as Mask R-CNN, SOLOv2, etc., or other types of instance segmentation algorithms such as U-Net can be used.

[0059] Step S02. Random non-uniform sampling is performed in the visual instance segmentation region, so that the sampling density of the central region is lower than that of the peripheral region and the sampling points are inside the segmentation mask. The multiple sampling points obtained by sampling are used as visual matching points.

[0060] After obtaining the visual instance segmentation region, further sampling is performed in the visual instance segmentation region to determine the visual matching point. In order to match the radar target and the visual target as much as possible to the target point outside the target detection frame (the region around the target detection frame), so that the radar target tends to match to the edge point of the vision, the embodiment adopts a random non-uniform sampling manner to sample, and at the same time, the sampling density of the central region is lower than that of the surrounding region and the sampling points are inside the segmentation mask, so that the distribution of the matching points is more in line with the reflection characteristics of the radar, the edge contour region is detected as completely as possible, it is ensured that each key part can be detected, and the risk of missing detection of irregular targets is avoided, so as to reduce the risk of missing detection and improve the matching success rate.

[0061] Specifically, when internally randomly sampling, the embodiment performs "random point scattering" on the polygon inside the segmentation mask to simulate the reflection target point position of the radar as the visual matching point to match the real millimeter wave radar target. By creating a density function, the value at the center point is small, and gradually increases towards the edge, so as to meet the requirement of "high periphery and low center". For example, random non-uniform sampling can be performed in the visual instance segmentation region according to an inverse Gaussian distribution function, so that the sampling density of the central region is lower than that of the surrounding region, that is, it has the characteristics of "high periphery and low center".

[0062] For any radially symmetric two-dimensional probability density function:

[0063] (3)

[0064] If it is required to have the characteristics of "high periphery and low center", the following conditions need to be met:

[0065] 1. Smoothness: At least two orders of derivative within the required range.

[0066] 2. The low density at the center satisfies:

[0067] (4)

[0068] 3. Monotonically increasing from the center to :

[0069] (5)

[0070] Therefore, in addition to using the inverse Gaussian distribution function, other types of probability distribution functions that meet the above conditions can also be used, such as uniform ring distribution, radial power distribution, etc., so that the sampling points have the characteristics of "high periphery and low center".

[0071] The following takes random sampling using the inverse Gaussian distribution function as an example. The embodiment can construct the inverse Gaussian function as follows:

[0072] (6)

[0073] wherein, is a probability function, is a logarithm function, is a sample mean position, is a sample standard deviation, and let the inverse of the probability function be taken and a bias be added to obtain the inverse Gaussian probability function :

[0074] (7)

[0075] wherein, is a correction term, .

[0076] That is, random sampling is performed in the visual instance segmentation region so as to make the sampling points conform to the inverse Gaussian probability function distribution. In the random non-uniform sampling, the sample mean position is set according to the center point position of the instance segmentation region, and specifically, the sample mean may be set as By introducing the correction term , the center weight can be prevented from being too small.

[0077] In the embodiment, the standard deviation follows the Gaussian distribution criterion, and the Gaussian distribution criterion is used to describe the probability of data distribution around the mean, wherein the calculation expression of the standard deviation is as follows:

[0078] (8)

[0079] wherein is an adjustable proportion coefficient.

[0080] In the embodiment, the adjustable proportion coefficient is introduced into the inverse Gaussian distribution function to calculate the standard deviation, so that the distribution of the matching points can be adjusted according to different data distribution states, and in combination with the correction term , the matching points can be more in line with the reflection characteristics of the radar, so that the radar target is more inclined to match to the edge points of the vision, to avoid missing detection.

[0081] The effect of the above-mentioned manner on random sampling in the instance segmentation region in the specific application embodiment is as shown in Figure 4 . As can be seen from the figure, the sampling points are more distributed in the edge region, which can be more in line with the characteristics of the appearance of the irregular target, so that the edge contour region of the entire target can be sampled, and then the sampling points are used as the visual matching points to match the point targets collected by the radar, so that the success rate of matching can be improved.

[0082] The embodiment can make the distribution of the matching points present the characteristics of high in the periphery and low in the center by the random sampling method according to the inverse Gaussian distribution function in the target region of the instance segmentation. The sampling point distribution obtained by using the inverse Gaussian distribution function for sampling in the specific application embodiment is shown in FIG. 8. Figure 5 、 Figure 6 The sampling point distribution can be more consistent with the reflection characteristics of the radar, so that the radar target tends to match to the points on the edge of the vision, greatly reduces the risk of missing detection, and further improves the matching success rate.

[0083] In the embodiment, the number of the visual matching points can be determined according to the radar performance parameter, the target size parameter and the target type parameter, so as to comprehensively consider the factors such as the distance resolution, the angle resolution, the target size and the target type of the radar, dynamically adjust the number of the sampling points, make the matching more reasonable, and thus improve the matching accuracy under different target categories.

[0084] Specifically, the matching point number calculation model can be constructed according to the radar performance parameter, the target size parameter and the target type parameter, and the visual matching point number is calculated by using the matching point number calculation model. For example, the matching point number calculation model can be configured as:

[0085] (9)

[0086] wherein, N is the generated visual matching point number, is the radar performance parameter, which is related to the distance resolution and the angle resolution of the radar; is the target size parameter, which is proportional to the area of the segmentation region, is the target type parameter, which is related to the material and type of the target outer surface.

[0087] In the embodiment, the random non-uniform sampling in the visual instance segmentation region also includes using the number of revolutions method to determine whether the sampling point is in the segmentation mask, wherein the number of revolutions is calculated according to the position relationship between the current sampling point and the edge by traversing all edges of the polygon formed by the outer contour of the segmentation mask, and it is determined whether the current sampling point is inside the segmentation mask according to the calculated number of revolutions.

[0088] Specifically, the step of using the number of revolutions method to determine whether the sampling point is in the segmentation mask includes:

[0089] initializing the number of revolutions;

[0090] If the horizontal line of the edge passes through the sampling point, it is determined whether the current sampling point is on the left side of the edge, and if so, the number of turns is increased; if the horizontal line of the edge passes below the sampling point, it is determined whether the current sampling point is on the left side of the edge, and if so, the number of turns is decreased.

[0091] After all the edges are traversed, if the final number of turns is not 0 or exceeds a preset threshold, it is determined that the current sampling point is inside the segmentation mask, otherwise it is determined to be outside the segmentation mask.

[0092] The embodiment can be applied to various complex segmentation masks by using the number of turns method to determine whether the sampling point is inside the segmentation mask, and the accuracy and robustness of the determination are improved.

[0093] For a given point , all edges of the segmentation mask polygon are traversed, the angle change from the point to each vertex of the polygon is calculated, and the number of turns of the point is obtained by accumulating these angle changes, so as to determine whether the point is inside the polygon segmentation mask. The number of turns can be defined as , that is, the number of turns of the polygon boundary around the point P, when the horizontal line passes through an edge, it is equivalent to the vector from “almost horizontal” around the edge vertex to occur a small, directed angle jump, so the accumulation of all jumps is the complete (or 0) rotation. In the accumulation process, if the polygon vertices are in counterclockwise order, the accumulated value is positive when the angle method (determine the number of turns by calculating the angle turned during rotation) is used; when the ray method (calculate the number of turns by calculating the relative position change of the ray from a fixed point to a point on the rotating object) is used, 1 is added for the “upward passing” case, and similarly, “downward passing” corresponds to negative rotation, so 1 is subtracted. The principle of determining whether the generated point is inside the segmentation mask is shown in Figure 7 .

[0094] In a specific application embodiment, the number of turns method is used to determine whether the sampling point is inside the segmentation mask, and the ray method is used to accumulate the angle, and the detailed steps include:

[0095] 1) Initialize the number of turns .

[0096] 2) Traverse each edge of the polygon , where :

[0097] If , it indicates that the horizontal line of the edge passes through the point, and it is determined whether is on the left side of the edge, and if so, the number of turns is increased; if the horizontal line of the edge passes below the sampling point, it is determined whether the current sampling point is on the left side of the edge, and if so, the number of turns is decreased. wherein let if then determine is on the left of the edge, otherwise determine not on the left of the edge.

[0098] if then it means that the edge goes under the horizontal line of the point determine whether it is on the left of the edge, if yes, then turn the number ;

[0099] 3) after traversing all the edges, if the final turn number then the point is inside the polygon, otherwise outside.

[0100] The following is an example of using the inverse Gaussian function and the turn number method to randomly sample in a segmentation mask with a specific probability distribution, as shown in Figure 8 , the detailed steps are as follows:

[0101] Step 2.1: randomly generate two-dimensional data in the range

[0102] Step 2.2: calculate the probability function using the inverse Gaussian distribution , generate a uniform random number , where .

[0103] Step 2.3: if , keep the point and go to the next step, otherwise discard.

[0104] Step 2.4: use the turn number method to determine whether the generated point is located inside the polygon, if yes, add the point to the set , otherwise discard.

[0105] Step 2.5: get all points that meet the conditions until , the algorithm ends. Where is the number of elements in the calculation set.

[0106] The effect of using the above sampling method in the specific application example is shown in Figure 9 .

[0107] It can be understood that in addition to the above method, other methods such as the ray method and the angle method can also be used to determine whether the sampling point is in the segmentation area according to actual needs.

[0108] Step S03. Obtain a plurality of radar matching points generated according to the detection data of the target object by the radar, match each visual matching point with each radar matching point, and obtain the parameter information of the target point.

[0109] In this embodiment, the parameter information of the target point specifically includes visual parameter information obtained according to a visual image and radar parameter information obtained according to radar data, wherein the visual parameter information includes a classification category and segmentation mask information, and the radar parameter information includes any one or more of distance information, angle information and speed information.

[0110] Specifically, an instance segmentation region in the visual image is sampled points as visual matching points, and the millimeter-wave radar detection generates radar matching points, the visual target and the target sampled by the millimeter-wave radar are matched based on a matching algorithm such as the Hungarian algorithm, and the parameter information of the actual target point is approximately obtained , wherein is distance information, is angle information, is speed information, is spatial motion parameter derived from the millimeter-wave radar, is a classification category, is a segmentation mask set containing boundary coordinate pairs of all segmentation masks, wherein is a contour and appearance feature derived from the visual, and the target matching of the radar and the visual is completed.

[0111] The present application realizes the fusion matching of radar and visual sensor targets by combining instance segmentation and random non-uniform sampling methods, can reduce the computational complexity under the premise of ensuring the matching accuracy, and enables the radar-visual matching to realize real-time operation under limited computing resources, and can be applied to various real-time application scenarios such as automatic driving, unmanned aerial vehicle perception and intelligent monitoring.

[0112] The embodiment further provides a computer device including a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to perform the method as described above.

[0113] It can be understood that the above method of the embodiment can be executed by a single device, such as a computer or a server, or can be applied to a distributed scenario and completed by multiple devices in cooperation. In the distributed scenario, one of the multiple devices can only execute one or more steps in the above method of the embodiment, and the multiple devices interact to complete the above method. The processor can be implemented in a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, and is configured to execute a related program to implement the above method of the embodiment. The memory can be implemented in a read-only memory (ROM), a random access memory (RAM), a static storage device, and a dynamic storage device. The memory can store an operating system and other application programs. When the above method of the embodiment is implemented by software or firmware, the related program code is stored in the memory and executed by the processor.

[0114] The embodiment further provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the above method.

[0115] Those skilled in the art should understand that the above-mentioned embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code. The present application is described with reference to flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions that are executed by the processor of the computer or other programmable data processing apparatus generate an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus with a function specified in one or more flows and / or blocks. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0116] The above merely preferred embodiments of the present application and not intended to limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present application without departing from the technical solution of the present application shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A radar and vision fusion matching method based on instance segmentation, characterized in that the steps are as follows: include: Obtain visual image data of the target object and perform target instance segmentation to obtain visual instance segmentation regions; Random non-uniform sampling is performed within the visual instance segmentation area using a sampling function with a probability distribution that has a lower probability in the center than in the surrounding areas, so that the sampling density in the center area is lower than that in the surrounding areas, and the sampling points are inside the segmentation mask. Multiple sampling points obtained by sampling are used as visual matching points. Multiple radar matching points generated based on radar detection data of the target object are obtained, and each visual matching point is matched with each radar matching point to obtain the parameter information of the target point; Within the visual instance segmentation region, random non-uniform sampling is performed according to any one of the sampling functions: inverse Gaussian distribution, uniform ring distribution, and radial power distribution, so that the sampling density of the central region is lower than that of the surrounding regions.

2. The radar and vision fusion matching method based on instance segmentation according to claim 1, characterized in that, The inverse Gaussian distribution function is obtained by using the inverted Gaussian function. The probability function of the inverse Gaussian distribution function is constructed. for: in, For correction items, The location of the sample mean. This represents the sample standard deviation.

3. The radar and vision fusion matching method based on instance segmentation according to claim 1, characterized in that, In performing random non-uniform sampling within the visual instance segmentation region, the method also includes setting the sample mean position based on the center point position of the instance segmentation region.

4. The radar and vision fusion matching method based on instance segmentation according to claim 1, characterized in that, The number of visual matching points is determined based on radar performance parameters, target size parameters, and target type parameters.

5. The radar and vision fusion matching method based on instance segmentation according to any one of claims 1 to 4, characterized in that, In the random non-uniform sampling within the visual instance segmentation region, the rotation number method is used to determine whether the sampling point is inside the segmentation mask. Specifically, by traversing all the edges of the polygon formed by the outer contour of the segmentation mask, the rotation number is calculated based on the positional relationship between the current sampling point and the edge. The calculated rotation number is then used to determine whether the current sampling point is inside the segmentation mask.

6. The radar and vision fusion matching method based on instance segmentation according to claim 5, characterized in that, The step of determining whether a sampling point is within the segmentation mask using the gyration method includes: Initialize the rotation number; Traverse each edge of the polygon formed by the outer contour of the segmentation mask. If the edge crosses the horizontal line of the sampling point, determine whether the current sampling point is to the left of the edge. If so, increase the number of revolutions. If the edge crosses the horizontal line of the sampling point, determine whether the current sampling point is to the left of the edge. If so, decrease the number of revolutions. After traversing all edges, if the final rotation number is not 0, the current sampling point is determined to be inside the segmentation mask; otherwise, it is determined to be outside the segmentation mask.

7. The radar and vision fusion matching method based on instance segmentation according to any one of claims 1 to 4, characterized in that, The parameter information of the target point includes visual parameter information obtained from visual images and radar parameter information obtained from radar data. The visual parameter information includes classification category and segmentation mask information, and the radar parameter information includes any one or more of distance information, angle information and velocity information.

8. A computer device comprising a processor and a memory, the memory being used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images