Radar and vision fusion matching method and device based on instance segmentation and medium

Abstract

The invention discloses a radar and vision fusion matching method and device based on instance segmentation, and a medium. The method comprises the following steps: obtaining visual image data of a target object, and carrying out target instance segmentation to obtain a visual instance segmentation region; carrying out random non-uniform sampling in the visual instance segmentation area by adopting a sampling function with the central probability lower than the probability distribution of the periphery so as to enable the sampling density of the central area to be lower than the sampling density of the periphery area and the sampling points to be in the segmentation mask, and taking the sampling points obtained by sampling as visual matching points; and obtaining a plurality of radar matching points generated according to detection data of the radar on the target object, and matching each visual matching point with each radar matching point to obtain parameter information of the target point. According to the invention, the matching accuracy and success rate in the radar target amount and visual target detection process can be improved, and the missed detection risk is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a radar and vision fusion matching method based on instance segmentation, a device and a medium. BACKGROUND

[0002] In application fields such as automatic driving, intelligent monitoring and robot navigation, radar (such as millimeter wave radar) and vision sensors (such as cameras) are often used in combination to detect targets by using radar and vision sensors respectively, so as to make up for the shortcomings of a single sensor, and thus to achieve more accurate and comprehensive environmental perception and target identification. Among them, radar can provide stable distance and speed information, while a camera can provide rich texture and target appearance information. However, due to the different perception mechanisms of radar and camera, how to match the detection results of the two, and construct a high-precision multi-sensor fusion system, is the key to realizing high-precision target detection.

[0003] For matching and fusion between radar targets and targets detected by vision sensors (radar-vision fusion), the existing technology usually adopts a uniform sampling method based on target detection boxes, that is, a series of uniformly distributed sampling points are generated by uniformly sampling in the target detection box of the vision sensor according to a fixed form, and then the sampling points of the vision sensor are matched with the radar point cloud data to determine the matching relationship between the vision sampling points and the radar points. However, since the target detection of radar is based on electromagnetic wave reflection, the detected target point positions are usually randomly distributed, and can correspond to any point on the target surface, so if the target detection box of the vision sensor is sampled by using the uniform sampling method, when the radar detection points are mapped to the image, the target points can not be at the center position of the actual target, and the reflected target position can come from any position of the object, causing a large mapping error of the radar target points, and thus leading to matching deviation and reducing the matching accuracy. Especially when the appearance of the detected target is irregular, the traditional uniform sampling method based on target detection boxes can not sample the key part of the target, resulting in target mismatch, and thus causing missed detection and false alarm. If a low-resolution millimeter wave radar is used, it cannot provide complete target size information, and can also cause deviation in the matching method based on the target center point. For example, when the edge of the target enters the warning area but the center does not enter, it can cause a missed detection problem. SUMMARY

[0004] The technical problem to be solved by the present application is that in view of the technical problems existing in the prior art, the present application provides a radar and vision fusion matching method based on instance segmentation, a device and a medium, which can improve the accuracy and success rate of matching in the radar target and vision target detection process, and reduce the risk of missed detection.

[0005] To solve the above technical problems, the technical scheme provided by the present application is: A radar and vision fusion matching method based on instance segmentation, comprising the following steps: Obtaining visual image data of a target object and performing target instance segmentation to obtain a visual instance segmentation region; In the visual instance segmentation region, a sampling function with a probability distribution having a center probability lower than that of the periphery is used to perform random non-uniform sampling, so that the sampling density of the center region is lower than that of the periphery region and the sampling points are inside the segmentation mask, and the plurality of sampling points obtained by sampling are used as visual matching points; Obtaining a plurality of radar matching points generated according to the detection data of the target object by radar, matching each visual matching point with each radar matching point, and obtaining the parameter information of the target point.

[0006] Further, in the visual instance segmentation region, a sampling function of any one of an inverse Gaussian distribution function, a uniform ring distribution, and a radial power distribution is used to perform random non-uniform sampling, so that the sampling density of the center region is lower than that of the periphery region.

[0007] Further, the inverse Gaussian distribution function is a reverse Gaussian function is constructed, and the probability function of the inverse Gaussian distribution function is is:

[0008]

[0009] wherein, is a correction term, is a sample mean position, is a sample standard deviation.

[0010] Further, in the random non-uniform sampling in the visual instance segmentation region, the sample mean position is set according to the center point position of the instance segmentation region.

[0011] Further, the number of visual matching points is determined according to radar performance parameters, target size parameters, and target type parameters.

[0012] Further, in the random non-uniform sampling in the visual instance segmentation region, the number of visual matching points is determined according to radar performance parameters, target size parameters, and target type parameters.

[0013] Further, the step of determining whether the sampling point is inside the segmentation mask by using the number of turns comprises: initializing the number of turns; traversing each side of the polygon formed by the outer contour of the segmentation mask, if the horizontal line of the side passes through the sampling point, determining whether the current sampling point is on the left side of the side, if yes, increasing the number of turns; if the horizontal line of the side passes below the sampling point, determining whether the current sampling point is on the left side of the side, if yes, decreasing the number of turns; after traversing all the sides, if the final number of turns is not 0, determining that the current sampling point is inside the segmentation mask, otherwise, determining that the current sampling point is outside the segmentation mask.

[0014] Further, the parameter information of the target point comprises first type parameter information obtained according to the visual image and second type parameter information obtained according to the radar data, the first type parameter information comprises classification category and segmentation mask information, and the second type parameter information comprises any one or more of distance information, angle information and speed information.

[0015] A computer device comprises a processor and a memory, the memory is used for storing a computer program, and the processor is used for executing the computer program to perform the above method.

[0016] A computer readable storage medium storing a computer program, the computer program is executed by a processor to implement the above method.

[0017] Compared with the prior art, the beneficial effects of the present application are as follows: the present application realizes radar and visual fusion matching based on instance segmentation, obtains visual image data of a target object and performs target instance segmentation to obtain a visual instance segmentation region, adopts a random non-uniform sampling manner to perform random non-uniform sampling in the visual instance segmentation region, simultaneously makes the sampling density of the central region lower than that of the peripheral region and the sampling points inside the segmentation mask, takes the plurality of sampling points obtained by sampling as visual matching points and radar matching points to perform matching, since the random non-uniform sampling manner is adopted, the distribution of the matching points can be more consistent with the reflection characteristics of the radar, the radar target and the visual target are matched to the target points outside as much as possible, that is, the radar target tends to be matched to the points on the edge of the visual target, the risk of missing detection of the target with irregular appearance can be avoided, and the matching success rate is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 It is a sampling point distribution diagram in a specific application embodiment using a random uniform sampling manner.

[0019] Figure 2 It is an effect diagram on an image in a specific application embodiment using a random uniform sampling manner.

[0020] Figure 3 is the implementation flow diagram of the radar and visual fusion matching method based on instance segmentation of the embodiment.

[0021] Figure 4 is the effect diagram of random sampling in the instance segmentation region obtained in the specific application embodiment.

[0022] Figure 5 is the inverse Gaussian function distribution diagram obtained in the specific application embodiment.

[0023] Figure 6 is the sampling point distribution effect diagram obtained by using the inverse Gaussian function distribution in the specific application embodiment.

[0024] Figure 7 is the principle diagram for judging whether the generated point is inside the segmentation mask, wherein (a) corresponds to the point outside the polygon, and (b) corresponds to the point inside the polygon.

[0025] Figure 8 is the complete flow diagram of the radar and visual target matching realized by using the inverse Gaussian function and the number of revolutions method in the specific application embodiment of the application.

[0026] Figure 9 is the sampling result diagram obtained in the specific application embodiment of the application. DETAILED DESCRIPTION

[0027] The application will be further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the application is not limited thereby.

[0028] For the convenience of understanding, first, the related technical background involved in the application is exemplarily introduced.

[0029] The radar and visual matching method based on instance segmentation can enrich the target size information by combining instance segmentation and radar and visual sensor data, and can realize accurate matching and recognition of the target. The steps of the radar and visual matching method based on instance segmentation can be divided into: 1. The distance, speed and angle information of the target is obtained by the millimeter wave radar sensor, which is converted into point cloud data in the radar coordinate system, and the image data is collected by the camera, and the instance segmentation algorithm is used to perform instance segmentation on the target in the image to generate a segmentation mask for each target, and each mask identifies all the pixels of a target in the image; 2. The visual features of the target are extracted from the segmentation mask, and the radar features such as distance, speed, angle, radial velocity of the target are extracted from the radar point cloud data; 3. Fusing the features extracted by vision and radar, determining the matching relationship between the visual segmentation mask and the radar points according to the similarity measurement between the fused features.

[0030] Currently, the instance segmentation matching method is usually based on random uniform sampling to select matching points in the target region, and the matching weight of the center point is usually configured to be higher than that of the edge region to highlight the target semantic information of the center region. However, this method may not sample the key parts of the target, resulting in target mismatch, and thus still has the risk of missing detection, especially when the target is irregular or the radar points are distributed discretely.

[0031] Taking the random uniform sampling method for sampling the visual image target detection data of a certain target in a specific application embodiment as an example, the sampling point distribution diagram of the random uniform sampling method is shown in FIG. 1, wherein the sampling points are uniformly distributed in the entire target detection frame region, and the target point is at the center position. The sampling effect on the image using this sampling method is shown in FIG. 2. Figure 1 As can be clearly seen from FIG. 1, since the sampling points are uniformly distributed, and the appearance of the target is irregular, there may be a lack of sampling points in some key parts of the target (for example, the forearm of the target in FIG. 1), which makes it impossible to sample the key parts and results in target mismatch, thereby causing missing detection. Figure 2 Figure 2 As can be clearly seen from FIG. 1, since the sampling points are uniformly distributed, and the appearance of the target is irregular, there may be a lack of sampling points in some key parts of the target (for example, the forearm of the target in FIG. 1), which makes it impossible to sample the key parts and results in target mismatch, thereby causing missing detection. Figure 2

[0032] Considering that in the process of matching the radar target with the visual image target, the target point position detected by the radar is randomly distributed, which may correspond to any point on the target surface, and the appearance shape of the target may be irregular. For such irregular targets, it is necessary to detect the edge contour region as completely as possible to ensure that each key part can be detected. Meanwhile, considering that compared with missing detection, false alarm has less impact on target detection, that is, the acceptance of false alarm is usually higher than that of missing detection, the present application realizes radar and visual fusion matching based on instance segmentation. First, visual image data of a target object is obtained and target instance segmentation is performed to obtain a visual instance segmentation region. Then, a random non-uniform sampling method is used for random non-uniform sampling in the visual instance segmentation region, so that the sampling density of the center region is lower than that of the surrounding region, and the sampling points are inside the segmentation mask. Finally, the multiple sampling points obtained by sampling are matched with the radar matching points as visual matching points. Since the random non-uniform sampling method is used, the distribution of the matching points can better conform to the reflection characteristics of the radar, and the target points of the radar target and the visual target on the outside (the surrounding region of the target detection frame) are matched as much as possible, so that the radar target tends to match the points on the edge of the vision, which can avoid missing detection of irregular targets and greatly reduce the risk of missing detection, thereby improving the matching success rate.

[0033] ​​The application will be further described below in conjunction with specific embodiments.

[0034] As shown in the embodiment based on the radar and visual fusion matching method of instance segmentation, the steps include: Figure 3 Step S01. Obtain visual image data of the target object and perform target instance segmentation to obtain a visual instance segmentation region.

[0035] In this embodiment, image data of the target object can be obtained based on a high-resolution optical camera to ensure that the target is clear and complete, and then the collected data is preprocessed, for example, including picture resolution setting, data labeling, and segmentation model building.

[0036] Preferably, the YOLO framework can be used as a segmentation model for target instance segmentation, based on the existing pre-trained parameters on the COCO dataset, fine-tuned on the local dataset, to realize a high-identification-rate real-time instance segmentation network model. Compared with the traditional target detection frame method, the YOLO framework for target instance segmentation can more accurately obtain the boundary information of the target, and the efficiency and accuracy of segmentation are higher, and the segmentation result can provide the complete contour of the target, rather than just a rectangular frame, thereby further improving the matching accuracy and reducing the matching error.

[0037] Taking the YOLO framework for target instance segmentation as an example, the generated segmentation data is a plurality of lines of text, and each line of data corresponds to an instance, which can be organized in the following format, for example:

[0038] Wherein, class-index is the index of the class, The boundary coordinates of the segmented mask are n coordinates.

[0039] For convenience of description, let the vertex sequence be Wherein , is connected to , is connected to . Connecting the coordinates in order is the outer contour of the instance segmentation mask, and the target class is output at the same time.

[0040] Then establish the pixel coordinates in the image, with the top left corner of the picture as (0, 0) and the bottom right corner as , which is equal in value to the pixel size of the picture , using the horizontal minimum value, horizontal maximum value, vertical minimum value, and vertical maximum value of the instance segmentation mask to represent the outermost region of the target, that is: (1)​ In this embodiment, the mask center point is set The center point position is, that is, (2) In a specific application embodiment, the above method is used for instance segmentation of the collected visual image. The model can directly output the area where the target is located in the picture, and provide target classification categories and classification confidence.

[0041] It can be understood that other types of instance segmentation deep learning networks can also be used to implement target instance segmentation according to actual needs, such as Mask R-CNN, SOLOv2, etc., or other types of instance segmentation algorithms such as U-Net can be used.

[0042] Step S02. Random non-uniform sampling is performed in the visual instance segmentation area, so that the sampling density of the central region is lower than that of the peripheral region, and the sampling points are inside the segmentation mask. The plurality of sampling points obtained by sampling are used as visual matching points.

[0043] After obtaining the visual instance segmentation area, further sampling is performed in the visual instance segmentation area to determine the visual matching points. In order to match the radar target and the visual target as much as possible to the target points outside the target detection frame (the peripheral region of the target detection frame), so that the radar target tends to match to the points on the edge of the vision, this embodiment uses a random non-uniform sampling method to sample, while the sampling density of the central region is lower than that of the peripheral region and the sampling points are inside the segmentation mask, so that the distribution of the matching points is more in line with the reflection characteristics of the radar, and the edge contour area is detected as completely as possible, ensuring that each key part can be detected, avoiding missing detection of irregular targets, thereby reducing the risk of missing detection and improving the matching success rate.

[0044] Specifically, when internally randomly sampling, the inside of the polygon of the segmentation mask is "randomly scattered" to simulate the reflection target point position of the radar as the visual matching point to match the real millimeter wave radar target. By creating a density function, the value at the center point is small, and gradually increases towards the edge, which can better meet the "high periphery, low center" requirement. For example, random non-uniform sampling can be performed according to an inverse Gaussian distribution function in the visual instance segmentation area, so that the sampling density of the central region is lower than that of the peripheral region, that is, it has the characteristics of "high periphery, low center".

[0045] For any radially symmetric two-dimensional probability density function: (3) If it is to have the characteristics of "high periphery, low center", the following conditions need to be met: 1. Smoothness: At least two orders of derivable in the required range.

[0046] 2. The low density at the center satisfies: (4) 3. Monotonically increasing from the center to : (5) Therefore, in addition to using the inverse Gaussian distribution function, other types of probability distribution functions that satisfy the above conditions can also be used, such as uniform ring distribution, radial power distribution, etc., to make the sampling points have the characteristics of "high in the periphery and low in the center".

[0047] The following is an example of random sampling using the inverse Gaussian distribution function. This embodiment can be constructed by inverting the Gaussian function: (6) wherein, is a probability function, is a logarithmic function, is a sample mean position, is a sample standard deviation, and let the inverse of the probability function be taken and add a bias to obtain the inverse Gaussian probability function : (7) wherein, is a correction term, .

[0048] That is, random sampling is performed within the visual instance segmentation region so as to conform to the above inverse Gaussian probability function distribution. In random non-uniform sampling, the sample mean position is set according to the center point position of the instance segmentation region, and specifically the sample mean can be set to By introducing the correction term , the center weight can be prevented from being too small.

[0049] In this embodiment, the standard deviation follows the Gaussian distribution criteria, which describes the probability of data distribution around the mean using the criteria, wherein the calculation expression of the standard deviation is: (8) wherein is an adjustable proportionality coefficient.

[0050] In this embodiment, the adjustable proportionality coefficient is introduced to calculate the standard deviation in the inverse Gaussian distribution function, which can be adjusted according to different data distribution states, in combination with the correction term The distribution of the matching points can be made to be more consistent with the reflection characteristics of the radar, so that the radar target is more likely to match to the edge points of the vision, to avoid missing detection.

[0051] In a specific application embodiment, the effect of the above-mentioned way of randomly sampling inside the instance segmentation region is as shown in Figure 4 As can be seen from the figure, the sampling points are more distributed in the edge region, which can be more consistent with the characteristics of the appearance of irregular targets, so that the edge contour region of the entire target can be sampled, and then using the sampling points as visual matching points and the points collected by the radar for matching can improve the success rate of matching.

[0052] In this embodiment, the random sampling method according to the inverse Gaussian distribution function in the target region segmented by the instance can make the distribution of the matching points present the characteristics of high periphery and low center. In a specific application embodiment, the sampling point distribution obtained by using the inverse Gaussian distribution function for sampling is as shown in Figure 5 , Figure 6 Using the sampling point distribution can be more consistent with the reflection characteristics of the radar, so that the radar target is more likely to match to the edge points of the vision, greatly reducing the risk of missing detection and further improving the success rate of matching.

[0053] In this embodiment, the number of visual matching points can be determined according to the radar performance parameters, target size parameters and target type parameters, so as to comprehensively consider the factors such as the distance and angle resolution of the radar, the target size and the target type, dynamically adjust the number of sampling points, make the matching more reasonable, and thus improve the matching accuracy under different target categories.

[0054] Specifically, a matching point number calculation model can be constructed according to the radar performance parameters, target size parameters and target type parameters, and the number of visual matching points is calculated by using the matching point number calculation model. For example, the matching point number calculation model can be configured as: (9) wherein, N is the number of generated visual matching points, is a radar performance parameter, which is related to the distance and angle resolution of the radar; is a target size parameter, which is proportional to the area of the segmentation region, is a target type parameter, which is related to the target surface material and type.

[0055] In the random non-uniform sampling in the visual instance segmentation region in the embodiment, the number of rotations is also used to determine whether the sampling point is in the segmentation mask, wherein the number of rotations is calculated according to the positional relationship between the current sampling point and the edge by traversing all edges of the polygon formed by the outer contour of the segmentation mask, and it is determined whether the current sampling point is inside the segmentation mask according to the calculated number of rotations.

[0056] Specifically, the step of determining whether the sampling point is in the segmentation mask by using the number of rotations includes: initializing the number of rotations; traversing each edge of the polygon formed by the outer contour of the segmentation mask, if the horizontal line passing through the sampling point is on the edge, it is determined whether the current sampling point is on the left side of the edge, and if so, the number of rotations is increased; if the horizontal line passing through the sampling point is below the edge, it is determined whether the current sampling point is on the left side of the edge, and if so, the number of rotations is decreased; after traversing all edges, if the final number of rotations is not 0 or exceeds a preset threshold, it is determined that the current sampling point is inside the segmentation mask, otherwise it is determined that the current sampling point is outside the segmentation mask.

[0057] By using the number of rotations to determine whether the sampling point is in the segmentation mask, the embodiment can be applied to various complex segmentation masks, and the accuracy and robustness of the determination are improved.

[0058] For a given point , all edges of the segmentation mask polygon are traversed, the angle change from the point to each vertex of the polygon is calculated, and the number of rotations of the point is obtained by accumulating these angle changes, so as to determine whether the point is inside the polygon segmentation mask. The number of rotations can be defined as , that is, the number of rotations of the polygon boundary around the point P, when the horizontal ray passes through an edge, it is equivalent to that the vector from “almost horizontal” around the vertex of the edge has a small, directed angle jump, so the accumulation of all jumps is the complete (or 0) rotation. In the accumulation process, if the polygon vertices are in counterclockwise order, the accumulated value is positive when the angle method (determine the number of rotations by calculating the angle turned during rotation) is used; when the ray method (calculate the number of rotations by calculating the relative position change of the ray from a fixed point to a point on the rotating object) is used, 1 is added for the case of “passing upwards”, and by analogy, negative rotation corresponds to subtracting 1. The principle of determining whether the generated point is inside the segmentation mask is shown in Figure 7 .

[0059] In a specific application embodiment, the way of determining whether the sampling point is in the segmentation mask based on the number of rotations and using the ray method to accumulate the angle includes the following detailed steps: 1) Initialize the number of turns .

[0060] 2) Traverse each edge of the polygon , where it is considered : If , it means that the horizontal line through the point crosses the edge, and it is determined whether is on the left side of the edge. If it is, the number of turns is , where , and if , it is determined that is on the left side of the edge, otherwise it is determined that it is not on the left side of the edge.

[0061] If , it means that the horizontal line through the point crosses the edge, and it is determined whether is on the left side of the edge. If it is, the number of turns is . 3) After traversing all edges, if the final number of turns is , the point is inside the polygon, otherwise it is outside.

[0062] The following uses the inverse Gaussian function and the number of turns method to randomly sample in the segmentation mask with a specific probability distribution in a specific application example, as shown in Figure 8 , the detailed steps are as follows: Step 2.1: Randomly generate two-dimensional data in the range

[0063] Step 2.2: Calculate the probability function using the inverse Gaussian distribution , generate a uniform random number , where .

[0064] Step 2.3: If , keep the point and go to the next step, otherwise discard it.

[0065] Step 2.4: Use the number of turns method to determine whether the generated point is located inside the polygon. If it is, add the point to the set , otherwise discard it.

[0066] Step 2.5: Get all points that meet the conditions , until , the algorithm ends. Where is the number of elements in the set.

[0067] The effect obtained by using the above sampling method in a specific application embodiment is as shown in Figure 9

[0068] It can be understood that, in addition to the above method, other methods such as the ray method and the angle method can also be used to determine whether the sampling point is in the segmentation region according to actual needs.

[0069] Step S03. Obtain a plurality of radar matching points generated according to the detection data of the target object by the radar, match each visual matching point with each radar matching point, and obtain the parameter information of the target point.

[0070] In this embodiment, the parameter information of the target point specifically includes visual parameter information obtained according to the visual image and radar parameter information obtained according to the radar data, wherein the visual parameter information includes a classification category and segmentation mask information, and the radar parameter information includes any one or more of distance information, angle information, and speed information.

[0071] Specifically, the instance segmentation region in the visual image is sampled points as visual matching points, and the millimeter wave radar detection generates radar matching points, the visual target and the target sampled by the millimeter wave radar are matched based on a matching algorithm such as the Hungarian algorithm, and the parameter information of the actual target point is approximately obtained , wherein is distance information, is angle information, is speed information, is a spatial motion parameter derived from the millimeter wave radar, is a classification category, is a segmentation mask set containing boundary coordinate pairs of all segmentation masks, wherein is a contour and appearance feature derived from the visual, and the radar and visual target matching is completed.

[0072] The present application realizes the fusion matching of radar and visual sensor targets by combining instance segmentation and random non-uniform sampling method, can reduce the computational complexity on the premise of ensuring the matching accuracy. The radar and visual matching can realize real-time operation under limited computing resources, and can be applied to various real-time application scenarios such as automatic driving, unmanned aerial vehicle perception, intelligent monitoring, etc.

[0073] The present embodiment further provides a computer device comprising a processor and a memory, the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above method.

[0074] ​It can be understood that the above method of the embodiment can be executed by a single device, such as a computer or a server, or can be applied to a distributed scenario and completed by multiple devices in cooperation. In the distributed scenario, one of the multiple devices can only execute one or more steps in the above method of the embodiment, and the multiple devices interact to complete the above method. The processor can be implemented in a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, and is configured to execute a related program to implement the above method of the embodiment. The memory can be implemented in a read-only memory (ROM), a random access memory (RAM), a static storage device, and a dynamic storage device. The memory can store an operating system and other application programs. When the above method of the embodiment is implemented by software or firmware, the related program code is stored in the memory and executed by the processor.

[0075] The embodiment further provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the above method.

[0076] Those skilled in the art should understand that the above-mentioned embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer readable storage media (including but not limited to a disk storage, a CD-ROM, an optical storage, etc.) containing computer usable program code. The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and a combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0077] The above merely preferred embodiments of the present application and not intended to limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification of the above embodiments made without departing from the technical solution of the present application, according to the technical essence of the present application, should fall within the scope of protection of the technical solution of the present application.

Claims

1. A radar and vision fusion matching method based on instance segmentation, characterized in that the steps of The method comprises the following steps: obtaining visual image data of a target object and performing target instance segmentation to obtain a visual instance segmentation region; performing random non-uniform sampling in the visual instance segmentation region by using a sampling function having a center probability lower than a peripheral probability, so that the sampling density of a center region is lower than the sampling density of a peripheral region, and the sampling points are inside the segmentation mask, and using the obtained multiple sampling points as visual matching points; obtaining multiple radar matching points generated according to radar detection data of the target object, matching each visual matching point with each radar matching point, and obtaining parameter information of the target point.

2. The instance segmentation based radar and vision fusion matching method according to claim 1, characterized in that, In the visual instance segmentation region, random non-uniform sampling is performed according to a sampling function of any one of an inverse Gaussian distribution function, a uniform ring distribution, and a radial power distribution, so that the sampling density of a center region is lower than the sampling density of a peripheral region.

3. The instance segmentation based radar and vision fusion matching method according to claim 2, characterized in that, The inverse Gaussian distribution function is a reversed Gaussian function The probability function of the inverse Gaussian distribution function is constructed as is wherein, is the correction term, is the sample mean position, is the sample standard deviation.

4. The instance segmentation based radar and vision fusion matching method according to claim 1, characterized in that, In the random non-uniform sampling in the visual instance segmentation region, the sample mean position is set according to the center point position of the instance segmentation region.

5. The instance segmentation based radar and vision fusion matching method according to claim 1, characterized in that, The number of the visual matching points is determined according to radar performance parameters, target size parameters, and target type parameters.

6. The instance segmentation based radar and vision fusion matching method according to any one of claims 1-5, characterized in that, In the random non-uniform sampling in the visual instance segmentation region, the number of turns method is used to determine whether the sampling point is inside the segmentation mask, wherein, by traversing all sides of a polygon formed by the outer contour of the segmentation mask, the number of turns is calculated according to the positional relationship between the current sampling point and the side, and it is determined whether the current sampling point is inside the segmentation mask according to the calculated number of turns.

7. The instance segmentation based radar and vision fusion matching method according to claim 6, characterized in that, The step of determining whether the sampling point is inside the segmentation mask by using the number of turns method comprises: initializing the number of turns; traversing each side of the polygon formed by the outer contour of the segmentation mask, if a horizontal line passing through the sampling point is on the side, it is determined whether the current sampling point is on the left side of the side, and if so, the number of turns is increased; if a horizontal line passing through the sampling point is below the side, it is determined whether the current sampling point is on the left side of the side, and if so, the number of turns is decreased; after traversing all sides, if the final number of turns is not 0, it is determined that the current sampling point is inside the segmentation mask, otherwise it is determined that the current sampling point is outside the segmentation mask.

8. The instance segmentation based radar and vision fusion matching method according to any one of claims 1-5, characterized in that, The parameter information of the target point comprises visual parameter information obtained according to visual image data and radar parameter information obtained according to radar data, the visual parameter information comprises classification categories and segmentation mask information, and the radar parameter information comprises any one or more of distance information, angle information, and speed information.

9. A computer device comprising a processor and a memory for storing a computer program, characterized in that, The processor is configured to execute the computer program to perform the method of any one of claims 1-8.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is configured to be executed by the processor to perform the method of any one of claims 1-8.

Images