Vehicle grounding point detection method and device
By integrating the perception results of vision sensors and radar sensing equipment, the position of the vehicle grounding point is optimized, and the problem of low labeling efficiency and accuracy of invisible points in the prior art is solved, and the accuracy of labeling and driving perception algorithm are improved. Prediction accuracy.
Patent Information
- Application Number
- CN202310522563.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-19
AI Technical Summary
The prior art has limitations in the labeling of vehicle grounding points, especially the accurate prediction and labeling efficiency of invisible points, resulting in a reduction in the distance measurement accuracy of occlusion and truncation targets.
By fusing the perception results of vision sensors and radar sensing devices, we determine the image coordinates and detection coordinates of the vehicle grounding point, and use internal and external fusion parameters to optimize the position of the grounding point, realize rectangular constraint correction, and improve the accuracy of labeling and efficiency.
Accurate detection and position correction of invisible points are realized, the accuracy and efficiency of labeling are improved, and the prediction accuracy of driving perception algorithm is enhanced.
Smart Images

Figure CN118980308A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information technology, and in particular to a method and device for detecting a vehicle grounding point. Background Art
[0002] The Advanced Driving Assistance System (ADAS) uses a variety of sensors installed on the car (millimeter wave radar, lidar, monocular / binocular cameras and satellite navigation) to sense the surrounding environment at any time during driving, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform systematic calculations and analysis, so as to allow the driver to be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving.
[0003] Normally, the driving perception algorithm of ADAS needs to use the position of the target grounding point for accurate ranging and close-range cut-in judgment. The grounding point prediction method based on deep learning technology requires a large number of manually labeled samples. Since the grounding point needs to be labeled with the specific position, orientation, occlusion and other information of each target key point, it is particularly important to use auxiliary labeling methods to improve labeling efficiency. According to the occlusion attribute, the grounding point can be divided into visible points and invisible points. In practical applications, the accurate prediction of invisible points plays an important role in the ranging of occluded and truncated targets. However, at present, the center point of the lower edge of the tire outer contour detection frame is usually used directly to roughly approximate the grounding point position, or the invisible grounding point is manually labeled. This method has limitations in labeling accuracy and efficiency. Summary of the invention
[0004] In view of this, the purpose of the embodiments of the present application is to provide a method and device for detecting a vehicle grounding point. This solution obtains a vehicle grounding point for auxiliary marking by fusing the perception results of different sensor devices. It can not only accurately determine the invisible points, but also correct the position of the grounding point through rectangular constraints, so that the position of the grounding point used for marking is more accurate and the marking efficiency is higher.
[0005] In a first aspect, a method for detecting a grounding point of a vehicle is provided, the method comprising:
[0006] Based on the visual image of the target vehicle acquired by the visual sensor, the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined;
[0007] Based on the detection data of the target vehicle collected by the target radar, the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined;
[0008] The target grounding point coordinates of the target vehicle for auxiliary marking are obtained by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
[0009] Optionally, the step of obtaining the coordinates of the target grounding point of the target vehicle for auxiliary marking by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates includes:
[0010] Determine internal fusion parameters; the internal fusion parameters include the weighted sum of angle fusion factors corresponding to all target ground points to be searched;
[0011] Based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system, an external fusion parameter is determined; the external fusion parameter includes a weighted sum value of distance fusion factors corresponding to all target grounding points to be searched;
[0012] The coordinates of the target ground point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition are determined as the coordinates of the target ground point for auxiliary marking of the target vehicle;
[0013] The target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
[0014] Optionally, the angle fusion factor corresponding to the target ground point to be searched includes: an absolute angle error of an inner angle corresponding to the search target ground point among the inner angles of a quadrilateral formed by sequentially connecting the target ground points to be searched;
[0015] The distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
[0016] Optionally, the step of determining the coordinates of the target ground point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition as the coordinates of the target ground point for auxiliary marking of the target vehicle includes:
[0017] In the search space of each target ground point to be searched, each target ground point to be searched is controlled to be searched in the horizontal and vertical directions respectively according to the same preset step parameters;
[0018] The target grounding point to be searched that minimizes the internal fusion parameter and / or the external fusion parameter is determined as the target grounding point coordinates of the target vehicle for auxiliary marking.
[0019] Optionally, the step of determining the image coordinates of the touchdown point of the target vehicle in the vehicle body coordinate system based on the visual image of the target vehicle acquired by the visual sensor includes:
[0020] Based on the image of the target vehicle acquired by the visual sensor, the coordinates of the grounding point of the target vehicle in the image coordinate system are determined;
[0021] The coordinates of the grounding point of the target vehicle in the image coordinate system are transformed to transform the coordinates of the grounding point of the target vehicle in the image coordinate system into the vehicle body coordinate system to obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0022] In a second aspect, a detection device for a vehicle grounding point is provided, the detection device comprising:
[0023] An image coordinate determination module determines the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the visual image of the target vehicle acquired by the visual sensor;
[0024] A detection coordinate determination module determines the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the detection data of the target vehicle collected by the target radar;
[0025] The fusion module obtains the target grounding point coordinates of the target vehicle for auxiliary marking by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
[0026] Optionally, the fusion module includes:
[0027] An internal parameter determination unit determines an internal fusion parameter; the internal fusion parameter includes a weighted sum of angle fusion factors corresponding to all target ground points to be searched;
[0028] An external parameter determination unit determines an external fusion parameter based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system; the external fusion parameter includes a weighted sum of distance fusion factors corresponding to all target grounding points to be searched;
[0029] A grounding point determination unit determines the coordinates of the target grounding point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition as the target grounding point coordinates for auxiliary marking of the target vehicle;
[0030] The target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
[0031] Optionally, the angle fusion factor corresponding to the target ground point to be searched includes: an absolute angle error of an inner angle corresponding to the search target ground point among the inner angles of a quadrilateral formed by sequentially connecting the target ground points to be searched;
[0032] The distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
[0033] Optionally, the grounding point determination unit is specifically used to:
[0034] In the search space of each target ground point to be searched, each target ground point to be searched is controlled to be searched in the horizontal and vertical directions respectively according to the same preset step parameters;
[0035] The target grounding point to be searched that minimizes the internal fusion parameter and / or the external fusion parameter is determined as the target grounding point coordinates of the target vehicle for auxiliary marking.
[0036] Optionally, the image coordinate determination module is specifically used to:
[0037] Based on the image of the target vehicle acquired by the visual sensor, the coordinates of the grounding point of the target vehicle in the image coordinate system are determined;
[0038] The coordinates of the grounding point of the target vehicle in the image coordinate system are transformed to transform the coordinates of the grounding point of the target vehicle in the image coordinate system into the vehicle body coordinate system to obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0039] In the third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of any possible implementation of the first aspect or the first aspect of the first aspect are performed.
[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any possible implementation of the above-mentioned first aspect, or the first aspect of the first aspect are executed.
[0041] The embodiment of the present application obtains the vehicle grounding point for auxiliary marking by fusing the perception results of different sensor devices, which can not only accurately determine the invisible points, but also correct the positions of the grounding points through rectangular constraints, so that the positions of the grounding points used for marking are more accurate and the marking efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A flow chart of a vehicle grounding point detection method provided by an embodiment of the present application is shown;
[0044] Figure 2 A flowchart showing the steps of determining the image coordinates of a grounding point provided by an exemplary embodiment of the present application;
[0045] Figure 3 A flowchart showing the steps of determining the target grounding point coordinates of the target vehicle for auxiliary marking provided by an exemplary embodiment of the present application;
[0046] Figure 4 A schematic diagram showing a method of searching for a target grounding point in a vehicle body coordinate system provided by an exemplary embodiment of the present application is shown;
[0047] Figure 5 A schematic diagram showing the structure of a vehicle grounding point detection device provided in an embodiment of the present application is shown;
[0048] Figure 6 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0049] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.
[0050] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0051] In order to enable those skilled in the art to use the contents of this application, the following implementation methods are provided in conjunction with the specific application scenario of "vehicle grounding point detection". For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around vehicle grounding point detection, it should be understood that the embodiments of this application can be applied to the scenario of auxiliary marking of vehicle grounding points in addition to the application scenario of automatic detection of vehicle grounding points mentioned above.
[0052] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0053] The Advanced Driving Assistance System (ADAS) uses a variety of sensors installed on the car (millimeter wave radar, lidar, monocular / binocular cameras and satellite navigation) to sense the surrounding environment at any time during driving, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform systematic calculations and analysis, so as to allow the driver to be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving.
[0054] Normally, the driving perception algorithm of ADAS needs to use the location of the target grounding point for accurate ranging and close-range entry judgment. The grounding point method based on deep learning technology requires a large number of manually labeled samples. Since the grounding point needs to be labeled with the specific location, orientation, occlusion and other information of each target key point, it is particularly important to use auxiliary labeling methods to improve labeling efficiency.
[0055] Specifically, according to the occlusion attribute, the grounding point can be divided into visible points and invisible points. In practical applications, accurate prediction of invisible points plays an important role in the ranging of occluded and truncated targets. In the prior art, the grounding point is determined by the following methods:
[0056] 1) Do not mark the grounding point, and directly use the center point of the lower edge of the tire detection frame
[0057] This method does not need to mark the grounding point, but uses the tire detection model to detect the outer contour of the target vehicle tire. The rectangular box, and then uses the lower boundary midpoint of the rectangular box as the grounding point. One of the disadvantages of this method is that the tire detection model needs to be trained first, and then the tire detection results are converted into the grounding point position, which will cause the prediction results to be unable to distinguish the inner and outer edges of the tire, and do not have the actual physical meaning of "grounding". It also does not take into account that the lower boundary midpoint of the detection box and the actual grounding point may not coincide at certain observation angles, thereby reducing the prediction accuracy; another disadvantage is that this method can only detect the grounding point detection of tires, which is not suitable for pedestrians and other targets.
[0058] 2) Only mark visible points
[0059] This method uses manually annotated visible grounding points to train the grounding point detection model. There are some limitations in practical applications such as target ranging: when the grounding point of the target vehicle tire or pedestrian is not fully exposed, the actual grounding point position cannot be accurately predicted, resulting in the inability to use the grounding point for accurate ranging. In particular, when two or more grounding positions of the target are blocked, there is a serious impact on target ranging and size estimation.
[0060] 3) Manually mark invisible points
[0061] It is difficult to determine the actual location of invisible points, and it is difficult to determine a unified labeling standard, so manual labeling is required. In actual operations, improving the efficiency of labeling, quality inspection, and acceptance of invisible points, producing high-quality labeled samples, and ultimately efficiently enabling various driving perception applications are common problems encountered in the industry.
[0062] It can be seen that the existing method of determining the grounding point has limitations in terms of accuracy and efficiency.
[0063] In response to at least one of the above-mentioned problems, an embodiment of the present application provides a method and device for detecting a vehicle grounding point. This scheme fuses the perception results of different sensor devices to obtain a vehicle grounding point for auxiliary marking. This scheme can not only accurately determine the invisible points, but also correct the positions of the grounding points through rectangular constraints, so that the positions of the grounding points used for marking are more accurate and the marking efficiency is higher.
[0064] To facilitate understanding of this embodiment, a vehicle grounding point detection method disclosed in an embodiment of the present application is first introduced in detail.
[0065] See also Figure 1, is a flow chart of a method for detecting a vehicle grounding point provided by an exemplary embodiment of the present application. Exemplarily, the method for detecting a vehicle grounding point may be executed by an onboard controller of a vehicle. The method for detecting a vehicle grounding point specifically includes:
[0066] Step S100: Based on the visual image of the target vehicle collected by the visual sensor, the image coordinates of the touchdown point of the target vehicle in the vehicle body coordinate system are determined.
[0067] In an exemplary embodiment of the present application, the visual sensor may be a variety of sensor devices that can be installed on a vehicle and can capture the original image of obstacles around the vehicle. Exemplarily, the visual sensor may include but is not limited to at least one of the following items: at least one graphic sensor, at least one image sensor, a single / binocular camera, a surround camera, etc. The above-mentioned obstacles may refer to dynamic and / or static objects indoors and outdoors that can hinder the normal travel of vehicles and other equipment. Exemplarily, objects located indoors may include but are not limited to at least one of the following items: people, walls, tables and chairs, and objects located outdoors may include but are not limited to at least one of the following items: people, vehicles (cars, bicycles, power-assisted vehicles), fences, and curbs. It should be understood here that, as an example, the main obstacle studied in this application is a vehicle, but other obstacles can also be studied, and this application does not make any restrictions here.
[0068] In addition, the body coordinate system is a coordinate system used to describe the relative positional relationship between the objects around the vehicle and the vehicle. As an example, the body coordinate system in the exemplary embodiment of the present application is the body coordinate system defined by the IMU. Under unified calibration in a specific scenario, the body coordinate system may also be other coordinate systems besides the IMU body coordinate system, such as the ISO body coordinate system or the SAE body coordinate system.
[0069] Usually, the visual image collected by the visual sensor is based on the camera coordinate system (i.e., the image coordinate system), and the obtained image tends to be the front view of the object. However, since the camera coordinate system and the vehicle coordinate system are different coordinate systems, the same point in the image needs to be converted to obtain the position coordinates in another coordinate system. The following will describe in detail how to obtain the image coordinates of the ground point in the visual image in the vehicle coordinate system.
[0070] Figure 2 A flow chart showing steps of determining image coordinates of a grounding point provided by an exemplary embodiment of the present application.
[0071] Reference Figure 2 In step S110, based on the visual image of the target vehicle collected by the visual sensor, the coordinates of the grounding point of the target vehicle in the image coordinate system are determined.
[0072] Here, as described above, the visual sensor can be set at different positions of the vehicle. For example, the visual sensor can be set on at least one of the front, rear, left and right sides of the vehicle, and the visual images of the vehicle on the corresponding sides are captured by the cameras included in the visual sensor. For example, if a target vehicle is around the vehicle, the visual image of the target vehicle can be captured by the visual sensor on one side of the vehicle.
[0073] In addition, the image coordinate system refers to a rectangular coordinate system defined on a digital image captured by a camera placed at a certain position in the environment, and the coordinates of each pixel are the column number and row number of the pixel in the array (the storage form of the digital image in the computer).
[0074] Accordingly, after the visual image of the target vehicle is collected from the visual sensor, since the visual image is a raw image, it needs to undergo specific processing to obtain the relevant information of the grounding point of the target vehicle. Here, in the example of the present application, the visual image collected by the visual sensor can be used in any existing way to obtain the grounding point coordinates of the target vehicle in the image coordinate system.
[0075] For example, the coordinates of the grounding point of the target vehicle in the image coordinate system can be outputted through a pre-trained machine learning model. Specifically, the visual image of the target vehicle acquired by the visual sensor can be inputted as input data into the pre-trained visual prediction model, and the visual prediction model outputs the image coordinates of the grounding point of the target vehicle in the image coordinate system. In addition, other methods other than machine learning model prediction can also be used to determine the image coordinates of the grounding point of the target vehicle in the image coordinate system, and this application does not impose any restrictions on this.
[0076] In step S120, coordinate transformation is performed on the grounding point coordinates of the target vehicle in the image coordinate system to transform the grounding point coordinates of the target vehicle in the image coordinate system into the vehicle body coordinate system, thereby obtaining the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0077] For example, the coordinate conversion method can be used to convert the grounding point coordinates of the target vehicle in the image coordinate system (horizontal coordinate u, vertical coordinate v, vertical coordinate k; where k=1) to the vehicle body coordinate system to obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system (horizontal coordinate X, vertical coordinate Y, vertical coordinate Z; where Y=0). The specific coordinate conversion formula is as follows:
[0078]
[0079] Where X is the horizontal coordinate of the grounding point of the target vehicle in the vehicle body coordinate system; Z is the vertical coordinate of the grounding point of the target vehicle in the vehicle body coordinate system; u is the horizontal coordinate of the target vehicle in the image coordinate system; v is the vertical coordinate of the target vehicle in the image coordinate system; f x 、f y 、c x and c y is the camera internal parameter, R is the rotation matrix of the camera external parameter, and T is the translation matrix of the camera external parameter.
[0080] Through the above conversion formula (1), the grounding point coordinates of the target vehicle in the vehicle body coordinate system can be determined according to the grounding point coordinates of the target vehicle in the image coordinate system and the camera parameters.
[0081] Return to reference Figure 1 In step S200, based on the detection data of the target vehicle collected by the target radar, the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined.
[0082] In an exemplary embodiment of the present application, the target radar may be any radar system that can be installed on a vehicle and can detect characteristic quantities such as the position and speed of a target, such as a laser radar, an ultrasonic radar, a millimeter wave radar, and the like.
[0083] Preferably, in the exemplary embodiment of the present application, the target radar is a laser radar. Here, the installation position of the laser radar and the target vehicle are in the same coordinate system, and the coordinate system can be any predefined coordinate system (such as a vehicle body coordinate system). The laser radar uses laser as a signal source. The pulsed laser emitted by the laser hits the target vehicle, causing scattering. A part of the light wave will be reflected on the receiver of the laser radar. According to the laser ranging principle, the distance from the laser radar to the target vehicle is calculated. The pulsed laser continuously scans the target vehicle, and the data of all target points on the target vehicle can be obtained. After imaging processing with this data, an accurate three-dimensional image of the target vehicle can be obtained. Since the coordinate system of the three-dimensional image is a predefined vehicle body coordinate system, the coordinates of the grounding point of the target vehicle can be included in the three-dimensional image. As an example, the four corner points of the bottom quadrilateral of the three-dimensional image can be used as the grounding point of the target vehicle to obtain the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0084] In this way, the image coordinates and detection coordinates of the grounding point of the target vehicle are obtained in the same coordinate system.
[0085] In step S300, the coordinates of the target grounding point of the target vehicle for auxiliary marking are obtained by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
[0086] In the embodiment of the present application, the following may be performed for the grounding point of the target vehicle: Figure 3 The processing steps shown are used to obtain the target grounding point coordinates of the target vehicle for auxiliary marking.
[0087] Figure 3 A flowchart showing steps for determining the target ground point coordinates of a target vehicle for auxiliary marking provided by an exemplary embodiment of the present application.
[0088] Reference Figure 3 In step S310, internal fusion parameters are determined; the internal fusion parameters include the weighted sum of angle fusion factors corresponding to all target ground points to be searched.
[0089] Here, it should be noted that the target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
[0090] Figure 4 A schematic diagram showing a method of searching for a target grounding point in a vehicle body coordinate system provided by an exemplary embodiment of the present application is shown; Figure 4 , Figure 4 It shows that there are four search spaces in the vehicle body coordinate system, namely search space A, search space B, search space C and search space D; wherein the target grounding points A0, B0, C0 and D0 to be searched are four grounding points pre-set in the search space A, search space B, search space C and search space D, respectively. Figure 4 A1, B1 and C1 are the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system, and A2, B2, C2 and D2 are the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0091] Among them, the detection coordinates of the grounding point in the vehicle body coordinate system need to be adapted to the image coordinates of the grounding point obtained previously, so as to determine the matching relationship between the image coordinates and the detection coordinates of the target vehicle in the vehicle coordinate system, as well as the type of the grounding point, for example, which grounding point is a visible grounding point and which grounding point is an invisible grounding point.
[0092] As an example, the matching correspondence between the image coordinates and the detection coordinates of the grounding point of the target vehicle can be obtained by an associative matching algorithm, such as the Hungarian matching algorithm, the Gale-Shapley algorithm, the KM algorithm, and the like.
[0093] For example, when using the Hungarian matching algorithm, the initial element value in the Hungarian matrix can be determined as the distance value between the image coordinates of each ground point and the detection coordinates of each ground point, and then after traversing to obtain the distance value between the image coordinates and the detection coordinates of each ground point, Hungarian matching is performed, and the image coordinates and detection coordinates corresponding to the matched elements being 1 are determined as matches. The ground points that match each other are visible ground points, and the ground points that do not find a match are invisible ground points.
[0094] For example, after Hungarian matching, Figure 4 In the figure, A1 and A2 match each other, B1 and B2 match each other, and C1 and C2 match each other. A1 and A2 are the image coordinates and detection coordinates of the target ground point A0 to be searched for the target vehicle, and A1 and A2 are both visible ground points; B1 and B2 are the image coordinates and detection coordinates of the target ground point B0 to be searched for the target vehicle, and B1 and B2 are visible ground points; similarly, C1 and C2 are also visible ground points, and D2 is the detection coordinate of the target ground point D0 to be searched for the target vehicle. D2 does not have a matching image coordinate, so D2 is an invisible ground point.
[0095] In addition, as an example, the angle fusion factor corresponding to the target ground point to be searched may include: the absolute angle error of the inner angle corresponding to the search target ground point among the inner angles of a quadrilateral formed by sequentially connecting the target ground points to be searched.
[0096] Specifically, in Figure 4 In the example shown, the target grounding points A0, B0, C0 and D0 to be searched are respectively located at the sampling positions of their respective search spaces, wherein a quadrilateral can be obtained by connecting them in order or in reverse order, wherein two adjacent sides form an inner angle corresponding to the target grounding point to be searched, for example, Figure 4 The inner angle corresponding to the target ground point A0 to be searched is ∠BoAoCo, the inner angle corresponding to the target ground point B0 to be searched is ∠DoBoAo, the inner angle corresponding to the target ground point C0 to be searched is ∠DoCoAo, and the inner angle corresponding to the target ground point D0 to be searched is ∠CoDoBo.
[0097] After obtaining the inner angle corresponding to each target ground point to be searched, for each search target ground point, the difference between the inner angle corresponding to the target ground point to be searched and the right angle can be determined, and the absolute value of the difference is the angle absolute error of the inner angle corresponding to the search target ground point. Based on the weighted summation of the angle absolute error of the inner angle corresponding to each search target ground point and the corresponding weight parameter, the internal fusion parameter can be obtained.
[0098] Here, the weight parameter corresponding to any inner angle can be determined in the following manner: first, the target ground point to be searched corresponding to the inner angle can be determined, then the image contrast of the ground point of the image coordinates corresponding to the target ground point to be searched in a predetermined area in the visual image can be determined, and finally, the weight parameter corresponding to the inner angle can be determined based on the image contrast.
[0099] Image contrast refers to the measurement of different brightness levels between the brightest white and the darkest black in the light and dark areas of an image. It generally refers to the size of the grayscale contrast of an image. The larger the difference range, the greater the contrast, and the smaller the difference range, the smaller the contrast. The reason why the weight parameter is determined by image contrast is that the image contrast in a predetermined area of the image can reflect the clarity of the image texture in the area. Furthermore, the clarity of the image will directly affect the accuracy of the coordinate transformation and projection of the image coordinates to the vehicle body coordinate system. Therefore, the image contrast can be used directly or indirectly as a weight parameter for evaluating the absolute angle error of each grounding point. The internal fusion parameters corresponding to all internal angles can be obtained using the weight parameters and the average angle error.
[0100] At the same time, in step S320, based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system, an external fusion parameter is determined; the external fusion parameter includes a weighted sum of distance fusion factors corresponding to all target grounding points to be searched.
[0101] As an example, the distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
[0102] Specifically, based on the matching image coordinates and detection coordinates as described above, the distance between the image coordinates and the corresponding ground point of the target to be searched can be determined. Similarly, the distance between the detection coordinates and the corresponding ground point of the target to be searched can also be determined. The above-mentioned absolute value of the distance is the absolute error of the coordinates corresponding to the image coordinates or the detection coordinates.
[0103] Accordingly, the external fusion parameter can be obtained by weighted summing the absolute coordinate error corresponding to the image coordinate or the detection coordinate and the corresponding weight parameter. The weight parameter is determined in the same way as the weight parameter described above, which is not repeated here. Accordingly, the image coordinates of each grounding point of the target vehicle correspond to a corresponding weight parameter.
[0104] In step S330, the coordinates of the target ground point to be searched corresponding to the internal fusion parameters and / or the external fusion parameters satisfying their corresponding target conditions are determined as the coordinates of the target ground point for auxiliary marking of the target vehicle.
[0105] Specifically, after the internal fusion parameters and / or the external fusion parameters are determined, a search method may be used to find the target ground point to be searched that satisfies the internal fusion parameters and / or the external fusion parameters when the target solution is satisfied.
[0106] As an example, regarding step S330, in specific implementation, it can be implemented in the following manner:
[0107] In the search space of each target ground point to be searched, each target ground point to be searched can be controlled to be searched in the horizontal and vertical directions respectively according to the same preset step parameters, and the target ground point to be searched that minimizes the internal fusion parameter and / or the external fusion parameter is determined as the target ground point coordinates of the target vehicle for auxiliary marking.
[0108] Here, the preset step parameters may include: step length and / or step number. Figure 4 When searching, the target grounding points A0, B0, C0 and D0 can be controlled as search sampling points, and the search can be started from their respective search spaces. In the initial search, A0 can be controlled to take A1 as the origin, B0 to take B1 as the origin, C0 to take C1 as the origin, and D0 to take D2 as the origin. The search is performed according to the step length d for N times, and from these N searches, the sampling points that make the internal fusion parameters and the external fusion parameters meet the target conditions at the same time are found.
[0109] Here, the target condition of the internal fusion parameter is that the internal fusion parameter is minimized; the target condition of the external fusion parameter is that the external fusion parameter is minimized. Specifically, the minimum external fusion parameter indicates that the position deviation between the image coordinates corresponding to the grounding point and the detection coordinates is minimized, and the minimum internal fusion parameter indicates that the quadrilateral formed by sequentially connecting the target grounding points to be searched is closest to a rectangle. When the internal fusion parameter is minimized and the external fusion parameter is minimized, the optimal solution for the target grounding point to be searched can be searched, thereby obtaining the target grounding point coordinates for auxiliary annotation of the target vehicle. It should be understood here that in the example of the present application, only one fusion parameter may be considered, for example, the internal fusion parameter may be considered alone, or the external fusion parameter may be considered, and the target grounding point to be annotated when one fusion parameter meets the target condition is determined as the target grounding point for auxiliary annotation.
[0110] As an example, the external fusion function and the internal fusion function can be stored in the form of a target fusion function, and the target fusion function is called by a computer to complete the search process to obtain the optimal solution:
[0111] For example, the target fusion function OBJ can be expressed by the following formula (2):
[0112]
[0113] Among them, w1 is the weight parameter corresponding to A1, w2 is the weight parameter corresponding to B1, w3 is the weight parameter corresponding to C1, A1 is the image coordinates of the first grounding point of the target vehicle in the vehicle body coordinate system; B1 is the image coordinates of the second grounding point of the target vehicle in the vehicle body coordinate system; C1 is the image coordinates of the third grounding point of the target vehicle in the vehicle body coordinate system; A2 is the detection coordinates of the first grounding point of the target vehicle in the vehicle body coordinate system; B2 is the detection coordinates of the second grounding point of the target vehicle in the vehicle body coordinate system; C2 is the detection coordinates of the third grounding point of the target vehicle in the vehicle body coordinate system. The detection coordinates in the coordinate system of the target vehicle are as follows: D2 is the detection coordinates of the fourth grounding point of the target vehicle in the vehicle body coordinate system; A0 is the target grounding point to be searched for the first grounding point of the target vehicle in the vehicle body coordinate system; B0 is the target grounding point to be searched for the second grounding point of the target vehicle in the vehicle body coordinate system; C0 is the target grounding point to be searched for the third grounding point of the target vehicle in the vehicle body coordinate system; D0 is the target grounding point to be searched for the fourth grounding point of the target vehicle in the vehicle body coordinate system; sig is the coefficient of balancing displacement deviation and angle deviation, which can be preset by the user.
[0114] here,
[0115] is the displacement deviation, i.e., the external fusion parameter.
[0116] w1×sig×|∠C0A0B0-90 0 |+w2×sig×|∠A0B0C0-90 0 |+w3×sig×|∠D0C0A0-90 0 |+(1-w1-w2-w3)×sig×|∠B0D0C0-90 0 |
[0117] is the angle deviation, i.e., the internal fusion parameter.
[0118] When A0, B0, C0 and D0 simultaneously minimize the displacement deviation and the angle deviation within the predetermined search range, A0, B0, C0 and D0 can be determined as the target grounding point coordinates of the target vehicle for auxiliary marking.
[0119] According to the detection method of the vehicle grounding point provided by the exemplary embodiment of the present application, first, based on the visual image of the target vehicle collected by the visual sensor, the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; then, based on the detection data of the target vehicle collected by the target radar, the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; finally, by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates, the target grounding point coordinates of the target vehicle for auxiliary marking are obtained. By fusing the perception results of different sensor devices to obtain the vehicle grounding point for auxiliary marking, not only can the invisible points be accurately determined, but also the position of the grounding point can be corrected by rectangular constraints, so that the position of the grounding point for marking is more accurate and the marking efficiency is higher.
[0120] Based on the same application concept, the embodiments of the present application also provide a detection device for a vehicle grounding point corresponding to the method provided in the above embodiments. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the detection method for a vehicle grounding point in the above embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0121] Figure 5 This is a schematic diagram of the structure of a vehicle grounding point detection device provided by an exemplary embodiment of the present application. Figure 5 As shown, the vehicle grounding point inspection device 500 includes:
[0122] An image coordinate determination module 510 determines the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the visual image of the target vehicle acquired by the visual sensor;
[0123] The detection coordinate determination module 520 determines the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the detection data of the target vehicle collected by the target radar;
[0124] The fusion module 530 obtains the coordinates of the target grounding point of the target vehicle for auxiliary marking by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
[0125] Optionally, the fusion module 530 includes:
[0126] An internal parameter determination unit determines an internal fusion parameter; the internal fusion parameter includes a weighted sum of angle fusion factors corresponding to all target ground points to be searched;
[0127] An external parameter determination unit determines an external fusion parameter based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system; the external fusion parameter includes a weighted sum of distance fusion factors corresponding to all target grounding points to be searched;
[0128] A grounding point determination unit determines the coordinates of the target grounding point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition as the target grounding point coordinates for auxiliary marking of the target vehicle;
[0129] The target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
[0130] Among them, the angle fusion factor corresponding to the target ground point to be searched includes: the angle absolute error of the inner angle corresponding to the search target ground point in the inner angle of the quadrilateral formed by connecting the target ground points to be searched in sequence; the distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
[0131] Optionally, the grounding point determination unit is specifically used to: in the search space of each target grounding point to be searched, control each target grounding point to be searched to be searched in the horizontal and vertical directions respectively according to the same preset stepping parameter; and determine the target grounding point to be searched that makes the internal fusion parameter minimum and / or the external fusion parameter minimum as the target grounding point coordinates of the target vehicle for auxiliary marking.
[0132] Optionally, the image coordinate determination module 510 is specifically used to: determine the grounding point coordinates of the target vehicle in the image coordinate system based on the image of the target vehicle captured by the visual sensor; perform coordinate conversion on the grounding point coordinates of the target vehicle in the image coordinate system to convert the grounding point coordinates of the target vehicle in the image coordinate system to the vehicle body coordinate system, and obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
[0133] See also Figure 6 , Figure 6 The structural diagram of the electronic device provided by the exemplary embodiment of the present application. Figure 6 As shown, the electronic device 900 includes a processor 910 , a memory 920 , and a bus 930 .
[0134] The memory 920 stores machine-readable instructions executable by the processor 910. When the electronic device 900 is running, the processor 910 communicates with the memory 920 via the bus 930. When the machine-readable instructions are executed by the processor 910, the steps of the vehicle grounding point detection method in any of the above embodiments can be performed, which are specifically as follows:
[0135] Based on the visual image of the target vehicle acquired by the visual sensor, the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; based on the detection data of the target vehicle acquired by the target radar, the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; and by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates, the target grounding point coordinates of the target vehicle for auxiliary marking are obtained.
[0136] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle grounding point detection method in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.
[0137] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned vehicle grounding point detection method can be executed.
[0138] The embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the vehicle grounding point detection method in the above method embodiment. The specific implementation method can be found in the method embodiment and will not be repeated here.
[0139] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0142] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0143] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for detecting a vehicle grounding point, characterized in that: The detection method comprises: Based on the visual image of the target vehicle acquired by the visual sensor, the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; Based on the detection data of the target vehicle collected by the target radar, the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system are determined; The target grounding point coordinates of the target vehicle for auxiliary marking are obtained by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
2. The detection method according to claim 1, characterized in that The step of fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates to obtain the coordinates of the target grounding point of the target vehicle for auxiliary marking includes: Determine internal fusion parameters; the internal fusion parameters include the weighted sum of angle fusion factors corresponding to all target ground points to be searched; Based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system, an external fusion parameter is determined; the external fusion parameter includes a weighted sum value of distance fusion factors corresponding to all target grounding points to be searched; The coordinates of the target ground point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition are determined as the coordinates of the target ground point for auxiliary marking of the target vehicle; The target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
3. The detection method according to claim 2, characterized in that The angle fusion factor corresponding to the target ground point to be searched includes: the angle absolute error of the inner angle corresponding to the search target ground point among the inner angles of the quadrilateral formed by sequentially connecting the target ground points to be searched; The distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
4. The detection method according to claim 2, characterized in that The step of determining the coordinates of the target ground point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition as the coordinates of the target ground point for auxiliary marking of the target vehicle includes: In the search space of each target ground point to be searched, each target ground point to be searched is controlled to be searched in the horizontal and vertical directions respectively according to the same preset step parameters; The target grounding point to be searched that minimizes the internal fusion parameter and / or the external fusion parameter is determined as the target grounding point coordinates of the target vehicle for auxiliary marking.
5. The detection method according to claim 1, characterized in that The step of determining the image coordinates of the touchdown point of the target vehicle in the vehicle body coordinate system based on the visual image of the target vehicle collected by the visual sensor comprises: Based on the image of the target vehicle acquired by the visual sensor, the coordinates of the grounding point of the target vehicle in the image coordinate system are determined; The coordinates of the grounding point of the target vehicle in the image coordinate system are transformed to transform the coordinates of the grounding point of the target vehicle in the image coordinate system into the vehicle body coordinate system to obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
6. A vehicle grounding point detection device, characterized in that: The detection device comprises: An image coordinate determination module determines the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the visual image of the target vehicle acquired by the visual sensor; A detection coordinate determination module determines the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system based on the detection data of the target vehicle collected by the target radar; The fusion module obtains the target grounding point coordinates of the target vehicle for auxiliary marking by fusing the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system with the detection coordinates.
7. The detection device according to claim 1, characterized in that: The fusion module includes: An internal parameter determination unit determines an internal fusion parameter; the internal fusion parameter includes a weighted sum of angle fusion factors corresponding to all target ground points to be searched; An external parameter determination unit determines an external fusion parameter based on the image coordinates and the detection coordinates of the grounding point of the target vehicle in the vehicle body coordinate system; the external fusion parameter includes a weighted sum of distance fusion factors corresponding to all target grounding points to be searched; A grounding point determination unit determines the coordinates of the target grounding point to be searched corresponding to the internal fusion parameter and / or the external fusion parameter satisfying the corresponding target condition as the target grounding point coordinates for auxiliary marking of the target vehicle; The target grounding point to be searched is a grounding point of the target vehicle preset in different search spaces under the vehicle body coordinate system for auxiliary marking.
8. The detection device according to claim 7, characterized in that: The angle fusion factor corresponding to the target ground point to be searched includes: the angle absolute error of the inner angle corresponding to the search target ground point among the inner angles of the quadrilateral formed by sequentially connecting the target ground points to be searched; The distance fusion factor corresponding to the target ground point to be searched includes: the absolute coordinate error between the image coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched, and the absolute coordinate error between the detection coordinates of each ground point of the target vehicle and the coordinates of the corresponding target ground point to be searched.
9. The detection device according to claim 7, characterized in that: The grounding point determination unit is specifically used for: In the search space of each target ground point to be searched, each target ground point to be searched is controlled to be searched in the horizontal and vertical directions respectively according to the same preset step parameters; The target grounding point to be searched that minimizes the internal fusion parameter and / or the external fusion parameter is determined as the target grounding point coordinates of the target vehicle for auxiliary marking.
10. The detection device according to claim 6, characterized in that: The image coordinate determination module is specifically used for: Based on the image of the target vehicle acquired by the visual sensor, the coordinates of the grounding point of the target vehicle in the image coordinate system are determined; The coordinates of the grounding point of the target vehicle in the image coordinate system are transformed to transform the coordinates of the grounding point of the target vehicle in the image coordinate system into the vehicle body coordinate system to obtain the image coordinates of the grounding point of the target vehicle in the vehicle body coordinate system.
11. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the vehicle grounding point detection method according to any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle grounding point detection method according to any one of claims 1 to 5 are executed.
