Perception information verification method and device, electronic equipment and storage medium

Abstract

The application provides a perception information verification method and device, electronic equipment and a storage medium. The perception information verification method comprises: performing perception element identification according to perception information in a vehicle perception range to obtain identified perception elements; and performing verification on the identified perception elements according to a mutual relationship between specified targets in the identified perception elements. The method can improve the reliability of the identified perception elements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of driving assistance, and in particular to a perception information verification method and device, an electronic device, and a storage medium. BACKGROUND

[0002] An automated parking assistance system (APA) is an automatic parking system based on sensors such as cameras, radars (ultrasonic radars or millimeter wave radars, etc.).

[0003] In an automatic parking process, it is necessary to automatically identify parking spaces and obstacles and other targets (which can be referred to as perception elements) according to perception information of sensors such as cameras, radars (ultrasonic radars or millimeter wave radars, etc.), and to automatically control a vehicle to enter a parking space according to position information of the perception elements.

[0004] In order to improve the safety of automatic parking, how to improve the accuracy of the perception elements has become a technical problem to be solved. SUMMARY

[0005] Therefore, the present application provides a perception information verification method and device, an electronic device, and a storage medium.

[0006] Specifically, the present application is implemented by the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a perception information verification method is provided, comprising:

[0008] performing perception element identification according to perception information in a vehicle perception range to obtain identified perception elements;

[0009] performing verification on the identified perception elements according to a mutual relationship between specified targets in the identified perception elements.

[0010] According to a second aspect of an embodiment of the present application, a perception information verification device is provided, comprising:

[0011] an identification unit configured to perform perception element identification according to perception information in a vehicle perception range to obtain identified perception elements;

[0012] a verification unit configured to perform verification on the identified perception elements according to a mutual relationship between specified targets in the identified perception elements.

[0013] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor is configured to execute the machine executable instructions to implement the method provided in the first aspect.

[0014] According to a fourth aspect of the embodiments of the present application, a storage medium is provided, and the storage medium stores machine executable instructions, and the machine executable instructions are executed by a processor to implement the method provided in the first aspect.

[0015] The technical solutions provided in the present application can bring at least the following beneficial effects:

[0016] By considering the mutual relationship between the perception elements based on more abundant information in the perception link, and verifying the identified perception elements according to the mutual relationship between the specified targets in the identified perception tuples, the reliability of the identified perception elements is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flow diagram of a perception information verification method according to an example embodiment of the present application;

[0018] Figure 2 is a flow diagram of a perception result verification method combining the explicit rule-based verification module and the data-driven verification module according to an example embodiment of the present application;

[0019] Figure 3A and Figure 3B are schematic diagrams of the grid map before and after correction according to an example embodiment of the present application;

[0020] Figure 4 is a structural diagram of a perception information verification device according to an example embodiment of the present application;

[0021] Figure 5 is a structural diagram of another perception information verification device according to an example embodiment of the present application;

[0022] Figure 6 is a hardware structural diagram of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0023] The example embodiments will be described in detail herein, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0024] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0025] In order to better understand the technical solutions provided by the embodiments of the application, and to make the above-mentioned purposes, characteristics and advantages of the embodiments of the application more apparent and easy to understand, the technical solutions in the embodiments of the application will be further described in detail below with reference to the drawings.

[0026] Please refer to Figure 1 , a flowchart of a perception information verification method provided by the embodiments of the application is shown, which can include the following steps: Figure 1

[0027] Step S100, identifying perception elements according to perception information within a vehicle perception range, to obtain identified perception elements.

[0028] Step S110, verifying the identified perception elements according to the mutual relationship between specified targets in the identified perception elements.

[0029] In the embodiments of the application, considering that a parking scene often appears several fixed combinations of perception elements, and each perception element in the fixed perception element group has a specific correlation relationship with a high probability, based on this, the real-time identification result of the perception elements can be verified based on the scene modeling and priori, the mutual relationship between the perception elements is considered based on more abundant information in the perception link, to optimize the overall perception effect and improve the reliability of the identified perception elements.

[0030] Correspondingly, the perception elements can be identified according to the perception information (such as image information collected by a camera, point cloud data of a millimeter wave radar, etc.) within the vehicle perception range, to obtain the identified perception elements.

[0031] Illustratively, the perception elements can include but are not limited to movable obstacles (motor vehicles, pedestrians, non-motor vehicles, etc.), immovable obstacles (no parking signs, road edges, wheel stops, ground locks, pillars, etc.), and road surface information (road surface, no parking area, guide signs, parking space lines, lane lines, etc.).

[0032] Illustratively, the identified perception elements can be verified according to the mutual relationship between the specified targets in the identified perception elements.

[0033] ​For example, since the slopes of adjacent lane lines are generally consistent in actual scenarios, in the case that the lane line is included in the perception element, the consistency of the slopes of adjacent lane lines can be used to determine whether the recognized lane line is accurate.

[0034] It can be seen that, in the method flow shown in Figure 1 In the method flow shown in

[0035] In some embodiments, the verification of the recognized perception element according to the mutual relationship between the specified targets in the recognized perception element can include:

[0036] In the case that the first specified target is included in the recognized perception element, the first specified target is verified according to the consistency between the same type of first specified targets; wherein the first specified target is a perception element that has a periodic regularity.

[0037] For example, considering that most of the same type of targets in the parking scenario have a periodic regularity, such as the slopes of adjacent lane lines, the sizes of parking spaces in the parking lot, the orientations of vehicles in different parking spaces, and the like. Therefore, the specified target (referred to as the first specified target herein) in the perception element can be verified based on the periodic regularity between the same type of targets.

[0038] Correspondingly, in the case that the first specified target is included in the recognized perception element, the first specified target can be verified according to the consistency between the same type of first specified targets

[0039] For example, the consistency between the same type of first specified targets can include:

[0040] In the case that the first specified target is a lane line or a road edge line, the consistency of the slopes, curvatures, orientations, and / or length information of adjacent first specified targets;

[0041] In the case that the first specified target is a parking space, the consistency of the sizes, orientations, and / or types of adjacent first specified targets;

[0042] In the case that the first specified target is a vehicle in a parking space, the consistency of the orientations of adjacent first specified targets;

[0043] In the case that the first specified target is a number, a ground lock, or a wheel stop in a parking space, the consistency of the position information of adjacent first specified targets in the parking space.

[0044] In one example, the verifying the first specified target according to the consistency between the first specified targets of the same type can include:

[0045] For any first specified target, in a case where it is determined that the consistency between the first specified target and other first specified targets of the same type does not meet the requirement, the confidence of the first specified target is reduced, or the relevant attribute of the first specified target is adjusted to make the consistency between the first specified target and other first specified targets of the same type meet the requirement.

[0046] For example, for any first specified target in the recognized perception elements, it can be determined whether the consistency between the first specified target and other first specified targets of the same type (including the same type of first specified targets that are currently perceived in real time or historically perceived) meets the requirement.

[0047] For example, it is determined whether the difference between the first specified target and other first specified targets of the same type exceeds a preset threshold value (which can be referred to as a first threshold value), and in a case where the threshold value is exceeded, it is determined that the requirement is not met; otherwise, it is determined that the requirement is met.

[0048] For example, the difference between the first specified targets can be represented by the difference between the attribute information of the first specified targets having periodic regularity.

[0049] For example, the difference between the slopes of adjacent lane lines, the difference between the sizes of adjacent parking spaces in a parking lot, the difference between the orientations of vehicles in different parking spaces (which can be represented by an angle), and the like.

[0050] For example, for any first specified target, in a case where it is determined that the consistency between the first specified target and other first specified targets of the same type does not meet the requirement, the confidence of the first specified target can be reduced.

[0051] For example, the confidence of the first specified target can be reduced according to the difference between the first specified target and other first specified targets of the same type; the greater the difference, the more the confidence can be reduced.

[0052] For example, when the confidence of the first specified target is reduced according to the difference between the first specified target and other first specified targets of the same type, the reduction can be performed in a linear manner or in a non-linear manner.

[0053] For example, in a linear reduction, the confidence U = u-k*(d-t), u is the confidence of the target before adjustment, k is a set threshold value (such as the first threshold value described above), d is the difference between the first specified target and other first specified targets of the same type, t is a distance threshold value, and k and t can be preset.

[0054] For example, in the case of nonlinear reduction, the target can be directly set to 0 or other set confidence values according to the difference size.

[0055] As another example, for any first specified target, in the case of determining that the consistency between the first specified target and other first specified targets of the same type does not meet the requirements, the relevant attributes of the first specified target can be adjusted to make the consistency between the first specified target and other first specified targets of the same type meet the requirements.

[0056] For example, taking the first specified target as a lane line as an example, in the case that the difference between the slope of a certain lane line and the slopes of other adjacent lane lines exceeds a first threshold, the slope of the lane line can be adjusted to make the difference between the slope of the lane line and the slopes of other adjacent lane lines not exceed the first threshold.

[0057] In some embodiments, the above verification of the identified perception elements according to the mutual relationship between the specified targets in the identified perception elements can include:

[0058] In the case that the identified perception elements include a second specified target, the second specified target is verified according to the relative positional relationship between the second specified target and the specified components of the identified second specified target.

[0059] For example, considering that in actual parking scenarios, part of the targets identified may belong to the components of other targets, such as the corner points of the parking space and the parking space, the wheels and the vehicle, and the relative positional relationship between the part of the targets and the other targets to which they belong needs to meet certain conditions.

[0060] For example, the identified parking space can usually be represented by a rectangular frame or a parallelogram frame, and the positions of the four corner points of the rectangular frame or the parallelogram frame can be determined according to the position of the rectangular frame or the parallelogram frame; the position of the corner point identified when the corner point is identified should match the position of the corner point determined according to the position of the rectangular frame or the parallelogram frame, and the relative positional difference between each related corner point should not be too large.

[0061] For example, assuming that parking space 1 is identified according to perception information, the coordinates of the four corner points of parking space 1 (assuming A1B1C1D1) can be derived according to the coordinates of parking space 1, and in addition, the coordinates of the four corner points of parking space 1 (assuming A2B2C2D2) can be identified by corner point identification, then the difference between the coordinates of A1 and A2 should not exceed a preset threshold, and similarly, B1 and B2, C1 and C2, and D1 and D2 should also meet the above conditions.

[0062] Accordingly, in a case where the second specified target is included in the identified perception element, the second specified target can be verified according to a relative positional relationship between the second specified target and a specified component of the identified second specified target.

[0063] For example, in a case where the second specified target is a parking space, the specified component can include a corner point of the parking space.

[0064] In a case where the second specified target is a vehicle, the specified component includes a wheel of the vehicle.

[0065] In an example, the verifying the second specified target according to the relative positional relationship between the second specified target and the specified component of the identified second specified target can include:

[0066] For any second specified target, in a case where a relative positional relationship between the second specified target and the specified component of the second specified target does not meet a requirement, a confidence of the second specified target is reduced, or a position of the second specified target is adjusted so that the relative positional relationship between the second specified target and the specified component of the second specified target meets the requirement.

[0067] For example, in a case where the second specified target is a parking space, the relative positional relationship between the second specified target and the specified component of the second specified target can be a relative positional relationship between corner point coordinates derived according to parking space frame coordinates and identified corner point coordinates.

[0068] For example, in a case where the second specified target is a vehicle, the relative positional relationship between the second specified target and the specified component of the second specified target can be a relative positional relationship between a vehicle landing point coordinate derived according to vehicle frame coordinates and a landing point coordinate derived according to identified wheel coordinates.

[0069] For example, for any second specified target in the identified perception element, it can be determined whether a relative positional relationship between the second specified target and a specified component of the second specified target meets a requirement.

[0070] For example, it can be determined whether the relative positional relationship between the second specified target and the specified component of the second specified target meets the requirement according to whether a difference in relative position between the second specified target and the specified component of the second specified target exceeds a preset threshold value (which can be referred to as a second threshold value).

[0071] For example, the difference in relative position between the second specified target and the specified component of the second specified target can be represented by a distance between a position of the specified component derived according to the identified second specified target and a position of the identified specified component.

[0072] For example, taking the second specified target as a parking space, it can be determined whether the distance between the corner point derived according to the recognized parking space frame and the recognized corner point exceeds a preset threshold (which can be referred to as a second threshold). If the second threshold is exceeded, it is determined that the relative positional relationship between the second specified target and the specified component of the second specified target does not meet the requirement. Otherwise, it is determined that the relative positional relationship between the second specified target and the specified component of the second specified target meets the requirement.

[0073] In one example, for any second specified target, in a case where it is determined that the relative positional relationship between the second specified target and the specified component of the second specified target does not meet the requirement, the confidence of the second specified target is reduced.

[0074] For example, the confidence of the second specified target can be reduced according to the difference between the relative positions of the second specified target and the specified component of the second specified target.

[0075] For example, when the confidence of the second specified target is reduced according to the difference between the relative positions of the second specified target and the specified component of the second specified target, the reduction can be performed in a linear manner or in a non-linear manner. The specific implementation is similar to the implementation of reducing the confidence of the first target, and will not be described here in detail.

[0076] In another example, for any second specified target, in a case where it is determined that the relative positional relationship between the second specified target and the specified component of the second specified target does not meet the requirement, the position of the second specified target can be adjusted to make the relative positional relationship between the second specified target and the specified component of the second specified target meet the requirement.

[0077] For example, taking the second specified target as a parking space, in a case where the recognized parking space corner point coordinates derived according to the recognized parking space do not match the recognized parking space corner point coordinates, the recognized parking space can be adjusted, such as taking the recognized parking space corner point as the actual parking space corner point to determine a new parking space frame.

[0078] In some embodiments, the above verification of the recognized perception element according to the mutual relationship between the specified targets in the recognized perception element can include:

[0079] In a case where the recognized perception element includes a third specified target and a fourth specified target, the third specified target or the fourth specified target is verified according to the relative positional relationship between the third specified target and the fourth specified target. The third specified target and the fourth specified target are different categories of perception elements that are associated in relative position.

[0080] For example, considering that in real-world parking scenarios, there may be specific correlations between different categories of targets, such as the constant relative relationship between pillars and parking spaces in a parking lot (e.g., the number of parking spaces between two adjacent pillars is usually the same, and the distance between a pillar and an adjacent parking space is usually the same); the parking space entrance line is parallel to the lane lines in the road; and the relationship between vehicles traveling in the lane and the road (e.g., the vehicle's orientation is parallel to the lane lines). Therefore, the identified perceptual elements can also be validated based on the relative positional relationships between different categories of targets specified in the identified perceptual elements.

[0081] Accordingly, if the identified perceptual elements include a third designated target and a fourth designated target, the third designated target or the fourth designated target is verified based on the relative positional relationship between the third designated target and the fourth designated target.

[0082] For example, if the third designated target is a column, the fourth designated target can be a parking space between adjacent columns;

[0083] If the third designated target is the parking space entrance line, the fourth designated target can be the adjacent lane line;

[0084] If the third designated target is a vehicle in a parking space, the fourth designated target can be either a parking space or a lane.

[0085] In one example, the verification of the third or fourth specified target based on the relative positional relationship between the third and fourth specified targets may include:

[0086] If the relative positional relationship between the third and fourth designated targets does not meet the requirements, the confidence level of the third or fourth designated target shall be reduced, or the position of the third or fourth designated target shall be adjusted so that the relative positional relationship between the third and fourth designated targets meets the requirements.

[0087] For example, taking the third designated target as a pillar and the fourth designated target as parking spaces between adjacent pillars, the relative positional relationship between the third and fourth designated targets can be the number of parking spaces between adjacent pillars. The relative positional relationship between the third and fourth designated targets may not meet the requirements if the number of parking spaces between one pair of pillars is inconsistent with the number of parking spaces between other pairs of pillars.

[0088] In a case where the third specified target is a parking space entrance line and the fourth specified target is a neighboring lane line, the relative positional relationship between the third specified target and the fourth specified target can be a difference between a slope of the parking space entrance line and a slope of the neighboring lane line. The third specified target and the fourth specified target not satisfying the requirement can be that the difference between the slope of the parking space entrance line and the slope of the neighboring lane line exceeds a threshold value (which can be referred to as a third threshold value).

[0089] As an example, in a case where the third specified target and the fourth specified target do not satisfy the requirement, the confidence of the third specified target or the fourth specified target is reduced.

[0090] Taking a case where the third specified target is a parking space entrance line and the fourth specified target is a lane line as an example, assuming that the difference between the slope of the recognized parking space entrance line and the slope of the recognized lane line exceeds the third threshold value, the confidence of the parking space entrance line can be reduced according to the difference between the slope of the parking space entrance line and the slope of the recognized lane line.

[0091] As another example, in a case where the third specified target and the fourth specified target do not satisfy the requirement, the position of the third specified target or the fourth specified target can be adjusted to make the relative positional relationship between the third specified target and the fourth specified target satisfy the requirement.

[0092] Still taking a case where the third specified target is a parking space entrance line and the fourth specified target is a lane line as an example, assuming that the difference between the slope of the recognized parking space entrance line and the slope of the recognized lane line exceeds the third threshold value, the slope of the parking space entrance line can be adjusted to make the difference between the slope of the parking space entrance line and the slope of the recognized lane line not exceed the third threshold value.

[0093] In some embodiments, the method provided by the embodiments of the present application can further include:

[0094] The identification results of various types of perception elements are normalized to obtain normalized representations of the identification results of the various types of perception elements;

[0095] The normalized representations are input into a pre-trained deep learning model to obtain corrected identification results of the various types of perception elements; wherein the deep learning model takes normalized representations of training samples as input, the training samples are obtained by manually labeling perception information in a vehicle perception range, and the normalized representations of the training samples are obtained by normalizing the training samples.

[0096] Exemplarily, in the embodiments of the present application, the deep learning model training can also be performed in a deep learning manner, the real state of each target in the actual parking scene is learned by the deep learning model, and then the perception elements recognized according to the perception information are verified by using the trained deep learning model.

[0097] Considering that the output forms of different categories of perception elements are different in the actual parking scene, if normalization is not performed, it will cause difficulty for network learning.

[0098] Exemplarily, in the output of the perception elements, three types of targets can be included:

[0099] The first type: targets that can be output in the form of an envelope box, such as pedestrians, motor vehicles, parking lines, road signs, etc.

[0100] The second type: targets that can be output in the form of a curve equation, such as lane lines, road edges, etc.

[0101] The third type: targets that can be output in the form of a point, such as parking angle points, wheel contact points, etc.

[0102] In order to optimize the deep learning effect, the recognition results of each type of perception element can be normalized to obtain the normalized representation of the recognition results of each type of perception element.

[0103] In an example, the recognition results of each type of perception element are normalized to obtain the normalized representation of the recognition results of each type of perception element, which can include:

[0104] According to the coordinates of each type of perception element in a specified coordinate system, each type of perception element is filled into a grid map matching the vehicle perception range; wherein the filling value of the same type of perception element in the grid map is the same.

[0105] Exemplarily, in order to realize the normalization of the recognition results of each type of perception element, the vehicle perception range can be mapped to a grid map of a predetermined size.

[0106] For example, assuming that the vehicle perception range is M*N (unit: meters), it is mapped to a grid map of W*H (unit: grid), then the distance represented by one grid is M / W in the horizontal direction and N / H in the vertical direction.

[0107] Exemplarily, the representation of the placeholder grid can be used, according to the coordinates of the perception elements in a specified coordinate system (such as a vehicle body coordinate system), each type of perception element is filled into a grid map matching the vehicle perception range; wherein the filling value of the same type of perception element in the grid map is the same.

[0108] Exemplarily, in the deep learning model training process, the perception information acquired through the sensor can be manually labeled to obtain real information such as motor vehicles, pedestrians, non-motor vehicles, parking spaces, lane lines, road edges, road signs, general obstacles, and the like in the scene, and then the real information of each type of target is filled in the grid map using the same color (the same target filling value), to obtain a grid map true value, which is input into the deep learning model, the model is trained through the grid map true value, and a trained model is obtained.

[0109] Exemplarily, the deep learning model can include but is not limited to a Transformers model or a segmentation network model.

[0110] The trained deep learning model can be deployed in actual products, and the perception elements perceived in real time are normalized to obtain normalized representations such as grid maps, which are input into the trained deep learning model, the recognition results of the perception elements are corrected using the trained deep learning model, and the corrected recognition results output by the model are restored to the original perception element description mode.

[0111] In order for those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the technical solutions provided by the embodiments of the present application will be described below in conjunction with specific embodiments.

[0112] In this embodiment, target information in a parking scene can be obtained based on sensors and intelligent algorithms. The target information can include movable obstacles (motor vehicles, pedestrians, non-motor vehicles, etc.), immovable obstacles (no parking signs, road edges, wheel stops, ground locks, pillars, etc.), and road surface information (road surface, no parking area, guide signs, parking space lines, lane lines, etc.). When these information is output, it will have the distance information of the vehicle system relative to the vehicle.

[0113] In this embodiment, two modules can be combined to verify the reliability of the perception results and adjust the distance measurement. The schematic diagram can be as shown in Figure 2

[0114] Exemplarily, in the verification module based on explicit rules, the following three rules can be used to verify the target perception:

[0115] 1. Periodic rule based on the same type of target

[0116] Exemplarily, considering that in actual parking scenarios, most of the same type of targets have periodicity, such as the slope of adjacent lane lines, the size of parking spaces in a parking lot, the orientation of vehicles in different parking spaces, and the like. Based on this, the periodicity between the same type of targets can be used to verify the specified target (i.e., the first specified target) in the perception elements:

[0117] ​1.1, consistency judgment of information such as slope, curvature, orientation, length, etc. of adjacent lane lines and curb lines;

[0118] 1.2, consistency judgment of information such as size, orientation, type, etc. of adjacent parking spaces;

[0119] 1.3, consistency judgment of orientation of vehicles in adjacent parking spaces;

[0120] 1.4, consistency judgment of information such as numbered position, ground lock position, wheel stop position, etc. in adjacent parking spaces.

[0121] For example, if the first specified target has a difference from other similar targets or historical information that exceeds a preset threshold (such as the first threshold described above), the confidence of the first specified target can be reduced according to the difference, or the relevant attributes of the first specified target can be adjusted to meet the periodicity rule.

[0122] 2, relevance rule based on target component detection

[0123] For example, considering the actual parking scene, part of the identified target may belong to the components of other targets, such as parking corner points and parking spaces, wheels and vehicles, and the relative position relationship between the part of the target and the other target it belongs to needs to meet certain conditions. Based on this, the second specified target in the identified perception element can be verified according to the relative position relationship between the specified target (i.e. the second specified target described above) and the specified component in the identified perception element:

[0124] 2.1, judging the position relationship between the parking corner point and the parking target one by one;

[0125] 2.2, judging the position relationship between the wheel and the vehicle target one by one.

[0126] For example, if the difference between the relative position relationship between the second specified target and the specified component of the second specified target exceeds a set threshold (such as the second threshold described above), the confidence of the non-component target is reduced, or the position of the second specified target is adjusted to meet the relevance rule.

[0127] 3, relevance rule based on non-similar targets

[0128] For example, in a real parking scenario, there are some specific correlations between different categories of targets, such as the relative relationship between a column and a parking space in the same parking lot is constant (e.g., the number of parking spaces between two adjacent columns is usually consistent, the distance between the column and the adjacent parking space is usually consistent), the parking space entrance line is parallel to the lane line in the road, the vehicle driving in the lane and the road relationship (e.g., the vehicle orientation is parallel to the lane line), and the like. Thus, the identified perception elements can also be verified according to the relative position relationship between the specified different categories of targets in the identified perception elements:

[0129] 3.1, calculate the relative position relationship between each pair of columns and adjacent parking spaces;

[0130] 3.2, calculate the relative position relationship between each parking space entrance line and adjacent lane line;

[0131] 3.3, calculate the relative position relationship between each vehicle orientation and the parking space, lane.

[0132] For example, if the relative position difference between the third specified target (e.g., a column) and the fourth specified target (e.g., a parking space) in the identified perception elements exceeds a set threshold (e.g., the third threshold described above), the confidence of the third specified target can be reduced, or the related attributes of the third specified target can be adjusted to meet the correlation rule.

[0133] For example, in the data-driven verification module, a deep learning method can be used for automatic verification.

[0134] For example, in the output of the perception elements, three categories of targets can be included:

[0135] Category 1: targets that can be output in the form of an envelope box, such as pedestrians, motor vehicles, parking space lines, road signs, and the like;

[0136] Category 2: targets that can be output in the form of a curve equation, such as lane lines, road edges, and the like;

[0137] Category 3: targets that can be output in the form of a point, such as a parking space corner point, a wheel contact point, and the like.

[0138] Due to the differences in the output forms of the targets, if not normalized, it will cause difficulties for the learning of the network.

[0139] In order to optimize the deep learning effect, the identification results of various perception elements can be normalized to obtain the normalized representation of the identification results of the identified various perception elements.

[0140] For example, in the normalized representation of various types of targets, the representation of the placeholder grid can be used. In the vehicle body coordinate system, a WxH grid is constructed according to the vehicle perception range MxN (in meters, rectangular area with a preset distance range centered on the vehicle center). Each grid represents a distance of M / W in the horizontal direction and N / H in the vertical direction. Each type of target perceived is filled in the grid using the same color (for example, different fill patterns in the figure) (the same target filling value is the same), and the schematic diagram can be as shown in Figure 3A

[0141] In the deep learning model training process, the perception information obtained through the sensor can be manually labeled to obtain real information such as motor vehicles, pedestrians, non-motor vehicles, parking spaces, lane lines, road edges, road signs, and general obstacles in the scene. Each type of target in the real information is filled in the grid using the same color (the same target filling value is the same), and the grid true value is obtained and input into the deep learning model. The model is trained through the grid true value to obtain a trained model.

[0142] The trained deep learning model can be deployed in actual products. The real-time perception elements are normalized to obtain a normalized representation such as a grid, and are input into the trained deep learning model. The trained deep learning model is used to correct the recognition results of the perception elements, and the corrected recognition results output by the model are restored to the original perception element description method.

[0143] For example, as shown in the grid in Figure 3A , after automatic correction by the deep learning model, the output grid can be as shown in Figure 3B .

[0144] Among them, the grid in the box represents the lane line. The original lane line in the upper half of the figure is not a straight line, and the lane line grid after model adjustment is more consistent with the intuitive feeling.

[0145] The above describes the method provided by the present application. The device provided by the present application is described below:

[0146] Please refer to Figure 4 , a structure diagram of a perception information verification device provided by an embodiment of the present application, as shown in Figure 4 , the perception information verification device can include:

[0147] The identification unit 410 is configured to identify the perception elements according to the perception information in the vehicle perception range to obtain the identified perception elements.

[0148] The verification unit 420 is configured to verify the identified perception elements according to the mutual relationship between the specified targets in the identified perception elements.​

[0149] In some embodiments, the verification unit 420 verifies the identified perception elements according to the inter-relationship between the specified targets in the identified perception elements, including:

[0150] In the case that the identified perception elements include first specified targets, the first specified targets are verified according to the consistency between the same type of first specified targets; wherein the first specified targets are perception elements that exist periodically.

[0151] In some embodiments, the consistency between the same type of first specified targets includes:

[0152] In the case that the first specified targets are lane lines or road edge lines, the consistency of the slope, curvature, orientation and / or length information of adjacent first specified targets;

[0153] In the case that the first specified targets are parking spaces, the consistency of the size, orientation and / or type information of adjacent first specified targets;

[0154] In the case that the first specified targets are vehicles in parking spaces, the consistency of the orientation of adjacent first specified targets;

[0155] In the case that the first specified targets are numbers, ground locks or wheel stops in parking spaces, the consistency of the position information of adjacent first specified targets in the parking spaces.

[0156] In some embodiments, the verification unit 420 verifies the first specified targets according to the consistency between the same type of first specified targets, including:

[0157] For any first specified target, in the case that the consistency between the first specified target and other same type of first specified targets does not meet the requirements, the confidence of the first specified target is reduced, or the related attributes of the first specified target are adjusted to make the consistency between the first specified target and other same type of first specified targets meet the requirements.

[0158] In some embodiments, the verification unit 420 verifies the identified perception elements according to the inter-relationship between the specified targets in the identified perception elements, including:

[0159] In the case that the identified perception elements include second specified targets, the second specified targets are verified according to the relative position relationship between the second specified targets and the specified components of the identified second specified targets.

[0160] In some embodiments, in the case that the second specified targets are parking spaces, the specified components include the corner points of the parking spaces;

[0161] In a case where the second specified target is a vehicle, the specified component includes a wheel.

[0162] In some embodiments, the verification unit 420 verifies the second specified target according to a relative positional relationship between the second specified target and the specified component of the second specified target, including:

[0163] For any second specified target, in a case where a relative positional relationship between the second specified target and the specified component of the second specified target does not meet a requirement, the confidence of the second specified target is reduced, or the position of the second specified target is adjusted to make the relative positional relationship between the second specified target and the specified component of the second specified target meet the requirement.

[0164] In some embodiments, the verification unit 420 verifies the identified perception element according to a mutual relationship between specified targets in the identified perception element, including:

[0165] In a case where the identified perception element includes a third specified target and a fourth specified target, the third specified target or the fourth specified target is verified according to a relative positional relationship between the third specified target and the fourth specified target; wherein the third specified target and the fourth specified target are different categories of perception elements that are associated in relative position.

[0166] In some embodiments, in a case where the third specified target is a column, the fourth specified target is a parking space between adjacent columns;

[0167] In a case where the third specified target is a parking space entrance line, the fourth specified target is an adjacent lane line;

[0168] In a case where the third specified target is a vehicle in a parking space, the fourth specified target is a parking space or a lane.

[0169] In some embodiments, the verification unit 420 verifies the third specified target or the fourth specified target according to a relative positional relationship between the third specified target and the fourth specified target, including:

[0170] In a case where the relative positional relationship between the third specified target and the fourth specified target does not meet a requirement, the confidence of the third specified target or the fourth specified target is reduced, or the position of the third specified target or the fourth specified target is adjusted to make the relative positional relationship between the third specified target and the fourth specified target meet the requirement.

[0171] In some embodiments, asFigure 5 As shown, the perception information verification apparatus can further include:

[0172] The correction unit 430 is configured to normalize the recognition results of the perception elements of each type to obtain normalized representations of the recognition results of the perception elements of each type, and input the normalized representations into a pre-trained deep learning model to obtain corrected recognition results of the perception elements of each type. The deep learning model takes normalized representations of training samples as input, and the training samples are obtained by manually labeling the perception information within the vehicle perception range. The normalized representations of the training samples are obtained by normalizing the training samples.

[0173] In some embodiments, the correction unit 430 normalizes the recognition results of the perception elements of each type to obtain normalized representations of the recognition results of the perception elements of each type, including:

[0174] According to the coordinates of the perception elements of each type in the specified coordinate system, the perception elements of each type are filled into a grid map matching the vehicle perception range. The filling values of the perception elements of the same type in the grid map are the same.

[0175] Embodiments of the present application provide an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions capable of being executed by the processor, and the processor is configured to execute the machine executable instructions to implement the perception information verification method described above.

[0176] Please refer to Figure 6 A hardware structure schematic diagram of an electronic device is provided for embodiments of the present application. The electronic device can include a processor 601 and a memory 602 storing machine executable instructions. The processor 601 and the memory 602 can communicate via a system bus 603. By reading and executing the machine executable instructions corresponding to the perception information verification logic in the memory 602, the processor 601 can execute the perception information verification method described above.

[0177] The memory 602 mentioned herein can be any electronic, magnetic, optical or other physical storage device, and can contain or store information such as executable instructions, data, etc. For example, the machine readable storage medium can be RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard drive), solid state disk, any type of storage disk (such as optical disk, dvd, etc.), or similar storage medium, or combination thereof.

[0178] In some embodiments, a storage medium is also provided, such as Figure 6The storage medium in the memory 602 is a machine readable storage medium, and machine executable instructions are stored in the storage medium. The machine executable instructions are executed by the processor to implement the above-mentioned perception information verification method. For example, the storage medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one entity or action from another, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0180] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method of verifying perception information, characterized by, The method comprises: performing perception element recognition according to perception information within a perception range of a vehicle to obtain recognized perception elements; performing verification on the recognized perception elements according to a mutual relationship between specified targets in the recognized perception elements; wherein the performing verification on the recognized perception elements according to the mutual relationship between the specified targets in the recognized perception elements comprises: in a case where the recognized perception elements include a first specified target, performing verification on the first specified target according to consistency between same-type first specified targets; wherein the first specified target is a perception element that has a periodic rule; the performing verification on the first specified target according to the consistency between the same-type first specified targets comprises: for any first specified target, in a case where it is determined that consistency between the first specified target and other same-type first specified targets does not meet a requirement, reducing confidence of the first specified target, or adjusting a related attribute of the first specified target to make the consistency between the first specified target and the other same-type first specified targets meet the requirement.

2. The method of claim 1, wherein, the consistency between the same-type first specified targets comprises: in a case where the first specified target is a lane line or a road edge line, consistency of slope, curvature, orientation and / or length information of adjacent first specified targets; in a case where the first specified target is a parking space, consistency of size, orientation and / or type information of adjacent first specified targets; in a case where the first specified target is a vehicle in a parking space, consistency of orientation of adjacent first specified targets; in a case where the first specified target is a number, a ground lock or a wheel stop in a parking space, consistency of position information of adjacent first specified targets in the parking space.

3. The method of claim 1, wherein, the performing verification on the recognized perception elements according to the mutual relationship between the specified targets in the recognized perception elements further comprises: in a case where the recognized perception elements further include a second specified target, performing verification on the second specified target according to a relative position relationship between the second specified target and a specified component of the recognized second specified target.

4. The method of claim 3, wherein, in a case where the second specified target is a parking space, the specified component comprises a corner point of the parking space; in a case where the second specified target is a vehicle, the specified component comprises a wheel.

5. The method of claim 3, wherein, the performing verification on the second specified target according to the relative position relationship between the second specified target and the specified component of the recognized second specified target comprises: for any second specified target, in a case where it is determined that the relative position relationship between the second specified target and the specified component of the second specified target does not meet a requirement, reducing confidence of the second specified target, or adjusting a position of the second specified target to make the relative position relationship between the second specified target and the specified component of the second specified target meet the requirement.

6. The method of claim 1, wherein, the performing verification on the recognized perception elements according to the mutual relationship between the specified targets in the recognized perception elements further comprises: In a case where the identified perception elements further include a third specified target and a fourth specified target, the third specified target or the fourth specified target is verified according to a relative positional relationship between the third specified target and the fourth specified target; the third specified target and the fourth specified target are different categories of perception elements that are associated in relative positions.

7. The method of claim 6, wherein, In a case where the third specified target is a column, the fourth specified target is a parking space between adjacent columns. In a case where the third specified target is a parking space entrance line, the fourth specified target is an adjacent lane line. In a case where the third specified target is a vehicle in a parking space, the fourth specified target is a parking space or a lane.

8. The method of claim 6, wherein, The verification of the third specified target or the fourth specified target according to the relative positional relationship between the third specified target and the fourth specified target includes: In a case where the relative positional relationship between the third specified target and the fourth specified target does not meet a requirement, a confidence of the third specified target or the fourth specified target is reduced, or a position of the third specified target or the fourth specified target is adjusted to make the relative positional relationship between the third specified target and the fourth specified target meet the requirement.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: normalizing the identification results of the perception elements to obtain normalized representations of the identification results of the perception elements; inputting the normalized representations into a pre-trained deep learning model to obtain corrected identification results of the perception elements; the deep learning model takes normalized representations of training samples as inputs, the training samples are obtained by manually labeling perception information in a vehicle perception range, and the normalized representations of the training samples are obtained by normalizing the training samples.

10. The method of claim 9, wherein, The normalization of the identification results of the perception elements to obtain the normalized representations of the identification results of the perception elements includes: filling the perception elements into a grid map matching the vehicle perception range according to coordinates of the perception elements in a specified coordinate system; the same category of perception elements has the same filling value in the grid map.

11. A perception information verification apparatus characterized by comprising: The method includes: an identification unit configured to identify perception elements according to perception information in a vehicle perception range to obtain identified perception elements; a verification unit configured to verify the identified perception elements according to mutual relationships between specified targets in the identified perception elements; The verification of the identified perception elements according to the mutual relationships between the specified targets in the identified perception elements includes: in a case where the identified perception elements include a first specified target, the first specified target is verified according to consistency between the same category of first specified targets; the first specified target is a perception element with a periodic law configured in advance; The verification of the first specified target according to the consistency between the same category of first specified targets includes: For any first specified target, in a case where it is determined that the consistency between the first specified target and other first specified targets of the same type does not meet the requirement, the confidence of the first specified target is reduced, or the relevant attribute of the first specified target is adjusted so that the consistency between the first specified target and other first specified targets of the same type meets the requirement.

12. The apparatus of claim 11, wherein, the consistency between the first specified targets of the same type comprises: in a case where the first specified target is a lane line or a road edge line, the consistency of the slope, curvature, orientation and / or length information of adjacent first specified targets; in a case where the first specified target is a parking space, the consistency of the size, orientation and / or type information of adjacent first specified targets; in a case where the first specified target is a vehicle in a parking space, the consistency of the orientation of adjacent first specified targets; in a case where the first specified target is a number, a ground lock or a wheel stop in a parking space, the consistency of the position information of adjacent first specified targets in the parking space; and / or, the verification unit verifies the identified perception elements according to the mutual relationship between the specified targets in the identified perception elements, and further comprises: in a case where the identified perception elements further comprise a second specified target, the second specified target is verified according to the relative position relationship between the second specified target and a specified part of the second specified target; wherein, in a case where the second specified target is a parking space, the specified part comprises a corner point of the parking space; in a case where the second specified target is a vehicle, the specified part comprises a wheel; wherein, the verification unit verifies the second specified target according to the relative position relationship between the second specified target and the specified part of the second specified target, comprises: for any second specified target, in a case where it is determined that the relative position relationship between the second specified target and the specified part of the second specified target does not meet the requirement, the confidence of the second specified target is reduced, or the position of the second specified target is adjusted so that the relative position relationship between the second specified target and the specified part of the second specified target meets the requirement; and / or, the verification unit verifies the identified perception elements according to the mutual relationship between the specified targets in the identified perception elements, and further comprises: in a case where the identified perception elements further comprise a third specified target and a fourth specified target, the third specified target or the fourth specified target is verified according to the relative position relationship between the third specified target and the fourth specified target; wherein, the third specified target and the fourth specified target are different types of perception elements that are associated in relative position; wherein, in a case where the third specified target is a column, the fourth specified target is a parking space between adjacent columns; in a case where the third specified target is a parking entrance line, the fourth specified target is an adjacent lane line; in a case where the third specified target is a vehicle in a parking space, the fourth specified target is a parking space or a lane; The verification unit verifies the third specified target or the fourth specified target according to a relative position relationship between the third specified target and the fourth specified target, and the verification includes: In a case where the relative position relationship between the third specified target and the fourth specified target does not meet a requirement, the confidence of the third specified target or the fourth specified target is reduced, or the position of the third specified target or the fourth specified target is adjusted to make the relative position relationship between the third specified target and the fourth specified target meet the requirement. And / or, The apparatus further includes: The correction unit performs normalization processing on the recognition results of the various types of perception elements to obtain normalized representations of the recognition results of the various types of perception elements, and inputs the normalized representations into a pre-trained deep learning model to obtain corrected recognition results of the various types of perception elements, wherein the deep learning model takes normalized representations of training samples as input, the training samples are obtained by manually annotating perception information within a vehicle perception range, and the normalized representations of the training samples are obtained by performing normalization processing on the training samples. The correction unit performs normalization processing on the recognition results of the various types of perception elements to obtain normalized representations of the recognition results of the various types of perception elements, and inputs the normalized representations into a pre-trained deep learning model to obtain corrected recognition results of the various types of perception elements, wherein the deep learning model takes normalized representations of training samples as input, the training samples are obtained by manually annotating perception information within a vehicle perception range, and the normalized representations of the training samples are obtained by performing normalization processing on the training samples. The correction unit performs normalization processing on the recognition results of the various types of perception elements to obtain normalized representations of the recognition results of the various types of perception elements, and inputs the normalized representations into a pre-trained deep learning model to obtain corrected recognition results of the various types of perception elements, wherein the deep learning model takes normalized representations of training samples as input, the training samples are obtained by manually annotating perception information within a vehicle perception range, and the normalized representations of the training samples are obtained by performing normalization processing on the training samples.

13. An electronic device, comprising: The processor is configured to execute the machine-executable instructions to implement the method of any one of claims 1-10.

14. A storage medium, characterized by The storage medium stores machine-executable instructions, and the machine-executable instructions are executed by the processor to implement the method of any one of claims 1-10.

Images