Intersection key point detection method, intelligent device and storage medium
By integrating multimodal information in complex low-speed scenarios and inputting the intersection key point detection model, the problem of inaccurate intersection detection in the existing technology is solved, precise detection of intersection key points is achieved, and the perception and decision-making capabilities of autonomous driving are improved.
Patent Information
- Application Number
- CN202411855788.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-11
AI Technical Summary
The prior art is difficult to achieve accurate intersection detection in complex low-speed scenarios, resulting in autonomous vehicles being in a state where no graphs are available in these scenarios.
By obtaining multimodal information of the target scene, including vector maps, lidar point cloud data and vehicle driving trajectory, and performing heterogeneous fusion, feature images are generated. Then, the feature image is input into the trained intersection key point detection model to detect the intersection key point.
Accurate detection of key intersections in complex low-speed scenarios has been achieved, and the perception ability and decision-making reliability of autonomous driving in these scenarios has been improved.
Smart Images
Figure CN119810793A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a method for detecting key points at intersections, an intelligent device, and a storage medium. Background Art
[0002] In the field of autonomous driving, accurate intersection detection is crucial in tasks such as high-precision map construction, driving navigation, and planning and control. At the same time, autonomous driving technology has a strong reliance on high-precision vector maps, which play an important role in positioning, perception, planning and control, and have become an indispensable part of the autonomous driving technology stack.
[0003] However, the existing high-precision map coverage mainly includes structured roads such as urban areas and highways. In complex low-speed scenes such as parking lots and battery swap stations, autonomous vehicles are in a stage where no maps are available. The construction of high-precision maps for complex low-speed scenes is inseparable from intersection detection. The accuracy of intersection key point detection directly determines the topological accuracy of high-precision maps.
[0004] Accordingly, the art needs a key point detection solution for intersections suitable for complex low-speed scenes to solve the above problems. Summary of the invention
[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem of how to accurately detect intersections in complex low-speed scenes.
[0006] In a first aspect, a method for detecting key points at an intersection is provided, wherein the method comprises:
[0007] Acquire multimodal information of a target scene; the multimodal information includes a vector map of the target scene, laser radar point cloud data, and a vehicle driving trajectory;
[0008] Performing heterogeneous fusion on the multimodal information to obtain a feature image of the target scene;
[0009] The characteristic image of the target scene is input into a trained intersection key point detection model to obtain the intersection key point detection result of the target scene.
[0010] In a technical solution of the above-mentioned intersection key point detection method, the heterogeneous fusion of the multimodal information to obtain the characteristic image of the target scene includes:
[0011] Projecting the multimodal information into a unified plane coordinate system;
[0012] In the unified plane coordinate system, the multimodal information is heterogeneously fused to obtain a feature image of the target scene. In a technical solution of the above-mentioned intersection key point detection method, the feature image of the target scene includes a hierarchical feature image, and the method further includes obtaining the hierarchical feature image based on the following steps: Acquiring height information of the target scene; stratifying the target scene based on the height information;
[0013] The multimodal information of each layer of target scene is heterogeneously fused to obtain the hierarchical feature image corresponding to each layer of target scene.
[0014] In a technical solution of the above-mentioned intersection key point detection method, the heterogeneous fusion of multimodal information of each layer of target scene to obtain a hierarchical feature image corresponding to each layer of target scene includes:
[0015] Perform voxel downsampling on the lidar point cloud data of each layer of the target scene to obtain voxelized point cloud data;
[0016] Obtain vector parking space information based on the vector map of each target scene;
[0017] rasterizing the voxelized point cloud data, the vector parking space information and the vehicle driving trajectory of each layer of the target scene to obtain a plurality of raster units;
[0018] A hierarchical feature image corresponding to each layer of the target scene is obtained based on the multiple grid units.
[0019] In a technical solution of the above-mentioned intersection key point detection method, inputting the feature image of the target scene into a trained intersection key point detection model to obtain the intersection key point detection result of the target scene includes:
[0020] Inputting the hierarchical feature images corresponding to each layer of the target scene into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of the target scene;
[0021] The intersection key point detection result of the target scene is obtained based on the intersection key point detection results corresponding to all layers of the target scene.
[0022] In a technical solution of the above-mentioned intersection key point detection method, the hierarchical feature image corresponding to each layer of target scene includes a plurality of sub-feature images; the hierarchical feature image corresponding to each layer of target scene is input into the intersection key point detection model to obtain the intersection key point detection result corresponding to each layer of target scene, which includes:
[0023] Inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image;
[0024] The key point detection results of the intersections corresponding to each layer of the target scene are obtained based on the key point detection results of the intersections of all the sub-feature images corresponding to each layer of the target scene.
[0025] In a technical solution of the above-mentioned intersection key point detection method, the method further includes obtaining the multiple sub-feature images based on the following steps:
[0026] The layered feature image is segmented by a sliding window based on a preset size and a preset step size to obtain a plurality of sub-feature images corresponding to each layer of the target scene.
[0027] In a technical solution of the above-mentioned intersection key point detection method, the intersection key point detection model includes a feature extraction module and a key point decoding module; the step of inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image includes:
[0028] Inputting each sub-feature image into the feature extraction module to perform multi-scale feature extraction to obtain feature images of multiple scales;
[0029] Inputting the feature images of the multiple scales into a key point decoding module to obtain coordinates of multiple key points;
[0030] Based on the coordinates of the multiple key points, a detection result of the intersection key points of each sub-feature image is obtained.
[0031] In a technical solution of the above-mentioned intersection key point detection method, the key point decoding module includes an initialization module, a self-attention module, a cross-attention module and a multi-layer perceptron; the inputting the feature images of the multiple scales into the key point decoding module to obtain the coordinates of the multiple key points includes:
[0032] a. Based on the initialization module, the feature images of the multiple scales are randomly initialized to obtain multiple key points and initial features of each key point;
[0033] b. performing feature fusion on the initial features of the multiple key points based on the self-attention module;
[0034] c. acquiring feature information of the plurality of key points after the feature fusion on the feature images of the plurality of scales based on the cross attention module;
[0035] d. Repeat step bc based on the multilayer perceptron until the number of executions reaches a preset number of layers of the multilayer perceptron, and determine the coordinates of the multiple key points based on the feature information of the multiple key points obtained by repeatedly executing step bc.
[0036] In a technical solution of the above-mentioned intersection key point detection method, the intersection key point detection results corresponding to each layer of the target scene are obtained based on the intersection key point detection results of all sub-feature images corresponding to each layer of the target scene, including:
[0037] Projecting all sub-feature images corresponding to each layer of the target scene onto the layered feature image, and splicing all the projected sub-feature images to obtain a spliced image;
[0038] Performing non-maximum suppression processing on the key points of the intersection in the overlapping area in the stitched image to obtain the key points of the intersection in the overlapping area;
[0039] Based on the intersection key points in the overlapped area and the intersection key points in the non-overlapped area, the intersection key point detection results corresponding to each layer of the target scene are obtained.
[0040] In a technical solution of the above-mentioned intersection key point detection method, the intersection key point detection result of each sub-feature image includes the confidence of each intersection key point in the sub-feature image; the non-maximum suppression processing is performed on the intersection key points in the overlapping area of the spliced image to obtain the intersection key points in the overlapping area, including:
[0041] Acquire a point set corresponding to the intersection key point in the overlapped area; wherein the point set corresponding to the intersection key point includes the intersection key point and other intersection key points within a preset distance range around the intersection key point;
[0042] Obtaining the confidence of all key points at the intersection in the point set;
[0043] Taking the point with the highest confidence in each point set as the intersection key point corresponding to each point set;
[0044] Based on the intersection key points corresponding to all point sets, the intersection key points of the overlapped area are obtained.
[0045] In a technical solution of the above-mentioned intersection key point detection method, the intersection key point detection model is trained based on at least the following steps:
[0046] Obtaining a training sample set; the training sample set is a true value feature image with key points of the intersection marked;
[0047] Training the feature extraction module and the key point decoding module in the intersection key point detection model based on the training sample set;
[0048] The loss function of the intersection key point detection model is obtained, and when the loss function reaches a preset threshold, the training of the intersection key point detection model is completed.
[0049] In a second aspect, a smart device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned intersection key point detection method is implemented.
[0050] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned intersection key point detection method.
[0051] Solution 1. A method for detecting key points at intersections, characterized in that the method comprises:
[0052] Acquire multimodal information of a target scene; the multimodal information includes a vector map of the target scene, laser radar point cloud data, and a vehicle driving trajectory;
[0053] Performing heterogeneous fusion on the multimodal information to obtain a feature image of the target scene;
[0054] The characteristic image of the target scene is input into a trained intersection key point detection model to obtain the intersection key point detection result of the target scene.
[0055] Solution 2. The method for detecting key points at intersections according to Solution 1 is characterized in that the step of performing heterogeneous fusion on the multimodal information to obtain a characteristic image of the target scene comprises:
[0056] Projecting the multimodal information into a unified plane coordinate system;
[0057] In the unified plane coordinate system, the multimodal information is heterogeneously fused to obtain a feature image of the target scene.
[0058] Solution 3. The method for detecting key points at intersections according to Solution 2 is characterized in that the feature image of the target scene includes a hierarchical feature image, and the method further comprises obtaining the hierarchical feature image based on the following steps:
[0059] Acquiring height information of the target scene;
[0060] stratifying the target scene based on the height information;
[0061] The multimodal information of each layer of target scene is heterogeneously fused to obtain the hierarchical feature image corresponding to each layer of target scene.
[0062] Solution 4. The method for detecting key points at intersections according to Solution 3 is characterized in that the multimodal information of each layer of target scenes is heterogeneously fused to obtain a hierarchical feature image corresponding to each layer of target scenes, which includes:
[0063] Perform voxel downsampling on the lidar point cloud data of each layer of the target scene to obtain voxelized point cloud data;
[0064] Obtain vector parking space information based on the vector map of each target scene;
[0065] rasterizing the voxelized point cloud data, the vector parking space information and the vehicle driving trajectory of each layer of the target scene to obtain a plurality of raster units;
[0066] A hierarchical feature image corresponding to each layer of the target scene is obtained based on the multiple grid units.
[0067] Solution 5. The method for detecting key points at intersections according to Solution 3 is characterized in that the step of inputting the feature image of the target scene into a trained key point detection model for intersections to obtain the key point detection result of the target scene comprises:
[0068] Inputting the hierarchical feature images corresponding to each layer of the target scene into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of the target scene;
[0069] The intersection key point detection result of the target scene is obtained based on the intersection key point detection results corresponding to all layers of the target scene.
[0070] Solution 6. The method for detecting key points at intersections according to Solution 5 is characterized in that the hierarchical feature image corresponding to each layer of target scene includes a plurality of sub-feature images; the hierarchical feature image corresponding to each layer of target scene is input into the key point detection model for intersections to obtain the key point detection results for each layer of target scene, including:
[0071] Inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image;
[0072] The key point detection results of the intersections corresponding to each layer of the target scene are obtained based on the key point detection results of the intersections of all the sub-feature images corresponding to each layer of the target scene.
[0073] Solution 7. The method for detecting key points at intersections according to Solution 6 is characterized in that the method further comprises obtaining the multiple sub-feature images based on the following steps:
[0074] The layered feature image is segmented by a sliding window based on a preset size and a preset step size to obtain a plurality of sub-feature images corresponding to each layer of the target scene.
[0075] Solution 8. The intersection key point detection method according to Solution 6 is characterized in that the intersection key point detection model includes a feature extraction module and a key point decoding module; the step of inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image includes:
[0076] Inputting each sub-feature image into the feature extraction module to perform multi-scale feature extraction to obtain feature images of multiple scales;
[0077] Inputting the feature images of the multiple scales into a key point decoding module to obtain coordinates of multiple key points;
[0078] Based on the coordinates of the multiple key points, a detection result of the intersection key points of each sub-feature image is obtained.
[0079] Solution 9. The method for detecting key points at intersections according to Solution 8 is characterized in that the key point decoding module includes an initialization module, a self-attention module, a cross-attention module and a multi-layer perceptron; the step of inputting the feature images of the multiple scales into the key point decoding module to obtain the coordinates of the multiple key points includes:
[0080] a. Based on the initialization module, the feature images of the multiple scales are randomly initialized to obtain multiple key points and initial features of each key point;
[0081] b. performing feature fusion on the initial features of the multiple key points based on the self-attention module;
[0082] c. acquiring feature information of the plurality of key points after the feature fusion on the feature images of the plurality of scales based on the cross attention module;
[0083] d. Repeat step bc based on the multilayer perceptron until the number of executions reaches a preset number of layers of the multilayer perceptron, and determine the coordinates of the multiple key points based on the feature information of the multiple key points obtained by repeatedly executing step bc.
[0084] Solution 10. The method for detecting key points at intersections according to Solution 6 is characterized in that the key point detection results at intersections corresponding to each layer of target scenes obtained based on the key point detection results at intersections of all sub-feature images corresponding to each layer of target scenes include:
[0085] Projecting all sub-feature images corresponding to each layer of the target scene onto the layered feature image, and splicing all the projected sub-feature images to obtain a spliced image;
[0086] Performing non-maximum suppression processing on the key points of the intersection in the overlapping area in the stitched image to obtain the key points of the intersection in the overlapping area;
[0087] Based on the intersection key points in the overlapped area and the intersection key points in the non-overlapped area, the intersection key point detection results corresponding to each layer of the target scene are obtained.
[0088] Solution 11. The method for detecting intersection key points according to Solution 10 is characterized in that the intersection key point detection result of each sub-feature image includes the confidence of each intersection key point in the sub-feature image; the intersection key points in the overlapping area of the spliced image are subjected to non-maximum suppression processing to obtain the intersection key points in the overlapping area, which includes:
[0089] Acquire a point set corresponding to the intersection key point in the overlapped area; wherein the point set corresponding to the intersection key point includes the intersection key point and other intersection key points within a preset distance range around the intersection key point;
[0090] Obtaining the confidence of all key points at the intersection in the point set;
[0091] Taking the point with the highest confidence in each point set as the intersection key point corresponding to each point set;
[0092] Based on the intersection key points corresponding to all point sets, the intersection key points of the overlapped area are obtained.
[0093] Solution 12. The method for detecting key points at intersections according to Solution 8 is characterized in that the key point detection model for intersections is trained based on at least the following steps:
[0094] Obtaining a training sample set; the training sample set is a true value feature image with key points of the intersection marked;
[0095] Training the feature extraction module and the key point decoding module in the intersection key point detection model based on the training sample set;
[0096] The loss function of the intersection key point detection model is obtained, and when the loss function reaches a preset threshold, the training of the intersection key point detection model is completed.
[0097] Solution 13. A smart device, comprising:
[0098] at least one processor;
[0099] and, a memory communicatively coupled to the at least one processor;
[0100] Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the intersection key point detection method described in any one of Schemes 1 to 12 is implemented.
[0101] Solution 14. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are suitable for being loaded and run by a processor to execute the intersection key point detection method described in any one of Solutions 1 to 12.
[0102] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:
[0103] In the implementation of the technical solution provided by this application, multimodal information such as vector maps of target scenes, LiDAR point cloud data, and vehicle driving trajectories are fully integrated. Among them, the vector map covers the geometric information of parking spaces, the LiDAR point cloud data accurately depicts the three-dimensional structure of the environment, and the vehicle driving trajectories record the actual driving paths of many vehicles in detail. These information sources are rich and diverse, which can enhance the comprehensiveness of the perception of complex low-speed scenes and improve the accuracy of the data. Furthermore, inputting the fused multimodal information into the intersection key point detection model of the Transformer architecture can give full play to the model's powerful ability to process complex data and extract features, realize accurate detection of intersection key points, and provide more reliable decision-making basis for autonomous driving in complex low-speed scenes. BRIEF DESCRIPTION OF THE DRAWINGS
[0104] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:
[0105] Figure 1 It is a schematic flow chart of the main steps of a method for detecting key points at intersections according to an embodiment of the present application;
[0106] Figure 2 is a schematic diagram of heterogeneous fusion of multimodal information according to an embodiment of the present application;
[0107] Figure 3 is a schematic flow chart of the main steps of obtaining a layered feature image according to an embodiment of the present application;
[0108] Figure 4 It is a schematic flow chart of the main steps of inputting the hierarchical feature images corresponding to each layer of target scene into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of target scene according to an embodiment of the present application;
[0109] Figure 5 is a schematic diagram of obtaining a key point detection result of an intersection of each sub-feature image through a key point detection model of an intersection according to an embodiment of the present application;
[0110] Figure 6 is a schematic diagram of stitching images according to an embodiment of the present application;
[0111] Figure 7 is a schematic diagram of a method for detecting key points at intersections according to an embodiment of the present application;
[0112] Figure 8 is a schematic diagram of a smart device according to an embodiment of the present application.
[0113] Reference numerals:
[0114] 81: memory; 82: processor. DETAILED DESCRIPTION
[0115] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.
[0116] In the description of the present application, the term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B, or A and B. The singular terms "a", "the" and "the" may also include plural forms.
[0117] As mentioned in the background technology, the existing high-precision map coverage mainly includes urban areas, highways and other structured roads. In complex low-speed scenarios such as parking lots, battery swap stations, and industrial parks, autonomous driving vehicles are in a stage where no maps are available.
[0118] The construction of high-precision maps for complex low-speed scenes is inseparable from intersection detection. If the high-precision map is regarded as a directed graph, the intersection represents the vertices in the graph. Therefore, the accuracy of intersection key point detection directly determines the topological accuracy of the high-precision map.
[0119] In order to solve the above problems, the present application provides a method for detecting key points at intersections, an intelligent device and a storage medium.
[0120] See attached Figure 1 , Figure 1 FIG. 1 is a schematic flow chart of the main steps of a method for detecting key points at intersections according to an embodiment of the present application. Figure 1As shown, the intersection key point detection method in the embodiment of the present application mainly includes the following steps S101 to S103.
[0121] Step S101: Acquire multimodal information of a target scene;
[0122] Among them, the target scenes can include complex low-speed scenes such as parking lots, battery swap stations, and inside industrial parks, as well as structured scenes including urban areas and highways.
[0123] The multimodal information includes vector maps of the target scene, lidar point cloud data, and vehicle trajectory.
[0124] Step S102: performing heterogeneous fusion on multimodal information to obtain a feature image of the target scene;
[0125] Step S103: inputting the characteristic image of the target scene into the trained intersection key point detection model to obtain the intersection key point detection result of the target scene.
[0126] Among them, the key points of intersections are the points where different roads converge, intersect and diverge in the traffic network of the target scene.
[0127] Based on the method described in the above steps S101 to S103, multimodal information such as the vector map of the target scene, the LiDAR point cloud data and the vehicle driving trajectory are fully integrated. Among them, the vector map covers the geometric information of the parking space, the LiDAR point cloud data accurately depicts the three-dimensional structure of the environment, and the vehicle driving trajectory records the actual driving paths of many vehicles in detail. These information sources are rich and diverse, which can enhance the comprehensiveness of the perception of complex low-speed scenes and improve the accuracy of the data. Furthermore, inputting the fused multimodal information into the intersection key point detection model of the Transformer architecture can give full play to the model's powerful ability to process complex data and extract features, realize accurate detection of intersection key points, and provide more reliable decision-making basis for autonomous driving in complex low-speed scenes.
[0128] The above steps S101 to S103 are further explained below.
[0129] In some implementations of step S101, vector maps, lidar point cloud data, and vehicle driving trajectories of complex low-speed scenes such as parking lots, battery swap stations, and inside industrial parks may be obtained.
[0130] The vector map can be a high-precision map provided by a map data supplier or surveyed by a professional surveying and mapping agency. A vector map is a map that stores geographic information in a vector data format. It accurately describes the location, shape and attributes of geographic entities such as roads, traffic signs and buildings through geometric elements such as points, lines and surfaces. The vector map in the target scene contains vector parking space information, which can mainly include the shape (such as rectangle, parallelogram, etc.) and size (length, width and other dimensions) of the parking space, as well as the relative position relationship between parking spaces (such as distance, angle), etc.
[0131] LiDAR point cloud data is the distance information of the target object obtained by the LiDAR sensor emitting a laser beam and receiving the reflected light. It is point cloud data composed of a large number of three-dimensional points. LiDAR point cloud data accurately depicts the three-dimensional structure of the target scene environment and can present the position, shape and height information of various objects in the scene.
[0132] The vehicle trajectory is the actual driving path of the vehicle in the target scene obtained by sensors such as the global positioning system and inertial measurement unit equipped by many vehicles. The vehicle trajectory mainly includes the change information of the vehicle position over time, and can also include the coordinate points passed by the vehicle, the change information of driving speed, the steering angle information, etc.
[0133] The above multimodal information sources are rich and diverse, which can improve the comprehensiveness of perception of complex low-speed scenes and provide accurate data for subsequent processing.
[0134] The above is a further description of step S101 , and the following is a further description of step S102 .
[0135] In some implementations of the above step S102, see the attached Figure 2 , Figure 2 FIG. 1 is a schematic diagram of heterogeneous fusion of multimodal information according to an embodiment of the present application. Figure 2 As shown, the multimodal information can be projected into a unified plane coordinate system, and the multimodal information can be heterogeneously fused in the unified plane coordinate system to obtain a feature image of the target scene.
[0136] Among them, the vector map of the target scene, the lidar point cloud data and the vehicle driving trajectory are all saved in the geographic coordinate system (GCS). This coordinate system is a spherical coordinate system. It uses the earth as a reference sphere and uses longitude, latitude and altitude to determine the position of any point on the earth.
[0137] In order to obtain the characteristic image of the target scene, the vector map, lidar point cloud data and vehicle driving trajectory in the GCS coordinate system can be projected into a unified plane coordinate system, such as the East-North-Up (EUN) coordinate system, the Universal Transverse Mercator Coordinate System (UTM) coordinate system, etc.
[0138] Taking the EUN coordinate system as an example, we can select a fixed projection reference point (called the station center, denoted as x_ref), and use this point as a reference to project the above multimodal information into the same ENU plane coordinate system, so that information of different modes can be in a unified feature space, preparing for subsequent heterogeneous fusion.
[0139] Furthermore, considering that in actual low-speed scenes, there may be multi-layer spaces such as multi-layer underground parking lots. Since the spatial layout, object distribution, and traffic conditions of different floors are often different, in order to avoid confusion of multi-layer information, the target scene can be layered first to obtain the layered feature image corresponding to each layer of the target scene.
[0140] For details, see the attached Figure 3 , Figure 3 FIG. 1 is a flow chart of the main steps of obtaining a hierarchical feature image according to an embodiment of the present application. Figure 3 As shown, it mainly includes the following steps S301 to S303.
[0141] Step S301: Acquire height information of the target scene;
[0142] In some implementations, the height data of different objects and different locations in the target scene can be acquired through the LiDAR point cloud data, thereby obtaining the height information of the entire target scene.
[0143] Step S302: layering the target scene based on the height information;
[0144] Specifically, the entire target scene can be divided into different levels according to certain height range standards. For example, in a multi-storey parking lot, the height of 0-4 meters can be divided into the first floor, the height of 4-7.8 meters can be divided into the second floor, and the height of 7.8-11.3 meters can be divided into the third floor.
[0145] If the target scene is determined to be a single-layer space based on the height information, no stratification is performed and the target scene is treated as a single layer for processing.
[0146] Step S303: heterogeneously fuse the multimodal information of each layer of target scene to obtain a hierarchical feature image corresponding to each layer of target scene.
[0147] In some implementations, step S303 may include the following steps S3031 to S3034:
[0148] Step S3031: performing voxel downsampling processing on the laser radar point cloud data of each layer of the target scene to obtain voxelized point cloud data;
[0149] Since the LiDAR point cloud data is very dense, it is difficult to process directly and the data volume is huge, so it needs to be voxelized and downsampled.
[0150] Voxel downsampling of dense point cloud data is to divide the entire three-dimensional space into multiple voxel units of equal size, and classify the numerous point cloud data into corresponding voxels. Each voxel can be regarded as an aggregation and simplified representation of the original point cloud data within a certain spatial range. This can reduce the amount of data and reduce the complexity of subsequent processing.
[0151] Step S3032: obtaining vector parking space information based on the vector map of each layer of the target scene;
[0152] Since the vector map of the target scene contains vector parking space information, the vector parking space information can be obtained through the vector map of each layer of the target scene, which mainly includes the shape (such as rectangle, parallelogram, etc.), size (length, width and other dimensions) of the parking space, and the relative position relationship between the parking spaces (such as distance, angle), etc.
[0153] Step S3033: performing rasterization processing on the voxelized point cloud data, vector parking space information and vehicle driving trajectory of each layer of the target scene to obtain a plurality of raster units;
[0154] Rasterization processing is to convert voxelized point cloud data, vector parking space information and vehicle driving trajectory into a grid-based (small squares) representation.
[0155] Specifically, a suitable grid size may be determined first, such as setting the resolution of each grid to 0.24*0.24 meters, and then dividing the plane area where the entire target scene is located into a plurality of such grid units.
[0156] Furthermore, the corresponding points in the voxelized point cloud data are assigned to the corresponding grid cells according to their locations. Similarly, the vector parking space information is also mapped to the corresponding grid cells to determine which grids contain parking spaces and the specific range of parking spaces. For vehicle driving trajectories, the grid cells that the trajectory passes through are marked according to the position coordinates of the vehicle. Through such rasterization processing, information of different modalities is integrated into a unified grid unit framework.
[0157] Step S3034: obtaining a hierarchical feature image corresponding to each layer of the target scene based on multiple grid units.
[0158] That is, multiple grid units are processed into the form of images to generate hierarchical feature images corresponding to each layer of the target scene.
[0159] Among them, the feature image can represent different information through different colors, such as gray for non-drivable areas in vector maps and lidar point cloud data, red for vehicle driving tracks, white for drivable areas, etc. Different lines can also be used to represent different information, such as solid lines for non-drivable areas in vector maps and lidar point cloud data, dotted lines for vehicle driving tracks, and blanks for drivable areas, etc., which are not limited here.
[0160] Through the above implementation, a hierarchical feature image corresponding to each layer of the target scene can be obtained.
[0161] Furthermore, since different target scenes vary in size, the model cannot handle feature images of different scales in subsequent processing. Therefore, the layered feature image can be segmented by a sliding window based on a preset size and a preset step size to obtain multiple sub-feature images corresponding to each layer of the target scene, and unify the size of the sub-feature images of each input model.
[0162] Among them, sliding window segmentation is a technology widely used in many fields such as image processing and signal processing. It mainly defines a "window" of fixed size and lets the window slide on the data set (such as image, signal sequence, etc.) according to certain rules to achieve data segmentation.
[0163] The preset size is a pre-determined window size. This window is usually a rectangular area, such as 8*8 pixels, and can also be set to other shapes according to specific needs. The preset step size is the distance the window moves each time it slides, such as 6 pixels.
[0164] It should be pointed out that the above examples of preset sizes and preset step sizes for sliding window segmentation are only schematic illustrations. In practical applications, when the sliding step size is smaller than the width of the sliding window, the multiple sub-feature images obtained by segmentation will have overlapping areas. For example, assuming that the size of the sliding window is 10*10 pixels, the sliding step size is 6 pixels, and the first feature sub-image is divided from the upper left corner of the layered feature image, when the window slides 6 pixels to the right to divide the second sub-feature image, the two sub-images will have an overlapping area of 4 pixels in width. When the sliding step size is equal to the width of the sliding window, there is no overlapping area between the sub-feature images. Those skilled in the art can set the preset size and preset step size according to the specific scenario, which is not limited here.
[0165] The above is a further description of step S102 , and the following is a further description of step S103 .
[0166] In some implementations of the above step S103, a key point detection model for intersections may be constructed based on the Transformer architecture, and the powerful capabilities of the Transformer architecture for complex data processing and feature extraction may be utilized to achieve higher accuracy and recall rates.
[0167] The intersection key point detection model may include a feature extraction module and a key point decoding module. The feature extraction module may be designed based on a convolutional neural network (CNN), and may perform multi-scale feature extraction on a feature image through multiple convolution kernels of different sizes, and output multiple layers of feature images of different scales.
[0168] The key point decoding module may include an initialization module, a self-attention module, a cross-attention module, and a multi-layer perceptron.
[0169] Among them, the initialization module can randomly initialize the feature images of multiple scales output by the feature extraction module to obtain multiple key points and the initial features of each key point; the self-attention module can perform feature fusion on the initial features of multiple key points; the cross-attention module can obtain the feature information of multiple key points on feature images of multiple scales; the multi-layer perceptron can regress the information of the key points and finally obtain the coordinates of each key point.
[0170] After the intersection key point detection model is constructed, the intersection key point detection model can be trained.
[0171] Specifically, a training sample set may be obtained, and the feature extraction module and the key point decoding module in the intersection key point detection model may be trained based on the training sample set. The training sample set may be a true feature image with an intersection label, and may be a feature image with manually annotated intersection key points.
[0172] Furthermore, during the training process, the loss function of the intersection key point detection model can be obtained, and when the loss function reaches a preset threshold, the training of the intersection key point detection model is completed.
[0173] That is, the output of the intersection key point detection model is effectively supervised through the true value feature image. Specifically, for multiple intersection key points output by the model, the corresponding key points in the true value feature image can be matched, and supervision can be achieved through a loss function, such as the mean absolute error loss function (also known as the L1 loss function).
[0174] In some implementations, the L1 loss function of the intersection key point detection model may be calculated according to the following formula (1):
[0175]
[0176] Among them, L1loss avg Indicates that the L1 loss function is to calculate the average loss of multiple samples (i.e., intersection key points); Σ indicates the sum of all samples; n indicates the number of samples; i is the index of the sample, indicating the sum from 1 to i; Represents the coordinates of the key points of the intersection predicted by the model, x i ,y i Represents the real coordinates of the intersection key points in the true feature image.
[0177] The loss function is used to measure the difference between the model output and the real data, so as to guide the continuous optimization of the model until the loss function of the intersection key point detection model reaches the preset threshold. The model is considered to have been trained. At this point, the model can detect the intersection key points of the feature image more stably.
[0178] The above is an explanation of the intersection key point detection model.
[0179] Furthermore, the hierarchical feature images corresponding to each layer of the target scene may be input into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of the target scene.
[0180] See attached Figure 4 , Figure 4 This is a flow chart of the main steps of inputting the hierarchical feature images corresponding to each layer of target scene into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of target scene according to an embodiment of the present application. Figure 4 As shown, it mainly includes the following steps S401 to S402.
[0181] Step S401: input each sub-feature image of the hierarchical feature image corresponding to each layer of the target scene into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image;
[0182] In some implementations, step S401 may include the following steps S4011 to S4013.
[0183] Step S4011: input each sub-feature image into a feature extraction module to perform multi-scale feature extraction to obtain feature images of multiple scales;
[0184] In some embodiments, see Figure 5 , Figure 5It is a schematic diagram of obtaining the intersection key point detection result of each sub-feature image through the intersection key point detection model according to an embodiment of the present application.
[0185] like Figure 5 As shown, each sub-feature image 51 of the layered feature image can be input into the feature extraction module 521 of the intersection key point detection model 52. The feature extraction module 521 can perform multi-scale feature extraction on the feature image through multiple convolution kernels of different sizes, and output feature images of multiple scales to the key point decoding module.
[0186] Among them, feature images of different scales can capture feature information of different sizes and details, which helps to more comprehensively obtain various features in the input image and provide a rich feature basis for the subsequent accurate detection of key points at intersections.
[0187] Step S4012: inputting feature images of multiple scales into a key point decoding module to obtain coordinates of multiple key points;
[0188] After the feature extraction module 521 extracts the relevant information on the feature image, the task of the key point decoder is to further extract the feature information related to each key point.
[0189] Among them, the feature information related to each key point may include the position information of the key point (such as the coordinates of the key point in the feature image), appearance features (such as the color features and texture features of the key point), semantic features (such as the category attributes of the key point and the contextual information with the surrounding environment), direction features and angle features, etc., which are not limited here.
[0190] In some implementations, step S4012 may include the following steps a to d.
[0191] Step a: Based on the initialization module 522, the feature images of multiple scales are randomly initialized to obtain multiple key points and the initial features of each key point;
[0192] Specifically, a default number of initial key points, such as N, may be set for the intersection key point detection model, and then feature images of multiple scales may be randomly initialized to obtain N key points, and each key point may be assigned some initial features.
[0193] Step b: perform feature fusion on the initial features of multiple key points based on the self-attention 523 module;
[0194] The self-attention module 523 can perform feature fusion on the initial features between N key points, allowing the features of different key points to influence and complement each other, explore the intrinsic correlation and feature relationship between the key points, and enhance the feature expression capability.
[0195] Step c: acquiring feature information of multiple key points after feature fusion on feature images of multiple scales based on the cross attention module 524;
[0196] The cross-attention module 524 can query the information related to each key point on the feature images of multiple scales output by the feature extraction module, and use the existing feature images to further enrich and refine the feature information of the key points, so that the key points can better correspond to the specific positions and feature expressions in the feature images.
[0197] Step d: Repeat step bc based on the multi-layer perceptron 525 until the number of executions reaches a preset number of layers of the multi-layer perceptron, and determine the coordinates of multiple key points based on the feature information of multiple key points obtained by repeatedly executing step bc.
[0198] A preset number of layers, such as 6 layers, 7 layers, etc., will be set in the multi-layer perceptron 525. The multi-layer perceptron will repeatedly execute steps bc until the number of executions reaches the preset number of layers of the multi-layer perceptron, that is, repeatedly perform feature fusion on the initial features of multiple key points, obtain feature information of the multiple key points after feature fusion on feature images of multiple scales, and finally obtain feature information of each key point on feature images of multiple scales, and obtain the coordinates (x i ,y i ).
[0199] Step S4013: Based on the coordinates of the multiple key points, the intersection key point detection result 53 of each sub-feature image is obtained.
[0200] By adding the coordinates of the above-mentioned multiple key points to each sub-feature image input to the intersection key point detection model, the intersection key point detection result of each sub-feature image can be obtained.
[0201] The intersection key point detection result of each sub-feature image includes the confidence of each intersection key point in the sub-feature image. Confidence is an indicator to measure the reliability of the detection result. A high confidence indicates that the intersection key point is likely to be accurately detected.
[0202] The above is a further description of step S401.
[0203] Step S402: Obtain the intersection key point detection result corresponding to each layer of the target scene based on the intersection key point detection results of all sub-feature images corresponding to each layer of the target scene.
[0204] In some implementations, step S402 may include the following steps S4021 to S4023.
[0205] Step S4021: projecting all sub-feature images corresponding to each layer of the target scene to the layered feature image, and splicing all the projected sub-feature images to obtain a spliced image;
[0206] That is, a plurality of sub-feature images are projected back into the layered feature image, and further, all the sub-feature images are spliced according to the positions of the sub-feature images to obtain a spliced image.
[0207] Step S4022: performing non-maximum suppression processing on the intersection key points in the overlapping area in the stitched image to obtain the intersection key points in the overlapping area;
[0208] Since the preset sizes and preset step sizes when the layered feature image is segmented by the sliding window are different, partial overlap may exist between the multiple sub-images.
[0209] See attached Figure 6 , Figure 6 FIG. 1 is a schematic diagram of stitching images according to an embodiment of the present application. Figure 6 As shown, after the sub-images 1-3 are spliced, there will be two overlapping areas, and the intersection key points in the overlapping areas will also be repeated. Therefore, it is necessary to perform non-maximum suppression (NMS) processing on the intersection key points in the overlapping areas.
[0210] NMS is a technology widely used in computer vision and target detection. Its main purpose is to eliminate redundant detection boxes (in target detection scenarios) or key points (in key point detection scenarios) and retain only those detection results that are most likely to be real targets or the most representative. In step S4022, by performing NMS processing on the key points of the intersection in the overlapping area, repeated and inaccurate key points in the overlapping area can be removed.
[0211] Specifically, the following steps (1)-(4) can be used to perform non-maximum suppression on the key points of the intersection in the overlapped area:
[0212] Step (1): Obtain the point set corresponding to the key points of the intersection in the overlapped area;
[0213] The point set corresponding to the intersection key point includes the intersection key point and other intersection key points within a preset distance range around the intersection key point;
[0214] That is, for any key intersection point in the overlapped area, obtain its corresponding point set P = {p1, p2, ..., pn}, where the point set P includes the key intersection point and other key intersection points within a preset distance range (such as 5 meters) around it.
[0215] Step (2): Obtain the confidence of all key points at the intersection in the point set;
[0216] That is, obtain the confidence of each intersection key point output by the intersection key point detection model.
[0217] Step (3): The point with the highest confidence in each point set is taken as the intersection key point corresponding to each point set;
[0218] The higher the confidence level, the greater the possibility that this point is the real intersection key point. Therefore, the point with the highest confidence level in the point set P can be used as the output of the final intersection key point.
[0219] Step (4): Based on the intersection key points corresponding to all point sets, obtain the intersection key points of the overlapping area.
[0220] After the previous steps (1)-(3), the corresponding intersection key points with the highest confidence have been determined for each point set P. Finally, by integrating these relatively optimal intersection key points selected from each point set, the intersection key points of the overlapping area can be obtained.
[0221] Step S4023: Based on the intersection key points in the overlapped area and the intersection key points in the non-overlapped area, the intersection key point detection results corresponding to each layer of the target scene are obtained.
[0222] In the stitched image, in addition to the overlapping areas, there are also a large number of non-overlapping areas. The intersection key points in the overlapping areas obtained after non-maximum suppression processing and the intersection key points originally detected in the non-overlapping areas are integrated together to form the intersection key point detection results corresponding to each layer of the target scene.
[0223] Furthermore, after obtaining the intersection key point detection results corresponding to each layer of the target scene, the intersection key point detection results of the target scene can be obtained based on the intersection key point detection results corresponding to all layers of the target scene.
[0224] See attached Figure 7 , Figure 7 It is a schematic diagram of a method for detecting key points at intersections according to an embodiment of the present application.
[0225] like Figure 7 As shown in the figure, the vector map of the target scene, the lidar point cloud data and the vehicle driving trajectory are first heterogeneously fused to provide the overall layout, local details and drivable area of the complex low-speed scene for intersection key point detection.
[0226] Furthermore, the above-mentioned fused multimodal information is input into the Transformer-based intersection key point detection model to realize end-to-end intersection key point detection, and finally the intersection key points of low-speed scenes are obtained, providing a more reliable decision-making basis for autonomous driving in complex low-speed scenes.
[0227] Among them, since there is significant uncertainty in the scale of each low-speed scene, the sliding window technology can be used based on the divide-and-conquer idea to splice and integrate the multiple sub-feature image results obtained by detection, and finally accurately output the key points of the intersection of the target scene.
[0228] The above is a further explanation of the intersection key point detection method provided in this application.
[0229] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.
[0230] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.
[0231] Furthermore, the present application also provides a smart device.
[0232] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by at least one processor, the method described in any of the above embodiments is implemented. The processor may include hardware, software, or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. The intelligent device described in the present application may include driving devices, smart cars, robots, and other devices.
[0233] The smart devices described in this application may include driving devices, smart cars, robots and other devices. Figure 8, Figure 8 is a schematic diagram of a smart device according to an embodiment of the present application. Figure 8 8 shows by way of example that the memory 81 and the processor 82 are communicatively connected via a bus.
[0234] In some embodiments of the present application, the smart device may further include at least one sensor, which is used to sense information. The sensor is connected to any type of processor mentioned in the present application. Optionally, the smart device may also include an autonomous driving system, which is used to guide the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments. (The electronic device described in the present application may be, but is not limited to, a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc., and the embodiments of the present application are not limited to this.)
[0235] Furthermore, the present application also provides a computer-readable storage medium.
[0236] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the method for detecting key points at intersections in the above-mentioned method embodiment, and the program may be loaded and run by a processor to implement the above-mentioned method for detecting key points at intersections. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.
[0237] It should be noted that the relevant user personal information that may be involved in the various embodiments of the present application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0238] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.
[0239] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0240] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.
Claims
1. A method for detecting key points at intersections, characterized in that: The method comprises: Acquire multimodal information of a target scene; the multimodal information includes a vector map of the target scene, laser radar point cloud data, and a vehicle driving trajectory; Performing heterogeneous fusion on the multimodal information to obtain a feature image of the target scene; The characteristic image of the target scene is input into a trained intersection key point detection model to obtain the intersection key point detection result of the target scene.
2. The method for detecting key points at intersections according to claim 1, characterized in that: The performing heterogeneous fusion on the multimodal information to obtain the characteristic image of the target scene includes: Projecting the multimodal information into a unified plane coordinate system; In the unified plane coordinate system, the multimodal information is heterogeneously fused to obtain a feature image of the target scene.
3. The method for detecting key points at intersections according to claim 2, characterized in that: The characteristic image of the target scene includes a hierarchical characteristic image, and the method further includes obtaining the hierarchical characteristic image based on the following steps: Acquiring height information of the target scene; stratifying the target scene based on the height information; The multimodal information of each layer of target scene is heterogeneously fused to obtain the hierarchical feature image corresponding to each layer of target scene.
4. The method for detecting key points at intersections according to claim 3, characterized in that: The heterogeneous fusion of multimodal information of each layer of target scene to obtain a hierarchical feature image corresponding to each layer of target scene includes: Perform voxel downsampling on the lidar point cloud data of each layer of the target scene to obtain voxelized point cloud data; Obtain vector parking space information based on the vector map of each target scene; rasterizing the voxelized point cloud data, the vector parking space information and the vehicle driving trajectory of each layer of the target scene to obtain a plurality of raster units; A hierarchical feature image corresponding to each layer of the target scene is obtained based on the multiple grid units.
5. The method for detecting key points at intersections according to claim 3, characterized in that: The step of inputting the characteristic image of the target scene into a trained intersection key point detection model to obtain the intersection key point detection result of the target scene comprises: Inputting the hierarchical feature images corresponding to each layer of the target scene into the intersection key point detection model to obtain the intersection key point detection results corresponding to each layer of the target scene; The intersection key point detection result of the target scene is obtained based on the intersection key point detection results corresponding to all layers of the target scene.
6. The method for detecting key points at intersections according to claim 5, characterized in that: The hierarchical feature image corresponding to each layer of the target scene includes a plurality of sub-feature images; the step of inputting the hierarchical feature image corresponding to each layer of the target scene into the intersection key point detection model to obtain the intersection key point detection result corresponding to each layer of the target scene includes: Inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image; The key point detection results of the intersections corresponding to each layer of the target scene are obtained based on the key point detection results of the intersections of all the sub-feature images corresponding to each layer of the target scene.
7. The method for detecting key points at intersections according to claim 6, characterized in that: The method further comprises obtaining the plurality of sub-feature images based on the following steps: The layered feature image is segmented by a sliding window based on a preset size and a preset step size to obtain a plurality of sub-feature images corresponding to each layer of the target scene.
8. The method for detecting key points at intersections according to claim 6, characterized in that: The intersection key point detection model includes a feature extraction module and a key point decoding module; The step of inputting each sub-feature image into the intersection key point detection model to obtain the intersection key point detection result of each sub-feature image comprises: Inputting each sub-feature image into the feature extraction module to perform multi-scale feature extraction to obtain feature images of multiple scales; Inputting the feature images of the multiple scales into a key point decoding module to obtain coordinates of multiple key points; Based on the coordinates of the multiple key points, a detection result of the intersection key points of each sub-feature image is obtained.
9. The method for detecting key points at intersections according to claim 8, characterized in that: The key point decoding module includes an initialization module, a self-attention module, a cross-attention module and a multi-layer perceptron; the inputting the feature images of the multiple scales into the key point decoding module to obtain the coordinates of the multiple key points includes: a. Based on the initialization module, the feature images of the multiple scales are randomly initialized to obtain multiple key points and initial features of each key point; b. performing feature fusion on the initial features of the multiple key points based on the self-attention module; c. acquiring feature information of the plurality of key points after the feature fusion on the feature images of the plurality of scales based on the cross attention module; d. Repeat step bc based on the multilayer perceptron until the number of executions reaches a preset number of layers of the multilayer perceptron, and determine the coordinates of the multiple key points based on the feature information of the multiple key points obtained by repeatedly executing step bc.
10. The method for detecting key points at intersections according to claim 6, characterized in that: The key point detection results of the intersections corresponding to each layer of the target scene are obtained based on the key point detection results of the intersections of all sub-feature images corresponding to each layer of the target scene, including: Projecting all sub-feature images corresponding to each layer of the target scene onto the layered feature image, and splicing all the projected sub-feature images to obtain a spliced image; Performing non-maximum suppression processing on the key points of the intersection in the overlapping area in the stitched image to obtain the key points of the intersection in the overlapping area; Based on the intersection key points in the overlapped area and the intersection key points in the non-overlapped area, the intersection key point detection results corresponding to each layer of the target scene are obtained.