Target detection and map construction device and method, equipment and medium

Abstract

The invention provides a target detection and map construction device and method, equipment and a medium, and the method comprises the steps: carrying out the feature extraction of perception data, obtaining a multi-scale feature, determining the initial position and the target size of a target object based on the perception data, determining a target scale feature matched with the target object based on the target size and the multi-scale feature, and carrying out the detection of a map. And based on the initial position and the target scale feature, determining a local feature of the target object, and based on the local feature and the initial position, determining a target detection result of the target object. Therefore, the target scale features are dynamically determined through the target size, the features with more matched scales can be dynamically selected for the targets with different sizes, the problem that details of small targets are lost or redundant information of large targets is interfered by fixed single-scale features is avoided, the pertinence and effectiveness of feature expression are enhanced, and the accuracy of feature expression is improved. Therefore, the accuracy of target detection can be improved.

Description

Technical Field

[0001] This disclosure relates to computer vision technology, and in particular to an object detection and map building apparatus, method, device, and medium. Background Technology

[0002] Object detection is one of the core tasks in computer vision and is widely used in fields such as map building, autonomous driving, robot navigation, and intelligent monitoring.

[0003] In some existing map building schemes, map elements are uniformly modeled as point sets and topological arrangements through a Transformer-based encoder-decoder architecture, and end-to-end vectorized output is achieved by using instance queries, thereby realizing target detection.

[0004] However, the above scheme relies on global feature regression point coordinates, resulting in low accuracy of target detection. Summary of the Invention

[0005] To address the aforementioned technical problems, this disclosure provides a target detection and map building apparatus, method, device, and medium.

[0006] A first aspect of this disclosure provides a target detection apparatus, including a processor configured to: Feature extraction is performed on the perceived data to obtain multi-scale features; Based on the sensing data, the initial position and size of the target object are determined; Based on the target size and multi-scale features, determine the target scale features that match the target object; Based on the initial position and target scale characteristics, determine the local features of the target object; Based on local features and initial position, the target detection result of the target object is determined.

[0007] A second aspect of this disclosure provides a map building apparatus, including a processor configured to: The target detection result of the target object is determined, wherein the target object is a map element, and the target detection result is determined by any of the target detection devices mentioned in the first aspect above; Based on the target detection results, a vector map containing the target objects is constructed.

[0008] A third aspect of this disclosure provides a target detection method, comprising: Feature extraction is performed on the perceived data to obtain multi-scale features; Based on the sensing data, the initial position and size of the target object are determined; Based on the target size and multi-scale features, determine the target scale features that match the target object; Based on the initial position and target scale characteristics, determine the local features of the target object; Based on local features and initial position, the target detection result of the target object is determined.

[0009] A fourth aspect of this disclosure provides a map construction method, including: The target detection result is determined, where the target is a map element, and the target detection result is determined by the target detection method in the third aspect mentioned above; Based on the target detection results, a vector map containing the target objects is constructed.

[0010] A fifth aspect of this disclosure provides an electronic device, comprising: Memory, used to store computer programs; A processor is configured to execute a computer program stored in a memory, wherein when the computer program is executed, it implements the method of any embodiment of the target detection method of the third aspect of the present disclosure, or implements the method of any embodiment of the map construction method of the fourth aspect of the present disclosure.

[0011] A sixth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any embodiment of the target detection method of the third aspect of this disclosure, or implements the method of any embodiment of the map construction method of the fourth aspect of this disclosure.

[0012] A seventh aspect of this disclosure provides a computer program including computer-readable code, wherein when executed by a processor, the computer program instructions implement the method of any embodiment of the target detection method of the third aspect of this disclosure, or implement the method of any embodiment of the map construction method of the fourth aspect of this disclosure.

[0013] Based on the embodiments of this disclosure, feature extraction is performed on the perceived data to obtain multi-scale features. After determining the initial position and size of the target object based on the perceived data, target scale features matching the target object are determined based on the target size and the multi-scale features. Local features of the target object are determined based on the initial position and the target scale features. Finally, the target detection result of the target object is determined based on the local features and the initial position. Therefore, by dynamically determining the target scale features based on the target size, more suitable scale features can be dynamically selected for targets of different sizes. This avoids the problem of lost details for small targets or interference with redundant information for large targets caused by fixed single-scale features, enhancing the specificity and effectiveness of feature representation, thereby improving the accuracy of target detection. Attached Figure Description

[0014] Figure 1A This is a schematic diagram illustrating the application scenarios to which this disclosure applies.

[0015] Figure 1B This is a schematic diagram of another application scenario to which this disclosure applies.

[0016] Figure 2A This is a schematic diagram of the structure of a target detection device provided in an exemplary embodiment of the present disclosure.

[0017] Figure 2B This is a schematic diagram of the structure of a map building apparatus provided in an exemplary embodiment of the present disclosure.

[0018] Figure 3 This is a schematic flowchart illustrating a target detection method configured to be executed by a target detection apparatus provided in an exemplary embodiment of this disclosure.

[0019] Figure 4 This is a schematic diagram of a process for determining the initial position and size of a target object by a target detection device configured to perform an exemplary embodiment of the present disclosure.

[0020] Figure 5 This is a schematic flowchart illustrating the process of determining the minimum circumscribed rotation rectangle of a target object by a target detection device configured to perform an action according to an exemplary embodiment of this disclosure.

[0021] Figure 6 This is a schematic flowchart illustrating another step in determining the minimum circumscribed rectangle of a target object by a target detection device configured to perform an exemplary embodiment of the present disclosure.

[0022] Figure 7 This is a schematic diagram illustrating the process of a target detection apparatus provided in an exemplary embodiment of the present disclosure being configured to execute the determination of feature data of feature points and global feature data.

[0023] Figure 8 This is a schematic diagram of a process for determining fusion features by a target detection apparatus configured to perform an exemplary embodiment of the present disclosure.

[0024] Figure 9 This is a schematic diagram of a process for determining the target size of a target object by a target detection device configured to perform an exemplary embodiment of the present disclosure.

[0025] Figure 10 This is a schematic flowchart illustrating the process of determining local features of a target object by a target detection apparatus configured to perform an exemplary embodiment of the present disclosure.

[0026] Figure 11 This is a schematic diagram of the process of determining the target detection result by a target detection device configured to perform an exemplary embodiment of the present disclosure.

[0027] Figure 12 This is a schematic diagram of another process for determining a target detection result by a target detection device provided in an exemplary embodiment of this disclosure.

[0028] Figure 13 This is a schematic flowchart of a map building method configured to be executed by a map building apparatus provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0029] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0030] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0031] Application Overview In the process of realizing this disclosure, the inventors discovered that in some existing map building schemes, map elements are uniformly modeled as point sets and topological arrangements through a Transformer-based encoder-decoder architecture, and end-to-end vectorized output is achieved by using instance queries, thereby realizing target detection.

[0032] However, the above scheme relies on global feature regression point coordinates, resulting in low accuracy of target detection.

[0033] Exemplary Applications Figure 1A This is a schematic diagram illustrating an application scenario applicable to an exemplary embodiment of this disclosure. Any target detection method provided in this embodiment can be executed by any suitable electronic device equipped with the target detection device 101, including but not limited to: terminal devices, servers, etc. The terminal device can be a robot or a vehicle.

[0034] like Figure 2A As shown, the target detection device 101 described above may be equipped with at least one processor 111.

[0035] The processor installed on the target detection device 101 can be used to process various types of data. The processor can transmit data with internal or external components such as memory and sensors of the target detection device 101 via a serial data bus or a controller area network (CAN) bus to obtain perception data. After determining the target detection result of the target object based on the perception data using the target detection method of this embodiment, the target detection result is transmitted to downstream processing devices, such as, but not limited to, internal or external memory or map building devices of the target detection device 101.

[0036] In one example, the aforementioned perception data can be acquired from a sensor. This sensor can be used to collect the perception data. As an example, the sensor can include at least one of the following types: ultrasonic radar, millimeter-wave radar, lidar, image sensor, etc. In some cases, the sensor can be an onboard sensor. The number of sensors can be one or more; when multiple sensors are installed on the target detection device 101, the types of the multiple sensors can be the same or different. The perception data can be raw data collected by the sensor. As an example, the perception data can be visual data collected by a camera, or it can include visual data collected by a camera and point cloud data collected by lidar.

[0037] In one embodiment, in a target detection scenario, the target detection device 101 can employ the target detection method provided in this disclosure to extract features from the perceived data, obtain multi-scale features, and determine the initial position and size of the target object based on the perceived data. Then, based on the target size and multi-scale features, it determines target scale features that match the target object. Next, based on the initial position and target scale features, it determines the local features of the target object. Finally, based on the local features and the initial position, it determines the target detection result. Thus, by dynamically determining the target scale features through target size, more suitable scale features can be dynamically selected for targets of different sizes, avoiding the problem of lost details for small targets or interference with redundant information for large targets caused by fixed single-scale features. This enhances the specificity and effectiveness of feature representation, thereby improving the accuracy of target detection.

[0038] Figure 1B This is a schematic diagram illustrating another application scenario to which an exemplary embodiment of this disclosure applies.

[0039] In one example, the downstream processing device described above may be, for example, a map building apparatus 102. Any map building method provided in this disclosure can be executed by any suitable electronic device equipped with the map building apparatus 102, including but not limited to: terminal devices, servers, etc. The terminal device may be a robot or a vehicle.

[0040] like Figure 2B As shown, the map building device 102 described above may be equipped with at least one processor 111.

[0041] The processor on the map building device 102 can process various types of data. The processor can transmit data with the target detection device 101 via a serial data bus or a controller area network (CAN) bus to obtain the target detection results of the target objects (such as map elements). Then, based on the target detection results, a vector map containing the target objects can be constructed so that the vector map can be transmitted to terminal devices such as vehicles and robots for environmental perception and path planning.

[0042] The processor 111 in the target detection device 101 and / or map building device 102 may be a central processing unit (CPU) or other form of processing unit with data processing capability and / or instruction execution capability, and may control other components in the target detection device 101 or map building device 102 to perform the desired function.

[0043] In one example, such as Figure 2A As shown, the target detection device 101 may further include at least one memory 112, an input device 113, and an input device 114. For example... Figure 2B As shown, the map building apparatus 102 may also include at least one memory 112, an input device 113, and an input device 114. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0044] The memory 112 in the target detection device 101 and / or map building device 102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may execute one or more computer program instructions to implement the target detection method and / or map building method of the various embodiments of this disclosure above, as well as other desired functions.

[0045] The input device 113 in the target detection device 101 and / or map building device 102 may also include, for example, a keyboard, a mouse, etc.

[0046] The input device 114 in the target detection device 101 and / or map building device 102 can output various information to the outside, which may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0047] Of course, for the sake of simplicity, Figure 2A , Figure 2B Only some of the components of the target detection device 101 and map building device 102 relevant to this disclosure are shown in this illustration, omitting components such as buses, input / output interfaces, etc. In addition, depending on the specific application, the target detection device 101 and / or map building device 102 may include any other suitable components.

[0048] Figure 1A , Figure 1B This is merely an exemplary application scenario implementation of the embodiments of this disclosure. Those skilled in the art will understand from the description of the embodiments of this disclosure that other feasible implementation methods may also be adopted.

[0049] Exemplary embodiments This disclosure provides a target detection device 101, which may include a processor 111. The processor 111 can read executable instructions from a memory 112 and execute the executable instructions to implement a target detection method. The target detection device 101 of this embodiment can be applied to electronic devices such as terminal devices and servers.

[0050] Figure 3 This is a schematic flowchart illustrating a target detection method configured to be executed by a target detection apparatus provided in an exemplary embodiment of this disclosure. The target detection apparatus 101 includes a processor 111. Figure 3 As shown, the processor 111 included in the target detection device 101 can be configured as follows: Step 301: Extract features from the perceived data to obtain multi-scale features.

[0051] In this embodiment, the sensing data can be data about the surrounding environment collected by devices such as sensors. In some optional implementations, the sensing data may include, but is not limited to, at least one of the following: image data, point cloud data, LiDAR data, etc.

[0052] Multi-scale features can be features obtained by extracting features from perceived data at different resolutions. In some optional implementations, multi-scale features can be multi-scale BEV (Bird's-Eye View) feature pyramids, i.e., BEV feature layers at different resolutions. In other optional implementations, multi-scale features can be other multi-scale features.

[0053] In some optional implementations, feature extraction modules can be used to extract features from the perceived data. In other optional implementations, multi-scale PV (Perspective View) features of the perceived data can be extracted and converted into BEV features. These feature extraction modules may include, but are not limited to, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Graph Neural Networks (GNNs). Furthermore, these feature extraction modules may include at least one of the following multi-scale structures: Bidirectional Feature Pyramid Network (BiFPN), Path Aggregation Network (PAN), U-Net, and Transformer.

[0054] Step 302: Based on the sensing data, determine the initial position and size of the target object.

[0055] In this embodiment, the target object can be any object in the perception data. In some optional implementations, the target object can include, but is not limited to, at least one of the following: dynamic targets and static elements. Static elements can include road markings such as zebra crossings and stop lines, as well as traffic lights, curbs, etc., which can be used for map construction and are also called map elements. Dynamic targets can include other vehicles, pedestrians, cyclists, etc.

[0056] The initial position of the target object can be a preliminarily determined position. In some optional implementations, the initial position of the target object can be represented by at least one of the following methods: the center position of the target object, the bounding box (Bbox) position of the target object, or the position of the rotated bounding box (RBBox) of the target object. The rotated bounding box is determined based on the minimum bounding rectangle of the target object, and therefore can also be called the minimum bounding rectangle.

[0057] The target size can characterize the size of the target object. In some alternative implementations, the target size can be represented in at least one of the following ways, but is not limited to: the size of the target object's detection box, the size of the target object's rotated bounding box. In addition, the target size can include, but is not limited to, at least one of the following: the length of the target object (e.g., the length of the target object's detection box, the length of the target object's minimum bounding rotated rectangle), and the width of the target object (e.g., the width of the target object's detection box, the width of the target object's minimum bounding rotated rectangle).

[0058] In some alternative implementations, a Transformer network can be used to determine the initial position and size of the target object based on the perceived data. Alternatively, the perceived data can be input into the Transformer network to obtain the coordinates, attributes (e.g., zebra crossings, lane lines), and scores (representing the confidence level of the attributes) of the target object's feature points (e.g., any point within the target object). Based on the coordinates, attributes, and scores, the initial position and size of the target object can then be determined.

[0059] In some alternative implementations, CNN or RNN networks can be used to determine the initial position and size of the target based on the perception data.

[0060] It should be noted that, in this embodiment, the execution order of steps 301 and 302 is not limited. For example, step 301 can be executed first and then step 302, or step 302 can be executed first and then step 301, or steps 301 and 302 can be executed simultaneously.

[0061] Step 303: Based on the target size and multi-scale features, determine the target scale features that match the target object.

[0062] In this embodiment, the target scale feature can be a specific scale feature among the multi-scale features that matches the target size. In some optional implementations, the target scale feature can be a scale feature among the multi-scale features that matches at least one of the length, width, and area of ​​the target size.

[0063] In some alternative implementations, the target scale features that match the target object can be determined based on the target size and multi-scale features by establishing a pre-established correspondence between the target size and target scale features. For example, the above correspondence can be expressed by the following formula (1): Formula (1) In formula (1), k represents the index of the target scale feature. The index represents the baseline scale feature in the multi-scale features, w represents the width of the minimum bounding rotation rectangle of the target object, h represents the height of the minimum bounding rotation rectangle of the target object, and S represents the size of the baseline image corresponding to the baseline scale feature (e.g., the arithmetic square root of the area of ​​the baseline image, side length, length, or width). Used to determine the image size (i.e., target size) during the inference phase. Normalize to a relative size with reference to the baseline image size S, that is... Converted to a relative ratio with S as the reference, [] indicates rounding up, rounding down, or rounding to the nearest integer, and M represents the scaling relationship between adjacent scale features in multi-scale features, for example, M can be 2. In one embodiment, the reference image size can be the size of the training sample image used to train the machine learning model (e.g., the training sample image used for the backbone network in the pre-trained machine learning model). Thus, the mapping between the target size and the target scale features can be realized through the above formula (1), and the image size in the inference stage can be represented by an equivalent scale consistent with the training stage. Therefore, the accuracy of determining the target scale features can be improved through the equivalent scale, thereby improving the accuracy of the target detection results.

[0064] In some alternative implementations, the above correspondence can also be represented using a correspondence table. Specifically, the target size, multi-scale features, and target scale features that have a correspondence can be represented as a record in the correspondence table.

[0065] Step 304: Determine the local features of the target object based on the initial position and target scale features.

[0066] In this embodiment, local features can be features within a specific region of the target object. In some optional implementations, local features may include at least one of the following: geometric features and semantic features.

[0067] In some alternative implementations, when the initial position is the detection box or the minimum bounding rectangle of the target object, features of the corresponding region of the detection box or the rotated bounding box in the target scale features can be extracted to obtain the local features of the target object.

[0068] In some alternative implementations, after obtaining the local features of the target object, the local features of the target object can be further rotated to a preset orientation, thereby constructing a vector map by stitching together the local features of multiple rotated target objects.

[0069] Step 305: Determine the target detection result based on local features and initial position.

[0070] In this embodiment, the target detection result can be the detection result of the target object. In some optional implementations, the target detection result may include at least one of the following: the location, size, and category of the target object.

[0071] In some alternative implementations, a regressor can predict the offset of the initial position based on local features, and then determine the final position based on the initial position and the offset, and use it as the target detection result.

[0072] In some alternative implementations, the offset of the initial position can be predicted based on local features using an MLP (Multilayer Perceptron), and then the final position can be determined based on the initial position and the offset predicted by the MLP, and used as the target detection result.

[0073] Based on this embodiment, feature extraction is performed on the perceived data to obtain multi-scale features. After determining the initial position and size of the target object based on the perceived data, target scale features matching the target object are determined based on the target size and the multi-scale features. Local features of the target object are then determined based on the initial position and the target scale features. Finally, the target detection result is determined based on the local features and the initial position. Therefore, by dynamically determining the target scale features based on the target size, more suitable scale features can be dynamically selected for targets of different sizes. This avoids the problem of lost details for small targets or interference with redundant information for large targets caused by fixed single-scale features, enhancing the specificity and effectiveness of feature representation, thereby improving the accuracy of target detection.

[0074] In some alternative implementations, such as Figure 4 As shown above, in the above Figure 3 Based on the illustrated embodiment, step 302 may include the following steps: Step 3021: Determine the initial positions of the feature points of the target object based on the perception data.

[0075] In this embodiment, the feature point can be a structural point that characterizes the geometric shape or semantic attributes of the target object, or it can be any point of the target object.

[0076] The initial position of a feature point can be a preliminarily determined location. In some alternative implementations, the initial position of a feature point can be represented by its coordinates, and the initial positions of multiple feature points of the target object can form a coarse-grained point set.

[0077] In some alternative implementations, the Transformer network can be used to perform feature analysis on the perceived data and predict the initial positions of the feature points of the target object.

[0078] In some alternative implementations, convolutional neural networks can also be used to determine the initial positions of feature points of the target object based on perceptual data.

[0079] Step 3022: Based on the initial position, determine the minimum circumscribed rectangle of the target object.

[0080] In this embodiment, the minimum bounding rectangle is the aforementioned rotating bounding box. The minimum bounding rectangle can be the smallest rectangle enclosing the feature points of the target object. In some optional implementations, the minimum bounding rectangle can be the smallest rectangle enclosing all feature points of the target object. In some optional implementations, the minimum bounding rectangle can also be the smallest rectangle enclosing only some feature points of the target object. For example, the initial position of the feature points of the target object can include the coordinates of all feature points of the target object, with each feature point's coordinate corresponding to a confidence level, which characterizes the reliability of the feature point's coordinates. Then, the coordinates of feature points with confidence levels higher than a threshold can be selected, and the minimum bounding rectangle enclosing each selected coordinate is determined as the minimum bounding rectangle of the target object.

[0081] In some alternative implementations, the initial positions of the feature points of the target object can be calculated using geometric algorithms to obtain the minimum circumscribed rectangle that encloses all or part of the feature points.

[0082] Step 3023: Determine the size of the smallest circumscribed rotating rectangle as the target size of the target object.

[0083] In this embodiment, after determining the minimum bounding rectangle, the dimensions of the minimum bounding rectangle (including at least one of height and width) can be determined as the target size of the target object.

[0084] Based on this embodiment, the size of the smallest outermost rotating rectangle that better fits the target object is determined as the target size. This avoids the problem in related technologies where the target size is not accurate enough due to matching irregular or slender targets with the detection box. Therefore, the target size of the target object can be determined more accurately, thus improving the accuracy of the target detection results.

[0085] In some alternative implementations, such as Figure 5 As shown above, in the above Figure 4 Based on the illustrated embodiment, step 3022 may include the following steps: Step 30221: Based on the perception data, determine the coordinates, category, and confidence level of the feature points of the target object.

[0086] In this embodiment, the above category can represent the semantic category to which the target object belongs. In some optional implementations, the above category may include, but is not limited to, at least one of the following: zebra crossing, vehicle, pedestrian, etc.

[0087] Confidence level can characterize the reliability of the prediction results for feature point coordinates or target object category.

[0088] In some alternative implementations, a Transformer network can be used to extract features from the perceived data to determine the feature points of the target object, and then the coordinates, category, and confidence level of the feature points can be determined. Alternatively, a Convolutional Neural Network can be used to extract features from the perceived data to determine the feature points of the target object, and then the coordinates, category, and confidence level of the feature points can be determined.

[0089] Step 30222: Determine the initial location based on coordinates, category, and confidence level.

[0090] In some optional implementations, the coordinates of feature points that meet preset requirements (e.g., belong to the category of map elements) and have a confidence level higher than a threshold can be selected, and the initial position of the target object can be determined based on the selected coordinates.

[0091] Step 30223: Based on the initial position, determine the minimum circumscribed rectangle of the target object.

[0092] In this embodiment, the smallest circumscribed rectangle can be the smallest rectangle that encloses all or part of the feature points of the target object.

[0093] In some alternative implementations, the initial positions of the feature points of the target object can be calculated using geometric algorithms to obtain the minimum circumscribed rectangle that encloses all or part of the feature points.

[0094] Based on this embodiment, multiple dimensions of data, such as coordinates, category, and confidence level of feature points, are determined through perceptual data. Then, the minimum bounding rectangle of the target object is determined based on the multiple dimensions of data, such as coordinates, category, and confidence level. Feature points with low confidence or other categories can be excluded, thereby improving the accuracy of determining the minimum bounding rectangle of the target object. In turn, the accuracy of determining the target size of the target object can be improved, thus enhancing the accuracy of the target detection results.

[0095] In some alternative implementations, such as Figure 6 As shown above, in the above Figure 4 Based on the illustrated embodiment, step 3021 may include the following steps: Step 30214: Based on the perception data, determine the multi-scale bird's-eye view features.

[0096] In this embodiment, the multi-scale bird's-eye view features can be multiple hierarchical features obtained by converting the perspective of the perceived data into a bird's-eye view and then extracting them through multi-scale feature extraction.

[0097] In some optional implementations, the PV (Perspective-to-Perception) features of the perceived data can be extracted first and converted into multi-scale BEV (Bird's Eye View) features to obtain multi-scale bird's-eye view features. Alternatively, multi-scale PV features can be extracted from the perceived data first to obtain multi-scale PV features. Then, the extracted multi-scale PV features are converted into bird's-eye view features to obtain multi-scale bird's-eye view features. In some optional implementations, single-scale PV features can also be extracted from the perceived data to obtain single-scale PV features. Then, based on the single-scale PV features, upsampling and downsampling are performed to obtain multi-scale bird's-eye view features.

[0098] Step 30215: Based on the self-attention mechanism, the multi-scale bird's-eye view features are processed to obtain the feature data of the feature points of the target object and the global feature data of the target object.

[0099] In this embodiment, self-attention is an attention mechanism that can capture the dependencies between different locations within a feature. In some optional implementations, a self-attention network can be used to process multi-scale bird's-eye view features to obtain feature data of the target object's feature points and global feature data of the target object. The self-attention network is a deep learning network built based on a self-attention module. In some optional implementations, the self-attention network may include, but is not limited to, at least one of the following: DETR (Detection Transformer) network and BEiT (Bidirectional Encoder representation from Image Transformers) network.

[0100] Feature data of feature points can characterize the feature information of feature points (any location of the target object). In some optional implementations, self-attention can be used to extract feature data of feature points from global feature data, so as to associate feature points of different target objects through interaction.

[0101] Global feature data can characterize the feature information of the entire target object. In some alternative implementations, multi-scale bird's-eye view features can be used as global feature data.

[0102] In some alternative implementations, a self-attention mechanism can be used to process the overall features of the multi-scale bird's-eye view to obtain the feature data of the feature points of the target object and the global feature data of the target object.

[0103] In some alternative implementations, a self-attention mechanism can be used to process the bird's-eye view features at any scale within the multi-scale bird's-eye view features. Then, the processing results from each scale are stitched together to obtain the feature data of the target object's feature points and the target object's global feature data.

[0104] In some alternative implementations, global feature data can be used as the Q-factor in the self-attention mechanism. enc K enc V enc Among them, Q enc =K enc =V enc =F bev Q enc K enc V enc These represent queries, matches, and information carriers from different bird's-eye view locations regarding themselves and other locations. F bev This represents multi-scale bird's-eye view features. This attention is used to model the correlation information between various locations in the BEV space, enabling each multi-scale bird's-eye view feature to incorporate semantic and spatial information from the entire scene.

[0105] The formula for calculating self-attention can be expressed as the following formula (2): Formula (2) Formula (2) enables global interaction of multi-scale bird's-eye view features, allowing the encoder to capture spatial context and cross-scale dependencies, and output the encoded multi-scale bird's-eye view features F. enc (i.e., the calculation result of formula (2), also known as visual features), and F enc As input to the decoder.

[0106] The decoder employs a hierarchical query mechanism and an iterative coordinate refinement strategy to decode the visual features output by the encoder (Fenc) into structured map elements (i.e., objects). Each map element is represented by a hierarchical query embedding, with each query embedding containing instance-level (object-level) embeddings and point-level (feature point-level) embeddings. In each decoder layer, self-attention is first calculated: Q... self = K self = V self = query embedding. Q self K represents the query requirement of a given structural point (i.e., feature point) for other structural points and structural instance context. self V represents the matching structural and geometric indices provided by all structural points. selfThis represents the instance-level (i.e., target-level) semantic and point-level (i.e., feature-point-level) geometric feature information carried by each structural point. Where, Q self K self V self That is, the feature data of the aforementioned feature points.

[0107] Step 30216: Based on the cross-attention mechanism, the feature data of the feature points and the global feature data are fused to obtain the fused features.

[0108] In this embodiment, the cross-attention mechanism is an attention mechanism that can capture the dependency relationship between two different features.

[0109] A fusion feature can be a feature that combines feature data of feature points and global feature data.

[0110] In some alternative implementations, the deformable cross-attention mechanism of the Transformer decoder can be used to fuse the feature data of feature points and global feature data to obtain fused features.

[0111] Step 30217: Determine the initial position based on the fusion features.

[0112] In some alternative implementations, the initial position can be determined based on the coordinate offset of the feature points of each decoder layer relative to the previous decoder layer by using a feedforward network or MLP based on the fused features.

[0113] Based on this embodiment, by processing multi-scale bird's-eye view features through a self-attention mechanism, the correlation between feature points of the target object and the global contextual dependency can be captured; by using a cross-attention mechanism to fuse feature point feature data (local features) and global feature data (global features), the fused features can simultaneously contain local details and global information, thereby improving the accuracy of initial position prediction.

[0114] In some alternative implementations, such as Figure 7 As shown above, in the above Figure 6 Based on the illustrated embodiment, step 30225 may include the following steps: Step 302251: Add location encoding to the multi-scale bird's-eye view features to obtain the encoded multi-scale bird's-eye view features.

[0115] In this embodiment, the location encoding can characterize the spatial location of the feature point.

[0116] The encoded multi-scale bird's-eye view features can represent the features obtained by adding positional encoding to the multi-scale bird's-eye view features.

[0117] In some alternative implementations, location encoding information can be combined with multi-scale bird's-eye view features, and the encoded multi-scale bird's-eye view features can be obtained through other operations such as overlay and splicing.

[0118] Step 302252: Determine the global feature data of the target object based on the encoded multi-scale bird's-eye view features.

[0119] In some alternative implementations, multi-head self-attention computation can be performed on the encoded multi-scale bird's-eye view features to determine the global feature data of the target object.

[0120] Step 302253: Based on the global feature data, update the feature data of the initial feature points to obtain the feature data of the target object's feature points.

[0121] In this embodiment, the value of the feature data of the initial feature points can be determined by random assignment during the initialization of the machine learning model.

[0122] In some alternative implementations, a cross-attention mechanism can be used to update the feature data of the initial feature points based on global feature data, thereby obtaining the feature data of the target object's feature points.

[0123] Based on this embodiment, by adding position encoding to the multi-scale bird's-eye view features, position information can be supplemented for the multi-scale bird's-eye view features, avoiding the loss of position information in subsequent processing; global feature data can be determined based on the encoded multi-scale bird's-eye view features, which can capture the contextual dependencies of the entire scene; the feature data of the initial feature points is updated based on the global feature data, so that the feature data of the updated feature points are integrated with the global feature data, thereby enhancing the representational ability of the feature data of the updated feature points.

[0124] In some alternative implementations, such as Figure 8 As shown above, in the above Figure 6 Based on the illustrated embodiment, step 30216 may further include the following steps: Step 302261: Based on the feature data of feature points, determine the enhanced features of multi-scale bird's-eye view features.

[0125] In this embodiment, the enhanced features can be multi-scale bird's-eye view features with improved representation capabilities obtained through information interaction. In some optional implementations, the enhanced features can be features obtained by interacting with feature information between different feature points or different targets based on feature data of feature points. Here, information interaction between different targets can enhance the representation capability of structural features. In some optional implementations, in the vectorized representation of map elements, there are often spatial associations and semantic dependencies between different targets (such as different lane lines, traffic signs, etc.). Specifically, the direction of adjacent lane lines may affect each other, and the position of traffic signs may have a specific spatial configuration with lane lines. Through information interaction between different targets, this cross-instance (target) information can be captured, thereby more accurately understanding the structure of the entire scene and improving the recognition and positioning accuracy of map elements.

[0126] In some alternative implementations, the feature data Q of the feature points of the target object is obtained using the above method. self K self V self Then, the enhanced features can be obtained using formula (3). : Formula (3) In formula (3), d represents the feature dimension of query (Q) and key (K) (i.e., the dimension of each attention head).

[0127] Formula (3) enables information interaction between different feature points within the same instance (i.e., the same target object) and between different instances (i.e., different target objects), thereby enhancing the representational power of structural features and obtaining enhanced features. .

[0128] Step 302262: Based on the global feature data, update the feature data of the initial feature points to obtain the feature data of the feature points of the target object.

[0129] In this embodiment, the initial feature data values ​​of the feature points can be randomly assigned during the initialization of the machine learning model.

[0130] In some alternative implementations, during the training of the machine learning model, the feature data of the feature points can be updated based on the global feature data through the backpropagation algorithm, thereby obtaining the feature data of the target object's feature points.

[0131] In some alternative implementations, a cross-attention mechanism can be used to update the feature data of the initial feature points based on global feature data, thereby obtaining the feature data of the target object's feature points.

[0132] It should be noted that, in this embodiment, the execution order of steps 302261 and 302262 is not limited. For example, step 302261 can be executed first and then step 302262, or step 302262 can be executed first and then step 302261, or steps 302261 and 302262 can be executed simultaneously.

[0133] Step 302263: Determine the fusion features based on the feature data of the enhanced features and feature points.

[0134] In some alternative implementations, the augmented features can be used as query features, and the feature data of the feature points can be used as key features and value features. The augmented features and feature data of the feature points can be fused through a cross-attention mechanism to obtain fused features.

[0135] Based on this embodiment, enhanced features are obtained through the feature data of feature points, which can enhance the representation of the internal structure of the target and the relationship between the target objects. By updating the multi-scale bird's-eye view features based on global feature data, updated feature data of feature points containing full scene context information can be obtained. By fusing the enhanced features with the feature data of feature points, the fused features can simultaneously possess local structural information, inter-target relationship information and global context information, which can improve the representation capability of the fused features and further improve the accuracy of determining the initial position.

[0136] In some alternative implementations, the enhanced features are obtained through formula (3). Then, based on enhanced features Deformable cross-attention is performed on the sampled BEV features, and each feature point-level query is added to the BEV features output by the encoder (F). enc Based on the predicted offset points, the features of the sampled points are used as keys and values, and a weighted sum is calculated and the point features are updated through an attention mechanism: Q cross =Q self K cross = V cross = F enc Q cross K represents the query requirement for visual evidence from structural points. cross V represents the visual index that can be matched in the BEV space output by the encoder. cross It then carries the visual and geometric feature information of the corresponding spatial location.

[0137] After that, through Determine the point features of the decoder output at each layer. The offset relative to the coordinates of the previous layer is then predicted using a feedforward network or MLP. This allows us to obtain the position coordinates of the current layer. Where i represents Q cross The index j represents K cross Value V cross The index, l represents the layer identifier.

[0138] This enables iterative coordinate refinement, allowing the structure query to gradually converge to the vectorized representation of the real map elements in a layer-by-layer iteration, ultimately outputting a coarse-grained point set (i.e., the initial position) for the first stage.

[0139] In some alternative implementations, such as Figure 9 As shown above, in the above Figures 3-8 Based on any of the embodiments shown, step 302 may include the following steps: Step 3024: Determine the initial positions of the feature points of the target object based on the perception data.

[0140] In this embodiment, the feature point can be a structural point that characterizes the geometric shape or semantic attributes of the target object, or it can be any point of the target object.

[0141] In some alternative implementations, the Transformer network can be used to perform feature analysis on the perceived data and predict the initial positions of the feature points of the target object.

[0142] In some alternative implementations, convolutional neural networks can also be used to determine the initial positions of feature points of the target object based on perceptual data.

[0143] Step 3025: Based on the initial position, determine the minimum circumscribed rectangle of the target object.

[0144] In this embodiment, the smallest circumscribed rectangle can be the smallest rectangle that encloses all or part of the feature points of the target object.

[0145] In some alternative implementations, the initial positions of the feature points of the target object can be calculated using geometric algorithms to obtain the minimum circumscribed rectangle that encloses all or part of the feature points.

[0146] Step 3026: Determine the length and width of the minimum circumscribed rotating rectangle.

[0147] In some alternative implementations, after determining the minimum bounding rectangle, the length and width of the minimum bounding rectangle can be further determined.

[0148] Step 3027: Determine the target size based on the length and width.

[0149] In some alternative implementations, the length and width of the minimum bounding rectangle can be directly used as the target size of the object. Alternatively, the product of the length and width of the minimum bounding rectangle can be calculated to obtain its area, which can then be used as the target size.

[0150] Based on this embodiment, the target size of the target object can be determined more accurately by using the length and width of the minimum circumscribed rotation rectangle. Based on this target size, the target scale feature in the multi-scale feature can be matched more accurately, thereby improving the accuracy of target detection.

[0151] In some alternative implementations, such as Figure 10 As shown above, in the above Figures 3-9 Based on any of the embodiments shown, step 304 may include the following steps: Step 3041: Based on the initial position, determine the minimum circumscribed rectangle of the target object.

[0152] In this embodiment, the smallest circumscribed rectangle can be the smallest rectangle that encloses all or part of the feature points of the target object.

[0153] In some alternative implementations, the initial positions of the feature points of the target object can be calculated using geometric algorithms to obtain the minimum circumscribed rectangle that encloses all or part of the feature points.

[0154] Step 3042: Determine the local features of the target object based on the position of the minimum bounding rectangle and the target scale features.

[0155] In some alternative implementations, the local features of the target object can be extracted by performing region cropping or feature sampling on the target-scale features based on the position of the minimum bounding rotation rectangle.

[0156] Based on this embodiment, by extracting the local features of the target object through the position of the minimum circumscribed rotation rectangle and the target scale features, the fine-grained features of the target object at the corresponding position can be focused, avoiding interference from irrelevant regional features, which can further improve the accuracy of determining the initial position.

[0157] In some alternative implementations, such as Figure 11 As shown above, in the above Figures 3-10 Based on any of the embodiments shown, step 305 may include the following steps: Step 3051: Based on local features, determine the position offset of the feature points of the target object.

[0158] In this embodiment, the position offset can be used to correct the initial position of the feature point, which can represent the offset between the actual position (i.e., the corrected position) of the feature point and the initial position.

[0159] In some alternative implementations, the positional offset of feature points can be predicted based on local features using a lightweight MLP regressor. Alternatively, a fully connected neural network can also be used to predict the positional offset of feature points based on local features.

[0160] Step 3052: Determine the initial position of the feature point based on the initial position.

[0161] In some alternative implementations, the coordinates of the feature points of the target object can be used as the initial position of the feature points. Thus, the initial position of the target object can be determined by the coordinates of multiple feature points of the target object.

[0162] It should be noted that, in this embodiment, the execution order of steps 3051 and 3052 is not limited. For example, step 3051 can be executed first and then step 3052, or step 3052 can be executed first and then step 3051, or steps 3051 and 3052 can be executed simultaneously.

[0163] Step 3053: Determine the corrected position of the feature point based on the position offset and the initial position of the feature point.

[0164] In this embodiment, the corrected position can be the position of the feature point obtained after correcting the initial position by the position offset.

[0165] In some alternative implementations, the initial position of the feature point can be added to its corresponding position offset to obtain the corrected position of the feature point.

[0166] Step 3054: Determine the target detection result based on the corrected position.

[0167] In some alternative implementations, the corrected position of the target object can be determined based on the corrected position of the feature points, thus obtaining the target detection result.

[0168] Based on this embodiment, the position offset of the feature point is predicted by local features, and then the corrected position of the feature point is determined. The initial position can be corrected by using fine-grained local information of the target object, thereby improving the accuracy of determining the position of the feature point (i.e., the corrected position). Furthermore, the target detection result can be determined based on the corrected position, which can improve the accuracy of determining the target detection result.

[0169] In some alternative implementations, step 3051 above includes: determining the positional offset of feature points of the target object based on local features using a pre-trained machine learning model.

[0170] The machine learning model can be used to predict the positional offset. In some optional implementations, the machine learning model may include, but is not limited to, at least one of the following: a lightweight MLP regressor, or a convolutional neural network regression model.

[0171] In some of these alternative implementations, local features can be input into the trained machine learning model, and the model can output the position offset through inference.

[0172] Based on this, the machine learning model is trained in the following way: Step 1: Using the initial machine learning model, the offset is obtained based on local features and the initial position.

[0173] The initial machine learning model can be an untrained machine learning model or a machine learning model that has been trained but does not meet the training termination criteria.

[0174] The true location value can be the precise coordinates of the feature points of the target object in the labeled data.

[0175] The aforementioned offset characterizes the difference between the initial position and the true position, and this offset is output by the initial machine learning model.

[0176] In some alternative implementations, local features and initial positions can be input into an initial machine learning model, which then calculates the output offset.

[0177] Step 2: Based on the target size, normalize the true value offset corresponding to the initial position to obtain the normalized true value offset, where the true value offset represents the actual offset of the initial position relative to the true position value.

[0178] The truth offset represents the positional deviation of the initial position relative to the true position value. In some optional implementations, the truth offset can be the difference between the initial position coordinates and the true position coordinates.

[0179] Normalization can be used to eliminate the influence of target size differences on offset.

[0180] The normalized truth offset can be the truth offset after normalization.

[0181] In some optional implementations, the equivalent scale can be calculated based on the target size, and then the true offset can be divided by this factor to obtain the normalized true offset. In some optional implementations, the equivalent scale F can be calculated based on the area of ​​the minimum bounding rectangle. In some optional implementations, The normalized truth offset is obtained by dividing the truth offset (the distance between the initial position and the true position of the feature point) by F. Here, w represents the width of the minimum bounding rectangle of the target object, and h represents the height of the minimum bounding rectangle of the target object.

[0182] Step 3: Determine the loss value based on the offset of the initial position relative to the true position value and the offset of the normalized true value.

[0183] The loss value can represent the difference between the offset of the initial position relative to the true position value and the offset of the normalized true position value.

[0184] In some alternative implementations, the loss value can be determined based on the offset of the initial position relative to the true position value and the offset of the normalized true position value using either the Smooth L1 loss function or the MSE (Mean Squared Error) loss function.

[0185] Step 4: Based on the loss value, train the initial machine learning model to obtain the trained machine learning model.

[0186] In some alternative implementations, the model parameters of the initial machine learning model can be optimized through backpropagation using the loss value to obtain the trained machine learning model.

[0187] It is understandable that predicting positional offsets through a pre-trained machine learning model can achieve accurate predictions of positional offsets by leveraging the fitting ability of the machine learning model. During the training process of the machine learning model, the offset is obtained based on local features and the initial position. After combining the true position value to calculate the true offset, the true offset is normalized based on the target size. This achieves supervised offset learning based on the adaptive normalization loss function of the target size (e.g., area). This can reduce the magnitude difference in true offsets caused by targets of different sizes, solve the gradient imbalance problem of slender elements, and thus improve the accuracy of target detection results.

[0188] In some optional implementations, taking a long, thin lane line as an example, and the target size of the target object as "6m long and 0.15m wide", if the true offset (i.e., the true position value) of a feature point (such as an endpoint) on the lane line is (0.04m, 0.03m), in related technologies, no equivalent scale is introduced. When normalizing the offset, the normalized offset in the x-direction (corresponding to the length direction) is 0.04m / 6m=1 / 150, and the normalized offset in the y-direction (corresponding to the width direction) is 0.03m / 0.15m=1 / 5. The normalized ratio of the x and y offsets exhibits scaling distortion compared to the original aspect ratio of the target object (the normalized offset ratio in the x and y directions is 1:30, while the original aspect ratio is 40:1). Furthermore, the normalized x and y offset ratio is distorted (the true offset ratio in the x and y directions is 4:3, while the normalized ratio is 1:30), and the magnitude relationship is reversed (the true offset has a larger x-direction offset, while the normalized offset has a smaller x-direction offset). Consequently, during model training, the model prioritizes y-direction errors, leading to a decrease in x-direction positioning accuracy. When normalizing the true offset based on the target size using this scheme, a unified equivalent scale can be calculated using the area of ​​the minimum bounding rectangle. Then, the equivalent scale F is used to process the above offsets (0.04m, 0.03m), resulting in the normalized offset in the x-direction as follows: The normalized offset in the y-direction is After normalization, the ratio and magnitude of the offset in the two directions remain unchanged. Compared with the normalization schemes of the above-mentioned related technologies, the model can simultaneously focus on the offset in the x and y directions (the gradient weights of the loss function on the errors in the x and y directions tend to be balanced). Thus, the equivalent scale F can be used to avoid gradient imbalance caused by the large difference in aspect ratio of slender elements, thereby improving the accuracy of endpoint positioning and the accuracy of target detection results.

[0189] In some alternative implementations, such as Figure 12 As shown above, in the above Figure 11 Based on the illustrated embodiment, step 3054 may include the following steps: Step 30541: Based on the corrected position, determine at least one of the target position and the target size of the target object.

[0190] In this embodiment, the target location can be the final determined location of the target object. In some optional implementations, the target location of the target object can be determined based on the corrected locations of all or some of the feature points of the target object. In some optional implementations, the smallest bounding rectangle containing the corrected locations of all or some of the feature points of the target object can be determined as the target location of the target object.

[0191] The target size can be the final determined size of the target object. In some alternative implementations, the target size can be determined based on the corrected positions of all or some of the feature points of the target object. In some alternative implementations, the target size can be determined as the size of the smallest bounding rectangle containing the corrected positions of all or some of the feature points of the target object.

[0192] Step 30542: Determine the target detection result based on at least one of the determined target location and target size.

[0193] In some optional implementations, the target location and / or target size can be used as the target detection result. In some optional implementations, the target detection result may also include semantic information of the target object.

[0194] Based on this embodiment, since the corrected position can more accurately represent the position of the feature point, the target position and / or target size of the target object can be determined by the corrected position, and then the target detection result can be determined based on at least one of the determined target position and target size, which can improve the accuracy of determining the target detection result.

[0195] Any of the target detection methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the target detection methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the target detection methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0196] This disclosure provides a map building apparatus, wherein the map building apparatus 102 may include a processor 111, which can read executable instructions from a memory 112 and execute the executable instructions to implement a target detection method. The map building apparatus 102 of this embodiment can be applied to electronic devices such as terminal devices and servers.

[0197] Figure 13 This is a schematic flowchart illustrating a map building method configured to be executed by a map building apparatus provided for an exemplary embodiment of this disclosure. The map building apparatus 102 includes a processor 111. Figure 13 As shown, the processor 111 included in the map building device 102 can be configured as follows: Step 401: Determine the target detection result of the target object, where the target object is a map element.

[0198] In this embodiment, the target detection result is obtained through the above... Figures 3-12 The target detection method configured to be executed by the target detection apparatus of any embodiment is determined.

[0199] Map elements can be geographic entities that need to be vectorized. In some alternative implementations, map elements can include, but are not limited to, at least one of the following: zebra crossings, sidewalks, curbs, and lane lines.

[0200] Step 402: Based on the target detection results, construct a vector map containing the target objects.

[0201] In this embodiment, the vector map can be a structured map composed of vector data (point sets) of map elements. In some optional implementations, the vector map may include, but is not limited to, at least one of the following: zebra crossings, sidewalks, curbs, and lane lines.

[0202] In some optional implementations, the target detection results (including the corrected positions of feature points of the target) can be organized into a structured vector map, thus obtaining a vector map containing the target. In some optional implementations, zebra crossing point sets (containing the coordinates of the corrected positions of feature points with the attribute of zebra crossing), lane line point sets (containing the coordinates of the corrected positions of feature points with the attribute of lane line), and curb point sets (containing the coordinates of the corrected positions of feature points with the attribute of curb) can be classified according to semantic attributes and integrated into a vector map.

[0203] Based on this embodiment, by dynamically determining the target scale features through target size, more suitable scale features can be dynamically selected for targets of different sizes. This avoids the problem of lost details for small targets or interference with redundant information for large targets caused by fixed single-scale features, enhancing the specificity and effectiveness of feature representation, thereby improving the accuracy of target detection results. Furthermore, constructing a vector map containing the target objects based on the above target detection results can improve the accuracy of the vector map.

[0204] In some optional implementations, a two-stage refinement mechanism based on orientation awareness (i.e., rotating the minimum bounding rectangle) can be introduced (i.e., determining the corrected position of the feature point based on the position offset and the initial position of the feature point). Combined with an adaptive loss function (i.e., normalizing the true value offset corresponding to the initial position based on the target size), the positioning accuracy and geometric fidelity of map element point coordinates can be significantly improved. For example, it can improve the boundary accuracy and positioning accuracy of some long and thin map elements.

[0205] Any of the map construction methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the map construction methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the map construction methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0206] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the target detection method or map construction method of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0207] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0208] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the target detection method or map construction method of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0209] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0210] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0211] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A target detection device, comprising a processor, the processor being configured to: Feature extraction is performed on the perceived data to obtain multi-scale features; Based on the perceived data, the initial position and size of the target object are determined; Based on the target size and the multi-scale features, determine the target scale features that match the target object; Based on the initial position and the target scale features, determine the local features of the target object; Based on the local features and the initial position, the target detection result of the target object is determined.

2. The apparatus according to claim 1, wherein, The step of determining the target detection result of the target object based on the local features and the initial position includes: Based on the local features, determine the positional offset of the feature points of the target object; Based on the initial position, determine the initial position of the feature point; Based on the position offset and the initial position of the feature point, determine the corrected position of the feature point; The target detection result is determined based on the corrected position.

3. The apparatus according to claim 2, wherein, The step of determining the positional offset of the feature points of the target object based on the local features includes: Based on the local features, the positional offset of the feature points of the target object is determined using a pre-trained machine learning model. The machine learning model was trained in the following manner: Using the initial machine learning model, based on local features and the initial position, the offset of the initial position relative to the true position value is obtained; Based on the target size, the truth offset corresponding to the initial position is normalized to obtain the normalized truth offset, wherein the truth offset represents the actual offset of the initial position relative to the position truth. The loss value is determined based on the offset of the initial position relative to the true position value and the offset of the normalized true value; Based on the loss value, the initial machine learning model is trained to obtain a trained machine learning model.

4. The apparatus according to claim 2, wherein, Determining the target detection result based on the corrected position includes: Based on the correction position, at least one of the target position and target size of the target object is determined; The target detection result is determined based on at least one of the determined target location and target size.

5. The apparatus according to claim 1, wherein, Determining the initial position and size of the target object based on the perceived data includes: Based on the perceived data, the initial positions of the feature points of the target object are determined; Based on the initial position, determine the minimum circumscribed rectangle of the target object; The size of the minimum circumscribed rotating rectangle is determined as the target size of the target object.

6. The apparatus according to claim 5, wherein, Determining the minimum circumscribed rectangle of the target object based on the initial position includes: Based on the perceived data, the coordinates, category, and confidence level of the feature points of the target object are determined; The initial position is determined based on the coordinates, the category, and the confidence level. Based on the initial position, the minimum circumscribed rotation rectangle of the target object is determined.

7. The apparatus according to claim 5, wherein, Determining the initial position of feature points of the target object based on the perceived data includes: Based on the perceived data, multi-scale bird's-eye view features are determined; Based on the self-attention mechanism, the multi-scale bird's-eye view features are processed to obtain the feature data of the feature points of the target object and the global feature data of the target object; Based on the cross-attention mechanism, the feature data of the feature points and the global feature data are fused to obtain fused features; The initial position is determined based on the fusion features.

8. The apparatus according to claim 7, wherein, The self-attention mechanism is used to process the multi-scale bird's-eye view features to obtain feature data of the target object's feature points and global feature data of the target object, including: Add position encoding to the multi-scale bird's-eye view features to obtain the encoded multi-scale bird's-eye view features; Based on the encoded multi-scale bird's-eye view features, the global feature data of the target object is determined; Based on the global feature data, the feature data of the initial feature points are updated to obtain the feature data of the feature points of the target object.

9. The apparatus according to claim 7, wherein, The method based on cross-attention fuses the feature data of the feature points and the global feature data to obtain fused features, including: Based on the feature data of the feature points, the enhanced features of the multi-scale bird's-eye view feature are determined; Based on the global feature data, the feature data of the initial feature points are updated to obtain the feature data of the feature points of the target object. Based on the enhanced features and the feature data of the feature points, the fusion features are determined.

10. The apparatus according to claim 1, wherein, Based on the perceived data, the target size of the target object is determined, including: Based on the perceived data, the initial positions of the feature points of the target object are determined; Based on the initial position, determine the minimum circumscribed rectangle of the target object; Determine the length and width of the minimum circumscribed rotation rectangle; The target size is determined based on the length and the width.

11. The apparatus according to any one of claims 1-10, wherein, The step of determining the local features of the target object based on the initial position and the target scale features includes: Based on the initial position, determine the minimum circumscribed rectangle of the target object; Based on the position of the minimum circumscribed rectangle and the target scale features, the local features of the target object are determined.

12. A map building apparatus, comprising a processor, the processor being configured to: Determine the target detection results for the target object, among which, The target object is a map element, and the target detection result is determined by the target detection device according to any one of claims 1-11. Based on the target detection results, a vector map containing the target object is constructed.

13. A target detection method, comprising: Feature extraction is performed on the perceived data to obtain multi-scale features; Based on the perceived data, the initial position and size of the target object are determined; Based on the target size and the multi-scale features, determine the target scale features that match the target object; Based on the initial position and the target scale features, determine the local features of the target object; Based on the local features and the initial position, the target detection result of the target object is determined.

14. A map construction method, comprising: The target detection result of the target object is determined, wherein the target object is a map element, and the target detection result is determined by the target detection method described in claim 13 above; Based on the target detection results, a vector map containing the target object is constructed.

15. A computer-readable storage medium storing a computer program, which, when executed, is used to implement the target detection method of claim 13 or the map construction method of claim 14.

16. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the target detection method of claim 13, or to implement the map construction method of claim 14.

Images