Target size marking method and device, equipment, storage medium and program product
By using image and point cloud data to calculate target observation frames, the method addresses inefficiencies and inaccuracies in existing target size labeling, achieving higher quality and reduced manual correction costs.
Patent Information
- Application Number
- CN202411042895.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-31
AI Technical Summary
The existing methods for target size labeling in autonomous driving face challenges of low efficiency and poor quality due to variance in pre-labeling predictions and the difficulty in quantifying optimal size attributes, leading to high error correction costs.
A method that utilizes image data from vehicle-mounted cameras and point cloud data from laser radars to generate and match target attribute information, calculate observation corner points and center points, and apply an objective function to determine the target observation frame for accurate size labeling.
Improves the efficiency and accuracy of target size labeling by reducing manual correction costs and enhancing the quality of labeled data.
Smart Images

Figure CN119068483A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology, and in particular to a target dimensioning method, device, equipment, storage medium and program product. Background Art
[0002] Currently, the true value of labeled data in the autonomous driving industry is mainly achieved by pre-labeling targets through training deep learning detection models, and then performing manual secondary quality inspection and error correction.
[0003] However, the quality of pre-labeling is highly correlated with the indicators of the pre-trained model, and the model often predicts large variances for the same target size due to the problem of observation angle, and manual quality inspection is difficult to quantify and confirm the optimal size attribute labeling value. Requiring consistent size labeling for the same target in the same data set will bring huge error correction costs, and the quality of the labeled data is also not high. Summary of the invention
[0004] The present application provides a target size labeling method, device, equipment, storage medium and program product to solve the problems of low target size labeling efficiency and poor labeling quality in the related art.
[0005] The first aspect of the present application provides a method for target size labeling, comprising the following steps: acquiring image data collected by a vehicle-mounted camera and point cloud data collected by a laser radar; generating first target attribute information and second target attribute information based on the image data and the point cloud data, respectively, and matching the first target attribute information and the second target attribute information to obtain a target list; calculating the observation corner points and center points of each target in the target list, and calculating the target observation frame corresponding to each target according to the objective function and the observation corner points and center points of each target, so as to label the size of the target based on the target observation frame.
[0006] Optionally, in one embodiment of the present application, matching the first target attribute information and the second target attribute information to obtain a target list includes: acquiring positioning information of the vehicle; and matching the first target attribute information and the second target attribute information based on the positioning information to obtain a target list.
[0007] Optionally, in one embodiment of the present application, a target observation frame of each target is calculated based on observation corner points, center points and an objective function, including: calculating the observation corner points and center points of each target in each frame; calculating the observation angle of each frame based on the observation corner points of each frame, and calculating the Euclidean distance between the target and the vehicle in each frame based on the center point of each frame; obtaining the maximum value of the observation angle and the minimum value of the Euclidean distance; and calculating the target observation frame corresponding to the target based on the maximum value of the observation angle, the minimum value of the Euclidean distance and the objective function.
[0008] Optionally, in one embodiment of the present application, the objective function is:
[0009] f(x)=Max(coeff a *theta / theta max +coeff b *dist min / dist);
[0010] Among them, coeff a is the first weight coefficient, coeff b is the second weight coefficient, coeff a +coeff b =1.0, theta is the observation angle, theta max is the maximum value of the observation angle, dist is the Euclidean distance between the target and the vehicle, and dist min is the minimum value of the Euclidean distance.
[0011] Optionally, in one embodiment of the present application, after marking the size of the target based on the target observation frame, it also includes: obtaining the target observation frame corresponding to each target; storing the target observation frame corresponding to each target and related information of the target observation frame in a preset database, wherein the related information includes at least one of the target size, observation angle, and center point.
[0012] Optionally, in one embodiment of the present application, after marking the size of the target based on the target observation frame, it also includes: obtaining basic information of the target to be marked and / or the target to be quality inspected; querying a preset database based on the basic information of the target to be marked and / or the target to be quality inspected, and obtaining the target observation frame corresponding to the target to be marked and / or the target to be quality inspected and related information of the target observation frame.
[0013] The second aspect of the present application provides a target size labeling device, including: an acquisition module, used to acquire image data collected by a vehicle-mounted camera and point cloud data collected by a laser radar; a matching module, used to generate first target attribute information and second target attribute information based on the image data and the point cloud data, respectively, and match the first target attribute information and the second target attribute information to obtain a target list; a calculation module, used to calculate the observation corner points and center points of each target in the target list, and calculate the target observation frame corresponding to each target according to the objective function, the observation corner points and center points of each target, so as to label the size of the target based on the target observation frame.
[0014] Optionally, in one embodiment of the present application, the matching module is further used to: obtain positioning information of the vehicle; and obtain a target list by matching the first target attribute information and the second target attribute information based on the positioning information.
[0015] Optionally, in one embodiment of the present application, the calculation module is further used to: calculate the observation corner points and center points of each target in each frame; calculate the observation angle of each frame based on the observation corner points of each frame, and calculate the Euclidean distance between the target and the vehicle in each frame based on the center point of each frame; obtain the maximum value of the observation angle and the minimum value of the Euclidean distance; calculate the target observation frame corresponding to the target based on the maximum value of the observation angle, the minimum value of the Euclidean distance and the objective function.
[0016] Optionally, in one embodiment of the present application, the objective function is:
[0017] f(x)=Max(coeff a *theta / theta max +coeff b *dist min / dist);
[0018] Among them, coeff a is the first weight coefficient, coeff b is the second weight coefficient, coeff a +coeff b =1.0, theta is the observation angle, theta max is the maximum value of the observation angle, dist is the Euclidean distance between the target and the vehicle, and dist min is the minimum value of the Euclidean distance.
[0019] Optionally, in one embodiment of the present application, it also includes: a storage module, used to obtain a target observation frame corresponding to each target after marking the size of the target based on the target observation frame; store the target observation frame corresponding to each target and related information of the target observation frame in a preset database, wherein the relevant information includes at least one of the target size, observation angle, and center point.
[0020] Optionally, in one embodiment of the present application, it also includes: a query module, which is used to obtain basic information of the target to be marked and / or the target to be quality inspected after marking the size of the target based on the target observation frame; query a preset database based on the basic information of the target to be marked and / or the target to be quality inspected, and obtain the target observation frame corresponding to the target to be marked and / or the target to be quality inspected and related information of the target observation frame.
[0021] A third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform a target dimension marking method as described in the above embodiment.
[0022] The fourth aspect of the present application provides a computer-readable storage medium having a computer program or instructions stored thereon, and the computer program or instructions are executed by a processor to perform a target dimensioning method.
[0023] The fifth aspect of the present application provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed, the target dimension marking method as described in the above embodiment is implemented.
[0024] Therefore, this application has at least the following beneficial effects:
[0025] The embodiment of the present application can generate first target attribute information and second target attribute information based on image data and point cloud data respectively, and match them to obtain a target list, and then calculate the observation corner point and center point of each target in the target list, and calculate the target observation frame corresponding to each target according to the target function, the observation corner point and center point of each target, so as to mark the size of the target based on the target observation frame, thereby improving the efficiency and accuracy of target size marking and reducing the cost of manual marking. Therefore, the technical problems of low efficiency of target size marking and poor marking quality in the related art are solved.
[0026] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0028] Figure 1 A flowchart of a target size marking method provided according to an embodiment of the present application;
[0029] Figure 2 A schematic diagram of observation angle calculation according to an embodiment of the present application;
[0030] Figure 3 A schematic diagram of Euclidean distance calculation according to an embodiment of the present application;
[0031] Figure 4 A schematic diagram of a target size marking method provided according to a specific embodiment of the present application;
[0032] Figure 5 This is an example diagram of a target size marking device provided according to an embodiment of the present application;
[0033] Figure 6 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0034] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0035] Before describing the solution of this application, the related technology is described as follows:
[0036] Related technology 1 provides a method for acquiring target annotation data, a target tracking method, an intelligent device and a medium, aiming to solve the problem of acquiring target annotation data conveniently and accurately, specifically: acquiring the first data frame collected by each sensor, acquiring the detection frame, the truth frame and its tracking ID of the target in the first data frame; modifying the truth frames representing the same target in the first data frames of different sensors into the same tracking ID to form the second data frames of each sensor; acquiring the truth frame representing the same target as the detection frame, and setting the tracking ID of the detection frame according to the tracking ID of the truth frame in the second data frame; acquiring the target annotation data according to the detection frame and its tracking ID. However, it only solves the problem of unifying the IDs of the same observed target by different sensors during the annotation process.
[0037] Related technology 2 performs target detection and tracking on the point cloud sequence collected by the radar to obtain the motion trajectory of the point cloud target; performs target detection on the image collected by the camera to obtain the category of the image target; obtains the position and posture of the point cloud target at the image time according to the motion trajectory of the point cloud target, obtains the image target matching the point cloud target on the image according to the position and posture, and obtains the annotation data according to the category of the image target and the trajectory information of the point cloud target. However, it cannot improve the detection efficiency and accuracy of the subsequent manual secondary quality inspection.
[0038] Related technology 3 first parses multiple consecutive point cloud frames to obtain the first point cloud set and initial bounding box corresponding to the target in each point cloud frame, and then aligns each first point cloud set to obtain the registration contour corresponding to the target and the registration matrix corresponding to each first point cloud set, and then adjusts the first initial bounding box based on the second point cloud set corresponding to the registration contour to obtain the adjusted first bounding box, and finally adjusts the other initial bounding boxes based on the adjusted first bounding box and each registration matrix to obtain the second bounding box. However, it only involves the correction of the previous and next frames, and lacks a specific quantitative calculation process for the optimal correction value.
[0039] Related technology 4 first obtains a point cloud map, in which the point cloud map is annotated with an initial target frame of the target to be detected; then, based on the point cloud distribution within the initial target frame, the degree of completeness of the initial target frame for the target to be detected is determined; finally, based on the degree of completeness, the initial target frame is adjusted to obtain a standard target frame. Therefore, on the basis of the initial target frame, the degree of completeness of the initial target frame is determined based on the point cloud distribution, and the degree of fit between the initial target frame and the point cloud of the target to be detected can be judged, so as to further adjust the initial target frame using the degree of completeness, improve the accuracy of target detection, and thus improve the annotation quality and annotation efficiency during target detection. However, the problem of quantitative statistical analysis and recommendation of target sizes for multiple frames of the same target is not solved, and the time cost of manual secondary quality inspection and error correction is still relatively high.
[0040] To this end, the present application provides a target size labeling method, in which the first target attribute information and the second target attribute information can be generated based on the image data and the point cloud data respectively, and the target list is obtained by matching them, and then the observation corner points and the center points of each target in the target list are calculated, and the target observation frame corresponding to each target is calculated according to the objective function and the observation corner points and the center points of each target, so as to label the size of the target based on the target observation frame, thereby improving the efficiency and accuracy of target size labeling and reducing the cost of manual labeling.
[0041] Specifically, Figure 1 A schematic flow chart of a target dimension marking method provided in an embodiment of the present application.
[0042] like Figure 1 As shown, the target dimensioning method includes the following steps:
[0043] In step S101, image data collected by the vehicle-mounted camera and point cloud data collected by the laser radar are obtained.
[0044] In step S102, first target attribute information and second target attribute information are generated based on the image data and the point cloud data, respectively, and the first target attribute information and the second target attribute information are matched to obtain a target list.
[0045] It can be understood that the embodiments of the present application can generate first target attribute information and second target attribute information based on image data and point cloud data respectively, and match the first target attribute information and the second target attribute information to obtain a target list, so that when the on-board camera and lidar detect the same target at the same time, the same target can be merged into one target, thereby avoiding duplication and confusion.
[0046] Specifically, the embodiment of the present application can input camera image data into an offline cloud-based visual reasoning model to generate 2D / 3D target attribute information, i.e., first target attribute information, and input point cloud information into an offline cloud-based 3D point cloud reasoning model to generate 3D target attribute information, i.e., second target attribute information, and match the first target attribute information and the second target attribute information to obtain a target list. In order to enable the on-board camera and lidar to detect the same target at the same time, these detection results will be merged into one tracking target, thereby avoiding duplication and confusion.
[0047] In an embodiment of the present application, matching the first target attribute information and the second target attribute information to obtain a target list includes: acquiring positioning information of the vehicle; and matching the first target attribute information and the second target attribute information based on the positioning information to obtain a target list.
[0048] It can be understood that the embodiment of the present application can obtain the positioning information of the vehicle, and obtain the target list by matching the first target attribute information and the second target attribute information based on the positioning information.
[0049] Specifically, the positioning information, the first target attribute information, and the second target attribute information may be used to obtain a target list through a multi-sensor fusion algorithm.
[0050] In step S103, the observation corner points and center points of each target in the target list are calculated, and the target observation frame corresponding to each target is calculated according to the objective function, the observation corner points and center points of each target, so as to mark the size of the target based on the target observation frame.
[0051] Among them, the target observation frame is also called the optimal observation frame.
[0052] It can be understood that the embodiments of the present application can calculate the observation corner points and midline points of each target in the target list, and calculate the target observation frame corresponding to each target based on the objective function, the observation corner points and the center point of each target, so that the size of the target can be annotated based on the observation frame to achieve optimal annotation of the target size.
[0053] In an embodiment of the present application, a target observation frame of each target is calculated based on observation corner points, center points and an objective function, including: calculating the observation corner points and center points of each target in each frame; calculating the observation angle of each frame based on the observation corner points of each frame, and calculating the Euclidean distance between the target and the vehicle in each frame based on the center point of each frame; obtaining the maximum value of the observation angle and the minimum value of the Euclidean distance; and calculating the target observation frame corresponding to the target based on the maximum value of the observation angle, the minimum value of the Euclidean distance and the objective function.
[0054] Among them, the objective function is:
[0055] f(x)=Max(coeff a*theta / theta max +coeff b *dist min / dist);
[0056] Among them, coeff a is the first weight coefficient, coeff b is the second weight coefficient, coeff a +coeff b =1.0, theta is the observation angle, theta max is the maximum value of the observation angle, dist is the Euclidean distance between the target and the vehicle, and dist min is the minimum value of the Euclidean distance.
[0057] Since the target is under continuous observation and the vehicle is in a moving state, the observation corner points and center points of the target are different in each frame. Therefore, the embodiment of the present application can calculate the observation corner points and center points of each target in each frame, and calculate the observation angle of the target in each frame based on the observation corner points of each frame, calculate the Euclidean distance between the target and the vehicle in each frame based on the midline point of each frame, and select the maximum value of the observation angle and the minimum value of the Euclidean distance in multiple frames, and further observe the maximum value of the observation angle, the minimum value of the Euclidean distance and the target observation frame corresponding to the target calculated by the objective function.
[0058] The observation angle theta in the embodiment of the present application is calculated as follows Figure 2 As shown, taking the target car as an example, for any target car and ego car, the ego car length l is known, and the target car frame vertex coordinates are known, so the line distance m, n between the ego car origin and the target car edge vertex AB can be obtained.
[0059] From the cosine theorem, we can get: theta = arccos((m 2 +n 2 -l 2 ) / 2mn);
[0060] The Euclidean distance dist is calculated as Figure 3 As shown, taking the target car as an example, for any tracking target, the center point position is known (x c ,y c ).
[0061] Therefore, the Euclidean distance between the target location and the vehicle is:
[0062] In an embodiment of the present application, after marking the size of the target based on the target observation frame, it also includes: obtaining the target observation frame corresponding to each target; storing the target observation frame corresponding to each target and related information of the target observation frame in a preset database, wherein the related information includes at least one of the target size, observation angle, and center point.
[0063] It can be understood that the embodiment of the present application can obtain the target observation frame corresponding to each target, and store the target observation frame corresponding to each target and the related information of the target observation frame in a preset database, so that subsequent labeling personnel can search for the required target information based on the preset database.
[0064] In an embodiment of the present application, after marking the size of the target based on the target observation frame, it also includes: obtaining basic information of the target to be marked and / or the target to be quality inspected; based on the basic information of the target to be marked and / or the target to be quality inspected, querying a preset database to obtain the target observation frame corresponding to the target to be marked and / or the target to be quality inspected and related information of the target observation frame.
[0065] It can be understood that the embodiments of the present application can also obtain basic information of the target to be labeled and / or the target to be quality inspected, and query the preset database based on the basic information of the target to be labeled and / or the target to be quality inspected to obtain the target observation frame corresponding to the target to be labeled and / or the target to be quality inspected and related information with the target observation frame, thereby directly obtaining the optimal labeling value of the target to be labeled or the target to be quality inspected without the need for complicated operations, thereby reducing the time and cost of manual quality inspection and error correction.
[0066] The target size marking method of the embodiment of the present application is described below through a specific embodiment. Figure 4 shown.
[0067] 1. The on-board camera collects image data, the laser radar collects point cloud data, and records RTK positioning information;
[0068] 2. Input the point cloud information into the offline cloud 3D point cloud inference model to generate 3D target attribute information;
[0069] 3. Input camera images into the offline cloud visual reasoning model to generate 2D / 3D target attribute information;
[0070] 4. Positioning information + 2D / 3D target attribute information is input into the offline multi-sensor fusion module to generate matching information between sensor detection targets and tracked targets;
[0071] 5. Calculate the observed corner points and target center points of each target matching chain;
[0072] 6. Calculate the observable angle theta based on the observed corner points, calculate the Euclidean distance dist between the target and the vehicle based on the center point, and calculate the maximum angle and minimum distance respectively;
[0073] 7. Take the objective function as f(x)=Max(coeff a *theta / theta max +coeff b *dist min / dist), (coeff a +coeff b =1.0), the maximum value result frame calculated by this function is the optimal observation point;
[0074] 8. Cache the result in step 7 in the detection target file, and cache the optimal frame observation value in step 7 as new information in each matching chain target;
[0075] 9. Enter the basic information of the target to be labeled or inspected in the interactive interface (data set, specific frame, initial center position, etc.);
[0076] 10. Obtain the input information in step 9 in the matching recommendation module, and query the optimal value in the cache file in step 8 based on the input information;
[0077] 11. Output the recommended result of the optimal observation frame of the target size.
[0078] In the above embodiment, an offline visual detection model, a lidar detection model, and multi-sensor fusion are integrated to pre-process the annotation data, and further perform tracking target-level statistics on the pre-processed data. An objective function is designed based on the statistical results, and the optimal observation frame of the pre-annotated tracking target relative to the vehicle size is quantitatively analyzed. The quantification results are cached and further presented to the annotation personnel through interactive query. The annotation personnel obtain the recommended results of the optimal size annotation frame through interactive query, thereby improving the efficiency and accuracy of target size quality inspection and annotation, and reducing labor costs.
[0079] According to the target size labeling method proposed in the embodiment of the present application, the first target attribute information and the second target attribute information can be generated based on the image data and the point cloud data respectively, and the target list can be obtained by matching them, and then the observation corner points and center points of each target in the target list are calculated, and the target observation frame corresponding to each target is calculated according to the objective function and the observation corner points and center points of each target, so as to label the size of the target based on the target observation frame, thereby improving the efficiency, accuracy and quality of target size labeling and reducing the cost of manual labeling.
[0080] Next, the target size marking device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.
[0081] Figure 5 It is a block diagram of a target size marking device according to an embodiment of the present application.
[0082] like Figure 5 As shown, the target dimension labeling device 10 includes: an acquisition module 100 , a matching module 200 and a calculation module 300 .
[0083] Among them, the acquisition module 100 is used to acquire the image data collected by the vehicle-mounted camera and the point cloud data collected by the laser radar; the matching module 200 is used to generate the first target attribute information and the second target attribute information based on the image data and the point cloud data, respectively, and match the first target attribute information and the second target attribute information to obtain a target list; the calculation module 300 is used to calculate the observation corner points and center points of each target in the target list, and calculate the target observation frame corresponding to each target according to the objective function and the observation corner points and center points of each target, so as to mark the size of the target based on the target observation frame.
[0084] In the embodiment of the present application, the matching module 200 is further used to: obtain the positioning information of the vehicle; and obtain a target list by matching the first target attribute information and the second target attribute information based on the positioning information.
[0085] In an embodiment of the present application, the calculation module 300 is further used to: calculate the observation corner points and center points of each target in each frame; calculate the observation angle of each frame based on the observation corner points of each frame, and calculate the Euclidean distance between the target and the vehicle in each frame based on the center point of each frame; obtain the maximum value of the observation angle and the minimum value of the Euclidean distance; calculate the target observation frame corresponding to the target based on the maximum value of the observation angle, the minimum value of the Euclidean distance and the objective function.
[0086] In the embodiment of the present application, the objective function is:
[0087] f(x)=Max(coeff a *theta / theta max +coeff b *dist min / dist);
[0088] Among them, oeff a is the first weight coefficient, coeff b is the second weight coefficient, coeff a +coeff b =1.0, theta is the observation angle, theta max is the maximum value of the observation angle, dist is the Euclidean distance between the target and the vehicle, and dist min is the minimum value of the Euclidean distance.
[0089] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a storage module.
[0090] Among them, the storage module is used to obtain the target observation frame corresponding to each target after marking the size of the target based on the target observation frame; store the target observation frame corresponding to each target and the related information of the target observation frame in a preset database, wherein the related information includes at least one of the target size, observation angle, and center point.
[0091] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a query module.
[0092] Among them, the query module is used to obtain the basic information of the target to be marked and / or the target to be quality inspected after marking the size of the target based on the target observation frame; query the preset database based on the basic information of the target to be marked and / or the target to be quality inspected, and obtain the target observation frame corresponding to the target to be marked and / or the target to be quality inspected and the relevant information of the target observation frame.
[0093] It should be noted that the above explanations of the target size marking method embodiment are also applicable to the target size marking device of this embodiment, and will not be repeated here.
[0094] According to the target size labeling device proposed in the embodiment of the present application, the first target attribute information and the second target attribute information can be generated based on the image data and the point cloud data respectively, and the target list can be obtained by matching them, and then the observation corner points and the center point of each target in the target list are calculated, and the target observation frame corresponding to each target is calculated according to the objective function and the observation corner points and the center point of each target, so as to label the size of the target based on the target observation frame, thereby improving the efficiency and accuracy of target size labeling and reducing the cost of manual labeling.
[0095] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:
[0096] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .
[0097] When the processor 602 executes the program, the target dimension marking method provided in the above embodiment is implemented.
[0098] Furthermore, the electronic device further comprises:
[0099] The communication interface 603 is used for communication between the memory 601 and the processor 602 .
[0100] The memory 601 is used to store computer programs that can be executed on the processor 602 .
[0101] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0102] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0103] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.
[0104] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
[0105] An embodiment of the present application also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the target dimension marking method as described above is implemented.
[0106] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed, implements the above target dimension marking method.
[0107] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0108] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0109] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0110] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0111] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
Claims
1. A target dimensioning method, characterized in that: The following steps are involved: Obtain image data collected by the vehicle-mounted camera and point cloud data collected by the lidar; Generate first target attribute information and second target attribute information based on the image data and the point cloud data respectively, and match the first target attribute information and the second target attribute information to obtain a target list; Calculate the observation corner points and center points of each target in the target list, calculate the target observation frame corresponding to each target according to the objective function and the observation corner points and center points of each target, and mark the size of the target based on the target observation frame.
2. The target dimension marking method according to claim 1, characterized in that: The matching the first target attribute information and the second target attribute information to obtain a target list includes: Get the vehicle's location information; The target list is obtained by matching the first target attribute information and the second target attribute information based on the positioning information.
3. The target dimension marking method according to claim 2, characterized in that: The calculating the target observation frame of each target according to the observation corner point, the center point and the target function comprises: Calculate the observed corner points and center points of each target in each frame; Calculating the observation angle of each frame according to the observation corner point of each frame, and calculating the Euclidean distance between the target and the vehicle in each frame according to the center point of each frame; Obtaining the maximum value of the observation angle and the minimum value of the Euclidean distance; A target observation frame corresponding to the target is calculated according to the maximum value of the observation angle, the minimum value of the Euclidean distance and the target function.
4. The target dimension marking method according to claim 3, characterized in that: The objective function is: f(x)=Max(coeff a *theta / theta max +coeff b *dist min / dist); Among them, coeff a is the first weight coefficient, coeff b is the second weight coefficient, coeff a +coeff b =1.0, theta is the observation angle, theta max is the maximum value of the observation angle, dist is the Euclidean distance between the target and the vehicle, and dist min is the minimum value of the Euclidean distance.
5. The target dimensioning method according to claim 1, characterized in that: After marking the size of the target based on the target observation frame, the method further includes: Obtaining a target observation frame corresponding to each target; The target observation frame corresponding to each target and related information of the target observation frame are stored in a preset database, wherein the related information includes at least one of target size, observation angle, and center point.
6. The target dimension marking method according to claim 5, characterized in that: After marking the size of the target based on the target observation frame, the method further includes: Obtain basic information of the target to be marked and / or the target to be quality inspected; Based on the basic information of the target to be labeled and / or the target to be quality inspected, a preset database is queried to obtain a target observation frame corresponding to the target to be labeled and / or the target to be quality inspected and related information of the target observation frame.
7. A target size marking device, characterized in that: include: The acquisition module is used to obtain the image data collected by the vehicle-mounted camera and the point cloud data collected by the laser radar; a matching module, configured to generate first target attribute information and second target attribute information based on the image data and the point cloud data, respectively, and to match the first target attribute information and the second target attribute information to obtain a target list; A calculation module is used to calculate the observation corner points and center points of each target in the target list, and calculate the target observation frame corresponding to each target according to the objective function and the observation corner points and center points of each target, so as to mark the size of the target based on the target observation frame.
8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target dimensioning method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instruction is executed by a processor to implement the target dimensioning method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the target dimension marking method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Target labeling method and device, electronic equipment and computer readable storage medium
CN115827906A
Data labeling method, system and device, electronic equipment and storage medium
CN116258687A
Parking space marking method, model training method, parking method and related devices
CN117593719A
Point cloud data labeling method and device, equipment and storage medium
CN117636343A
Point cloud labeling method, apparatus, and system, device, and storage medium
WO2021114884A1
Cited By
Data auditing interaction method and related device
CN119251652A
