A training set construction method, apparatus, and computer-readable storage medium

Abstract

This application discloses a training set construction method, apparatus, and computer-readable storage medium for achieving automated annotation, reducing the cost of manual annotation, and obtaining high-quality training samples. The method includes: acquiring point cloud data and image data of the current frame; acquiring object standard descriptions, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or image data is less than or equal to a preset threshold; processing the point cloud data to obtain point cloud detection results, the point cloud detection results containing point cloud detection annotations for multiple objects; processing the image data to obtain image detection results, the image detection results containing image detection annotations for multiple objects; establishing a binding relationship between the point cloud detection results, image detection results, and corresponding standard descriptions; and obtaining a set of training samples in the training set based on the binding relationship, the annotation results of the training samples including point cloud detection results, image detection results, and corresponding relationships.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a training set construction method, apparatus and computer-readable storage medium. Background Art

[0002] Currently, artificial intelligence algorithms demonstrate excellent performance across various fields and are widely applied, thanks to the support of datasets, such as deep learning algorithms. Generally, deep learning algorithms are trained using pre-prepared datasets, and then the trained deep learning algorithm is used to perform specific tasks.

[0003] Most current datasets suffer from inconsistent quality, insufficient content, and overly simplistic scenarios. For example, datasets containing images or point clouds typically require manual annotation to obtain high-quality, content-rich datasets. However, manual annotation is extremely time-consuming and labor-intensive. Summary of the Invention

[0004] This application provides a training set construction method, apparatus, and computer-readable storage medium, which can solve the problem of high manual annotation costs.

[0005] In a first aspect, embodiments of this application provide a training set construction method, including:

[0006] Obtain point cloud data and image data of the current frame. The timestamp of the point cloud data is the same as the timestamp of the image data. The point cloud data and the image data are data containing objects collected for the target area.

[0007] Obtain a standard description of the object, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or the image data is less than or equal to a preset threshold;

[0008] The point cloud data is processed to obtain point cloud detection results, which include point cloud detection annotations for multiple objects.

[0009] The image data is processed to obtain image detection results, which include image detection annotations for multiple objects;

[0010] Establish a binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions, where the standard descriptions are used to describe the corresponding single object;

[0011] A set of training samples is obtained based on the binding relationship. The annotation results of the training samples include point cloud detection results, image detection results, and correspondence relationships. The correspondence relationship is the correspondence between the point cloud detection annotation and the image detection annotation of a single object among the multiple objects.

[0012] The beneficial effects of the embodiments in this application compared with the prior art are:

[0013] This application embodiment processes the point cloud data of the current frame to obtain point cloud detection results; processes the image data of the current frame to obtain image detection results; establishes a binding relationship between the point cloud detection results, the image detection results, and corresponding standard descriptions, whereby the standard descriptions describe the corresponding single objects; this binding ensures a one-to-one correspondence between the point cloud detection results and the image detection results, thereby accurately labeling the objects in the point cloud detection results and image detection results. Based on the binding relationship, a set of training samples is obtained in the training set, thus achieving automated labeling, reducing the cost of manual labeling, and obtaining high-quality training samples.

[0014] In one possible implementation of the first aspect, the object is a vehicle, and the method further includes:

[0015] Receive an onboard OBU message, wherein the difference between the timestamp of the onboard OBU message and the timestamp of the point cloud data of the current frame is less than a preset threshold;

[0016] The information carried in the vehicle-mounted OBU message is used as the standard description of the corresponding object.

[0017] In one possible implementation of the first aspect, the standard description includes the motion state and attributes of a single object;

[0018] Correspondingly, the information carried in the vehicle-mounted OBU message is used as the standard description of the corresponding object, including:

[0019] The point cloud features are defined as any one or any combination of the position information, speed information, size information and steering angle information carried in the vehicle-mounted OBU message.

[0020] The image feature is defined as any one or any combination of the vehicle color information, model information, and size information carried in the OBU information.

[0021] In one possible implementation of the first aspect, establishing a binding relationship between the point cloud detection result, the image detection result, and the corresponding standard description includes:

[0022] The point cloud detection results and the image detection results are fused to obtain a fusion result; the fusion result includes the distribution status of multiple targets in the target region;

[0023] The standard description is matched with each target in the fusion result to obtain a first correspondence; the first correspondence is the correspondence between each object in the fusion result and the standard description.

[0024] Based on the first and second correspondences, a binding relationship is established between each target in the point cloud detection result and each target in the image detection result and the standard description; the second correspondence is the correspondence between the point cloud detection result, the image detection result and the fusion result.

[0025] In one possible implementation of the first aspect, the coordinate system of the fusion result is an image coordinate system. Correspondingly, based on the first and second correspondences, a binding relationship is established between each target in the point cloud detection result and each target in the image detection result and the standard description, including:

[0026] The point cloud detection results are converted to the image coordinate system using system calibration parameters;

[0027] In the image coordinate system, each target in the point cloud detection result, each target in the image detection result, and each target in the fusion result are matched, and the binding relationship between each target in the point cloud detection result, each target in the image detection result, and the standard description is established based on the matching result.

[0028] In one possible implementation of the first aspect, the point cloud detection annotation includes a three-dimensional detection box, and the image detection annotation includes a two-dimensional detection box.

[0029] In one possible implementation of the first aspect, the method further includes:

[0030] The training samples obtained based on the binding relationship are filtered to remove abnormal point cloud detection labels and image detection labels.

[0031] Secondly, embodiments of this application provide a target detection method, including:

[0032] Build machine learning models;

[0033] The machine learning model is trained using the training set obtained by any one of the methods in the first aspect above;

[0034] The acquired point cloud and image data are processed using a trained learning model to perform object detection.

[0035] Thirdly, embodiments of this application provide a training set construction apparatus, including:

[0036] The acquisition unit is used to acquire point cloud data and image data of the current frame. The timestamp of the point cloud data is the same as the timestamp of the image data. The point cloud data and the image data are data containing objects collected for a target area.

[0037] And obtain a standard description of the object, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or the image data is less than or equal to a preset threshold;

[0038] The processing unit is used to process the point cloud data to obtain point cloud detection results, wherein the point cloud detection results include point cloud detection annotations for multiple objects;

[0039] The image data is processed to obtain image detection results, which include image detection annotations for multiple objects.

[0040] A construction unit is used to establish the binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions, wherein the standard descriptions are used to describe the corresponding single objects.

[0041] And a set of training samples in the training set are obtained according to the binding relationship. The annotation results of the training samples include point cloud detection results, image detection results and correspondences. The correspondences are the correspondences between the point cloud detection annotations and the image detection annotations of a single object among the multiple objects.

[0042] Fourthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any one of the first or second aspects above.

[0043] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the first or second aspects above.

[0044] In a sixth aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first or second aspects above.

[0045] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a training set construction method provided in an embodiment of this application;

[0048] Figure 2 This is a flowchart illustrating a training set construction method provided in another embodiment of this application;

[0049] Figure 3 This is a flowchart illustrating a target detection method provided in another embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of a training set construction apparatus provided in another embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0053] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0054] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0055] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0056] Figure 1 This is a flowchart illustrating a training set construction method according to an embodiment of this application. It is provided as an example and not as a limitation. This method can create training samples for training sets used in object re-identification, tracking, detection, or path planning. The method includes:

[0057] S101: Obtain the point cloud data and image data of the current frame.

[0058] The timestamps of the point cloud data and the image data are the same. Both point cloud data and image data are data containing objects collected from the target area.

[0059] In some embodiments, point cloud data and image data of the current frame can be acquired through a smart base station (also known as a roadside fusion sensing system or roadside base station). A smart base station is a crucial infrastructure for intelligent transportation vehicle-road cooperation, serving as a service station integrating sensing, computing, and communication capabilities. It includes sensing devices / systems, roadside units, and processing units. Specifically, the data can be acquired through sensing devices / systems, for example, by acquiring point cloud data through LiDAR sensors installed on the roadside and image data through cameras installed on the roadside.

[0060] By acquiring point cloud data and image data with the same timestamp, it is ensured that the data obtained belongs to the same location and the same object.

[0061] S102: Obtain the standard description of the object.

[0062] Specifically, the difference between the timestamp in the standard description and the timestamp in the point cloud data or image data is less than or equal to a preset threshold. This ensures that the acquired data belongs to the same object. In this embodiment, the preset threshold ranges from 1s to 2s, but is not limited to this.

[0063] The above standard description can include the motion state and attributes of a single object. This standard description is an accurate description of a single object. Motion state includes any one or any combination of the target's position, velocity, and steering angle information, and attributes include any one or any combination of the target's size, color, and model information.

[0064] S103: Process the point cloud data to obtain point cloud detection results, which include point cloud detection annotations for multiple objects.

[0065] Specifically, point cloud detection algorithms are used to process point cloud data, detect all objects, and obtain point cloud detection results.

[0066] Point cloud detection annotation includes 3D bounding boxes. A 3D bounding box describes an object. It includes any one or any combination of information about the target in motion and / or any one or any combination of information about the target's attributes, providing the basis for the correspondence between point cloud detection annotation and annotation description.

[0067] S104: Process the image data to obtain the image detection results, which include image detection annotations for multiple objects.

[0068] Specifically, image detection algorithms are used to process image data, detect all objects, and obtain image detection results.

[0069] Image detection annotation includes two-dimensional detection boxes. A two-dimensional detection box describes an object. It includes any one or any combination of information about the target in motion and / or any one or any combination of information about the target's attributes, providing the basis for the correspondence between image detection annotation and annotation description.

[0070] S105: Establish the binding relationship between point cloud detection results, image detection results, and corresponding standard descriptions.

[0071] Specifically, the standard description is matched with each target in the point cloud detection result, that is, the standard description is matched with the 3D detection box of each target to obtain the first correspondence. The first correspondence is the correspondence between each object in the point cloud detection result and the standard description.

[0072] The standard descriptions are matched with each target in the image detection results, that is, the standard descriptions are matched with the two-dimensional detection boxes of each target to obtain a second correspondence. The second correspondence is the correspondence between each object in the image detection results and the standard description. Based on the first and second correspondences, each target in the point cloud detection results and each target in the image detection results are matched. According to the matching results, the binding relationship between each target in the point cloud detection results and each target in the image detection results and the standard description is established, thus obtaining the correspondence between the point cloud detection annotation and the image detection annotation of a single object among multiple objects.

[0073] S106: Based on the binding relationship, a set of training samples in the training set is obtained. The annotation results of the training samples include point cloud detection results, image detection results, and correspondence relationships. The correspondence relationship is the correspondence between the point cloud detection annotation and the image detection annotation of a single object among multiple objects.

[0074] The training samples contain two types of detection results, which can improve detection capabilities, thereby overcoming the problem of inaccurate detection by a single sensing device and improving the reliability of the training samples.

[0075] This application embodiment processes the point cloud data of the current frame to obtain point cloud detection results; processes the image data of the current frame to obtain image detection results; establishes a binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions, whereby the standard descriptions describe the corresponding single objects; this binding ensures a one-to-one correspondence between the point cloud detection results and the image detection results, thereby accurately labeling the objects in both the point cloud detection results and the image detection results. Based on the binding relationship, a set of training samples is obtained in the training set, thus achieving automated labeling, reducing the cost of manual labeling, and obtaining high-quality training samples.

[0076] Figure 2 This is a flowchart illustrating a training set construction method provided in another embodiment of this application, and is intended as an example rather than a limitation. Figure 2 As shown, the binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions is established, including:

[0077] S201: The point cloud detection results and the image detection results are fused to obtain the fusion result; the fusion result includes the distribution status of multiple targets in the target area.

[0078] In this embodiment, the point cloud detection results and image detection results are fused at the result level by using the intrinsic and extrinsic parameters of the camera and lidar sensor. The distribution state includes three-dimensional detection boxes and two-dimensional detection boxes.

[0079] Optionally, the vehicle-end perception results obtained by the vehicle-end perception device set on the vehicle end are received, and the point cloud detection results, image detection results and vehicle-end perception results are fused to obtain a fused result, so as to improve the reliability of the fused result.

[0080] Optionally, the point cloud detection results are processed using a target tracking algorithm to obtain point cloud tracking results, which include multiple frames of point cloud detection results. The image detection results are then processed using the same algorithm to obtain image tracking results, which include multiple frames of image detection results. The point cloud tracking results and image tracking results are then fused to obtain a fused result, further improving the reliability of the fused result.

[0081] Vehicle-mounted sensing devices may include at least one of a lidar sensor and a camera.

[0082] S202: Match the standard description with each target in the fusion result to obtain the first correspondence;

[0083] The first correspondence is the correspondence between each object in the fusion result and the standard description.

[0084] Specifically, the standard description is matched with the distribution status to obtain the first correspondence.

[0085] S203: Based on the first and second correspondence relationships, establish the binding relationship between each target in the point cloud detection results and each target in the image detection results and the standard description;

[0086] The second correspondence is the correspondence between point cloud detection results, image detection results, and fusion results.

[0087] The coordinate system of the fusion result is the image coordinate system.

[0088] Specifically, the point cloud detection results are converted to the image coordinate system using system calibration parameters.

[0089] By using the intrinsic and extrinsic parameters of the camera and LiDAR sensor, the point cloud detection results are mapped onto the image coordinate system.

[0090] In the image coordinate system, each target in the point cloud detection result, each target in the image detection result, and each target in the fusion result are matched, and the binding relationship between each target in the point cloud detection result, each target in the image detection result, and the standard description is established based on the matching results.

[0091] Specifically, the bounding boxes belonging to the same object in the point cloud detection results and the image detection results are bound together, and the bound 3D and 2D bounding boxes are bound to the standard description to establish a binding relationship.

[0092] This embodiment establishes a binding relationship between each target in the point cloud detection result and each target in the image detection result and the standard description based on the first correspondence and the second correspondence, which can correct the detection object error and further improve the reliability of the training sample.

[0093] In another embodiment, the object is a vehicle, and the method further includes:

[0094] Receive vehicle-mounted OBU messages; the difference between the timestamp of the vehicle-mounted OBU message and the timestamp of the point cloud data in the current frame is less than a preset threshold.

[0095] Among them, the on-board unit (OBU) messages can be obtained by the roadside unit.

[0096] The information carried in the vehicle OBU message is used as the standard description of the corresponding object.

[0097] Specifically, any one or any combination of the position information, speed information, size information, and steering angle information carried in the vehicle-mounted OBU message is used as point cloud features; the 3D detection box contains point cloud features, and the standard description is matched with each target in the point cloud detection result through the point cloud features.

[0098] The vehicle color information, model information, and size information carried in the OBU information are used as any one or any combination of them as image features; the two-dimensional detection box contains image features, and the standard description is matched with each target in the image detection result through the image features.

[0099] In another embodiment, the method further includes:

[0100] A set of training samples obtained based on the binding relationship is filtered to remove abnormal point cloud detection labels and image detection labels.

[0101] Specifically, if a single object is not fully displayed in the training samples, the point cloud detection annotation and image detection annotation corresponding to that object will be removed.

[0102] When the object is a vehicle, if a single vehicle is not displayed completely, such as only the front or rear of the vehicle is displayed, the point cloud detection annotations and image detection annotations corresponding to that vehicle are removed.

[0103] Figure 3 This is a schematic flowchart of a target detection method provided in another embodiment of this application. For example... Figure 3 As shown, target detection methods include:

[0104] S301: Building a machine learning model;

[0105] S302: Train the machine learning model using the training set obtained by the training set construction method;

[0106] S303: Use the trained learning model to process the acquired point cloud and image data to perform object detection.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] Figure 4 This is a schematic diagram of a training set construction apparatus provided in another embodiment of this application. For ease of explanation, only the parts related to the embodiments of this application are shown. Figure 4 As shown, the device 4 includes:

[0109] The acquisition unit 40 is used to acquire point cloud data and image data of the current frame. The timestamp of the point cloud data is the same as the timestamp of the image data. The point cloud data and image data are data containing objects collected for the target area.

[0110] And obtain the standard description of the object, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or image data is less than or equal to a preset threshold;

[0111] Processing unit 41 is used to process point cloud data to obtain point cloud detection results, which include point cloud detection annotations for multiple objects.

[0112] The image data is processed to obtain image detection results, which include image detection annotations for multiple objects.

[0113] Construction unit 42 is used to establish the binding relationship between point cloud detection results, image detection results and corresponding standard descriptions. The standard description is used to describe the corresponding single object.

[0114] Based on the binding relationship, a set of training samples in the training set is obtained. The annotation results of the training samples include point cloud detection results and image detection results, as well as the correspondence relationship, which is the correspondence between the point cloud detection annotation and the image detection annotation of a single object among multiple objects.

[0115] In one possible implementation, the object is a vehicle, and the device also includes a receiving unit;

[0116] The receiving unit is used to receive vehicle-mounted OBU messages, wherein the difference between the timestamp of the vehicle-mounted OBU message and the timestamp of the point cloud data of the current frame is less than a preset threshold.

[0117] The acquisition unit 41 is specifically used to acquire the information carried in the vehicle OBU message as a standard description of the corresponding object.

[0118] In one possible implementation, the construction unit 42 is specifically used to fuse the point cloud detection results and the image detection results to obtain a fusion result; the fusion result includes the distribution status of multiple targets in the target region;

[0119] The standard description is matched with each target in the fusion result to obtain a first correspondence; the first correspondence is the correspondence between each object in the fusion result and the standard description.

[0120] Based on the first and second correspondences, a binding relationship is established between each target in the point cloud detection result and each target in the image detection result and the standard description; the second correspondence is the correspondence between the point cloud detection result, the image detection result and the fusion result.

[0121] In one possible implementation, the device further includes a rejection unit;

[0122] The elimination unit is used to filter a set of training samples obtained according to the binding relationship, and eliminate abnormal point cloud detection labels and image detection labels.

[0123] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 5 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 (Only one is shown in the diagram), memory 51, and computer program 52 stored in said memory 51 and executable on said at least one processor 50, wherein said processor 50 executes said computer program 52 to implement the steps in any of the above method embodiments.

[0124] The electronic device 5 may be a desktop computer, a cloud server, a processing unit integrated into a smart base station, or other computing devices. This electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may be an external storage device of the electronic device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 5. Furthermore, the memory 51 may include both internal and external storage units of the electronic device 5. The memory 51 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0127] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0129] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0132] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0135] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training set construction method, characterized in that, include: Obtain point cloud data and image data of the current frame. The timestamp of the point cloud data is the same as the timestamp of the image data. The point cloud data and the image data are data containing objects collected for the target area. Obtain a standard description of the object, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or the image data is less than or equal to a preset threshold; the object is a vehicle. The point cloud data is processed to obtain point cloud detection results, which include point cloud detection annotations for multiple objects. The image data is processed to obtain image detection results, which include image detection annotations for multiple objects; Establish a binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions. The standard descriptions are used to describe the corresponding single object and include the motion state and attributes of the single object carried in the vehicle OBU message. A set of training samples is obtained based on the binding relationship. The annotation results of the training samples include point cloud detection results, image detection results, and correspondence relationships. The correspondence relationship is the correspondence between the point cloud detection annotation and the image detection annotation of a single object among the multiple objects. The step of establishing the binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions includes: fusing the point cloud detection results and the image detection results to obtain a fusion result; the fusion result includes the distribution state of multiple targets in the target region; matching the standard descriptions with each target in the fusion result to obtain a first correspondence; the first correspondence is the correspondence between each object in the fusion result and the standard description; based on the first correspondence and the second correspondence, establishing the binding relationship between each target in the point cloud detection results and each target in the image detection results and the standard description; the second correspondence is the correspondence between the point cloud detection results, the image detection results, and the fusion result.

2. The method as described in claim 1, characterized in that, The method further includes: Receive an onboard OBU message, wherein the difference between the timestamp of the onboard OBU message and the timestamp of the point cloud data of the current frame is less than a preset threshold; The information carried in the vehicle-mounted OBU message is used as the standard description of the corresponding object.

3. The method according to claim 2, characterized in that, The step of using the information carried in the vehicle-mounted OBU message as a standard description of the corresponding object includes: Use any one or any combination of the position information, speed information, size information and steering angle information carried in the vehicle-mounted OBU message as point cloud features; The vehicle color information, model information, and size information carried in the OBU information can be used as any one or any combination of them as image features.

4. The method according to claim 1, wherein the coordinate system of the fusion result is an image coordinate system, and correspondingly, based on the first and second correspondences, establishing a binding relationship between each target in the point cloud detection result and each target in the image detection result and the standard description, including: The point cloud detection results are converted to the image coordinate system using system calibration parameters; In the image coordinate system, each target in the point cloud detection result, each target in the image detection result, and each target in the fusion result are matched, and the binding relationship between each target in the point cloud detection result, each target in the image detection result, and the standard description is established based on the matching result.

5. The method according to claim 1, characterized in that, The point cloud detection annotation includes a three-dimensional detection box, and the image detection annotation includes a two-dimensional detection box.

6. The method as described in claim 1, characterized in that, The method further includes: The training samples obtained based on the binding relationship are filtered to remove abnormal point cloud detection labels and image detection labels.

7. A target detection method, characterized in that, The target detection method includes: Build machine learning models; The machine learning model is trained using the training set obtained by the method described in any one of claims 1-6; The acquired point cloud and image data are processed using a trained learning model to perform object detection.

8. A training set construction apparatus, characterized in that, include: The acquisition unit is used to acquire point cloud data and image data of the current frame. The timestamp of the point cloud data is the same as the timestamp of the image data. The point cloud data and the image data are data containing objects collected for a target area. The standard description of the object is obtained, wherein the difference between the timestamp of the standard description and the timestamp of the point cloud data or the image data is less than or equal to a preset threshold; the object is a vehicle. The processing unit is used to process the point cloud data to obtain point cloud detection results, wherein the point cloud detection results include point cloud detection annotations for multiple objects; The image data is processed to obtain image detection results, which include image detection annotations for multiple objects. The construction unit is used to establish the binding relationship between the point cloud detection results, the image detection results and the corresponding standard descriptions. The standard descriptions are used to describe the corresponding single object and include the motion state and attributes of the single object carried in the vehicle OBU message. And a set of training samples in the training set are obtained according to the binding relationship. The annotation results of the training samples include point cloud detection results, image detection results and correspondences. The correspondences are the correspondences between the point cloud detection annotations and the image detection annotations of a single object among the multiple objects. The step of establishing the binding relationship between the point cloud detection results, the image detection results, and the corresponding standard descriptions includes: fusing the point cloud detection results and the image detection results to obtain a fusion result; the fusion result includes the distribution state of multiple targets in the target region; matching the standard descriptions with each target in the fusion result to obtain a first correspondence; the first correspondence is the correspondence between each object in the fusion result and the standard description; based on the first correspondence and the second correspondence, establishing the binding relationship between each target in the point cloud detection results and each target in the image detection results and the standard description; the second correspondence is the correspondence between the point cloud detection results, the image detection results, and the fusion result.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements any one of claims 1 to 6 or the method as claimed in claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6 or as described in claim 7.

Images