Image data labeling method and device based on weakly supervised learning, equipment and medium

Abstract

The application provides an image data labeling method and device based on weakly supervised learning, equipment and medium. The method comprises: collecting depth point cloud data and color image data of the surrounding environment by using a depth camera on a robot; performing distance clustering processing on the depth point cloud data to separate the foreground image and the background image in the depth point cloud data, wherein the foreground image comprises a plurality of first target objects; determining whether the distance between the first target object and the depth camera of the robot is less than a distance threshold according to the distance information of the first target object in the depth point cloud data; regarding the first target object corresponding to the distance less than the distance threshold as a second target object, and performing bounding box labeling on the second target object in the color image data to obtain labeled color image data. The application reduces the labeling cost of image data, improves the labeling efficiency, and improves the quality and accuracy of image data labeling.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image data annotation method, apparatus, device and medium based on weakly supervised learning. Background Technology

[0002] In the field of robotics, robots need to perform high-intensity target detection tasks in order to achieve functions such as obstacle avoidance, localization, and navigation. Therefore, the efficiency and accuracy of robots in performing target detection tasks will directly affect the robot's business execution capabilities.

[0003] Current object detection tasks require manual pre-labeling of data to train an object detection model, which the robot then uses to perform object detection during subsequent task execution. However, this manual data labeling method not only increases labeling costs and reduces the efficiency of image data labeling, but also makes the detection performance of the object detection model susceptible to the quality of the image data labeling. Furthermore, manual labeling cannot quickly adapt to information about the external environment collected in real time by the robot, resulting in poor image labeling quality. Summary of the Invention

[0004] In view of this, embodiments of this application provide an image data annotation method, apparatus, device, and medium based on weakly supervised learning to solve the problems of high annotation cost, low annotation efficiency, and inability to quickly annotate image data based on environmental data collected in real time by robots in the prior art.

[0005] A first aspect of this application provides an image data annotation method based on weakly supervised learning, comprising: acquiring depth point cloud data and color image data of the surrounding environment using a depth camera on a robot; performing distance clustering processing on the depth point cloud data to separate foreground and background images in the depth point cloud data, wherein the foreground image contains multiple first target objects; determining whether the distance between the first target object and the robot's depth camera is less than a distance threshold based on the distance information of the first target object in the depth point cloud data; taking the first target object corresponding to the distance being less than the distance threshold as a second target object, and annotating the second target object with a bounding box in the color image data to obtain annotated color image data.

[0006] A second aspect of this application provides an image data annotation device based on weakly supervised learning, comprising: an acquisition module configured to acquire depth point cloud data and color image data of the surrounding environment using a depth camera on a robot; a separation module configured to perform distance clustering processing on the depth point cloud data to separate a foreground image and a background image in the depth point cloud data, wherein the foreground image contains a plurality of first target objects; a judgment module configured to determine whether the distance between the first target object and the depth camera of the robot is less than a distance threshold based on the distance information of the first target object in the depth point cloud data; and an annotation module configured to take the first target object corresponding to the distance being less than the distance threshold as a second target object, and to annotate the second target object with a bounding box in the color image data to obtain annotated color image data.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0010] This application utilizes a depth camera on a robot to collect depth point cloud data and color image data of the surrounding environment. Distance clustering is performed on the depth point cloud data to separate foreground and background images, with the foreground image containing multiple first target objects. Based on the distance information of the first target objects in the depth point cloud data, it is determined whether the distance between the first target object and the robot's depth camera is less than a distance threshold. The first target objects whose distance is less than the distance threshold are designated as second target objects, and bounding boxes are added to the color image data to obtain annotated color image data. This application enables automatic annotation using a robot based on depth point cloud data and color image data containing surrounding environment information collected by a depth camera. This not only reduces the cost and improves the efficiency of image data annotation but also allows the robot to annotate the collected environmental image data in real time, improving the quality and accuracy of image data annotation. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the image data annotation method based on weakly supervised learning provided in an embodiment of this application;

[0013] Figure 2 This is a schematic diagram of the structure of the image data annotation device based on weakly supervised learning provided in the embodiments of this application;

[0014] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0016] As mentioned in the background section, during the robot's task execution, it performs high-intensity target detection tasks. Currently, the annotation cost for target detection is too high, hindering the progress of these tasks. Existing robot-based target detection methods suffer from high labor and time costs, and are susceptible to the impact of data annotation quality.

[0017] In view of this, embodiments of this application provide an improved image data annotation method based on weakly supervised learning. This application enables robots to possess automatic and closed-loop learning characteristics in target detection without manual data annotation. By combining point cloud data and image data, the robot can automatically annotate the data to obtain high-quality labeled bounding boxes. This application provides an efficient, fast, and convenient method for acquiring labeled data for weakly supervised target detection, offering a new approach to realizing automatic data annotation by robots.

[0018] This application presents an image data annotation method based on weakly supervised learning, which can significantly reduce the labor, time, and error costs associated with sample annotation and greatly improve the iteration speed of the algorithm model. This application acquires depth point cloud data through a depth camera, then distinguishes between background and target, and combines this with image data to generate annotation data, enabling the robot to automatically annotate the data without manual annotation. Specifically targeting robot object detection tasks, this application combines deep learning, weakly supervised learning, and point cloud data processing methods, using a depth camera to annotate target bounding boxes using point cloud data and image data, thereby reducing labor and time costs.

[0019] Figure 1 This is a flowchart illustrating the image data annotation method based on weakly supervised learning provided in an embodiment of this application. Figure 1 Image data annotation methods based on weakly supervised learning can be performed by robots. For example... Figure 1 As shown, this image data annotation method based on weakly supervised learning can specifically include:

[0020] S101 uses a depth camera on the robot to collect depth point cloud data and color image data of the surrounding environment;

[0021] S102, perform distance clustering processing on the depth point cloud data to separate the foreground image and background image in the depth point cloud data, wherein the foreground image contains multiple first target objects;

[0022] S103, based on the distance information of the first target object in the depth point cloud data, determine whether the distance between the first target object and the robot's depth camera is less than the distance threshold;

[0023] S104, the first target object corresponding to the distance less than the distance threshold is taken as the second target object, and the second target object is marked with a border in the color image data to obtain the marked color image data.

[0024] Specifically, the depth camera in this application embodiment can be a depth camera (RGBD camera). A depth camera is a camera that can directly output depth, or can indirectly calculate depth through software, or directly measure depth through physical methods. By scanning the robot's surrounding environment with an RGBD camera, depth point cloud data (i.e., depth image) and RGB image (i.e., color image) containing information about the external environment can be directly obtained.

[0025] Furthermore, in this embodiment, the target object refers to the target contained in the foreground image. For example, assuming the robot's task scenario is a hotel service scenario, the depth point cloud data and color image data captured by the robot using a depth camera may contain information such as pedestrians and obstacles. In practical applications, objects such as pedestrians and obstacles in the depth point cloud data and color image data can be used as target objects. It should be understood that the above is only an example scenario, and the target objects in the depth point cloud data and color image data captured by the robot can be flexibly changed in different task scenarios.

[0026] In some embodiments, using a depth camera on a robot to collect depth point cloud data and color image data of the surrounding environment includes: using a backend server to locate the robot's position and sending an image data annotation instruction to the robot, so that after receiving the image data annotation instruction, the robot turns on the depth camera to collect information about the surrounding environment, thereby obtaining depth point cloud data and color image data containing information about the surrounding environment.

[0027] Specifically, the backend server is used for remote control or scheduling of robots. The robot and the backend server transmit data through a remote communication network or 5G network signal. The backend server can be a physical server or a virtual server. For example, in one scenario, the backend server adopts a cloud platform architecture, which can contain multiple servers.

[0028] Furthermore, before using the depth camera mounted on the robot to collect depth point cloud data and color image data of the surrounding environment, the backend server first locates the robot's position and sends image data annotation instructions to the robot. After receiving the image data annotation instructions, the robot automatically turns on the depth camera (i.e., the depth camera) to collect information about the surrounding environment, obtaining depth point cloud data and color image data containing the surrounding environment information. Therefore, the image data annotation in this embodiment is based on depth point cloud data and color image data of the surrounding environment information collected by the robot in real time, rather than manual annotation based on historical image data.

[0029] In some embodiments, distance clustering processing is performed on the depth point cloud data to separate the foreground image and the background image in the depth point cloud data, including: clustering the pixels according to the distance information of each pixel in the depth point cloud data using a predetermined distance clustering algorithm to obtain multiple clusters, including the cluster corresponding to the foreground image and the cluster corresponding to the background image, wherein the cluster corresponding to the background image contains multiple clusters of the first target object.

[0030] Specifically, the depth point cloud data in this application embodiment can actually be considered as a depth point cloud image. Each pixel in the depth point cloud image corresponds to its own distance information, which refers to the distance between the pixel and the depth camera.

[0031] Furthermore, the main principle of distance clustering algorithms is that, given a dataset with N objects, K clusters are constructed, where k ≤ n. These clusters must satisfy the following conditions: each cluster contains at least one object, and each object belongs to exactly one cluster. The k clusters that satisfy these conditions are considered a valid partition. In practical applications, the distance clustering algorithm in this embodiment can employ the k-Means algorithm, also known as the k-means algorithm. The k-Means algorithm requires that the data in each cluster follow a normal distribution, and the overall data follow a Gaussian mixture distribution; that is, the model is a Gaussian mixture model (GMM).

[0032] In some embodiments, determining whether the distance between the first target object and the robot's depth camera is less than a distance threshold based on the distance information of the first target object in the depth point cloud data includes: calculating the distance information corresponding to each first target object based on the distance information of each pixel in the cluster of the first target object in the depth point cloud data, and comparing the distance information corresponding to each first target object with the distance threshold to determine whether the distance between the first target object and the robot's depth camera is less than the distance threshold.

[0033] Specifically, each cluster of first target objects contains at least one first target object, and each cluster of first target objects contains n pixels. These n pixels together constitute a first target object. Therefore, based on the distance information of each pixel in the cluster of first target objects in the depth point cloud data, the distance information corresponding to each first target object can be automatically determined. By comparing the distance information corresponding to each first target object with a distance threshold, the first target objects whose distance (i.e., distance information) to the robot's depth camera is less than the distance threshold are taken as the objects to be finally labeled (i.e., the second target objects). Please refer to the following embodiment for the specific calculation method of the distance information corresponding to the first target object.

[0034] In some embodiments, calculating the distance information corresponding to each first target object includes: using the distance information corresponding to the pixel located in the middle position in the cluster of the first target object as the distance information of the first target object; or, using the average value of the distance information corresponding to each pixel in the cluster of the first target object as the distance information of the first target object.

[0035] Specifically, there are two methods for calculating the distance information of the first target object. The first method is to use the distance information corresponding to the pixel located in the middle position in the cluster of the first target object as the distance information of the first target object. The second method is to use the average value of the distance information of each pixel in the cluster of the first target object as the distance information of the first target object. The above two methods for calculating distance information are merely optional implementation methods and do not constitute a limitation on the technical solution of this application.

[0036] In some embodiments, the process of marking the second target object with a bounding box in the color image data to obtain marked color image data includes: mapping the second target object to the color image data based on the position of the second target object in the depth point cloud data and the mapping relationship between the depth point cloud data and the color image data, and marking the second target object with a bounding box in the color image data to obtain marked color image data.

[0037] Specifically, since both the depth point cloud image and the color image are acquired using the same depth camera, there is a certain positional mapping relationship between them. That is, the position of the second target object in the color image can be found by knowing its position in the depth point cloud image. Based on this mapping relationship, the second target object is mapped to the color image data, and bounding boxes are added to the second target object based on the color image data, thus obtaining the annotated color image data. In practical applications, bounding box annotation can be automatically performed on the second target object in the color image using a preset bounding box annotation algorithm.

[0038] In some embodiments, after obtaining the labeled color image data, the method further includes: uploading the labeled color image data to a backend server, so that the backend server uses the labeled color image data as training samples, trains a pre-configured target detection model using the training samples, and returns the trained target detection model to the robot.

[0039] Specifically, after labeling the second target object in the color image with bounding boxes, the labeled color image data is uploaded to a backend server. The backend server uses this labeled color image data as training samples to train a pre-configured target detection model, resulting in a trained target detection model. In practical applications, the target detection model can be YOLO or other target detection models. This application does not impose specific limitations on the target detection model; any bounding box-based target detection model is applicable to this application.

[0040] According to the technical solution provided in the embodiments of this application, this embodiment utilizes a depth camera on a robot to collect depth point cloud data and color image data of the surrounding environment; performs distance clustering processing on the depth point cloud data to separate the foreground image and background image in the depth point cloud data, wherein the foreground image contains multiple first target objects; based on the distance information of the first target objects in the depth point cloud data, determines whether the distance between the first target object and the robot's depth camera is less than a distance threshold; the first target object corresponding to the distance being less than the distance threshold is taken as a second target object, and a bounding box is marked on the second target object in the color image data to obtain the marked color image data. This application can utilize a robot to automatically mark depth point cloud data and color image data containing surrounding environment information collected by a depth camera, which not only reduces the cost of image data marking and improves the efficiency of image data marking, but also enables the robot to mark the collected environmental image data in real time, improving the quality and accuracy of image data marking.

[0041] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0042] Figure 2 This is a schematic diagram of the image data annotation device based on weakly supervised learning provided in an embodiment of this application. Figure 2 As shown, the image data annotation device based on weakly supervised learning includes:

[0043] The acquisition module 201 is configured to acquire depth point cloud data and color image data of the surrounding environment using a depth camera on the robot.

[0044] The separation module 202 is configured to perform distance clustering processing on the depth point cloud data to separate the foreground image and background image in the depth point cloud data, wherein the foreground image contains multiple first target objects;

[0045] The judgment module 203 is configured to determine whether the distance between the first target object and the robot's depth camera is less than a distance threshold based on the distance information of the first target object in the depth point cloud data.

[0046] The annotation module 204 is configured to take the first target object corresponding to the distance less than the distance threshold as the second target object, and to annotate the second target object in the color image data to obtain the annotated color image data.

[0047] In some embodiments, Figure 2The acquisition module 201 uses the backend server to locate the robot's position and sends image data annotation instructions to the robot. After receiving the image data annotation instructions, the robot turns on the depth camera to collect information about the surrounding environment and obtains depth point cloud data and color image data containing information about the surrounding environment.

[0048] In some embodiments, Figure 2 The separation module 202 clusters the pixels based on the distance information of each pixel in the depth point cloud data using a predetermined distance clustering algorithm to obtain multiple clusters. The clusters include the clusters corresponding to the foreground image and the clusters corresponding to the background image. The clusters corresponding to the background image contain multiple clusters of the first target object.

[0049] In some embodiments, Figure 2 The judgment module 203 calculates the distance information corresponding to each first target object based on the distance information of each pixel in the cluster of the first target object in the depth point cloud data, and compares the distance information corresponding to each first target object with the distance threshold to determine whether the distance between the first target object and the robot's depth camera is less than the distance threshold.

[0050] In some embodiments, Figure 2 The judgment module 203 uses the distance information corresponding to the pixel located in the middle position in the cluster of the first target object as the distance information of the first target object; or, it uses the average value of the distance information of each pixel in the cluster of the first target object as the distance information of the first target object.

[0051] In some embodiments, Figure 2 The annotation module 204 maps the second target object to the color image data based on the position of the second target object in the depth point cloud data and the mapping relationship between the depth point cloud data and the color image data, and annotates the second target object with a bounding box in the color image data to obtain the annotated color image data.

[0052] In some embodiments, Figure 2 After obtaining the annotated color image data, the annotation module 204 uploads the annotated color image data to the backend server, so that the backend server uses the annotated color image data as training samples to train the pre-configured target detection model, and returns the trained target detection model to the robot.

[0053] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0054] Figure 3This is a schematic diagram of the structure of the electronic device 3 provided in an embodiment of this application. Figure 3 As shown, the electronic device 3 of this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various device embodiments described above.

[0055] For example, computer program 303 may be divided into one or more modules / units, which are stored in memory 302 and executed by processor 301 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 303 in electronic device 3.

[0056] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0057] Processor 301 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0058] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 302 can include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0059] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0060] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0061] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0062] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0064] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0065] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0066] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image data annotation method based on weakly supervised learning, characterized in that, include: The robot uses a depth camera to collect depth point cloud data and color image data of the surrounding environment. Distance clustering is performed on the depth point cloud data to separate the foreground image and background image in the depth point cloud data, wherein the foreground image contains multiple first target objects; Based on the distance information of the first target object in the depth point cloud data, determine whether the distance between the first target object and the depth camera of the robot is less than a distance threshold; The first target object corresponding to the distance being less than the distance threshold is taken as the second target object, and the second target object is marked with a border in the color image data to obtain the marked color image data. The method of using a depth camera on the robot to collect depth point cloud data and color image data of the surrounding environment includes: The backend server is used to locate the position of the robot and send an image data annotation instruction to the robot. After receiving the image data annotation instruction, the robot turns on the depth camera to collect information about the surrounding environment and obtains depth point cloud data and color image data containing information about the surrounding environment. The step of performing distance clustering processing on the depth point cloud data to separate the foreground and background images in the depth point cloud data includes: Based on the distance information of each pixel in the depth point cloud data, the pixels are clustered using a predetermined distance clustering algorithm to obtain multiple clusters. The clusters include the clusters corresponding to the foreground image and the clusters corresponding to the background image. The clusters corresponding to the background image contain multiple clusters of the first target object. The step of determining whether the distance between the first target object and the robot's depth camera is less than a distance threshold based on the distance information of the first target object in the depth point cloud data includes: Based on the distance information of each pixel in the cluster of the first target object in the depth point cloud data, the distance information corresponding to each first target object is calculated, and the distance information corresponding to each first target object is compared with the distance threshold to determine whether the distance between the first target object and the depth camera of the robot is less than the distance threshold. The calculation of distance information corresponding to each of the first target objects includes: The distance information corresponding to the pixel located in the middle position in the cluster of the first target object is used as the distance information of the first target object; Alternatively, the average value of the distance information of each pixel in the cluster of the first target object can be used as the distance information of the first target object. The step of marking the second target object with a border in the color image data to obtain the marked color image data includes: Based on the position of the second target object in the depth point cloud data and the mapping relationship between the depth point cloud data and the color image data, the second target object is mapped to the color image data, and the second target object is marked with a border in the color image data to obtain the marked color image data.

2. The method according to claim 1, characterized in that, After obtaining the labeled color image data, the method further includes: The labeled color image data is uploaded to the backend server, so that the backend server uses the labeled color image data as training samples to train the pre-configured target detection model, and returns the trained target detection model to the robot.

3. An image data annotation device based on weakly supervised learning, wherein the device employs the method described in any one of claims 1-2, characterized in that, The device includes: The acquisition module is configured to use the depth camera on the robot to acquire depth point cloud data and color image data of the surrounding environment; The separation module is configured to perform distance clustering processing on the depth point cloud data to separate the foreground image and background image in the depth point cloud data, wherein the foreground image contains a plurality of first target objects; The judgment module is configured to determine whether the distance between the first target object and the robot's depth camera is less than a distance threshold based on the distance information of the first target object in the depth point cloud data. The annotation module is configured to take the first target object corresponding to the distance being less than the distance threshold as the second target object, and to annotate the second target object with a border in the color image data to obtain annotated color image data.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method of any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 2.

Images