Methods, apparatus, computer equipment and storage media for robot object recognition

Abstract

This invention relates to a method, apparatus, computer device, and storage medium for robot object recognition. The method includes: a robot acquiring a current environmental image and current distance data via a data acquisition device, wherein the current distance data is the current distance between the robot and the target object; the robot recognizing the target object based on the current environmental image, the current distance data, and a target object recognition model, and obtaining the center coordinates of the target object; and the robot obtaining the actual three-dimensional coordinates of the target object based on camera calibration parameters and the center coordinates of the target object, thereby generating a navigation path based on the actual three-dimensional coordinates of the target object. This invention can improve the accuracy of robot recognition.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a method, apparatus, computer device, and storage medium for a robot to identify target objects. Background Technology

[0002] With the development of information technology and the Internet, robotics technology has made great strides and is being applied in more and more fields.

[0003] There are many types of robots, including mining robots, inspection robots, warehouse robots, and humanoid robots. They can all simulate professional human operations or replace humans in various scenarios. When performing operations, robots need to identify two types of targets: one is the operation target, such as goods and shelves; the other is the supporting target, such as charging stations.

[0004] Robots need to identify the target of operation in order to perform the corresponding functions. For example, warehouse robots and handling robots need to identify goods and shelves in order to carry out handling and storage. At the same time, robots also need to identify supporting targets in order to work continuously without human intervention. For example, when the battery is low, the robot automatically identifies the charging station to charge.

[0005] Whether it's the operational target or the supporting target, it all relies on the robot's recognition of the target object. In current technologies, target objects are diverse, and insufficient differentiation between the background and the target object can affect the robot's recognition. Current target object recognition technologies are still insufficient to meet the robot's needs for target object identification. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, the present invention provides a method, apparatus, computer equipment and storage medium for robot identification of target objects.

[0007] In a first aspect, the present invention provides a method for a robot to identify a target object, the method comprising:

[0008] The robot acquires current environmental images and current distance data through a data acquisition device, wherein the current distance data is the current distance between the robot and the target object;

[0009] The robot identifies the target object and obtains the center coordinates of the target object based on the current environment image, the current distance data, and the target object recognition model.

[0010] The robot obtains the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, and generates a navigation path based on the actual three-dimensional coordinates of the target object.

[0011] The data acquisition device includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data.

[0012] Optionally, before the robot acquires the current environmental image and current distance data through the data acquisition device, the method further includes:

[0013] Build the initial model;

[0014] Obtain the raw training data;

[0015] The original training data is preprocessed to obtain sample training data;

[0016] The initial model is trained using the sample training data to obtain the target object recognition model.

[0017] Optionally, the original training data includes original images and original distances, wherein the original images contain the target object;

[0018] The preprocessing of the original training data to obtain sample training data includes:

[0019] Obtain the original bounding box, which is the bounding box between the target object and the background;

[0020] Convert the original bounding box into a center point;

[0021] Using a general artificial intelligence model, a first mask group and a corresponding first score set are generated based on the original bounding box;

[0022] Using the general artificial intelligence model, a second mask group and a corresponding second score set are generated based on the center point;

[0023] The first mask group and the second mask group are merged to obtain a merged mask group;

[0024] Merge the first score set and the second score set to obtain the merged score set corresponding to the merged mask group;

[0025] Based on the merged mask group, the merged score set, and the visual language model, obtain the optimal segmentation mask;

[0026] Based on the optimal segmentation mask, obtain the corrected bounding box;

[0027] The corrected bounding box is filtered to obtain the filtered bounding box;

[0028] Based on the original bounding box and the filtered bounding box, an interpolated bounding box is obtained to acquire the sample training data.

[0029] Optionally, obtaining the optimal segmentation mask based on the merged mask group, the merged score set, and the visual language model includes:

[0030] Based on the original bounding box and the visual language model, obtain the first score;

[0031] The second score is obtained based on the merged mask group and the merged score set;

[0032] The total score is obtained based on the first score and the second score;

[0033] The mask with the highest total score is taken as the optimal segmentation mask;

[0034] The total score is obtained in the following manner:

[0035]

[0036] For the total score, For the first The first score of the mask, It is the first The second score of the mask, Hyperparameters used to control the contribution of the first and second scores.

[0037] Optionally, obtaining the corrected bounding box based on the optimal segmentation mask includes:

[0038] Obtain the minimum X value, maximum X value, minimum Y value, and maximum Y value from the coordinates included in the optimal segmentation mask to obtain the corrected bounding box.

[0039] Optionally, filtering the corrected bounding box to obtain the filtered bounding box includes:

[0040] Obtain the intersection-union ratio (CUI) between the original bounding box and the corresponding corrected bounding box to obtain the position change parameters and size change parameters;

[0041] If the position change parameter exceeds a first preset value or the size change parameter exceeds a second preset value, then the original bounding box is used as the corresponding corrected bounding box.

[0042] Optionally, the step of obtaining interpolated bounding boxes based on the original bounding boxes and the filtered bounding boxes to obtain the sample training data is carried out in the following manner:

[0043]

[0044]

[0045] in, For the original bounding box, To correct the bounding box, For mixed parameters, Obtained from the hybrid parameter model It is a mixed parameter model. for The parameter ROI() is the alignment function for the ROI. x It is the first Features of the backbone network of the original image Indicates the concatenation of features. It is the Sigmoid activation function.

[0046] Secondly, a device for robot identification of target objects is provided, the device comprising:

[0047] The data acquisition device is used to acquire the current environmental image and current distance data, wherein the current distance data is the current distance between the robot and the target object;

[0048] The recognition unit is used to recognize the target object and obtain the center coordinates of the target object based on the current environment image, the current distance data and the target object recognition model;

[0049] The coordinate acquisition unit is used to acquire the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, so as to generate a navigation path based on the actual three-dimensional coordinates of the target object;

[0050] The data acquisition device includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data.

[0051] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any of the preceding claims.

[0052] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the preceding claims.

[0053] This invention provides a method, apparatus, computer device, and storage medium for robot object recognition. The method includes: a robot acquiring a current environmental image and current distance data via a data acquisition device, wherein the current distance data is the current distance between the robot and the target object; the robot recognizing the target object based on the current environmental image, the current distance data, and a target object recognition model, and obtaining the center coordinates of the target object; the robot obtaining the actual three-dimensional coordinates of the target object based on camera calibration parameters and the center coordinates of the target object, and generating a navigation path based on the actual three-dimensional coordinates of the target object. This invention can improve the accuracy of robot recognition. In this embodiment, the robot recognizes the target object based on the current environmental image, current distance data, and target object recognition model, comprehensively considering image processing and distance data recognition, resulting in high accuracy. Furthermore, in this embodiment, using the center coordinates as an additional input for recognition helps overcome the influence of noise in the bounding box. This embodiment obtains the center coordinates of the target object and generates the final navigation path using the center coordinates, simultaneously achieving recognition and navigation functions, resulting in high efficiency. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 The diagram shown illustrates the application environment of the robot object recognition method according to an embodiment of the present invention.

[0057] Figure 2 The diagram shown is a flowchart illustrating a method for robot object recognition according to an embodiment of the present invention.

[0058] Figure 3 The diagram shown is a flowchart illustrating a method for robot object recognition according to an embodiment of the present invention.

[0059] Figure 4 The diagram shown is a structural block diagram of a robot object recognition device according to an embodiment of the present invention.

[0060] Figure 5 The diagram shown is an internal structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Figure 1 This is an application environment diagram of a robot object recognition method in one embodiment. (Refer to...) Figure 1 This method for robot target object recognition is applied to a robot target object recognition system. The method includes a robot 110 and / or a server 120. The robot 110 and server 120 are connected via a network. The robot 110 can be categorized by shape, such as a humanoid robot or a wheeled robot, or by function, such as a warehouse robot or an inspection robot. The mobile terminal can be at least one of a mobile phone, tablet computer, or laptop computer. The server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. The server 120 can be set up independently of the robot 110 or integrated into the robot 110.

[0063] The present invention provides a method for a robot to identify a target object, which is applied to robot 110.

[0064] Figure 1 The diagram shown is a flowchart of a robot method for identifying target objects according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Step 210: The robot acquires the current environmental image and current distance data through the data acquisition device, wherein the current distance data is the current distance between the robot and the target object;

[0066] Step 220: The robot identifies the target object based on the current environment image, the current distance data, and the target object recognition model, and obtains the center coordinates of the target object;

[0067] Step 230: The robot obtains the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, and generates a navigation path based on the actual three-dimensional coordinates of the target object.

[0068] The data acquisition device includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data.

[0069] In this embodiment of the invention, the camera is mainly used to acquire RGB images and / or depth images, and the lidar is used to acquire point cloud data. The point cloud data can be used to acquire current environment images and current distance data.

[0070] In this embodiment of the invention, the target center coordinates obtained by the robot are based on the robot's camera as the coordinate system. Therefore, a conversion is required between the target center coordinates and the actual map coordinates, which needs to be performed according to the camera calibration parameters.

[0071] In this embodiment of the invention, the robot identifies the target object based on the current environmental image, current distance data, and a target object recognition model. By comprehensively considering image processing and distance data recognition, the accuracy is high. Furthermore, in this embodiment, using the center coordinates as an additional input for recognition helps overcome the influence of noise in the bounding box. This embodiment obtains the center coordinates of the target object and uses them to generate the final navigation path, simultaneously achieving recognition and navigation functions with high efficiency.

[0072] In this embodiment of the invention, before step 210, before the robot acquires the current environmental image and current distance data through the data acquisition device, the method further includes:

[0073] Build the initial model;

[0074] Obtain the raw training data;

[0075] The original training data is preprocessed to obtain sample training data;

[0076] The initial model is trained using the sample training data to obtain the target object recognition model.

[0077] In this embodiment of the invention, the original training data includes original images and original distances, and the original images contain the target object;

[0078] refer to Figure 3 As shown, the preprocessing of the original training data to obtain sample training data includes:

[0079] Step 310: Obtain the original bounding box, which is the bounding box between the target object and the background;

[0080] Step 320: Convert the original bounding box into a center point;

[0081] Step 330: Using a general artificial intelligence model, generate a first mask group and a corresponding first score set based on the original bounding box;

[0082] Step 340: Using the general artificial intelligence model, generate a second mask group and a corresponding second score set based on the center point;

[0083] Step 350: Merge the first mask group and the second mask group to obtain a merged mask group; merge the first score set and the second score set to obtain a merged score set corresponding to the merged mask group;

[0084] Step 360: Obtain the optimal segmentation mask based on the merged mask group, the merged score set, and the visual language model;

[0085] In the method of this embodiment of the invention, the original bounding box is converted into a center point, and then a mask and a score are generated based on the original bounding box and the center point respectively, and then merged. This fully considers both the bounding box and the center point, and uses the center point as an additional input to obtain the optimal segmentation mask, which helps to overcome the influence of noise in the bounding box.

[0086] Step 370: Obtain the corrected bounding box based on the optimal segmentation mask;

[0087] Step 380: Filter the corrected bounding box to obtain the filtered bounding box;

[0088] Step 390: Obtain interpolated bounding boxes based on the original bounding boxes and the filtered bounding boxes to obtain the sample training data.

[0089] In step 360, obtaining the optimal segmentation mask based on the merged mask group, the merged score set, and the visual language model includes:

[0090] Based on the original bounding box and the visual language model, obtain the prediction mask and the corresponding first score;

[0091] The second score is obtained based on the merged mask group and the merged score set;

[0092] The total score is obtained based on the first score and the second score;

[0093] The mask with the highest total score is taken as the optimal segmentation mask.

[0094] The total score is obtained in the following manner:

[0095]

[0096] For the total score, For the first The first score of the mask, It is the first The second score of the mask, Hyperparameters used to control the contribution of the first and second scores.

[0097] In this embodiment of the invention, in the visual language model, firstly, an image containing only the mask region is generated, then all predicted masks of the image containing only the mask region are obtained, and a score is generated for each predicted mask by calculating the softmax of the cosine similarity between each image encoding and the target category text embedding through the image encoder in the visual language model. This score is denoted as the first score.

[0098] In this embodiment of the invention, a first score is obtained through a visual language model, and a total score is obtained through the first score and a second score, which can make the bounding box closely fit the segmentation mask with the highest score.

[0099] In this embodiment of the invention, step 370, obtaining the corrected bounding box based on the optimal segmentation mask, includes:

[0100] Obtain the minimum X value, maximum X value, minimum Y value, and maximum Y value from the coordinates included in the optimal segmentation mask to obtain the corrected bounding box.

[0101] The optimal segmentation mask is actually a set of coordinates. The coordinates of the corrected bounding box can be obtained from the values ​​in this set of coordinates, thereby obtaining the corrected bounding box.

[0102] In this embodiment of the invention, step 380, filtering the corrected bounding box to obtain the filtered bounding box, includes:

[0103] Obtain the intersection-union ratio (CUI) between the original bounding box and the corresponding corrected bounding box to obtain the position change parameters and size change parameters;

[0104] If the position change parameter exceeds a first preset value or the size change parameter exceeds a second preset value, then the original bounding box is used as the corresponding corrected bounding box.

[0105] In certain scenarios, accurate masks may still not be generated because the bounding box might select only a portion of the target object or the background. Therefore, in this embodiment of the invention, the intersection-over-union (IoU) ratio between the original bounding box and the corresponding corrected bounding box is used to capture positional and dimensional change parameters. These parameters characterize the severity of positional and dimensional changes in the bounding box, respectively. If the positional and dimensional changes are too drastic, the original bounding box is retained and used as the corrected bounding box.

[0106] In this embodiment of the invention, in step 390, the interpolated bounding box is obtained based on the original bounding box and the filtered bounding box to obtain the sample training data, in the following manner:

[0107]

[0108]

[0109] in, For the original bounding box, To correct the bounding box, For mixed parameters, Obtained from the hybrid parameter model It is a mixed parameter model. for The parameter ROI() is the ROI alignment function. x It is the first Features of the backbone network of the original image Indicates the concatenation of features. It is the Sigmoid activation function.

[0110] In this embodiment of the invention, the difference between the original bounding box and the corrected bounding box is used to obtain the difference bounding box, which is equivalent to denoising. This can improve instance refinement and selection during the training process and enhance the feature extraction capability of the model.

[0111] The method of this invention obtains the center coordinates of the target object and uses the center point as an additional input, which helps to overcome the influence of noise in the bounding box and improve the accuracy of robot recognition.

[0112] In this embodiment of the invention, the model building, training and other processes can be completed by the robot, or by the server, or by the server set on the robot.

[0113] like Figure 4 As shown, the present invention also provides a device for robot identification of target objects, the device comprising:

[0114] Data acquisition device 410 is used to acquire current environmental image and current distance data, wherein the current distance data is the current distance between the robot and the target object;

[0115] The identification unit 420 is used to identify the target object and obtain the center coordinates of the target object based on the current environment image, the current distance data and the target object identification model;

[0116] The coordinate acquisition unit 430 is used to acquire the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, so as to generate a navigation path based on the actual three-dimensional coordinates of the target object;

[0117] The data acquisition device 410 includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data.

[0118] In this embodiment of the invention, the identification unit 420 is further configured to:

[0119] Build the initial model;

[0120] Obtain the raw training data;

[0121] The original training data is preprocessed to obtain sample training data;

[0122] The initial model is trained using the sample training data to obtain the target object recognition model.

[0123] In this embodiment of the invention, the original training data includes original images and original distances, and the original images contain the target object;

[0124] The identification unit 420 is further configured to:

[0125] Obtain the original bounding box, which is the bounding box between the target object and the background;

[0126] Convert the original bounding box into a center point;

[0127] Using a general artificial intelligence model, a first mask group and a corresponding first score set are generated based on the original bounding box;

[0128] Using the general artificial intelligence model, a second mask group and a corresponding second score set are generated based on the center point;

[0129] The first mask group and the second mask group are merged to obtain a merged mask group;

[0130] Merge the first score set and the second score set to obtain the merged score set corresponding to the merged mask group;

[0131] Based on the merged mask group, the merged score set, and the visual language model, obtain the optimal segmentation mask;

[0132] Based on the optimal segmentation mask, obtain the corrected bounding box;

[0133] The corrected bounding box is filtered to obtain the filtered bounding box;

[0134] Based on the original bounding box and the filtered bounding box, an interpolated bounding box is obtained to acquire the sample training data.

[0135] In this embodiment of the invention, the identification unit 420 is further configured to:

[0136] Based on the original bounding box and the visual language model, obtain the first score;

[0137] The second score is obtained based on the merged mask group and the merged score set;

[0138] The total score is obtained based on the first score and the second score;

[0139] The mask with the highest total score is taken as the optimal segmentation mask.

[0140] The total score is obtained in the following manner:

[0141]

[0142] For the total score, For the first The first score of the mask, It is the first The second score of the mask, Hyperparameters used to control the contribution of the first and second scores.

[0143] In this embodiment of the invention, the identification unit 420 is further configured to:

[0144] Obtain the minimum X value, maximum X value, minimum Y value, and maximum Y value from the coordinates included in the optimal segmentation mask to obtain the corrected bounding box.

[0145] In this embodiment of the invention, the identification unit 420 is further configured to:

[0146] Obtain the intersection-union ratio (CUI) between the original bounding box and the corresponding corrected bounding box to obtain the position change parameters and size change parameters;

[0147] If the position change parameter exceeds a first preset value or the size change parameter exceeds a second preset value, then the original bounding box is used as the corresponding corrected bounding box.

[0148] In this embodiment of the invention, the recognition unit 420 is further configured to obtain the interpolated bounding box in the following manner to obtain the sample training data:

[0149]

[0150]

[0151] in, For the original bounding box, To correct the bounding box, For mixed parameters, Obtained from the hybrid parameter model It is a mixed parameter model. for The parameter ROI() is the ROI alignment function. x It is the first Features of the backbone network of the original image Indicates the concatenation of features. It is the Sigmoid activation function.

[0152] The device described in this invention can improve the accuracy of robot recognition of target objects.

[0153] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following method: a robot acquires a current environmental image and current distance data through a data acquisition device, the current distance data being the current distance between the robot and a target object; the robot identifies the target object based on the current environmental image, the current distance data, and a target object recognition model, and acquires the center coordinates of the target object; the robot acquires the actual three-dimensional coordinates of the target object based on camera calibration parameters and the center coordinates of the target object, and generates a navigation path based on the actual three-dimensional coordinates of the target object; wherein the data acquisition device includes a camera and a lidar, the camera being used to acquire the current environmental image, and the lidar being used to acquire the current environmental image and the current distance data.

[0154] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the following method: a robot acquires a current environmental image and current distance data via a data acquisition device, wherein the current distance data is the current distance between the robot and a target object; the robot identifies the target object based on the current environmental image, the current distance data, and a target object recognition model, and acquires the center coordinates of the target object; the robot acquires the actual three-dimensional coordinates of the target object based on camera calibration parameters and the center coordinates of the target object, and generates a navigation path based on the actual three-dimensional coordinates of the target object; wherein the data acquisition device includes a camera and a lidar, the camera being used to acquire the current environmental image, and the lidar being used to acquire the current environmental image and the current distance data.

[0155] The aforementioned method for robot object recognition achieves the beneficial effect of solving the technical problems raised in the background art.

[0156] Figures 2 to 3 This is a flowchart illustrating a method for a robot to identify a target object in one embodiment. It should be understood that, although... Figures 2 to 3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2 to 3 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0157] Figure 5 An internal structural diagram of a computer device in one embodiment is shown. Specifically, this computer device may be... Figure 1 Robot 110 or server 120 in the middle. For example... Figure 5 As shown, the computer device includes a processor, memory, network interface, input device, and display screen connected via a system bus. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and may also store computer programs. When executed by the processor, these programs enable the processor to implement a method for robot object recognition. The internal memory may also store computer programs, which, when executed by the processor, enable the processor to implement the robot object recognition method. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0158] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0160] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for a robot to identify a target object, characterized in that, The method includes: The robot acquires current environmental images and current distance data through a data acquisition device, wherein the current distance data is the current distance between the robot and the target object; The robot identifies the target object and obtains the center coordinates of the target object based on the current environment image, the current distance data, and the target object recognition model. The robot obtains the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, and generates a navigation path based on the actual three-dimensional coordinates of the target object. The data acquisition device includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data. Before the robot acquires the current environmental image and current distance data through the data acquisition device, the method further includes: Build the initial model, Obtain the raw training data. The original training data is preprocessed to obtain sample training data. The initial model is trained using the sample training data to obtain the target object recognition model; The original training data includes original images and original distances, wherein the original images contain the target object; The preprocessing of the original training data to obtain sample training data includes: Obtain the original bounding box, which is the bounding box between the target object and the background. Convert the original bounding box into a center point. Using a general artificial intelligence model, a first mask group and a corresponding first score set are generated based on the original bounding box. Using the aforementioned general artificial intelligence model, a second mask group and a corresponding second score set are generated based on the center point. The first mask group and the second mask group are merged to obtain a merged mask group. Merge the first score set and the second score set to obtain the merged score set corresponding to the merged mask group. Based on the merged mask group, the merged score set, and the visual language model, obtain the optimal segmentation mask. Based on the optimal segmentation mask, obtain the corrected bounding box. The corrected bounding box is filtered to obtain the filtered bounding box. Based on the original bounding box and the filtered bounding box, an interpolated bounding box is obtained to acquire the sample training data; The step of obtaining interpolated bounding boxes based on the original bounding boxes and the filtered bounding boxes to obtain the sample training data is carried out in the following manner: in, For the original bounding box, To correct the bounding box, For mixed parameters, Obtained from the hybrid parameter model It is a mixed parameter model. for The parameter ROI() is the alignment function for the ROI. It is the first Features of the backbone network of the original image Indicates the concatenation of features. It is the Sigmoid activation function.

2. The method according to claim 1, characterized in that, The step of obtaining the optimal segmentation mask based on the merged mask group, the merged score set, and the visual language model includes: Based on the original bounding box and the visual language model, obtain the first score; The second score is obtained based on the merged mask group and the merged score set; The total score is obtained based on the first score and the second score; The mask with the highest total score is taken as the optimal segmentation mask; The total score is obtained in the following manner: For the total score, For the first The first score of the mask, For the first The second score of the mask, Hyperparameters used to control the contribution of the first and second scores.

3. The method according to claim 1, characterized in that, The step of obtaining the corrected bounding box based on the optimal segmentation mask includes: Obtain the minimum X value, maximum X value, minimum Y value, and maximum Y value from the coordinates included in the optimal segmentation mask to obtain the corrected bounding box.

4. The method according to claim 1, characterized in that, The step of filtering the corrected bounding box to obtain the filtered bounding box includes: Obtain the intersection-union ratio (CUI) between the original bounding box and the corresponding corrected bounding box to obtain the position change parameters and size change parameters; If the position change parameter exceeds a first preset value or the size change parameter exceeds a second preset value, then the original bounding box is used as the corresponding corrected bounding box.

5. A device for robot to identify target objects, characterized in that, The device includes: The data acquisition device is used to acquire the current environmental image and current distance data, wherein the current distance data is the current distance between the robot and the target object; The recognition unit is used to recognize the target object and obtain the center coordinates of the target object based on the current environment image, the current distance data and the target object recognition model; The coordinate acquisition unit is used to acquire the actual three-dimensional coordinates of the target object based on the camera calibration parameters and the center coordinates of the target object, so as to generate a navigation path based on the actual three-dimensional coordinates of the target object; The data acquisition device includes a camera and a lidar. The camera is used to acquire the current environment image, and the lidar is used to acquire the current environment image and the current distance data. The identification unit is also used for: Build the initial model, Obtain the raw training data. The original training data is preprocessed to obtain sample training data. The initial model is trained using the sample training data to obtain the target object recognition model; The original training data includes original images and original distances, wherein the original images contain the target object; The identification unit is also used for: Obtain the original bounding box, which is the bounding box between the target object and the background. Convert the original bounding box into a center point. Using a general artificial intelligence model, a first mask group and a corresponding first score set are generated based on the original bounding box. Using the aforementioned general artificial intelligence model, a second mask group and a corresponding second score set are generated based on the center point. The first mask group and the second mask group are merged to obtain a merged mask group. Merge the first score set and the second score set to obtain the merged score set corresponding to the merged mask group. Based on the merged mask group, the merged score set, and the visual language model, obtain the optimal segmentation mask. Based on the optimal segmentation mask, obtain the corrected bounding box. The corrected bounding box is filtered to obtain the filtered bounding box. Based on the original bounding box and the filtered bounding box, an interpolated bounding box is obtained to acquire the sample training data; The recognition unit is further configured to obtain the interpolated bounding box to acquire the sample training data in the following manner: in, For the original bounding box, To correct the bounding box, For mixed parameters, Obtained from the hybrid parameter model It is a mixed parameter model. for The parameter ROI() is the alignment function for the ROI. It is the first Features of the backbone network of the original image Indicates the concatenation of features. It is the Sigmoid activation function.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 4.

Images