Robot motion control method, apparatus, computing device, and storage medium

Abstract

The application provides a robot motion control method and device, a computing device and a storage medium, wherein the method is applied to a robot, the robot is provided with a visual sensor, and the method comprises the following steps: acquiring a motion scene image collected by the visual sensor; inputting the motion scene image into an identification recognition model to recognize a target identification, wherein the identification recognition model is obtained based on a plurality of identification samples, the identification samples comprise normal identification samples and abnormal identification samples, and the abnormal identification samples comprise identification samples generated based on the normal identification samples by using a generative adversarial network; obtaining target position information of the target identification; and moving to a target position where the target identification is located based on the target position information. By acquiring the motion scene image and based on the identification recognition model, the target position information of the target identification can be obtained and the target position can be reached. Whether the target identification is normal or abnormal, the target position can be obtained, and the accuracy and efficiency of robot path navigation and planning are improved.

Description

Technical Field

[0001] This invention relates to the field of computers, and in particular to a robot motion control method, apparatus, computing device, and storage medium. Background Technology

[0002] In logistics transportation or fixed-point navigation, robots typically need to plan their routes to determine the target location. Currently, most robots use downward-facing cameras located on their undersides, parallel to the ground, to recognize QR code information and pinpoint their current location based on the recognized QR code.

[0003] However, using a downward-facing camera to recognize QR code information for robot navigation has limitations. These limitations can arise when the QR code is dirty or damaged, misplaced in the area, or the downward-facing camera has low resolution. These issues can lead to the robot being unable to accurately recognize the QR code, preventing it from reaching the target location. Therefore, a method to improve the accuracy of robot path planning is urgently needed. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a robot motion control method. The present invention also relates to a robot motion control device, a computing device, and a computer-readable storage medium, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a robot motion control method is provided, applied to a robot equipped with a vision sensor, the method comprising:

[0006] Acquire images of moving scenes captured by a visual sensor;

[0007] The motion scene image is input into the identifier recognition model to identify the target identifier. The identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples. Abnormal identifier samples include identifier samples generated by generative adversarial networks based on normal identifier samples.

[0008] Obtain the target location information of the target identifier;

[0009] Based on the target location information, move to the target location where the target identifier is located.

[0010] According to a second aspect of the present invention, a robot motion control device is provided, disposed on a robot, the robot being provided with a vision sensor, the device comprising:

[0011] The first acquisition module is configured to acquire motion scene images collected by the visual sensor;

[0012] The recognition module is configured to input motion scene images into the identifier recognition model to identify target identifiers. The identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples. Abnormal identifier samples include identifier samples generated by a generative adversarial network based on normal identifier samples.

[0013] The second acquisition module is configured to obtain the target location information of the target identifier;

[0014] The motion module is configured to move to the target location where the target identifier is located based on the target location information.

[0015] According to a third aspect of the present invention, a computing device is provided, comprising: a memory and a processor;

[0016] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the method provided in the first aspect of the embodiments of the present invention.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided that stores computer instructions which, when executed by a processor, implement the method provided in the first aspect of the present invention.

[0018] The robot motion control method provided by this invention is applied to a robot equipped with a vision sensor to acquire motion scene images collected by the vision sensor; the motion scene images are input into an identifier recognition model to identify target identifiers, wherein the identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples, and the abnormal identifier samples include identifier samples generated by using a generative adversarial network based on normal identifier samples; the target position information of the target identifier is obtained; and based on the target position information, the robot moves to the target position where the target identifier is located.

[0019] Thus, by acquiring motion scene images from visual sensors and identifying target icons in the motion scene images based on an icon recognition model, the robot can obtain the target location information of the icon and move to the target location. By generating abnormal icon samples using a generative adversarial network based on normal icon samples, the number of abnormal icon samples can be increased, improving the icon recognition model's ability to identify abnormal icons. Training the icon recognition model with normal and abnormal icon samples enables it to identify target icons in motion scene images. Regardless of whether the target icon is normal or abnormal, the robot can obtain the target location information, allowing it to move to the target location based on this information. This avoids the inability to accurately identify the target icon in the case of an abnormal icon, which would prevent the robot from locating the target position, thus improving the accuracy and efficiency of robot path planning and navigation. Attached Figure Description

[0020] Figure 1 This is a flowchart of a robot motion control method provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the processing flow of a robot motion control method provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of a robot motion control device according to an embodiment of the present invention;

[0023] Figure 4 This is a structural block diagram of a computing device provided in an embodiment of the present invention. Detailed Implementation

[0024] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0025] The terminology used in one or more embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a,” “the,” and “the” used in one or more embodiments of the invention and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of the invention refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of the present invention, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0027] First, the terminology used in one or more embodiments of the present invention will be explained.

[0028] AMR (Autonomous Mobile Robot): refers to a highly autonomous mobile robot that can respond appropriately to various dynamic changes in the environment. It typically includes automatic obstacle avoidance, automatic identification of goods, shelves, docking stations, etc.

[0029] BEV space (Bird's Eye View space): Target view space is a bird's-eye view area in front of the direction of movement. It is a perspective or coordinate system used to describe the perceived world.

[0030] Visual sensor: refers to a sensor that acquires images of a target object to obtain image data.

[0031] Currently, most AMRs use downward-facing vision cameras to recognize QR codes. AMRs can receive task processing instructions, move to the corresponding task processing position, and use the downward-facing vision camera to recognize the QR code at the task processing position to obtain the robot's current position. After completing the task processing at the current position, it can move to the next task processing position according to the task processing instructions.

[0032] Because downward-facing vision cameras are typically mounted on the bottom of the robot, parallel to the ground, their low mounting height results in a small field of view. This means the area captured in the real-world scene is relatively small, requiring extremely accurate QR code recognition. Consequently, the robot heavily relies on the positioning results of previous QR codes. If the previous QR code has significant errors in its placement or low recognition accuracy, the robot may fail to step on the next code because it's not visible in the camera's field of view, hindering its ability to accurately locate its current position and affecting its movement to the next task location. Furthermore, if a downward-facing vision camera fails to recognize a QR code, causing it to miss it, the robot must rely on odometry to reach the next code. The accumulated error in odometry can lead to continuous code loss, resulting in severely inaccurate path navigation and planning. Moreover, since downward-facing cameras typically recognize discontinuous QR codes, they cannot perform object tracking to correct misrecognition, causing the robot to move along incorrect paths and unable to autonomously correct itself.

[0033] To address the aforementioned issues, a scheme has been proposed that uses a forward-looking camera to capture visual images of the scene for code finding. However, current forward-looking code finding schemes have poor adaptability. Most of them use edge detection and line fitting, which cannot identify dirty or blurry QR codes. They cannot use object detection frameworks to obtain the region information of the QR code, and can only locate the center position of the QR code. They cannot obtain the rotation angle of the QR code relative to the robot, thus they cannot adaptively adjust the robot's movement position and orientation to improve the accuracy of the robot's docking position and task execution efficiency.

[0034] Based on this, the present invention provides a robot motion control method applied to a robot. The robot is equipped with a vision sensor to acquire motion scene images collected by the vision sensor; the motion scene images are input into an identifier recognition model to identify target identifiers, wherein the identifier recognition model is trained based on multiple identifier samples, the identifier samples include normal identifier samples and abnormal identifier samples, the abnormal identifier samples include identifier samples generated by using a generative adversarial network based on normal identifier samples; the target position information of the target identifier is obtained; based on the target position information, the robot moves to the target position where the target identifier is located.

[0035] By acquiring motion scene images from visual sensors, a marker recognition model can identify target markers in the motion scene images, obtain the target location information of the markers, and move to the target location. Based on normal marker samples, a generative adversarial network is used to generate abnormal marker samples, increasing the number of abnormal marker samples and improving the marker recognition model's ability to identify abnormal markers. Training the marker recognition model with both normal and abnormal marker samples enables it to identify target markers in motion scene images. Regardless of whether the marker is normal or abnormal, the model can obtain the target location information, allowing the robot to move to the target location based on this information. This avoids the inability to accurately identify markers when they are abnormal, thus improving the accuracy and efficiency of robot path planning and navigation.

[0036] This invention provides a robot motion control method, and also relates to a robot motion control device, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0037] See Figure 1 , Figure 1 The flowchart illustrates a robot motion control method according to an embodiment of the present invention, applied to a robot equipped with a vision sensor, and specifically includes the following steps:

[0038] S102: Acquire motion scene images captured by the visual sensor.

[0039] In practical applications, autonomous mobile robots can be equipped with forward-looking vision sensors, which can acquire visual images with a larger field of view than downward-looking vision sensors.

[0040] Specifically, a visual sensor can be understood as a sensor that identifies an object by capturing visual images. Visual sensors typically analyze and judge the object by the number of pixels, thereby achieving the identification of the object.

[0041] Motion scene images can be understood as images of the scene in which an autonomous mobile robot is located. These are visual images captured by vision sensors mounted on the robot. Autonomous mobile robots can be applied to various scenarios that require the identification, movement, handling, and placement of target objects. Through vision sensors, autonomous mobile robots can identify target objects or obstacles, determine the target's movement position based on the identification results, and perform path planning and navigation.

[0042] For example, in a warehousing and logistics scenario, an autonomous mobile robot can transport goods to be processed to various workstations in the work area. Each workstation has a corresponding label to help the autonomous mobile robot locate itself. The autonomous mobile robot can use its forward-looking vision sensor to collect images of the moving scene in the work area, identify the labels in the moving scene images, determine the target location to move to, and perform path planning and navigation based on the target location.

[0043] It should be noted that since the markings in the field are usually placed manually, such as by sticking them at preset distances between workstations, there is a risk of inaccurate placement. This can easily lead to errors in distance, causing the robot to fail to capture the markings when it moves to the target location. As a result, the cumulative error can cause the robot's path planning to deviate more and more from the standard route. Furthermore, due to unforeseen circumstances, the markings will become damaged or dirty over time, altering their original shape and making them unrecognizable by the vision sensor. Therefore, the motion scene images captured by the vision sensor may contain both normal and abnormal markings. Normal markings can be understood as those that are undamaged or dirty and whose shape and pose have not changed, while abnormal markings can be understood as those that are damaged or dirty and whose shape or pose has changed to some extent, such as a QR code with a missing corner or a QR code partially obscured by dirt.

[0044] Currently, if the markers in a motion scene image are not normal markers, it can easily lead the robot to judge that there are no markers in the motion scene image, causing the robot to miss or fail to recognize them accurately.

[0045] S104: Input the motion scene image into the identifier recognition model to identify the target identifier. The identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples. Abnormal identifier samples include identifier samples generated by a generative adversarial network based on normal identifier samples.

[0046] In practical applications, to avoid the problem of robots missing or failing to accurately identify the location of labels due to abnormal labels, one or more embodiments of the present invention achieve accurate identification of labels in motion scene images by training a label recognition model.

[0047] It should be noted that the identifier can be a graphic with a code value, such as a QR code or barcode, or it can be a graphic with a preset shape, or a recognizable graphic formed by combining a graphic with a code value and a graphic with a preset shape.

[0048] Specifically, a marker recognition model can be understood as a model that identifies markers in moving scene images. This model is trained based on multiple marker samples and can identify markers in moving scene images, obtaining their location information. The model can identify both normal and abnormal markers. A target marker can be understood as a marker in a moving scene image identified by the marker recognition model; this marker may be normal or abnormal. Marker samples can be understood as samples used to train the marker recognition model. These samples can be obtained by collecting moving scene images of marker-containing samples in the application scenario's work area, captured by an autonomous mobile robot using a vision camera. Normal marker samples are those with normal markers in the sample moving scene images, while abnormal marker samples are those with abnormal markers. Markers in the work area can be placed on the ground or in other locations that can be collected and acquired by vision sensors.

[0049] In practical applications, normal markers constitute the majority of motion scene images collected at work sites, while the number of marker samples corresponding to abnormal markers is relatively small. This results in the model being unable to learn the features of abnormal marker samples effectively, leading to a weak ability to recognize abnormal markers. Therefore, in order to improve the ability of the marker recognition model to recognize abnormal markers, in one or more embodiments of the present invention, before inputting the motion scene image into the marker recognition model to recognize the target marker, the following steps may be included:

[0050] Obtain a first sample set, wherein the first sample set includes multiple normal label samples and multiple abnormal label samples, and the abnormal label samples include label samples generated by a generative adversarial network based on normal label samples;

[0051] The initial identifier recognition model is trained using multiple normal identifier samples and multiple abnormal identifier samples to obtain the trained identifier recognition model.

[0052] Specifically, the first sample set can be understood as a collection of label samples used to train the label recognition model. The first sample set includes multiple normal label samples and multiple abnormal label samples. Furthermore, the number of normal label samples and the number of abnormal label samples can be the same or similar. Since in real-world application scenarios, the number of normal labels in motion scene images is greater than the number of abnormal labels, the number of normal label samples will be much greater than the number of abnormal label samples. This results in a weaker ability of the model to recognize abnormal labels, making it unable to accurately identify abnormal labels and thus unable to obtain their location information. Therefore, in addition to collecting motion scene images containing abnormal labels from the work site, in one or more embodiments of this invention, a generative adversarial network (GAN) is pre-trained to generate abnormal label samples corresponding to normal label samples, thereby expanding the sample data volume of abnormal label samples. Therefore, the multiple abnormal label samples included in the first sample set can be motion scene images containing abnormal samples collected at the work site using a visual sensor, or abnormal label samples generated using a GAN based on normal label samples.

[0053] In practical applications, multiple normal and multiple abnormal label samples can be used as training samples, and the type corresponding to each sample can be used as a label to train the initial label recognition model. This enables the model to classify normal and abnormal samples, resulting in a label recognition model that can identify label types.

[0054] For example, the label of normal labeled samples can be set to 0, and the label of abnormal labeled samples can be set to 1. The model loss value is calculated based on the prediction type output by the initial label recognition model and the label corresponding to the sample. The model parameters are adjusted according to the loss value, and the model after parameter adjustment is trained until the training stopping condition is reached, thus obtaining the trained label recognition model.

[0055] By generating abnormal label samples using generative adversarial networks based on normal label samples, and using multiple normal label samples and multiple abnormal label samples as the first sample set to train the initial label recognition model, the trained label recognition model can be equipped with the ability to identify label types. It can not only identify normal labels but also abnormal labels, thus avoiding the inability to identify target labels due to the robot missing abnormal labels, which would lead to inaccurate path planning.

[0056] Furthermore, since only a small number of the anomaly marker samples in the first sample set are obtained from the work site using a visual camera, and the anomaly marker samples that cannot be obtained through acquisition in real-world scenarios need to be simulated and constructed using neural networks. Therefore, in order to increase the number of anomaly marker samples and expand the first sample set used for training the marker recognition model while ensuring the sample quality and accuracy of the anomaly marker samples, in one or more embodiments of the present invention, obtaining the first sample set may include the following steps:

[0057] Obtain multiple normal label samples;

[0058] Multiple normal labeled samples are input into a pre-trained generative adversarial network to generate multiple abnormal labeled samples corresponding to the multiple normal labeled samples.

[0059] The first sample set is composed of multiple normal label samples and multiple abnormal label samples.

[0060] Specifically, a Generative Adversarial Network (GAN) can be understood as a pre-trained neural network that can output abnormal labeled samples from input normal labeled samples. GANs can process normal labeled samples into abnormal labeled samples by adding noise to them. While ensuring the quality and accuracy of the abnormal labeled samples, GANs obtain a larger number of abnormal labeled samples corresponding to normal labeled samples, thereby expanding the number of abnormal labeled samples in the initial sample set. This allows the label recognition model to learn more abnormal labeled samples, thus improving its label recognition ability and its ability to classify label types.

[0061] In practical applications, due to the limited number of abnormal marker samples that can be collected by visual sensors in real working scenarios, the model cannot learn richer features of abnormal marker samples and has poor recognition ability. As a result, robots often cannot distinguish between motion scene images containing abnormal marker samples and motion scene images without any marker samples. This leads to the robot easily losing abnormal markers during task execution, resulting in inaccurate path planning and navigation, low robot position recognition efficiency, and unsatisfactory task processing effect.

[0062] By using normal label samples collected from real-world scenarios and generating corresponding abnormal label samples using generative adversarial networks, the number of abnormal label samples can be increased while ensuring sample quality and accuracy. This allows each normal label sample to have a corresponding abnormal label sample, enabling the model to learn more abnormal label samples and become more sensitive to their identification. This prevents the robot from losing abnormal labels, thereby improving the efficiency and accuracy of robot position recognition, the accuracy of path planning and navigation, and the overall performance of the robot in handling autonomous movement tasks.

[0063] To improve the accuracy of anomaly label samples generated by generative adversarial networks (GANs), one or more embodiments of the present invention may include the following steps before inputting multiple normal label samples into a pre-trained GAN to generate multiple anomaly label samples corresponding to the multiple normal label samples:

[0064] Obtain multiple training sample pairs, wherein each training sample pair includes a normal labeled sample and a label abnormal labeled sample corresponding to the normal labeled sample;

[0065] Normal labeled samples are input into the initial generative adversarial network to generate predicted abnormal labeled samples;

[0066] Calculate the loss value based on the labeled anomaly samples and the predicted anomaly samples;

[0067] The network parameters of the generative adversarial network are adjusted according to the loss value until the preset training conditions are met, and the trained generative adversarial network is obtained.

[0068] In practical applications, images containing markers in real-world work scenarios can be acquired using visual sensors. Specifically, these images can be motion scene images captured by a robot equipped with a visual sensor at the work site. Historically acquired motion scene images containing normal markers can be stored. When a motion scene image of an abnormal marker corresponding to a normal marker is acquired, the corresponding normal marker sample and abnormal marker sample can be associated to form training sample pairs. Specifically, historically acquired motion scene images of abnormal markers corresponding to normal markers can be stored, resulting in one normal marker sample and multiple abnormal marker samples corresponding to that marker. The normal marker sample and each abnormal marker sample can be associated separately, using the normal marker sample as a training sample and the abnormal marker samples as sample labels, to obtain multiple training sample pairs corresponding to that marker. Images containing markers can also be captured by workers using a visual camera.

[0069] In practical applications, normal labeled samples from the training sample pairs are used as training samples and input into the initial generative adversarial network (GAN). This generates predicted abnormal labeled samples. The labeled abnormal samples corresponding to the normal labeled samples from the training sample pairs are used as labels. The predicted abnormal labeled samples and the labeled abnormal labeled samples are compared. The loss value of the initial GAN ​​is calculated based on the predicted abnormal labeled samples and the labeled abnormal labeled samples. The network parameters are adjusted based on the loss value. The initial GAN ​​is trained repeatedly until the loss value reaches a preset training threshold or the number of training iterations reaches a preset convergence threshold, thus obtaining the trained GAN.

[0070] By training a generative adversarial network (GAN) using training sample pairs consisting of normal labeled samples and their corresponding labeled abnormal labeled samples, a more accurate GAN with better abnormal labeled sample construction can be obtained. This allows for the acquisition of more abnormal labeled samples based on normal labeled samples and the trained GAN, increasing the number of abnormal labeled samples and thus improving the label classification and identification capabilities of the label recognition model.

[0071] In practical applications, it is necessary to train the initial identifier recognition model using multiple normal identifier samples and multiple abnormal identifier samples with labels. This enables the trained identifier recognition model to have the ability to classify identifier types and to better identify abnormal identifiers. Therefore, in one or more embodiments of the present invention, before training the initial identifier recognition model using multiple normal identifier samples and multiple abnormal identifier samples to obtain the trained identifier recognition model, the following steps may be included:

[0072] Obtain a second sample set, wherein the second sample set includes a first preset number of first labeled samples, and the first labeled samples carry corresponding first labeling results;

[0073] Based on the first labeled sample and the first labeling result, an initial identifier recognition model is trained.

[0074] Specifically, the second sample set includes a first preset number of labeled samples and multiple unlabeled samples. Both the labeled and unlabeled samples in the second sample set can be used to train the identifier recognition model. To improve the model's learning ability and identifier recognition performance, in one embodiment, the total number of labeled and unlabeled samples in the collected second sample set can be greater than 100,000. Specifically, the labeled and unlabeled samples can be either motion scene images containing normal identifiers or motion scene images containing abnormal identifiers.

[0075] It should be noted that the first preset number of labeled samples can be obtained by the annotation object pre-annotating the unlabeled samples. Pre-annotating the unlabeled samples to obtain the first preset number of labeled samples can be understood as the fully labeled training phase in model training. The annotation object can be understood as the staff who annotate the unlabeled samples. By annotating the unlabeled samples to obtain the first preset number of labeled samples in the fully labeled training phase, accurately labeled training samples can be obtained, thereby improving the accuracy of model training.

[0076] In practical applications, the first preset number can be five percent of the total number of samples in the second sample set. Since the first labeled samples are all manually labeled, in order to improve the efficiency of model training, the first preset number of the first labeled samples in the fully labeled training stage is often small. For example, when the number of unlabeled samples is greater than 100,000, the first preset number can be no less than 5,000.

[0077] It should be noted that the first labeled sample in the second sample set carries the corresponding first labeling result, which makes it easier to train the initial label recognition model based on the first labeled sample and the first labeling result.

[0078] By training an initial label recognition model based on the first labeled sample and the corresponding first labeling result, the accuracy of the output result of the initial label recognition model can be improved, enabling the initial label recognition model to classify and recognize labels. Furthermore, the initial label recognition model can not only recognize normal labels, but also recognize abnormal labels, thus preventing the robot from losing abnormal labels during autonomous movement tasks.

[0079] In practical applications, after completing the training phase of full annotation, an initial marker recognition model can be obtained. Furthermore, to improve the marker recognition capability of this initial model and simultaneously increase its training efficiency, the following steps can be included after obtaining the initial marker recognition model:

[0080] Extract a second preset number of second unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples;

[0081] The second unlabeled sample is input into the initial label recognition model to obtain the first prediction result;

[0082] Generate a calibration instruction to instruct the labeled object to calibrate the first prediction result, thereby obtaining a second labeling result for the second unlabeled sample;

[0083] The initial identifier recognition model is trained based on the second unlabeled sample and the corresponding second labeled result.

[0084] In practical applications, after obtaining the initial label recognition model, we can enter the semi-automatic annotation stage. This can be understood as eliminating the need for manual annotation of unlabeled samples to train the model. Instead, we only need to calibrate the annotation content based on the prediction results output by the initial label recognition model, and then annotate the unlabeled samples based on the calibration results. This improves the accuracy of the model output results, increases the model training efficiency, and reduces manual costs and annotation time.

[0085] Specifically, a second preset number of second unlabeled samples are extracted from the unlabeled samples in the second sample set, excluding the first labeled samples. Since the first labeled samples have already been accurately labeled by the labeled objects, there is no need to label the first labeled samples again. The second preset number of second unlabeled samples are extracted from the unlabeled samples in the second sample set, and in the semi-automatic labeling stage, the second unlabeled samples are labeled. Based on the labeled second labeling results, the initial label recognition model is further trained.

[0086] In practical applications, the second preset number can be 25% of the sample data of the remaining unlabeled samples. The second preset number of second unlabeled samples can be extracted from the remaining unlabeled samples. The second unlabeled samples can be input into the initial label recognition model to obtain the first prediction result output by the initial label recognition model.

[0087] Specifically, the first prediction result can be understood as the sample type of each second unlabeled sample predicted by the identifier recognition model, and the sample type of the second unlabeled sample includes normal or abnormal.

[0088] In practical applications, based on the first prediction result, a calibration instruction can be generated to instruct the labeled object to calibrate the first prediction result and obtain the second labeling result for the second unlabeled sample.

[0089] Specifically, the calibration instruction is used to calibrate the first prediction result. Upon receiving the calibration instruction, the staff responsible for labeling can check the predicted sample type corresponding to each second unlabeled sample. If the predicted sample type is incorrect, it will be corrected, and the corresponding second unlabeled sample will be labeled based on the correct sample type. If the predicted sample type is correct, it can be directly used as the sample label for the second unlabeled sample. After calibration, the second labeling result for the second unlabeled sample can be obtained.

[0090] Based on the second unlabeled sample and the corresponding second labeled result, the initial label recognition model can be trained, which can further improve the label recognition model's ability to recognize labels, so that the label recognition model can not only recognize normal labels, but also abnormal labels.

[0091] Through a semi-automatic annotation stage, a second preset number of second unannotated samples are extracted from the unannotated samples in the second sample set, excluding the first annotated samples. These second unannotated samples are then input into the initial label recognition model to obtain a first prediction result, which improves the annotation efficiency of the second unannotated samples. According to the calibration instruction, the annotation object is instructed to calibrate the first prediction result to obtain a second annotation result for the second unannotated samples, which improves the accuracy of the second annotation result. Based on the second unannotated samples and the corresponding second annotation result, the initial label recognition model is trained, which improves the accuracy of the model's recognition result and enhances the model's ability to classify and recognize normal and abnormal labels.

[0092] Furthermore, the second sample set also contains remaining unlabeled and unused unlabeled samples. To improve the recognition capability of the identifier recognition model and the accuracy of the model output, according to one or more embodiments of this specification, after training the initial identifier recognition model, the following steps may be included:

[0093] Extract a third preset number of unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples and the second unlabeled samples;

[0094] The third unlabeled sample is input into the initial identifier recognition model to obtain the second prediction result, which includes the identifier recognition confidence score.

[0095] If the confidence level of the identifier recognition is lower than the preset confidence level, a calibration instruction is generated to instruct the labeled object to calibrate the second prediction result and obtain the third labeling result of the third unlabeled sample;

[0096] The initial identifier recognition model is trained based on the third unlabeled sample and the corresponding third labeled result.

[0097] In practical applications, after training the initial labeling model using the second unlabeled samples and the second labeled results, the third stage, namely the fully automatic labeling stage, can be entered. Excluding the first labeled samples and the second unlabeled samples, the third preset number of unlabeled samples extracted from the remaining unlabeled samples in the second sample set each time can be the same or increased progressively. Training the initial labeling recognition model in the third stage with a uniform amount of data can achieve better model training results. Increasing the amount of training data progressively in the cyclical training of the third stage can help reach the model convergence condition earlier, improving the model's training efficiency.

[0098] Specifically, the third preset quantity can be the remainder of the remaining unlabeled sample data and the preset number of training sessions in the third stage, or it can be determined that the third preset quantity used in each training session is 10% higher than the third preset quantity used in the previous training session.

[0099] In practical applications, a third unlabeled sample can be input into the initial label recognition model to obtain a second prediction result output by the initial label recognition model. Specifically, the second prediction result can include the label recognition confidence level, which can be understood as a preset accuracy threshold that allows for label errors. If the label recognition confidence level is lower than the preset confidence level, i.e., lower than the preset accuracy threshold, it indicates that the model's recognition result is not accurate enough, and the labels need to be recalibrated to improve the accuracy of model training. In the third stage of training, the labeled object does not need to calibrate the prediction results of each third unlabeled sample in the second prediction result. It only needs to check the accuracy of the output result of the initial label recognition model by sampling, and quantify the accuracy by calculating the confidence level. For example, in this training process, 20 prediction results of the third unlabeled sample are sampled from the third preset number. If 2 of the prediction results are inaccurate, the confidence level of this label recognition is 90%, while the preset confidence level is 95%. This indicates that after this training, the label recognition accuracy of the model is low. It is necessary to recalibrate the sample labels of the third unlabeled sample and retrain the model to improve the model accuracy.

[0100] By extracting a third preset number of unlabeled samples from the unlabeled samples in the second sample set (excluding the first labeled samples and the second unlabeled samples) to train the initial label recognition model, the labeling cost of manual labeling can be reduced, the sample labeling efficiency can be improved, and the model training efficiency can be accelerated. At the same time, when the label recognition confidence is lower than the preset confidence, a calibration instruction is generated to instruct the labeled object to calibrate the second prediction result and obtain the third labeling result of the third unlabeled samples. When the model accuracy is detected to be low, the labeling result of the third unlabeled samples can be calibrated in a timely manner, and the model can be trained based on the calibrated third labeling result and the third unlabeled samples, thereby further improving the label recognition accuracy of the label recognition model.

[0101] In another embodiment of the present invention, the confidence level of the identifier recognition included in the second prediction result may also reach a preset confidence level. Accordingly, after inputting the third unlabeled sample into the initial identifier recognition model and obtaining the second prediction result, the following steps may also be included:

[0102] If the confidence level of the identifier recognition reaches the preset confidence level, return to the step of extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set other than the first labeled samples and the second unlabeled samples, until all unlabeled samples in the second sample set have been traversed.

[0103] Specifically, when the label recognition confidence level reaches a preset confidence level, it can be understood that the accuracy of the label types predicted by the initial label recognition model meets the standard, and the moving scene images can be accurately classified and recognized based on this initial label recognition model. In this case, the model can be further trained based on the model output and the third unlabeled sample. After training is completed, the process returns to extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled sample and the second unlabeled sample, until all unlabeled samples in the second sample set have been traversed.

[0104] Alternatively, the process can directly return to the step of extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples and the second unlabeled samples, until all unlabeled samples in the second sample set have been traversed. Then, based on all the third unlabeled samples obtained from the third-stage annotation and the third annotation results corresponding to the third unlabeled samples, the initial label recognition model is trained to obtain a fully trained label recognition model.

[0105] When the confidence level of the identifier recognition reaches a preset confidence level, the process returns to the step of extracting a third preset number of unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples and the second unlabeled samples, until all unlabeled samples in the second sample set have been traversed. All remaining unlabeled samples in the second sample set, excluding the first labeled samples and the second unlabeled samples, can be labeled. The initial identifier recognition model is then trained based on the labeling results, increasing the amount of data in the training dataset of the identifier recognition model. This improves the identifier recognition model's ability to classify and recognize identifiers, enabling it to recognize not only normal identifiers but also abnormal identifiers, preventing the robot from losing abnormal identifiers, and improving the accuracy of path planning and navigation.

[0106] In practical applications, before an autonomous mobile robot moves to the designated location in a work environment, it can acquire images of the moving scene using its onboard vision sensors. These images are then input into a trained marker recognition model, which identifies the markers within the moving scene images. The marker recognition model can identify the type of marker present in the moving scene image, and also determine the marker's location within the visual image.

[0107] In one or more embodiments of the present invention, inputting a motion scene image into an identifier recognition model to identify a target identifier may include the following steps:

[0108] Input the moving scene image into the identifier recognition model to obtain the type recognition result for the target identifier;

[0109] If the type identification result is abnormal, obtain the target identifier's region information;

[0110] In practical applications, when a moving scene image is input into the label recognition model, and the moving scene image includes a target label, the label recognition model can identify the label type of the target label and obtain the type recognition result for the target label.

[0111] In one implementation, regardless of whether the type recognition result is normal or abnormal, the identifier recognition model can obtain the regional information of the target identifier and determine the target location information of the target identifier based on the regional information of the target identifier.

[0112] It should be noted that, in addition to a classification module that can identify the type of identifier, the identifier recognition model may also include a region detection module. By analyzing and detecting pixels in the visual image, the region information of the target identifier can be obtained. During the process of the identifier recognition model detecting the type of the target identifier, the region detection module can simultaneously identify the region corresponding to the pixel of the target identifier. The region detection module can also acquire and detect region information before or after identifying the type of the target identifier, depending on actual needs. This invention does not impose any limitations on this.

[0113] Specifically, region information can be understood as the region where the target identifier is located in the moving scene image. Region information can be obtained based on the pixel information of the target identifier. Region information can be the approximate location of the target identifier or the location of the target identifier's border.

[0114] Accordingly, obtaining the target location information of the target identifier may include the following steps:

[0115] Based on the regional information, determine the target location information of the target identifier.

[0116] In practical applications, when regional information is available, the target location information of the target identifier can be determined based on the regional information. The target location information can be understood as the location of the center of the target identifier.

[0117] In another implementation, the identifier recognition model can identify and detect the region of the target identifier in advance, but only obtain the region information of the target identifier if the type recognition result of the target identifier is abnormal.

[0118] Optionally, the identifier recognition model can also identify and detect the region of the target identifier through the region detection module only when the type recognition result of the target identifier is abnormal, thereby obtaining the region information of the target identifier.

[0119] It should be noted that when the type identification result of the target identifier is abnormal, the region information of the abnormal identifier in the motion scene image can be obtained, and the center position of the current identifier region of the abnormal identifier can be determined as the target position information; alternatively, according to the deep learning algorithm, the current identifier region of the abnormal identifier can be restored to the target identifier region of the normal identifier corresponding to the abnormal identifier in the motion scene image, and the center position of the target identifier region can be obtained and determined as the target position information.

[0120] According to one or more embodiments of the present invention, after inputting a motion scene image into an identifier recognition model and obtaining a type recognition result for the target identifier, the following steps may be further included:

[0121] If the type recognition result is normal, obtain the pose information of the target identifier;

[0122] Specifically, when the type recognition result is normal, more accurate pose information of the marker can be obtained through visual images. Based on the pose information, the robot's movement direction and rotation angle can be determined, so that the robot's running posture can better conform to the preset shape of the docking position. For example, the docking position may be between two shelves. During the robot's movement towards the marker's location, it is possible that because the movement posture is not parallel to the two shelves, one side of the robot may collide with one of the shelves, preventing the robot from moving to the docking position in one go.

[0123] Therefore, when the target marker is normal, the robot's running direction and rotation angle relative to the marker can be adjusted based on the target marker's pose information. This avoids the robot being unable to move to the target position in one go due to non-standard running posture, thereby improving the robot's task execution efficiency.

[0124] Accordingly, obtaining the target location information of the target identifier may include the following steps:

[0125] Based on the pose information, obtain the corner positions of each corner of the target identifier and the rotation angle of the target identifier relative to the robot;

[0126] The target location information of the target identifier is determined based on the corner point position and rotation angle.

[0127] Specifically, pose information can include the corner positions of each corner point of the target identifier in the motion scene image and the rotation angle of the target identifier relative to the robot. Based on the pose information, the corner positions of the target identifier at each angle are regressed to obtain the corner positions of each corner point of the target identifier in the real scene and the rotation angle of the target identifier relative to the robot. Based on the corner positions and rotation angles, the robot's motion direction can be adjusted, and the target position information of the target identifier can be determined. The target position information can include the location of each corner point of the target identifier in the real scene.

[0128] It should be noted that the label recognition model may also include a keypoint regression module. Based on the keypoint regression module, the corner points of the target label in the motion scene image can be regressed to obtain the position of the corner points. The keypoint regression module can process the feature information extracted from the motion scene image synchronously with the aforementioned region detection module and classification module, or it can perform processing under preset conditions. For example, the preset conditions can be that when the type recognition result is normal, the keypoint regression module regresses each corner point of the target label, and obtains the rotation angle of the label relative to the robot based on the precise corner point position information; or, when the type recognition result is abnormal, the keypoint regression module regresses each corner point of the target label. Since abnormal labels usually lack at least one corner point, the approximate center position of the abnormal label can be predicted through the regressed corner point position, and the target position information of the abnormal label can be determined based on the predicted center position.

[0129] By acquiring the pose information of the target identifier when the type recognition result is normal, and obtaining the corner positions of each corner of the target identifier and the rotation angle of the target identifier relative to the robot based on the pose information, the target position information of the target identifier can be determined based on the corner positions and rotation angles. When the identifier is normal, the corner positions can be regressed based on the pose information to obtain the precise corner positions of each corner and the rotation angle of the target identifier relative to the robot. This allows for adjustment of the robot's running direction, improving the robot's running efficiency and the accuracy of path planning.

[0130] S106: Obtain the target location information of the target identifier.

[0131] In practical applications, by inputting images of moving scenes into a sign recognition model, the target sign can be identified, and the target location information of the sign can be obtained.

[0132] Specifically, the target location information can be the center position of the target identifier in the target area of ​​the moving scene image; or it can be the approximate position outlined by the region border corresponding to the target area of ​​the moving scene image; for abnormal target identifiers, the target location information can also be the center position in the restored region after the target area of ​​the moving scene image is restored, or the approximate position outlined by the region border corresponding to the restored region.

[0133] Optionally, in another embodiment of this specification, the target position information may differ depending on the target identifier type identification result. If the target identifier type identification result indicates that the target identifier is a normal identifier, the target position information may be the precise corner positions of each corner point corresponding to the target identifier, as well as the rotation angle of the target identifier relative to the robot. If the target identifier type identification result indicates that the target identifier is an abnormal identifier, the target position information may be the current center position of the target identifier or the center position of the restored normal identifier, or the approximate area enclosed by the current border of the target identifier, or the approximate area enclosed by the border of the restored normal identifier.

[0134] S108: Based on the target location information, move to the target location where the target identifier is located.

[0135] In practical applications, based on the target location information, the robot can move to the target location where the target marker is located.

[0136] Specifically, the target location can be understood as the location of the target identifier in the real scene.

[0137] To improve the accuracy of target location positioning, thereby enhancing the accuracy and efficiency of robot path navigation and planning, in one or more embodiments of this specification, moving to the target location where the target identifier is located based on target location information may include the following steps:

[0138] Acquire the relative position information of the vision sensor with respect to the robot;

[0139] Based on the target location information and relative location information, determine the target position of the target identifier in the target view space;

[0140] Based on the target location, plan the target path to move to the target location;

[0141] Move to the target location according to the target path.

[0142] In practical applications, since motion scene images are visual images acquired by vision sensors and do not have depth features, the relative position information of the vision sensor relative to the robot can be obtained. Based on the relative position information, the depth value of the target position information in the BEV space can be predicted, thereby determining the target position of the target identifier in the target view space.

[0143] Based on this, the robot's perspective can be switched to the target BEV perspective corresponding to the target position. According to the target position and orientation information under the target BEV perspective, a target path to the target position can be planned. The orientation information can be the direction in which the robot moves towards the center of the target marker, or it can be the robot's movement direction calculated based on the rotation angle of the target marker relative to the robot.

[0144] In practical applications, the target path obtained from the planning can also be the path information obtained after obstacle avoidance planning based on obstacle information in the path from the target BEV's perspective. Based on the target path, movement can proceed to the target location in the real-world scenario.

[0145] This invention provides a robot motion control method applied to a robot equipped with a vision sensor to acquire motion scene images collected by the vision sensor; inputting the motion scene images into an identifier recognition model to identify target identifiers, wherein the identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples, and the abnormal identifier samples include identifier samples generated by using a generative adversarial network based on normal identifier samples; obtaining the target position information of the target identifier; and moving to the target position of the target identifier based on the target position information.

[0146] By applying the scheme of this invention, a motion scene image is acquired by a visual sensor, and a target identifier in the motion scene image is identified based on an identifier recognition model. This allows the robot to obtain the target location information of the identifier and move to that location. By generating abnormal identifier samples using a generative adversarial network based on normal identifier samples, the number of abnormal identifier samples is increased, improving the identifier recognition model's ability to identify abnormal identifiers. Training the identifier recognition model with both normal and abnormal identifier samples enables it to identify target identifiers in the motion scene image. Regardless of whether the target identifier is normal or abnormal, the robot can obtain the target location information, allowing it to move to the target location based on that information. This avoids the robot being unable to locate the target position due to inaccurate identification of abnormal identifiers, thus improving the accuracy and efficiency of robot path planning and navigation.

[0147] The following is in conjunction with the appendix Figure 2Taking the application of the robot motion control method provided in this application in an autonomous mobile robot performing autonomous mobile tasks as an example, the robot motion control method will be further explained. Figure 2 This illustration shows a schematic diagram of the processing flow of a robot motion control method for performing autonomous mobile tasks in an embodiment of this application.

[0148] Images of the moving scene are acquired using an image acquisition device mounted on the body of the autonomous mobile robot. This image acquisition device may include a forward-facing vision sensor.

[0149] Based on neural networks, image data preprocessing is performed on visual data in moving scene images. This preprocessing includes steps such as data cleaning, data integration, data reduction, and data transformation. Through preprocessing, higher-quality feature data can be selected, improving the efficiency and accuracy of subsequent feature extraction and processing.

[0150] Image features are extracted via a network, and the extracted visual image features are input into a pre-trained label recognition model. This model includes a region detection module, a classification module, and a keypoint regression module. The region detection module detects the regions of target labels in the moving scene image; the classification module determines the type of target label; and the keypoint regression module regresses the corner points of target labels in the moving scene image.

[0151] If the type recognition result is normal, the identification area detected by the region detection module and the key point regression module and the corner position obtained by regression can be obtained. The position information and posture information of the target identification can be calculated, and the rotation angle of the target identification relative to the autonomous mobile robot can be obtained based on the position information and posture information.

[0152] In the case of an abnormal type recognition result, the approximate identification area and approximate corner point position detected by the region detection module and the key point regression module can be obtained, the position information of the target identification can be calculated, and the approximate center position of the target identification can be obtained based on the position information.

[0153] Based on the approximate center position, or the precise corner position, and the rotation angle of the target marker relative to the autonomous mobile robot, the current position and orientation of the autonomous mobile robot can be determined. The relative position information of the visual sensor relative to the autonomous mobile robot can be obtained, converted to the BEV perspective, and path planning and navigation can be performed based on the position and orientation of the BEV perspective.

[0154] This invention provides a robot motion control method applied to a robot equipped with a vision sensor to acquire motion scene images collected by the vision sensor; inputting the motion scene images into an identifier recognition model to identify target identifiers, wherein the identifier recognition model is trained based on multiple identifier samples, including normal identifier samples and abnormal identifier samples, and the abnormal identifier samples include identifier samples generated by using a generative adversarial network based on normal identifier samples; obtaining the target position information of the target identifier; and moving to the target position of the target identifier based on the target position information.

[0155] By applying the scheme of this invention, a motion scene image is acquired by a visual sensor, and a target identifier in the motion scene image is identified based on an identifier recognition model. This allows the robot to obtain the target location information of the identifier and move to that location. By generating abnormal identifier samples using a generative adversarial network based on normal identifier samples, the number of abnormal identifier samples is increased, improving the identifier recognition model's ability to identify abnormal identifiers. Training the identifier recognition model with both normal and abnormal identifier samples enables it to identify target identifiers in the motion scene image. Regardless of whether the target identifier is normal or abnormal, the robot can obtain the target location information, allowing it to move to the target location based on that information. This avoids the robot being unable to locate the target position due to inaccurate identification of abnormal identifiers, thus improving the accuracy and efficiency of robot path planning and navigation.

[0156] Corresponding to the above method embodiments, this invention also provides a robot motion control device, such as... Figure 3 As shown, Figure 3 This diagram illustrates a structural schematic of a robot motion control device according to an embodiment of the present invention. The device is configured on a robot equipped with a vision sensor and includes:

[0157] The first acquisition module 302 is configured to acquire motion scene images collected by a visual sensor;

[0158] The recognition module 304 is configured to input a motion scene image into a label recognition model to recognize a target label. The label recognition model is trained based on multiple label samples, including normal label samples and abnormal label samples. Abnormal label samples include label samples generated by a generative adversarial network based on normal label samples.

[0159] The second acquisition module 306 is configured to acquire the target location information of the target identifier;

[0160] The motion module 308 is configured to move to the target location where the target identifier is located based on the target location information.

[0161] Optionally, the identification module 304 is further configured as follows:

[0162] Obtain a first sample set, wherein the first sample set includes multiple normal label samples and multiple abnormal label samples, and the abnormal label samples include label samples generated by a generative adversarial network based on normal label samples;

[0163] The initial identifier recognition model is trained using multiple normal identifier samples and multiple abnormal identifier samples to obtain the trained identifier recognition model.

[0164] Optionally, the identification module 304 is further configured as follows:

[0165] Obtain multiple normal label samples;

[0166] Multiple normal labeled samples are input into a pre-trained generative adversarial network to generate multiple abnormal labeled samples corresponding to the multiple normal labeled samples.

[0167] The first sample set is composed of multiple normal label samples and multiple abnormal label samples.

[0168] Optionally, the identification module 304 is further configured as follows:

[0169] Obtain multiple training sample pairs, wherein each training sample pair includes a normal labeled sample and a label abnormal labeled sample corresponding to the normal labeled sample;

[0170] Normal labeled samples are input into the initial generative adversarial network to generate predicted abnormal labeled samples;

[0171] Calculate the loss value based on the labeled anomaly samples and the predicted anomaly samples;

[0172] The network parameters of the generative adversarial network are adjusted according to the loss value until the preset training conditions are met, and the trained generative adversarial network is obtained.

[0173] Optionally, the identification module 304 is further configured as follows:

[0174] Obtain a second sample set, which includes multiple unlabeled samples;

[0175] Generate annotation instructions to instruct the annotation object to annotate the first preset number of first annotated samples in the second sample set;

[0176] Obtain the first annotation result of the first labeled sample;

[0177] Based on the first labeled sample and the corresponding first labeling result, an initial label recognition model is trained.

[0178] Optionally, the identification module 304 is further configured as follows:

[0179] Extract a second preset number of second unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples;

[0180] The second unlabeled sample is input into the initial label recognition model to obtain the first prediction result;

[0181] Generate a calibration instruction to instruct the labeled object to calibrate the first prediction result, thereby obtaining a second labeling result for the second unlabeled sample;

[0182] The initial identifier recognition model is trained based on the second unlabeled sample and the corresponding second labeled result.

[0183] Optionally, the identification module 304 is further configured as follows:

[0184] Extract a third preset number of unlabeled samples from the unlabeled samples in the second sample set, excluding the first labeled samples and the second unlabeled samples;

[0185] The third unlabeled sample is input into the initial identifier recognition model to obtain the second prediction result, which includes the identifier recognition confidence score.

[0186] If the confidence level of the identifier recognition is lower than the preset confidence level, a calibration instruction is generated to instruct the labeled object to calibrate the second prediction result and obtain the third labeling result of the third unlabeled sample;

[0187] The initial identifier recognition model is trained based on the third unlabeled sample and the corresponding third labeled result.

[0188] Optionally, the identification module 304 is further configured as follows:

[0189] If the confidence level of the identifier recognition reaches the preset confidence level, return to the step of extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set other than the first labeled samples and the second unlabeled samples, until all unlabeled samples in the second sample set have been traversed.

[0190] Optionally, the identification module 304 is further configured as follows:

[0191] Input the moving scene image into the identifier recognition model to obtain the type recognition result for the target identifier;

[0192] If the type identification result is abnormal, obtain the target identifier's region information;

[0193] Obtain the target location information of the target identifier, including:

[0194] Based on the regional information, determine the target location information of the target identifier.

[0195] Optionally, the identification module 304 is further configured as follows:

[0196] If the type recognition result is normal, obtain the pose information of the target identifier;

[0197] Obtain the target location information of the target identifier, including:

[0198] Based on the pose information, obtain the corner positions of each corner of the target identifier and the rotation angle of the target identifier relative to the robot;

[0199] The target location information of the target identifier is determined based on the corner point position and rotation angle.

[0200] Optionally, motion module 308 is further configured as follows:

[0201] Acquire the relative position information of the vision sensor with respect to the robot;

[0202] Based on the target location information and relative location information, determine the target position of the target identifier in the target view space;

[0203] Based on the target location, plan the target path to move to the target location;

[0204] Move to the target location according to the target path.

[0205] By applying the embodiments of this invention, by acquiring motion scene images collected by a visual sensor, and identifying target icons in the motion scene images based on an icon recognition model, the target location information of the icon can be obtained, and the robot can move to the target location where the icon is located. By generating abnormal icon samples using a generative adversarial network based on normal icon samples, the number of abnormal icon samples can be increased, improving the icon recognition model's ability to identify abnormal icons. By training the icon recognition model with normal and abnormal icon samples, the model can identify target icons in the motion scene images. Regardless of whether the target icon is normal or abnormal, the target location information of the icon can be obtained, enabling the robot to move to the target location based on the target location information. This avoids the inability to accurately identify the target icon when the icon is abnormal, thus preventing the robot from failing to locate the target position and improving the accuracy and efficiency of robot path planning and navigation.

[0206] The above is a schematic scheme of a robot motion control device according to this embodiment. It should be noted that the technical solution of this robot motion control device and the technical solution of the robot motion control device method described above belong to the same concept. For details not described in detail in the technical solution of the robot motion control device, please refer to the description of the technical solution of the robot motion control device method described above.

[0207] Figure 4 A structural block diagram of a computing device 400 according to an embodiment of the present invention is shown. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and a database 450 is used to store data.

[0208] The computing device 400 also includes an access device 440, which enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 440 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Controller (NIC)), such as an IEEE 802.12 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0209] In one embodiment of the present invention, the above-mentioned components of the computing device 400 and Figure 4 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 4 The illustrated block diagram of the computing device is for illustrative purposes only and is not intended to limit the scope of the invention. Those skilled in the art can add or replace other components as needed.

[0210] The computing device 400 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 400 can also be a mobile or stationary server.

[0211] The processor 420 is used to execute computer-executable instructions corresponding to the technical solutions described above.

[0212] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method.

[0213] An embodiment of the present invention also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are used to implement the steps of the above-described method.

[0214] The foregoing has described specific embodiments of the invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0215] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0216] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the invention are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0217] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0218] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of the present invention. These embodiments have been selected and specifically described to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A robot motion control method characterized by, The method is applied to a robot provided with a visual sensor, and comprises the following steps: acquiring a moving scene image collected by the visual sensor; inputting the moving scene image into an identification recognition model to identify a target identification, wherein the identification recognition model is trained based on a plurality of identification samples, the identification samples include normal identification samples and abnormal identification samples, the abnormal identification samples include identification samples generated based on the normal identification samples by using a generative adversarial network; a training process of the generative adversarial network comprises the following steps: acquiring a plurality of training sample pairs, wherein the training sample pairs include normal identification samples and label abnormal identification samples corresponding to the normal identification samples; inputting the normal identification samples into an initial generative adversarial network to generate predicted abnormal identification samples; calculating a loss value according to the label abnormal identification samples and the predicted abnormal identification samples; adjusting network parameters of the generative adversarial network according to the loss value until a preset training condition is reached to obtain a trained generative adversarial network; obtaining target position information of the target identification; moving to a target position where the target identification is located based on the target position information.

2. The method of claim 1, wherein, Before the step of inputting the moving scene image into the identification recognition model to identify the target identification, the method further comprises the following steps: acquiring a first sample set, wherein the first sample set includes a plurality of normal identification samples and a plurality of abnormal identification samples, the abnormal identification samples include identification samples generated based on the normal identification samples by using a generative adversarial network; training an initial identification recognition model by using the plurality of normal identification samples and the plurality of abnormal identification samples to obtain a trained identification recognition model.

3. The method of claim 2, wherein, The step of acquiring the first sample set comprises the following steps: acquiring a plurality of normal identification samples; inputting the plurality of normal identification samples into a pre-trained generative adversarial network to generate a plurality of abnormal identification samples corresponding to the plurality of normal identification samples; composing a first sample set according to the plurality of normal identification samples and the plurality of abnormal identification samples.

4. The method of claim 2, wherein, Before the step of training the initial identification recognition model by using the plurality of normal identification samples and the plurality of abnormal identification samples to obtain the trained identification recognition model, the method further comprises the following steps: acquiring a second sample set, wherein the second sample set includes a first preset number of first labeled samples, and the first labeled samples carry corresponding first labeling results; training an initial identification recognition model according to the first labeled samples and the first labeling results.

5. The method of claim 4, wherein, After the step of training the initial identification recognition model, the method further comprises the following steps: extracting a second preset number of second unlabeled samples from unlabeled samples other than the first labeled samples in the second sample set; inputting the second unlabeled samples into the initial identification recognition model to obtain first prediction results; generating a calibration instruction to instruct a labeling object that labels the unlabeled samples to calibrate the first prediction results to obtain second labeling results of the second unlabeled samples; training the initial identification recognition model according to the second unlabeled samples and the corresponding second labeling results.

6. The method of claim 5, wherein, After the initial identification recognition model is trained, the method further includes: extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set except the first labeled sample and the second unlabeled sample; inputting the third unlabeled samples into the initial identification recognition model to obtain a second prediction result, wherein the second prediction result includes an identification recognition confidence; in a case where the identification recognition confidence is lower than a preset confidence, generating a calibration instruction to instruct the labeling object to calibrate the second prediction result to obtain a third labeling result of the third unlabeled sample; training the initial identification recognition model according to the third unlabeled sample and the corresponding third labeling result.

7. The method of claim 6, wherein, After the third unlabeled samples are inputted into the initial identification recognition model to obtain a second prediction result, the method further includes: in a case where the identification recognition confidence reaches the preset confidence, returning to execute the step of extracting a third preset number of third unlabeled samples from the unlabeled samples in the second sample set except the first labeled sample and the second unlabeled sample until all the unlabeled samples in the second sample set are traversed.

8. The method of claim 1, wherein, The step of inputting the motion scene image into the identification recognition model to identify a target identification includes: inputting the motion scene image into the identification recognition model to obtain a type identification result for the target identification; in a case where the type identification result is abnormal, obtaining region information of the target identification; The step of obtaining target position information of the target identification includes: determining the target position information of the target identification according to the region information.

9. The method of claim 8, wherein, After the motion scene image is inputted into the identification recognition model to obtain a type identification result for the target identification, the method further includes: in a case where the type identification result is normal, obtaining pose information of the target identification; The step of obtaining target position information of the target identification includes: obtaining corner point positions of each corner point of the target identification and a rotation angle of the target identification relative to the robot according to the pose information; determining the target position information of the target identification according to the corner point positions and the rotation angle.

10. The method of claim 1, wherein, The step of moving to a target position where the target identification is located based on the target position information includes: obtaining relative position information of the visual sensor relative to the robot; determining a target position of the target identification in a target visual angle space according to the target position information and the relative position information; planning a target path for moving to the target position based on the target position; moving to the target position according to the target path.

11. A robot motion control apparatus characterized by comprising: The device is configured in a robot, and the robot is provided with a visual sensor, and the device includes: a first obtaining module configured to obtain a motion scene image collected by the visual sensor; The identification module is configured to input the motion scene image into an identification recognition model to identify a target identification, wherein the identification recognition model is trained based on a plurality of identification samples, the identification samples include normal identification samples and abnormal identification samples, the abnormal identification samples include identification samples generated based on the normal identification samples by using a generative adversarial network, and a training process of the generative adversarial network includes: obtaining a plurality of training sample pairs, wherein the training sample pairs include normal identification samples and label abnormal identification samples corresponding to the normal identification samples; inputting the normal identification samples into an initial generative adversarial network to generate predicted abnormal identification samples; calculating a loss value according to the label abnormal identification samples and the predicted abnormal identification samples; and adjusting network parameters of the generative adversarial network according to the loss value until a preset training condition is reached to obtain a trained generative adversarial network. The second acquisition module is configured to obtain target position information of the target identification. The motion module is configured to move to a target position where the target identification is located based on the target position information.

12. A computing device, comprising: Comprise: A memory and a processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, realize the method in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions, when executed by the processor, realize the method in any one of claims 1-10.

Images