Transparent object marking method for embodied robot, embodied robot and control device

Abstract

The present application relates to a transparent object marking method of an embodied robot, an embodied robot and a control device. The method includes: obtaining a current data point set collected by a sensor from the surrounding environment in real time during the movement of the embodied robot in a target scene; when a suspected transparent data point is detected in the current data point set, marking a transparent feature for the suspected transparent data point to obtain a first data point set; predicting the transparency probability corresponding to the suspected transparent data point through a target probability prediction model; the target probability prediction model is obtained by training a sample data point set, and the data points in the sample data point set that meet the straight line property are marked with a straight line feature; eliminating the transparent features of the suspected transparent data points in the first data point set whose transparency probability does not reach the probability threshold to obtain a second data point set; and using the second data point set to update the scene map of the embodied robot. The use of this method can improve the accuracy of transparent object marking.

Description

Technical Field

[0001] The present application relates to the technical field of embodied robots, and in particular to a transparent object marking method of an embodied robot, an embodied robot and a control device. Background Art

[0002] At present, when constructing a 2D (two-dimensional) map, a self-moving embodied robot can record information such as walls or obstacles in the environment in the map to facilitate navigation and obstacle avoidance. For example, the embodied robot can detect obstacle information based on sensors and mark the obstacle on the map based on the obstacle information.

[0003] In traditional technology, since there may be transparent objects in the environment, such as glass walls, and transparent objects have the characteristics of high transmittance and low reflectivity, they are easily ignored or misjudged as noise during detection, affecting the accuracy of transparent object marking in the map. Therefore, filtering technology can be used to reduce noise to improve the accuracy of marking transparent objects.

[0004] However, the use of filtering technology to reduce noise does not significantly improve the accuracy of marking transparent objects, so the accuracy of transparent object marking needs to be further improved. Summary of the invention

[0005] Based on this, it is necessary to provide a transparent object marking method for an embodied robot, an embodied robot, a control device, a computer-readable storage medium and a computer program product that can improve the accuracy of transparent object marking in order to address the above technical problems.

[0006] On the one hand, the present application provides a transparent object marking method for an embodied robot, comprising: while the embodied robot moves in a target scene, obtaining a current data point set collected by a sensor from the surrounding environment in real time; when it is determined through a preset detection algorithm that there are suspected transparent data points in the current data point set, marking transparent features for each suspected transparent data point in the current data point set to obtain a first data point set; inputting the first data point set into a target probability prediction model to predict the transparency probability corresponding to each suspected transparent data point in the first data point set, the transparency probability representing the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability The rate prediction model is trained using a set of sample data points, wherein transparent data points in the sample data point set are marked with transparent features, transparent data points in the sample data point set that conform to the properties of a straight line are marked with straight line features, non-transparent data points in the sample data point set are not marked with transparent features, the transparent data points are data points of transparent objects, and the non-transparent data points are data points of non-transparent objects; transparent features of suspected transparent data points whose transparency probability does not reach a probability threshold in the first data point set are eliminated, and transparent features of suspected transparent data points whose transparency probability reaches a probability threshold are retained to obtain a second data point set; and the second data point set is used to update the scene map of the embodied robot.

[0007] On the other hand, the present application also provides an embodied robot, which includes: a data acquisition module, which is used to acquire a current set of data points collected by sensors from the surrounding environment in real time while the embodied robot moves in a target scene; a feature marking module, which is used to mark transparent features for each suspected transparent data point in the current data point set when it is determined through a preset detection algorithm that there are suspected transparent data points in the current data point set, so as to obtain a first data point set; a probability prediction module, which is used to input the first data point set into a target probability prediction model to predict the transparency probability corresponding to each suspected transparent data point in the first data point set, wherein the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the The target probability prediction model is obtained by training using a set of sample data points, wherein transparent data points in the sample data point set are marked with transparent features, transparent data points in the sample data point set that conform to the properties of a straight line are marked with straight line features, and non-transparent data points in the sample data point set are not marked with transparent features, the transparent data points are data points of transparent objects, and the non-transparent data points are data points of non-transparent objects; a feature processing module is used to eliminate transparent features of suspected transparent data points whose transparency probability does not reach a probability threshold in the first data point set, and retain the transparent features of suspected transparent data points whose transparency probability reaches the probability threshold to obtain a second data point set; a map updating module is used to use the second data point set to update the scene map of the embodied robot.

[0008] On the other hand, the present application also provides a control device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the transparent object marking method of the embodied robot when executing the computer program.

[0009] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the transparent object marking method of the embodied robot are implemented.

[0010] On the other hand, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps in the transparent object marking method of the embodied robot.

[0011] The transparent object marking method of the embodied robot, the embodied robot, the control device, the computer-readable storage medium and the computer program product mentioned above, since the target probability prediction model is obtained by training using a sample data point set, and the transparent data points in the sample data point set are marked with transparent features, the transparent data points in the sample data point set that conform to the properties of a straight line are marked with straight line features, and the non-transparent data points in the sample data point set are not marked with transparent features, so that the sample data point set contains transparent features and straight line features, and different types of data points can be distinguished by transparent features and straight line features, so that the target probability prediction model obtained by training the sample data point set can learn transparent features and straight line features, which helps the target probability prediction model to more accurately distinguish data points of transparent objects from data points of non-transparent objects. When it is determined through a preset detection algorithm that there are suspected transparent data points, the suspected transparent data points are not directly marked in the scene map. Instead, after obtaining the first data point set, the target probability prediction model is used to predict the transparency probability of the suspected transparent data points in the first data point set. When the transparency probability does not reach the probability threshold, it means that the suspected transparent data point is very likely not a transparent data point, and the credibility of the suspected transparent data point whose transparency probability reaches the probability threshold is a transparent data point is higher. Therefore, the transparent features of the suspected transparent data points in the first data point set whose transparency probability does not reach the probability threshold are eliminated, and the transparent features of the suspected transparent data points whose transparency probability reaches the probability threshold are retained to obtain a second data point set. The second data point set is used to update the scene map of the embodied robot, so that transparent objects can be accurately marked in the scene map, thereby improving the marking accuracy of transparent objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 A diagram of an application environment of a transparent object marking method of an embodied robot in some embodiments;

[0014] Figure 2 is a schematic flow chart of a transparent object marking method of an embodied robot in some embodiments;

[0015] Figure 3 A schematic diagram of a process for training a probability prediction model in some embodiments;

[0016] Figure 4is a schematic flow chart of a transparent object marking method of an embodied robot in some other embodiments;

[0017] Figure 5 is a block diagram of modules included in an embodied robot in some embodiments;

[0018] Figure 6 1 is a diagram of the internal structure of a control device in some embodiments. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] The transparent object marking method of the embodied robot provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The application environment includes the embodied robot 102 and the surrounding environment of the embodied robot 102. The embodied robot in the present application is a robot system that has perception, movement and interaction capabilities, and can interact with the environment in real time, and can capture information about the surrounding environment through sensory organs such as cameras, lidars and tactile sensors. The structure of the embodied robot in the present application is not limited to the structure presented by the embodied robot 102. The embodied robot in the present application may be, but is not limited to, a sweeping robot, a mopping robot, a sweeping and mopping robot, a robotic arm and a humanoid robot in the field of household services. There is a control device in the embodied robot, and the transparent object marking method of the embodied robot provided in the present application may be executed by the control device.

[0021] Specifically, during the movement of the embodied robot in the target scene, the control device can obtain the current data point set collected by the sensor from the surrounding environment in real time, and when it is determined through a preset detection algorithm that there are suspected transparent data points in the current data point set, each suspected transparent data point in the current data point set is marked with a transparent feature to obtain a first data point set. The control device can input the first data point set into the target probability prediction model to predict the transparency probability corresponding to each suspected transparent data point in the first data point set, and the transparent probability represents the probability that the suspected transparent data point belongs to a transparent object. Among them, the target probability prediction model is obtained by training with a sample data point set, the transparent data points in the sample data point set are marked with a transparent feature, the transparent data points in the sample data point set that meet the straight line property are marked with a straight line feature, the non-transparent data points in the sample data point set are not marked with a transparent feature, the transparent data points are data points of transparent objects, and the non-transparent data points are data points of non-transparent objects. The control device can eliminate the transparent features of the suspected transparent data points whose transparent probability does not reach the probability threshold in the first data point set, and retain the transparent features of the suspected transparent data points whose transparent probability reaches the probability threshold, to obtain a second data point set. The controller may update the scene map of the embodied robot using the second set of data points.

[0022] In some embodiments, Figure 2 As shown, a transparent object marking method for an embodied robot is provided, wherein the method is Figure 1 The control device execution in the embodied robot 102 is used as an example to illustrate, including the following steps 202 to 208. Among them:

[0023] Step 202 , while the embodied robot is moving in the target scene, a current set of data points collected by the sensor from the surrounding environment in real time is obtained.

[0024] The target scene may be a home scene. The sensor may be, but is not limited to, a sensor for collecting point cloud data or a sensor for collecting images. The sensor may be, but is not limited to, at least one of a lidar sensor, a visual sensor, or an ultrasonic sensor, and the lidar sensor may be, but is not limited to, at least one of a DTOF (Direct Time of Flight) sensor or a triangulation radar. The visual sensor may be, but is not limited to, a depth camera. At least one sensor may be provided on the embodied robot. The moving process may be a process of building a map or a process of executing a task. The task may be any task, and may be, but is not limited to, carrying household items or cleaning.

[0025] The data point can be a point cloud data point or a pixel point in an image. The current data point set can be point cloud data or image pixel data. The surrounding environment is the environment within the sensor acquisition range. The acquisition range is determined by the sensor's own performance and can be understood as the sensor's detection range. It can be a range with a distance from the sensor that is less than or equal to a preset distance. The preset distance is determined by the sensor's performance. For example, the detection range of a lidar sensor is usually a circular direction with a radius of 5m centered on itself.

[0026] The data point may represent a point on the surface of an object in the target scene. Each data point may contain attribute information. The embodied robot may calculate the position information of the data point based on the attribute information. The position information may be the relative position between the data point and the embodied robot when the data point is collected. The attribute information includes but is not limited to at least one of the reflection intensity or color of the data point. The reflection intensity refers to the intensity of the signal emitted by the sensor, such as the laser pulse emitted by the laser radar, reflected by the object.

[0027] Specifically, in the process of the embodied robot building a scene map of the target scene, the control device can control the embodied robot to move, and collect data points from the surrounding environment in real time or at a fixed time during the movement. The current data point set can be a frame of data collected at the current moment. A frame of data can be understood as a set of data points collected by a sensor such as a laser radar after one rotation (360 degrees).

[0028] Step 204 , when it is determined by a preset detection algorithm that there are suspected transparent data points in the current data point set, a transparent feature is marked for each suspected transparent data point in the current data point set to obtain a first data point set.

[0029] The preset detection algorithm may be an algorithm for distinguishing data points of transparent objects from data points of non-transparent objects by means of data analysis, for example, an algorithm for distinguishing data points of transparent objects from data points of non-transparent objects based on the reflection intensity of the data points, etc. The specific preset detection algorithm is not limited in the present application.

[0030] Suspected transparent data points refer to data points that may be transparent objects. A transparent object is one that allows light to penetrate through it, and through it, the scene behind it can be observed. For example, a transparent object is a glass door. A transparent object can be fully transparent or semi-transparent, and the specific transparency of a transparent object is not limited in this application.

[0031] The transparency feature is used to characterize that the data point is collected from the surface of a transparent object. The difference between the first data point set and the current data point set is that the suspected transparent data points in the first data point set have been marked with the transparency feature, while the suspected transparent data points in the current data point set are not marked with the transparency feature. Marking the transparency feature can be understood as adding the transparency feature to the attribute information of the data points in the current data point set.

[0032] Specifically, the control device may identify data points that are likely to belong to transparent objects, namely, suspected transparent data points, from the current data point set through a preset detection algorithm.

[0033] Step 206, input the first data point set into the target probability prediction model, predict the transparency probability corresponding to each suspected transparent data point in the first data point set, and the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability prediction model is trained using the sample data point set, the transparent data points in the sample data point set are marked with transparent features, the transparent data points in the sample data point set that conform to the straight line property are marked with straight line features, the non-transparent data points in the sample data point set are not marked with transparent features, the transparent data points are data points of transparent objects, and the non-transparent data points are data points of non-transparent objects.

[0034] Among them, since the preset detection algorithm based on data analysis may have errors in identifying data points of transparent objects, a target probability prediction model is used in this application for further identification.

[0035] The marking of data points in the sample data point set is manual or machine marked and is accurate. Multiple data points in the sample data point set can be fitted into a straight line. If a transparent data point is on the fitted straight line or the distance between the data point and the straight line is close to 0, it can be considered that the transparent data point meets the straight line properties, and the straight line feature is marked. Otherwise, it is considered that it does not meet the straight line properties, and the straight line feature is not marked. Marking the straight line feature can be understood as adding the straight line feature to the attribute information of the data point.

[0036] Due to the characteristics of transparent objects (such as glass walls), the data points collected from transparent objects are more likely to conform to the properties of straight lines. Therefore, transparent data points marked with straight line features and transparent features are more likely to be data points of transparent objects than data points marked only with transparent features.

[0037] Specifically, the target probability prediction model may be deployed in the embodied robot after being trained by the computer device, or the target probability prediction model may be deployed in the computer device. The computer device may be a terminal or a server. The control device may send the first data point set to the computer device, and the computer device inputs the first data point set into the target probability prediction model.

[0038] Step 208 , eliminating transparent features of suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set, and retaining transparent features of suspected transparent data points whose transparency probability reaches the probability threshold, to obtain a second data point set.

[0039] Among them, there may be at least one suspected transparent data point. The probability threshold may be preset based on artificial experience or determined based on the transparency probability of the data point of the transparent object. The difference between the second data point set and the second data point set is that the suspected transparent data points in the second data point set whose transparency probability does not reach the probability threshold are not marked with transparent features.

[0040] Step 210: Use the second data point set to update the scene map of the embodied robot.

[0041] Among them, there are historical data points in the scene map (that is, data points historically added to the scene map). By collecting current data points in real time and adding them to the scene map, the number of data points in the scene map can be increased, thereby incrementally updating the data in the scene map.

[0042] In some embodiments, for each second data point in the second data point set, the control device can convert the relative position recorded by the second data point into a map position in the scene map according to the current position of the embodied robot (i.e., the current position of the embodied robot in the scene map), where the relative position is the relative position between the second data point and the embodied robot when the data point is collected. Then, the control device can update the relative position of the data point record to a map position, and add the second data point with the updated position to the scene map, thereby updating the scene map.

[0043] Among them, since the position recorded in the data point in the current data point set is a relative position, before adding the data point to the scene map, the relative position can be converted into a map position in the scene map. Then, the data point with the updated position is added to the scene map.

[0044] In this embodiment, the position of the data point can be accurately located in the scene map through position conversion, so that the data point can be accurately added to the scene map, thereby improving the accuracy of updating the scene map.

[0045] In the transparent object marking method of the above-mentioned embodied robot, since the target probability prediction model is obtained by training using a set of sample data points, and the transparent data points in the sample data point set are marked with transparent features, the transparent data points that conform to the properties of a straight line in the sample data point set are marked with straight line features, and the non-transparent data points in the sample data point set are not marked with transparent features, the sample data point set contains transparent features and straight line features, and different types of data points can be distinguished by transparent features and straight line features, so that the target probability prediction model trained by the sample data point set can learn transparent features and straight line features, which helps the target probability prediction model to more accurately distinguish data points of transparent objects from data points of non-transparent objects. When it is determined through a preset detection algorithm that there are suspected transparent data points, the suspected transparent data points are not directly marked in the scene map. Instead, after obtaining the first data point set, the target probability prediction model is used to predict the transparency probability of the suspected transparent data points in the first data point set. When the transparency probability does not reach the probability threshold, it means that the suspected transparent data point is very likely not a transparent data point, and the credibility of the suspected transparent data point whose transparency probability reaches the probability threshold is a transparent data point is higher. Therefore, the transparent features of the suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set are eliminated, and the transparent features of the suspected transparent data points whose transparency probability reaches the probability threshold are retained to obtain a second data point set. The second data point set is used to update the scene map of the embodied robot, so that transparent objects can be accurately marked in the scene map, thereby improving the marking accuracy of transparent objects.

[0046] In some embodiments, the training step of the target probability prediction model includes: inputting a set of sample data points into the probability prediction model to be trained, and predicting the predicted transparent probability corresponding to each data point in the sample data point set; determining a model loss value based on the predicted transparent probability corresponding to each data point in the sample data point set, the model loss value is positively correlated with the predicted transparent probability corresponding to the non-transparent data point, and the model loss value is negatively correlated with the predicted transparent probability corresponding to the transparent data point; using the model loss value to train the probability prediction model to obtain a target probability prediction model.

[0047] The probability prediction model can be any neural network model, which can be but not limited to a model based on the Transformer network. The Transformer network is a deep learning model based on the self-attention mechanism. The core structure of the Transformer includes an encoder and a decoder, and the encoder includes a multi-head self-attention mechanism. The predicted transparency probability corresponding to a data point is the probability that the data point belongs to a transparent object output by the probability prediction model.

[0048] Specifically, during the training process, the model parameters of the probability prediction model are updated in a direction that reduces the model loss value. For example, the computer device can train the probability prediction model based on a gradient descent algorithm and a model loss value. The computer device can obtain multiple sets of sample data points, and use the multiple sets of sample data points to iteratively train the probability prediction model until the probability prediction model converges to obtain a target probability prediction model.

[0049] In this embodiment, since the model loss value is positively correlated with the predicted transparent probability corresponding to the non-transparent data point, and the model loss value is negatively correlated with the predicted transparent probability corresponding to the transparent data point, in the process of training the probability prediction model, the predicted transparent probability output by the probability prediction model for the non-transparent data point can show a decreasing trend, while the predicted transparent probability output by the probability prediction model for the transparent data point can show an increasing trend, so that the probability prediction model can learn the ability to distinguish between transparent data points and non-transparent data points.

[0050] In some embodiments, a model loss value is determined based on the predicted transparent probability corresponding to each data point in the sample data point set, including: performing mean calculation on the predicted transparent probabilities corresponding to each non-transparent data point in the sample data point set to obtain a first mean; performing mean calculation on the predicted transparent probabilities corresponding to each first transparent data point in the sample data point set to obtain a second mean, wherein the first transparent data points are marked with straight line features and transparent features; performing mean calculation on the predicted transparent probabilities corresponding to each second transparent data point in the sample data point set to obtain a third mean, wherein the second transparent data points are marked with transparent features and are not marked with straight line features; determining a model loss value based on the first mean, the second mean and the third mean, wherein the model loss value is positively correlated with the first mean, and the model loss value is negatively correlated with the second mean and the third mean, respectively.

[0051] The difference between the first transparent data point and the second transparent data point is whether they are marked with a straight line feature.

[0052] In some embodiments, the model loss value = the first mean - the second mean - the third mean. Alternatively, the model loss value = a × the first mean - b × the second mean - c × the third mean. Wherein, a, b, and c are preset coefficients, and are all positive numbers.

[0053] In this embodiment, since the model loss value is positively correlated with the first mean, and the model loss value is negatively correlated with the second mean and the third mean respectively, in the process of training the probability prediction model, the predicted transparent probability output by the probability prediction model for non-transparent data points can show a decreasing trend, and the predicted transparent probability output by the probability prediction model for transparent data points can show an increasing trend, so that the probability prediction model can learn the ability to distinguish between transparent data points and non-transparent data points.

[0054] In some embodiments, the model loss value is determined based on the first mean, the second mean and the third mean, including: taking the difference obtained by subtracting the third mean from the second mean as a reference difference; determining the model loss value based on the first mean, the second mean, the third mean and the reference difference, and the model loss value is negatively correlated with the reference difference.

[0055] In some embodiments, the model loss value = the first mean - the second mean - the third mean - the reference difference. The reference difference = the second mean - the third mean. Alternatively, the model loss value = a × the first mean - b × the second mean - c × the third mean - d × the reference difference. Wherein, d is a preset coefficient and is a positive number.

[0056] In some embodiments, during the training process, the model parameters of the probability prediction model are updated in a direction that reduces the model loss value. Thus, by adjusting the model parameters, the first mean tends to decrease, and the second mean, the third mean and the reference difference tend to increase, so that the second mean tends to be greater than the third mean.

[0057] In some embodiments, Figure 3 As shown, a flow chart of the trained probability prediction model is provided, including:

[0058] Step 302: input the sample data point set into the probability prediction model to be trained, and predict the predicted transparency probability corresponding to each data point in the sample data point set.

[0059] Step 304 , calculating the mean of the predicted transparent probabilities corresponding to each non-transparent data point in the sample data point set to obtain a first mean.

[0060] Step 306, calculating the mean of the predicted transparent probabilities corresponding to the first transparent data points in the sample data point set to obtain a second mean, wherein the first transparent data points are marked with a straight line feature and a transparent feature.

[0061] Step 308 , calculating the mean of the predicted transparent probabilities corresponding to each second transparent data point in the sample data point set to obtain a third mean, wherein the second transparent data points are marked with transparent features and are not marked with straight line features.

[0062] Step 310: The difference obtained by subtracting the third mean from the second mean is used as a reference difference.

[0063] Step 312, determining a model loss value according to the first mean, the second mean, the third mean and the reference difference, wherein the model loss value is negatively correlated with the reference difference.

[0064] Step 314, using the model loss value to train the probability prediction model to obtain a target probability prediction model.

[0065] In this embodiment, due to the characteristics of transparent objects (such as glass walls), the data points collected from the transparent objects are more likely to conform to the properties of straight lines. Therefore, data points marked with straight line features and transparent features are more likely to be transparent data points than data points marked with only transparent features. Therefore, through the negative correlation between the model loss value and the reference difference value, the second mean can tend to be greater than the third mean, so that the predicted transparency probability output by the probability prediction model for the first transparent data point is greater than the predicted transparency probability output for the second transparent data point. Therefore, the probability prediction model can learn the knowledge of straight line features and transparent features, and can adjust the output predicted transparency probability according to whether there are straight line features, thereby improving the accuracy of the trained target probability prediction model in identifying transparent data points.

[0066] In some embodiments, after step 210, the method further includes: determining a preset range, the preset range being centered on the current position of the embodied robot and being larger than the collection range of the sensor; inputting a third data point set within the preset range in the updated scene map into the target probability prediction model, and updating the transparency probability corresponding to each third data point of each marked transparent feature in the third data point set within the preset range; and verifying the updated scene map based on the updated transparency probability corresponding to each third data point of each marked transparent feature.

[0067] Among them, the current position of the embodied robot refers to the position of the embodied robot in the map coordinate system. The map coordinate system is the coordinate system used by the scene map. The current position of the embodied robot can be understood as the current position of the embodied robot in the scene map.

[0068] The preset range is larger than the collection range. For example, if the collection range is a range within a circle with a radius of 5 meters centered on the sensor (or embodied robot), then the preset range can be a range within a circle with a radius of 7 meters centered on the embodied robot. The preset range can also be an area with the boundary proportionally expanded by a certain range (for example, 1.25 times) based on the boundary information of the second data point set in the scene map.

[0069] It should be noted that the third data point set includes the second data point set and historical data points before the current moment within the preset range; wherein, if the historical data points before the current moment within the preset range are marked with a transparent feature, the historical data points marked with the transparent feature have a historical transparency probability. The historical transparency probability of the historical data points is also predicted by the target probability prediction model.

[0070] In some embodiments, after the third data point set is input into the target probability prediction model, the target probability prediction model will process each third data point marked with a transparent feature. The specific processing method is: if the third data point has a straight line feature in the third data point set, the target probability prediction model will increase the transparency probability of the third data point; if the third data point does not have a straight line feature in the third data point set, the target probability prediction model will reduce the transparency probability of the third data point.

[0071] In this embodiment, since the current data point set is a frame of data collected within the collection range, and since the number of data points in a frame of data is usually sparse, the straight line properties of the data points in the current data point set (or the first data point set) cannot be well presented. Based on this, the third data point set within the preset range is obtained from the updated scene map. Since the preset range is larger than the collection range, the preset range includes the data points in the current data point set, and may include the data points historically collected within the preset range. The data points within the preset range are denser and more complete, and can better present the straight line properties of the data points. The target probability prediction model is used to predict the transparency probability corresponding to each data point within the preset range, thereby improving the accuracy of the transparency probability of each data point.

[0072] In some embodiments, based on the transparent probability corresponding to each of the third data points marked with transparent features after update, the updated scene map is verified, including: for each third data point marked with transparent features, if the transparent probability corresponding to the third data point reaches the identification threshold (e.g., 100%), then a transparent object identification is marked at the position of the third data point in the updated scene map; if the transparent probability corresponding to the third data point does not reach the probability threshold (e.g., 60%), then the transparent feature of the third data point is eliminated in the updated scene map. Wherein, the identification threshold is greater than the probability threshold, the transparent feature is a type of information in the attribute information of the data point, and the transparent object identification is an identification on the scene map.

[0073] Among them, if the transparency probability corresponding to the data point does not reach the probability threshold, and the data point is marked with a transparent feature in the updated scene map, it may be mislabeled. Therefore, eliminating the transparent feature of the data point in the updated scene map can reduce the number of mislabeling situations.

[0074] In this embodiment, if the transparency probability corresponding to the third data point marked with a transparent feature does not reach the probability threshold, the transparent feature of the third data point is eliminated in the updated scene map, thereby reducing the possibility of mislabeling.

[0075] In some embodiments, after step 202, the method further includes: determining a preset range when it is determined through a preset detection algorithm that there are no suspected transparent data points in the current data point set, and the number of data points in the current data point set that conform to the straight line property is greater than a quantity threshold, the preset range is centered on the current position of the embodied robot, and the preset range is greater than the collection range of the sensor; when there is a fourth data point marked with a transparent feature within the preset range in the scene map, combining the current data point set with the fourth data point to obtain a combined data point set; inputting the combined data point set into the target probability prediction model, and updating the transparency probability corresponding to the fourth data point in the combined data point set; and updating the scene map based on the transparency probability corresponding to the fourth data point in the updated combined data point set.

[0076] Among them, the fourth data point is a historical data point saved within a preset range in the scene map before the current moment, and the fourth data point is predicted by the target probability prediction model before the current moment to have a historical transparency probability; the data points in the combined data point set include the data points in the current data point set and the fourth data point. If there are multiple fourth data points marked with transparent features within the preset range in the scene map, then the combined data point set includes multiple fourth data points; the quantity threshold can be set according to manual experience.

[0077] It should be noted that after the fourth data point is input into the target probability prediction model, the target probability preset model will process the fourth data point. The specific processing method is: if the fourth data point has a straight line feature in the combined data point set, the target probability prediction model will increase the transparency probability of the fourth data point; if the fourth data point does not have a straight line feature in the combined data point set, the target probability prediction model will reduce the transparency probability of the fourth data point.

[0078] In some embodiments, when it is determined through a preset detection algorithm that there are no suspected transparent data points in the current data point set, and the number of data points in the current data point set that conform to the straight line property is greater than a quantity threshold, and before determining the preset range, the control device will add each data point of the current data point set to the corresponding position in the scene map.

[0079] In some embodiments, for the fourth data point in the combined data point set after being processed by the target probability preset model, if the transparency probability corresponding to the fourth data point does not reach the probability threshold, and the fourth data point in the scene map has been marked with a transparent feature, the transparent feature of the fourth data point is eliminated in the scene map, thereby reducing the possibility of mislabeling. If the transparency probability corresponding to the fourth data point reaches the identification threshold, a transparent object identification is marked at the location of the fourth data point in the scene map.

[0080] In this embodiment, if there are data points marked with transparent features within the preset range, it means that there may be transparent objects within the current collection range, and thus there may be transparent data points in the current data point set. Due to the angle limitation of the sensor, the preset detection algorithm has not successfully identified them, but the number of data points that meet the straight line property in the current data point set is greater than the quantity threshold, indicating that the current data points (i.e., the data points in the current data point set) have rich straight line features. Therefore, in this case, the current data point set is combined with the fourth data point within the preset range. Since the fourth data point within the preset range is a historically collected data point, the transparency probability of the fourth data point can be predicted as a whole by combining the historically collected fourth data point and the current data point set, thereby improving the accuracy of the transparency probability of the fourth data point.

[0081] In some embodiments, Figure 4 As shown, a transparent object marking method of an embodied robot is provided, comprising:

[0082] Step 402 , while the embodied robot is moving in the target scene, a current set of data points collected by the sensor from the surrounding environment in real time is obtained.

[0083] Step 404 , determining whether there are suspected transparent data points in the current data point set by a preset detection algorithm, if so, executing step 406 , if not, executing step 418 .

[0084] Step 406 , marking a transparent feature for each suspected transparent data point in the current data point set, to obtain a first data point set.

[0085] Step 408, input the first data point set into the target probability prediction model, and predict the transparency probability corresponding to each suspected transparent data point in the first data point set, wherein the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability prediction model is trained using a sample data point set, transparent data points in the sample data point set are marked with transparent features, transparent data points in the sample data point set that conform to the properties of a straight line are marked with a straight line feature, non-transparent data points in the sample data point set are not marked with transparent features, transparent data points are data points of transparent objects, and non-transparent data points are data points of non-transparent objects.

[0086] Step 410 , eliminating transparent features of suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set, and retaining transparent features of suspected transparent data points whose transparency probability reaches the probability threshold, to obtain a second data point set.

[0087] Step 412, determining a preset range, the preset range is centered on the current position of the embodied robot, and the preset range is larger than the collection range of the sensor.

[0088] Step 414, input the third data point set within the preset range in the updated scene map into the target probability prediction model, and update the transparency probability corresponding to each third data point marking the transparent object feature in the third data point set within the preset range.

[0089] Step 416: verify the updated scene map based on the updated transparency probability corresponding to each third data point with transparent features.

[0090] Step 418 , determining whether the number of data points in the current data point set that conform to the straight line property is greater than a quantity threshold, if so, executing step 420 , if not, executing step 430 .

[0091] Step 420, determining a preset range, the preset range is centered on the current position of the embodied robot, and the preset range is larger than the collection range of the sensor.

[0092] Step 422 , determine whether there is a fourth data point marked with a transparent feature within a preset range in the scene map. If so, execute step 424 ; if not, execute step 430 .

[0093] Step 424: Combine the current data point set with the fourth data point to obtain a combined data point set.

[0094] Step 426, input the combined data point set into the target probability prediction model, and update the transparent probability corresponding to the fourth data point in the combined data point set.

[0095] Step 428, updating the scene map based on the transparency probability corresponding to the fourth data point in the combined data point set.

[0096] Step 430, adding the current data point set to the scene map.

[0097] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0098] Based on the same inventive concept, the embodiment of the present application also provides an embodied robot for implementing the transparent object marking method of the embodied robot involved above. The implementation scheme for solving the problem provided by the embodied robot is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodied robot embodiments provided below can refer to the limitations of the transparent object marking method of the embodied robot above, and will not be repeated here.

[0099] In some embodiments, Figure 5 As shown, an embodied robot is provided, the embodied robot comprising: a data acquisition module 502, a feature marking module 504, a probability prediction module 506, a feature processing module 508 and a map updating module 510, wherein:

[0100] The data acquisition module 502 is used to acquire a current set of data points collected by sensors from the surrounding environment in real time while the embodied robot moves in the target scene.

[0101] The feature marking module 504 is used to mark a transparent feature for each suspected transparent data point in the current data point set when it is determined by a preset detection algorithm that there are suspected transparent data points in the current data point set, so as to obtain a first data point set.

[0102] The probability prediction module 506 is used to input the first data point set into the target probability prediction model, and predict the transparency probability corresponding to each suspected transparent data point in the first data point set, wherein the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability prediction model is trained using a sample data point set, wherein transparent data points in the sample data point set are marked with transparent features, transparent data points in the sample data point set that conform to the properties of a straight line are marked with a straight line feature, non-transparent data points in the sample data point set are not marked with transparent features, transparent data points are data points of transparent objects, and non-transparent data points are data points of non-transparent objects.

[0103] The feature processing module 508 is used to eliminate the transparent features of the suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set, and retain the transparent features of the suspected transparent data points whose transparency probability reaches the probability threshold, to obtain the second data point set.

[0104] The map updating module 510 is used to update the scene map of the embodied robot using the second data point set.

[0105] In some embodiments, a model training module for training a probability prediction model is used to input a set of sample data points into the probability prediction model to be trained, and predict the predicted transparency probability corresponding to each data point in the set of sample data points; determine a model loss value based on the predicted transparency probability corresponding to each data point in the set of sample data points, the model loss value is positively correlated with the predicted transparency probability corresponding to the non-transparent data point, and the model loss value is negatively correlated with the predicted transparency probability corresponding to the transparent data point; use the model loss value to train the probability prediction model to obtain a target probability prediction model.

[0106] In some embodiments, the model training module is also used to calculate the mean of the predicted transparent probabilities corresponding to each non-transparent data point in the sample data point set to obtain a first mean; calculate the mean of the predicted transparent probabilities corresponding to each first transparent data point in the sample data point set to obtain a second mean, and the first transparent data points are marked with straight line features and transparent features; calculate the mean of the predicted transparent probabilities corresponding to each second transparent data point in the sample data point set to obtain a third mean, and the second transparent data points are marked with transparent features and are not marked with straight line features; determine the model loss value based on the first mean, the second mean and the third mean, the model loss value is positively correlated with the first mean, and the model loss value is negatively correlated with the second mean and the third mean, respectively.

[0107] In some embodiments, the model training module is also used to use the difference obtained by subtracting the third mean from the second mean as a reference difference; determine the model loss value based on the first mean, the second mean, the third mean and the reference difference, and the model loss value is negatively correlated with the reference difference.

[0108] In some embodiments, the embodied robot also includes a map verification module, which is used to determine a preset range, the preset range is centered on the current position of the embodied robot, and the preset range is larger than the collection range of the sensor; the third data point set within the preset range in the updated scene map is input into the target probability prediction model, and the transparency probability corresponding to each third data point marked with a transparent feature in the third data point set within the preset range is updated; based on the updated transparency probability corresponding to each third data point marked with a transparent feature, the updated scene map is verified.

[0109] In some embodiments, the map verification module is further used to eliminate the transparent feature of each third data point marked with a transparent feature in the updated scene map if the transparency probability corresponding to the third data point does not reach the probability threshold.

[0110] In some embodiments, the map update module 510 is also used to determine a preset range when it is determined through a preset detection algorithm that there are no suspected transparent data points in the current data point set, and the number of data points that conform to the straight line property in the current data point set is greater than a quantity threshold. The preset range is centered on the current position of the embodied robot, and the preset range is greater than the collection range of the sensor; when there is a fourth data point marked with a transparent feature within the preset range in the scene map, the current data point set is combined with the fourth data point to obtain a combined data point set; the combined data point set is input into the target probability prediction model, and the transparent probabilities corresponding to the fourth data points are updated; and the scene map is updated based on the transparent probabilities corresponding to the updated fourth data points.

[0111] In some embodiments, the map update module 510 is also used to convert the relative position recorded by the second data point into a map position in the scene map for each second data point in the second data point set according to the current position of the embodied robot; the relative position is the relative position between the second data point and the embodied robot when the second data point is collected; the relative position recorded by the second data point is updated to a map position, and the second data point with the updated position is added to the scene map.

[0112] Each module in the above-mentioned embodied robot can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the control device in the form of hardware, or can be stored in the memory in the control device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0113] In some embodiments, a control device is provided. The control device is a device in an embodied robot for controlling the embodied robot. The internal structure diagram thereof can be as follows: Figure 6 As shown. The control device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the control device is used to provide computing and control capabilities. The memory of the control device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the control device is used to store data related to the transparent object marking method of the embodied robot. The input / output interface of the control device is used to exchange information between the processor and an external device. The communication interface of the control device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a transparent object marking method of an embodied robot is implemented.

[0114] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the control device to which the scheme of the present application is applied. The specific control device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In some embodiments, a control device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the transparent object marking method of the embodied robot are implemented.

[0116] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the transparent object marking method of the embodied robot are implemented.

[0117] In some embodiments, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps in the transparent object marking method of the embodied robot.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0119] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0121] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A transparent object marking method for an embodied robot, characterized in that: The method comprises: When the embodied robot moves in the target scene, it obtains the current data point set collected by the sensor from the surrounding environment in real time; When it is determined by a preset detection algorithm that there are suspected transparent data points in the current data point set, marking a transparent feature for each suspected transparent data point in the current data point set to obtain a first data point set; The first data point set is input into the target probability prediction model, and the transparency probability corresponding to each suspected transparent data point in the first data point set is predicted, and the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability prediction model is trained using a sample data point set, transparent data points in the sample data point set are marked with transparent features, transparent data points in the sample data point set that meet the straight line property are marked with straight line features, non-transparent data points in the sample data point set are not marked with transparent features, the transparent data points are data points of transparent objects, and the non-transparent data points are data points of non-transparent objects; Eliminating transparent features of suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set, and retaining transparent features of suspected transparent data points whose transparency probability reaches the probability threshold, to obtain a second data point set; updating a scene map of the embodied robot using the second set of data points; When it is determined by a preset detection algorithm that there are no suspected transparent data points in the current data point set, and the number of data points in the current data point set that meet the straight line property is greater than a quantity threshold, a preset range is determined, the preset range is centered on the current position of the embodied robot, and the preset range is greater than the collection range of the sensor; In a case where there is a fourth data point marked with a transparent feature within the preset range in the scene map, combining the current data point set with the fourth data point to obtain a combined data point set; Inputting the combined data point set into the target probability prediction model, and updating the transparent probability corresponding to the fourth data point; The scene map is updated based on the updated transparency probability corresponding to the fourth data point.

2. The method according to claim 1, characterized in that The target probability prediction model is trained using a set of sample data points, specifically including: Inputting the sample data point set into the probability prediction model to be trained, and predicting the predicted transparent probability corresponding to each data point in the sample data point set; Determine a model loss value based on the predicted transparency probability corresponding to each data point in the set of sample data points, wherein the model loss value is positively correlated with the predicted transparency probability corresponding to the non-transparent data point, and the model loss value is negatively correlated with the predicted transparency probability corresponding to the transparent data point; The probability prediction model is trained using the model loss value to obtain a target probability prediction model.

3. The method according to claim 2, characterized in that The determining of the model loss value based on the predicted transparency probability corresponding to each data point in the sample data point set includes: Calculating the mean of the predicted transparent probabilities corresponding to each non-transparent data point in the sample data point set to obtain a first mean; Calculating the mean of predicted transparent probabilities corresponding to first transparent data points in the sample data point set to obtain a second mean, wherein the first transparent data points are marked with a straight line feature and a transparent feature; Calculating the mean of the predicted transparency probabilities corresponding to the second transparent data points in the sample data point set, respectively, to obtain a third mean, wherein the second transparent data points are marked with transparent features and are not marked with straight line features; A model loss value is determined according to the first mean, the second mean and the third mean, wherein the model loss value is positively correlated with the first mean, and the model loss value is negatively correlated with the second mean and the third mean, respectively.

4. The method according to claim 3, characterized in that The determining the model loss value according to the first mean, the second mean and the third mean includes: subtracting the third mean from the second mean as a reference difference; A model loss value is determined according to the first mean, the second mean, the third mean and the reference difference, and the model loss value is negatively correlated with the reference difference.

5. The method according to any one of claims 1 to 4, characterized in that: After updating the scene map of the embodied robot using the second data point set, the method further includes: Determine a preset range, wherein the preset range is centered at the current position of the embodied robot and is larger than the acquisition range of the sensor; Inputting the third data point set within the preset range in the updated scene map into the target probability prediction model, and updating the transparency probability corresponding to each third data point marked with a transparent feature in the third data point set within the preset range; The updated scene map is verified based on the updated transparency probabilities respectively corresponding to the third data points of each marked transparent feature.

6. The method according to claim 5, characterized in that The verifying the updated scene map based on the updated transparency probabilities respectively corresponding to the third data points of each marked transparent feature includes: For each third data point marked with a transparent feature, if the transparency probability corresponding to the third data point does not reach the probability threshold, the transparent feature of the third data point is eliminated in the updated scene map.

7. The method according to any one of claims 1 to 4, characterized in that: The updating of the transparency probability corresponding to the fourth data point includes: If the fourth data point has a straight line feature in the set of combined data points, increasing the transparency probability of the fourth data point; If the fourth data point does not have a straight line feature in the combined data point set, the transparency probability of the fourth data point is reduced.

8. The method according to any one of claims 1 to 4, characterized in that The updating of the scene map of the embodied robot by using the second data point set includes: For each second data point in the second data point set, converting the relative position recorded by the second data point into a map position in the scene map according to the current position of the embodied robot; the relative position is the relative position between the second data point and the embodied robot when the second data point is collected; The relative position recorded by the second data point is updated to the map position, and the second data point with the updated position is added to the scene map.

9. An embodied robot, characterized in that: The embodied robot comprises: A data acquisition module is used to obtain a set of current data points collected by sensors from the surrounding environment in real time while the embodied robot moves in the target scene; a feature marking module, for marking a transparent feature for each suspected transparent data point in the current data point set respectively, when it is determined by a preset detection algorithm that there are suspected transparent data points in the current data point set, to obtain a first data point set; A probability prediction module, used for inputting the first data point set into a target probability prediction model, predicting the transparency probability corresponding to each suspected transparent data point in the first data point set, wherein the transparency probability represents the probability that the suspected transparent data point belongs to a transparent object; wherein the target probability prediction model is obtained by training with a sample data point set, wherein transparent data points in the sample data point set are marked with transparent features, wherein the transparent data points in the sample data point set that conform to the straight line property are marked with straight line features, wherein non-transparent data points in the sample data point set are not marked with transparent features, wherein the transparent data points are data points of transparent objects, and wherein the non-transparent data points are data points of non-transparent objects; a feature processing module, configured to eliminate transparent features of suspected transparent data points whose transparency probability does not reach the probability threshold in the first data point set, and retain transparent features of suspected transparent data points whose transparency probability reaches the probability threshold, to obtain a second data point set; A map updating module, configured to update a scene map of the embodied robot using the second set of data points; The map update module is also used for: determining a preset range when it is determined through a preset detection algorithm that there are no suspected transparent data points in the current data point set, and the number of data points that conform to the straight line property in the current data point set is greater than a quantity threshold, wherein the preset range is centered on the current position of the embodied robot, and the preset range is greater than the acquisition range of the sensor; when there is a fourth data point marked with a transparent feature within the preset range in the scene map, combining the current data point set with the fourth data point to obtain a combined data point set; inputting the combined data point set into the target probability prediction model to update the transparency probability corresponding to the fourth data point; and updating the scene map based on the updated transparency probability corresponding to the fourth data point.

10. A control device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images