Obstacle recognition method and device, storage medium and electronic device

Abstract

The application provides an obstacle identification method and device, a storage medium and an electronic device. The method comprises: performing object identification on to-be-identified data collected by a target sensor on a mobile body to obtain a target object; determining a matching object that matches the target object from a preset group of reference objects; and in a case where object features of the target object are determined to be missing according to object features of the matching object, complementing the object features of the target object using the object features of the matching object as an obstacle identification result. The above technical solution solves the problem of low safety of the mobile body caused by the inability to accurately identify obstacles, and improves the safety of the movement of the mobile body.

Description

[Technical Field]

[0001] This application relates to the field of smart homes, and more specifically, to an obstacle recognition method and apparatus, a storage medium, and an electronic device. [Background Technology]

[0002] Currently, when a mobile device moves within a certain area, it can collect environmental data through its sensors to detect obstacles such as furniture within the area, and then plan its movement path based on the detected obstacles. However, for objects with poor real-time data (e.g., furniture with thin legs such as tables and chairs), it is difficult for sensors to accurately identify the complete object data of the obstacle. This can lead to the planned movement path being too close to these obstacles, increasing the risk of collision. This can cause the mobile device to tip over or even be damaged, negatively impacting the user experience.

[0003] Therefore, it can be seen that obstacle recognition methods in related technologies have the problem of low mobility safety of moving objects due to the inability to accurately identify obstacles. [Summary of the Invention]

[0004] The purpose of this application is to provide an obstacle recognition method and apparatus, storage medium and electronic device, so as to at least solve the problem of low mobility safety of moving bodies due to the inability to accurately identify obstacles.

[0005] The purpose of this application is to achieve the following technical solution:

[0006] According to one aspect of the embodiments of this application, an obstacle recognition method is provided, comprising: performing object recognition on data to be recognized collected by a target sensor on a moving body to obtain a target object; determining a matching object that matches the target object from a preset set of reference objects; and, if it is determined from the object features of the matching object that the object features of the target object are missing, using the object features of the matching object to complete the object features of the target object, as an obstacle recognition result.

[0007] In an exemplary embodiment, determining the matching object that matches the target object from a preset set of reference objects includes: determining the reference objects from the set of reference objects whose object features are more than or equal to a preset similarity threshold as the matching objects.

[0008] In one exemplary embodiment, the object features of each reference object in the set of reference objects are object features pre-stored according to the object type and corresponding to the object type of each reference object.

[0009] In an exemplary embodiment, determining a reference object in the set of reference objects whose object features are similar to or greater than a preset similarity threshold with the object features of the target object as the matching object includes: if there is an associated object in the set of reference objects that is associated with the object position of the target object, determining the similarity between the object features of the target object and the object features of the associated object; and if the similarity between the object features of the associated object and the object features of the target object is greater than or equal to the similarity threshold, determining the associated object as the matching object.

[0010] In one exemplary embodiment, the method further includes: identifying reference objects in the set of reference objects whose distance between the associated position and the object position of the target object is less than or equal to a preset distance threshold as the associated objects.

[0011] In an exemplary embodiment, determining the reference object in the set of reference objects whose similarity to the object features of the target object is greater than or equal to a preset similarity threshold as the matching object includes: when there is no associated object in the set of reference objects that is associated with the object position of the target object, determining the similarity between the object features of each reference object in the set of reference objects and the object features of the target object as the similarity corresponding to each reference object; and determining the reference object in the set of reference objects whose corresponding similarity is greater than or equal to the preset similarity threshold and whose corresponding similarity is the largest as the matching object.

[0012] In an exemplary embodiment, before determining the matching object that matches the target object in a preset set of reference objects, the method further includes at least one of the following: fusing sensor data collected by a set of sensors on the moving body to obtain object features of each reference object in the set of reference objects, wherein the set of sensors includes the target sensor; fusing sensor data collected by the target sensor through multiple data acquisitions to obtain object features of each reference object in the set of reference objects.

[0013] In an exemplary embodiment, the step of using the object features of the matching object to complete the object features of the target object includes: when the first leg of the target object is missing an edge, using the leg features in the matching object that match the first leg to complete the missing edge of the first leg; and when the second leg of the target object is missing, using the leg features in the matching object that match the second leg and the leg position of the second leg in the target object to complete the second leg.

[0014] According to another aspect of the embodiments of this application, an obstacle recognition device is also provided, comprising: a recognition unit, configured to perform object recognition on the data to be recognized collected by a target sensor on a moving body to obtain a target object; a first determination unit, configured to determine a matching object that matches the target object from a preset set of reference objects; and a completion unit, configured to complete the object features of the target object using the object features of the matching object when it is determined from the object features of the matching object that the object features of the target object are missing, and use the object features of the matching object as the obstacle recognition result.

[0015] In an exemplary embodiment, the first determining unit includes: a first determining module, configured to determine the reference objects in the set of reference objects whose object features are more than or equal to the object features of the target object as the matching objects.

[0016] In one exemplary embodiment, the object features of each reference object in the set of reference objects are object features pre-stored according to the object type and corresponding to the object type of each reference object.

[0017] In an exemplary embodiment, the first determining module includes: a first determining submodule, configured to determine the similarity between the object features of the target object and the object features of the associated object when there is an associated object in the set of reference objects that is associated with the object position of the target object; and a second determining submodule, configured to determine the associated object as the matching object when the similarity between the object features of the associated object and the object features of the target object is greater than or equal to the similarity threshold.

[0018] In one exemplary embodiment, the apparatus further includes: a second determining unit, configured to determine, among the set of reference objects, reference objects whose distance between their associated positions and the object positions of the target object is less than or equal to a preset distance threshold as the associated objects.

[0019] In an exemplary embodiment, the first determining module includes: a third determining submodule, configured to, when there is no associated object in the set of reference objects that is associated with the object position of the target object, determine the similarity between the object features of each reference object in the set of reference objects and the object features of the target object, and use this as the similarity corresponding to each reference object; and a fourth determining submodule, configured to determine the reference object in the set of reference objects whose corresponding similarity is greater than or equal to the preset similarity threshold and whose corresponding similarity is the largest as the matching object.

[0020] In one exemplary embodiment, the apparatus further includes at least one of the following: a first fusion unit, configured to fuse sensor data collected by a set of sensors on the moving body before determining a matching object that matches the target object in a preset set of reference objects, to obtain object features of each reference object in the set of reference objects, wherein the set of sensors includes the target sensor; and a second fusion unit, configured to fuse sensor data collected by the target sensor through multiple data acquisitions to obtain object features of each reference object in the set of reference objects.

[0021] In an exemplary embodiment, the completion unit includes: a first completion module, configured to complete the missing edge of the first leg of the target object using a leg feature in the matching object that matches the first leg when the first leg of the target object is missing an edge; and a second completion module, configured to complete the second leg of the target object using a leg feature in the matching object that matches the second leg and the leg position of the second leg in the target object when the second leg of the target object is missing.

[0022] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described obstacle recognition method when running.

[0023] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the obstacle recognition method described above through the computer program.

[0024] In this embodiment, the object features of real-time detected obstacles are supplemented by pre-stored object features. Object recognition is performed on the data collected by the target sensor on the moving body to obtain the target object. A set of preset reference objects is then identified, and a matching object is found that matches the target object. If the object features of the target object are missing based on the matching object's features, the matching object's features are used to supplement the target object's features, serving as the obstacle recognition result. Since pre-stored object features are used to supplement the real-time detected object features when the detected obstacle's features are incomplete, the integrity of the obstacle's object data can be improved. Furthermore, using complete obstacle data for moving body route planning can reduce the possibility of the moving body tipping over due to excessively close proximity to objects, thus improving the moving body's safety and solving the problem of low moving body safety caused by inaccurate obstacle recognition. [Attached Image Description]

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the hardware environment of an optional obstacle recognition method according to an embodiment of this application;

[0028] Figure 2 This is a flowchart illustrating an optional obstacle recognition method according to an embodiment of this application;

[0029] Figure 3 This is a flowchart illustrating another optional obstacle recognition method according to an embodiment of this application;

[0030] Figure 4 This is a structural block diagram of an optional obstacle recognition device according to an embodiment of this application;

[0031] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application.

Detailed Implementation Methods

[0032] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0034] According to one aspect of the embodiments of this application, an obstacle recognition method is provided. Optionally, in this embodiment, the above-described obstacle recognition method can be applied to, for example... Figure 1 The hardware environment shown consists of mobile device 102, base station 104, and cloud platform 106. For example... Figure 1 As shown, the mobile body 102 can connect to the base station 104 and / or the cloud platform 106 (e.g., a voice cloud platform) via a network to enable interaction between the mobile body 102 and the base station 104 and / or the cloud platform 106.

[0035] The aforementioned networks may include, but are not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth, infrared. The network used by the mobile body 102 to communicate with the base station 104 and / or the cloud platform 106 may be the same as or different from the network used by the base station 104 to communicate with the cloud platform 106. The mobile body 102 may include, but is not limited to: cleaning robots, such as intelligent vacuum cleaners, intelligent sweeping robots, and intelligent sweeping robots that integrate vacuuming and sweeping; delivery robots, such as food delivery robots and item delivery robots.

[0036] The obstacle recognition method of this application embodiment can be executed by the mobile body 102 and the cloud platform 106 individually, or it can be executed jointly by the mobile body 102 and the cloud platform 106. The obstacle recognition method of this application embodiment can also be executed by a client installed on the mobile body 102.

[0037] Taking the obstacle recognition method in this embodiment executed by the moving body 102 as an example, Figure 2 This is a flowchart illustrating an optional obstacle recognition method according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0038] Step S202: Perform object recognition on the data to be identified collected by the target sensor on the moving body to obtain the target object.

[0039] The obstacle recognition method in this embodiment can be applied to scenarios where obstacles within a target area are identified when a mobile body (i.e., the aforementioned mobile body 102) moves within that area. The target area can be an indoor area, such as a user's home, office, or factory workshop. Functionally, the mobile body can be a cleaning robot, delivery robot, etc.; morphologically, it can be a biomimetic robot, such as a quadruped robot or a crawling robot. This embodiment does not impose any limitations on this aspect.

[0040] In this embodiment, the mobile body can be equipped with sensors for data acquisition, such as LDS (Laser Distance Sensor), edge sensors, and image acquisition components (e.g., monocular camera, binocular camera, etc.). The number of each type of sensor can be one or more, and the installation locations of different sensors can be the same or different; this embodiment does not limit this. When the mobile body performs a specific task (e.g., cleaning task, delivery task, etc.), it can use its sensors to acquire data to obtain the data to be collected. For example, a target sensor can be used to collect data in real time within its acquisition range to obtain the data to be identified.

[0041] The data to be identified can be data for obstacle identification, which can be point cloud data collected by the target sensor. The moving body can perform object identification on the data to be identified, obtaining object data of the target object (which can be an obstacle that can affect the movement of the moving body, for example, an obstacle whose size meets a preset size threshold) within the acquisition range of the target sensor. The number of target objects can be one or more. By extracting features from the object data of the target objects, the object features of the target objects are obtained.

[0042] There are various ways to perform object recognition. For example, point cloud data can be clustered using a clustering algorithm to obtain object data of the target object. Alternatively, the data to be recognized can be input into a pre-trained recognition model to obtain the object data of the target object output by the recognition model. Other methods can also be used for object recognition. In this embodiment, the method of object recognition is not limited.

[0043] Step S204: Determine the matching object that matches the target object from a preset set of reference objects.

[0044] In related technologies, for objects with poor real-time data, it is difficult for sensors to collect complete object data. The movement path planned for the moving body based on incomplete object data may be too close to objects in the area, making it easy to collide with objects. This not only affects the movement safety of the moving body, but the collision will also cause damage to the object and affect its service life.

[0045] For example, when dealing with obstacles like table or chair legs, the LDS (Local Disk System) may scan a circle around these obstacles but not necessarily detect any points on them. Furthermore, if the obstacle is in the LDS's blind spot, it may sometimes be completely missed. If the robot vacuum plans its path based on these detected obstacles, collisions are likely to occur, damaging the furniture and resulting in a poor user experience.

[0046] To at least partially address the aforementioned issues, in this embodiment, a set of object features for reference objects can be pre-defined. If the object features of the target object collected by the sensor in real time are incomplete, the pre-defined object features can be used to complete them, thereby reducing collisions caused by incomplete data, facilitating path planning, improving the efficiency of the moving object in performing tasks (e.g., cleaning efficiency), and enhancing the user experience.

[0047] For example, the point cloud information of chair and table legs has obvious features. When the robot vacuum cleaner is cleaning, it records these characteristic shape information in the global map. For example, it records the shape of a chair leg, the size of the chair leg, the spacing between the chair legs, and other feature information.

[0048] When performing cleaning tasks, due to blind spots, the real-time data collected by the robot vacuum cleaner is not good, and there may be incomplete feature information (i.e., object features) of chairs (or tables). In this case, the saved feature information can be called to complete the feature information of the chair and table legs. That is, the real-time feature information is completed by using the pre-stored complete feature information of the chair (or table).

[0049] Upon identifying a target object, a matching object can be determined from a pre-defined set of reference objects (which may be a pre-set set of obstacles). The matching object can be determined based on the similarity between object data, the similarity between object features, or other methods. Here, object data refers to the collected raw data, such as point cloud data, and object features refer to the object features extracted from the collected raw data, such as shape features.

[0050] Optionally, there may be multiple objects of the same type within the same scene. If each object of the same type were to have its own object features stored, it would consume significant storage resources, resulting in a waste of those resources. In some embodiments, object data can be stored according to object type; that is, for objects of the same type, a corresponding object feature can be stored. Correspondingly, the object features of each reference object are pre-stored object features corresponding to the object type of each reference object.

[0051] For example, based on the complete feature information of each chair (or table) in the global map, we can analyze the types of chairs (or tables) involved in the global map and the number of chairs (or tables) of each type. The feature information of chairs (or tables) of the same type is the same, so the pre-saved feature information set can be stored according to the type of chair (or table) to obtain the feature information subset corresponding to each type of chair (or table) (i.e., the object features of each reference object).

[0052] Step S206: If it is determined from the object features of the matched object that the object features of the target object are missing, the object features of the matched object are used to complete the object features of the target object, which is then used as the obstacle recognition result.

[0053] Based on the object features of the matched object, it can be determined that the target object's object features are incomplete; that is, the target object's object features are not complete, such as missing component features or missing line features of a component. If the object features of the matched object determine that the target object's object features are incomplete, then the object features of the matched object can be used to complete the target object's object features, for example, by completing missing component features or missing line features, thus obtaining the complete object features of the target object. The completed object features of the target object can be used as the recognition result for obstacle recognition, which can then be used in subsequent tasks such as path planning for moving objects.

[0054] For example, a robot vacuum cleaner can save the shape data of table and chair legs in advance. When the real-time data is incomplete, it can roughly determine the shape of the leg and use the saved data to complete the shape, thereby reducing collisions caused by incomplete data, facilitating path planning, improving the cleaning efficiency of the robot vacuum cleaner, and enhancing the user experience.

[0055] It should be noted that the target object may not match any of the reference objects in the set; that is, the target object may not belong to the set of reference objects. For example, the target object may not be furniture with long, thin legs. In this case, movement path planning can be performed directly based on the target object's object features. Furthermore, if it is determined from the object features of the matching objects that the target object's object features are not missing, movement path planning can also be performed directly based on the target object's object features.

[0056] Through steps S202 to S206 above, the object features of real-time detected obstacles are completed by using pre-stored object features. When the detected object features are incomplete, the pre-stored object features are used to complete the real-time detected object features, and the completed object features are output as the obstacle recognition result. This can improve the completeness of the object data of obstacles. Based on the complete object data, the movement route planning of the moving body can reduce the possibility of the moving body overturning or other dangers due to the distance between it and the object, improve the movement safety of the moving body, and solve the problem of low movement safety of the moving body caused by the inability to accurately identify obstacles.

[0057] In one exemplary embodiment, determining a matching object from a preset set of reference objects that matches the target object includes:

[0058] S11, among a set of reference objects, those whose object features are more than or equal to the object features of the target object are identified as matching objects.

[0059] In this embodiment, a set of matching objects that match the target object can be determined based on the similarity between object features. For example, the similarity between the object features of the target object and the object features of each reference object can be determined. The similarity here can be used to represent the shape similarity between the target object and the reference objects, and can include, but is not limited to, at least one of the following similarities: length similarity, width similarity, height similarity, object boundary similarity, etc., or other types of similarity.

[0060] To improve the accuracy of object matching, a similarity matching threshold can be preset, i.e., a preset similarity threshold. If the similarity between the object features of the reference object and the object features of the target object is greater than or equal to the preset similarity threshold, the reference object and the target object can be matched; otherwise, they are determined not to match. If there is a reference object in a set of reference objects that matches the target object, then that reference object can be determined as the matched object.

[0061] Optionally, if there are multiple reference objects that match the target object in a set of reference objects, the reference object with the highest similarity between the object features among the multiple reference objects can be determined as the matching object; or, a prompt message can be issued to the user, and the matching object can be determined based on the user's selection; or, the matching object can be determined by combining the similarity between other features, which is not limited in this embodiment.

[0062] In this embodiment, the accuracy of matching objects can be improved by determining matching objects based on a preset similarity threshold and the similarity between object features.

[0063] In an exemplary embodiment, determining reference objects from a set of reference objects whose object features are similar to the object features of the target object to a preset similarity threshold as matched objects includes:

[0064] S21, In the case that there are related objects in a set of reference objects that are associated with the object position of the target object, determine the similarity between the object features of the target object and the object features of the related objects.

[0065] S22, if the similarity between the object features of the associated object and the object features of the target object is greater than or equal to a preset similarity threshold, the associated object is identified as the matching object.

[0066] To improve the efficiency of object matching, object matching can be performed based on object position. For a set of reference objects, those that do not change frequently or whose positional changes are small can be associated with their usual locations or regions. In addition, for reference objects whose positions or regions change frequently, their positional change characteristics can be recorded (to record the change patterns of reference objects). Thus, a location or region can be associated with one or more reference objects.

[0067] For example, based on multiple scans, the positional change characteristics of various types of chairs (or tables) can be analyzed. For chair (or table) types that do not change frequently or whose positional changes are generally minor, their frequently placed locations or areas can be recorded in the global map. Furthermore, for chair (or table) types that frequently change location or area, the range of areas involved in the changes (e.g., frequently changing location in the living room or dining room, but rarely appearing in the bedroom) and change characteristics (e.g., random location, or mostly changing location in the dining room, occasionally appearing in the living room) can also be recorded as a reference for subsequent object recognition.

[0068] For a target object, based on its position, it can be determined whether there are any associated objects among a set of reference objects that are related to the target object's position. If so, the similarity between the target object's features and the associated objects' features can be determined first. If there are multiple associated objects, the similarity between each associated object and the target object's features can be determined separately.

[0069] If the similarity between the object features of an associated object and the object features of the target object is greater than or equal to a preset similarity threshold, the associated object can be directly identified as a matching object, ignoring all other reference objects in the set. If there are multiple associated objects, the similarity between the object features of each associated object and the target object can be determined separately to be greater than or equal to the preset similarity threshold. If the similarity between the object features of one associated object and the target object is greater than or equal to the preset similarity threshold, it can be directly identified as a matching object. If the similarity between the object features of multiple associated objects and the target object is greater than or equal to the preset similarity threshold, a similar method to that described above can be used to determine the matching objects, which will not be repeated here.

[0070] Optionally, if the similarity between the object features of the associated object and the object features of the target object is less than a preset similarity threshold, the similarity between the object features of other reference objects (excluding the associated object) and the object features of the target object can be determined, and then the matching object can be determined based on the determined similarity.

[0071] For example, for chair (or table) types whose location information does not change frequently or whose location information changes little, when the corresponding location is reached or near the corresponding location, the feature information of the chair (or table) associated with that location can be retrieved, and partial feature comparison can be performed on the chair (or table) data scanned in real time. If the feature comparison results meet certain requirements, the retrieved complete feature information can be used to complete the feature information collected in real time.

[0072] In this embodiment, by pre-configuring the correspondence between reference objects and their usual locations, the associated reference objects are determined based on the positions of objects detected in real time, and the matching objects are determined based on the similarity between the object features of the associated reference objects and the objects detected in real time, which can improve the efficiency of matching object determination.

[0073] In one exemplary embodiment, the above method further includes:

[0074] S31, among a group of reference objects, the reference object whose distance between the associated position and the object position of the target object is less than or equal to a preset distance threshold is identified as an associated object.

[0075] Because there is a difference between the determined object position and the actual object position, a distance threshold can be preset. If the distance between the target object's position and the associated position of the reference object is less than or equal to the preset distance threshold, the two can be considered a match. Based on this, reference objects in a set whose associated positions are less than or equal to the preset distance threshold can be identified as associated objects.

[0076] Furthermore, the location associated with the reference object may be a location region, and the distance between the object position of the target object and the location associated with the reference object may be the shortest distance between the object position of the target object and the location region associated with the reference object.

[0077] In this embodiment, by using the preset distance threshold and determining the associated object based on the relationship between the distance between the location of the reference object and the object location of the target object and the preset distance threshold, the rationality of the determination of the associated object can be improved.

[0078] In an exemplary embodiment, determining reference objects from a set of reference objects whose object features are similar to the object features of the target object to a preset similarity threshold as matched objects includes:

[0079] S41, In the case that there are no related objects in a set of reference objects that are associated with the object position of the target object, determine the similarity between the object features of each reference object in the set of reference objects and the object features of the target object, and use it as the similarity to each reference object.

[0080] S42, among a group of reference objects, the reference object with the highest similarity that is greater than or equal to the preset similarity threshold is identified as the matching object.

[0081] If there are no related objects in a set of reference objects that are associated with the object position of the target object, the similarity between the object features of each reference object and the object features of the target object can be determined separately, and used as the similarity score corresponding to each reference object.

[0082] After determining the similarity to each reference object, it can be determined whether there exists a reference object in the set with a similarity greater than or equal to a preset similarity threshold. If there is one such reference object, it can be identified as the matching object. If there are multiple such reference objects, a similar method can be used to identify the matching object from among the multiple reference objects. Alternatively, if no matching object exists, the object features of the target object can be directly used for path planning of the moving body, as already explained, and will not be repeated here.

[0083] For example, for chairs (or tables) whose location information changes frequently and whose location deviations are usually large, the real-time collected feature data can be compared with the pre-stored feature information set. Based on the comparison results, the chair (or table) type corresponding to the real-time collected feature data can be determined, and then the real-time collected feature data can be completed based on the feature information subset corresponding to the chair (or table) type.

[0084] In this embodiment, when it is determined that there are no associated objects based on the object's location, the reference object with the highest object feature similarity and reaching a preset similarity threshold is identified as the matching object, which can improve the efficiency of matching object determination.

[0085] In one exemplary embodiment, before determining the matching object that matches the target object from a preset set of reference objects, the method further includes at least one of the following:

[0086] S51, fuse the sensor data collected by a set of sensors on the moving body to obtain the object features of each reference object in a set of reference objects, the set of sensors including the target sensor.

[0087] S52, fuse the sensor data collected from multiple data acquisitions of the target sensor to obtain the object features of each reference object in a set of reference objects.

[0088] The object features of a set of reference objects can be obtained in advance. One way to obtain these features is by acquiring sensor data through a single data acquisition operation using the target sensor, and then extracting object features from that data. However, since a single object scan rarely captures complete object data, the extracted object features may be incomplete, reducing the accuracy of subsequent object recognition.

[0089] In this embodiment, in order to improve the completeness of obtaining the object features of the reference object, the object features of the reference object can be obtained through at least one of the following methods:

[0090] Method 1: A set of sensors can be deployed on the moving body. The moving body can fuse the sensor data collected by the set of sensors to obtain the object features of each reference object.

[0091] To ensure the completeness of object data acquisition, each sensor in a set of sensors deployed on the moving body can scan the same object separately, obtaining a set of sensor data corresponding to the same object. Different sensors have different acquisition angles, and the data types acquired are different. By fusing the sensor data acquired by different sensors, object data corresponding to the same reference object can be obtained from different acquisition angles or parameters. By fusing the sensor data acquired by a set of sensors, complete object features of the same reference object can be obtained.

[0092] When fusing sensor data collected from different sensors, the data can first be preprocessed to align the data from each sensor. Then, data transformation is performed on the data from each sensor, and the data is fused in the transform domain. Finally, an inverse image transform is performed on the fused data to obtain the object data of the fused reference object. After obtaining the fused reference object data, object features (e.g., shape features) are extracted to obtain the object features of the reference object.

[0093] For example, data from multiple sensors, such as laser data and AI (Artificial Intelligence) data, can be fused to obtain complete feature information of a chair (or table).

[0094] Method 2: The sensor data collected from the target sensor through multiple data acquisitions are fused to obtain the object features of each reference object in a set of reference objects.

[0095] To ensure the integrity of object data acquisition, multiple data acquisitions can be performed on the same object using a target sensor (optionally, a different sensor), and the sensor data acquired from the multiple acquisitions can be fused to obtain object data for each reference object.

[0096] When fusing sensor data acquired through multiple data acquisitions, the sensor data from each acquisition can be preprocessed first to align the data. The aligned sensor data is then fused, i.e., weighted superposition of object data at the same location, to obtain the fused object data of the reference object. After obtaining the fused reference object data, object features are extracted to obtain the object features of the reference object.

[0097] For example, the scan data obtained from multiple scans of a sensor can be fused to obtain complete feature information of a chair (or table).

[0098] In this embodiment, by fusing sensor data collected from different sensors or by fusing sensor data collected from multiple data acquisitions from the same sensor, object data of the reference object can be obtained, which can improve the comprehensiveness of object feature acquisition and also improve the flexibility of object feature acquisition.

[0099] In one exemplary embodiment, the object features of the target object are completed using the object features of the matching object, including:

[0100] S61, In the case that the first leg of the target object is missing an edge, the missing edge of the first leg is filled in using the leg feature in the matching object that matches the first leg.

[0101] S62, In the case that the second leg of the target object is missing, the second leg is completed by using the leg features in the matching object that match the second leg and the leg position of the second leg in the target object.

[0102] The object features used to complete the target object can be any feature of the target object. For example, for scanned tables and chairs, the features of the legs, seat, or tabletop can be completed. Considering the limited height of the moving object, whether or not the features of higher objects are completed will not affect the movement of the moving object. In this embodiment, only the leg features of the target object can be completed. Correspondingly, a pre-stored set of reference object features can be the complete object features of the target object or the leg features of the reference objects.

[0103] The missing features of the target object can be some legs of the target object (i.e., the legs are completely in the scanning blind zone and no feature information of the legs is scanned), or some edges of some legs of the target object (i.e., only one or more edges of the legs are scanned, but some edges are missing). The missing leg features of the target object can be completed by using the object features of the matching object.

[0104] If the first leg of a target object is missing an edge, the missing edge can be filled in using leg features from the matching object that match the first leg. During edge filling, the position and data of the missing edge can be determined based on the relative positions of the edges in the first leg, and the missing edge data can be filled in to the missing edge's position to obtain the filled first leg. There can be one or more first legs and missing edges, and similar methods can be used to fill in these missing edges.

[0105] For example, a robotic vacuum cleaner can use pre-saved shape data of table and chair legs to supplement the information of table and chair legs within a certain range of the robot body when real-time data is incomplete. On the one hand, it can supplement the information of a single leg, such as a square chair leg. However, LDS can only scan two sides at a time. It can use the pre-saved shape data of the chair leg to supplement the other two sides, that is, to supplement the three-dimensional data of a single leg.

[0106] If the second leg of the target object is completely missing, it can be completed using leg features from the matching object that match the second leg, along with the second leg's position within the target object. The leg position can be the second leg's position relative to a reference point on the target object, or it can be the position relative to other legs on the target object. The leg features from the matching object that match the second leg are then used to complete the second leg's position within the target object, resulting in a target object with a completed second leg. There can be one or more second legs, and similar methods can be used to complete such missing legs.

[0107] For example, if a robot vacuum only detects three legs of a chair and misses the other leg in a blind spot, the information about the four legs and their relative positions can be used to complete the scan.

[0108] In this embodiment, by matching the leg features of the object to complete the missing leg edges or missing legs, the resource consumption of feature completion can be guaranteed and the efficiency of feature completion can be improved. At the same time, since the legs of the object are completed, the risk of collision between the moving body and obstacles can be reduced, and the safety of the moving body can be improved.

[0109] The obstacle recognition method in this application embodiment will be explained below with reference to an optional example. In this optional example, the moving body is a robotic vacuum cleaner, a set of sensors includes an LDS and a camera, and the target object is a table and chairs.

[0110] This optional example provides a solution for completing the shape data of table and chair legs, such as... Figure 3 As shown, the obstacle recognition method in this optional example may include the following steps:

[0111] Step S302: Obtain and save the shape data of each table and chair leg in the user's home. The shape data of different table and chair legs (i.e., complete feature information) can be saved separately or saved according to the type of table and chair.

[0112] Step S304: If the shape data of the table and chair detected by the robot vacuum cleaner in real time is missing, the shape of the table and chair legs is determined and the saved shape data is used to complete the shape of the table and chair legs.

[0113] In most households, the number of table and chair legs is the same. When only a few table and chair legs are identified (for example, four chair legs, but only 2-3 chair legs are detected in real-time data), the saved data is used to complete the remaining table and chair legs that cannot be detected in real-time data.

[0114] In addition, some special objects, such as cabinet legs and bed legs, are also suitable for the above solutions. Considering that the positions of these objects are generally constant, completion is easier, and the legs of cabinets and beds are usually spaced far apart, the robot vacuum can treat each leg as a separate obstacle. Therefore, the completion solution in this optional example is often suitable for recognizing obstacles such as table and chair legs that frequently change position and whose gaps often obstruct the robot vacuum's movement.

[0115] In this optional example, the missing shape data of the table and chair legs is completed by using pre-saved shape data of the table and chair legs in real time. This can reduce collisions caused by incomplete shape data of the robot vacuum cleaner, facilitate path planning, improve the cleaning efficiency of the robot vacuum cleaner, and enhance the user experience.

[0116] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0118] According to another aspect of the embodiments of this application, an obstacle recognition device for implementing the above-described detection method is also provided. Figure 4 This is a structural block diagram of an optional obstacle recognition device according to an embodiment of this application, such as... Figure 4 As shown, the device may include:

[0119] The identification unit 402 is used to identify the target object by analyzing the data collected by the target sensor on the moving body.

[0120] The first determining unit 404 is connected to the recognizing unit 402 and is used to determine the matching object that matches the target object in a preset set of reference objects.

[0121] The completion unit 406 is connected to the first determining unit 404 and is used to complete the object features of the target object using the object features of the matching object when it is determined that the object features of the target object are missing based on the object features of the matching object, as an obstacle recognition result.

[0122] It should be noted that the identification unit 402 in this embodiment can be used to perform the above step S202, the first determination unit 404 in this embodiment can be used to perform the above step S204, and the completion unit 406 in this embodiment can be used to perform the above step S206.

[0123] The above module completes the object features of real-time detected obstacles by using pre-stored object features. When the detected object features are incomplete, the pre-stored object features are used to complete the real-time detected object features, and the completed object features are output as the obstacle recognition result. This can improve the completeness of the object data of obstacles. Based on the complete object data, the movement route planning of the moving body can reduce the possibility of the moving body tipping over due to the distance between it and the object, thus improving the movement safety of the moving body and solving the problem of low movement safety of the moving body caused by the inability to accurately identify obstacles.

[0124] In one exemplary embodiment, the first determining unit includes:

[0125] The first determining module is used to determine the reference objects in a set of reference objects whose similarity between the object features and the object features of the target object is greater than or equal to a preset similarity threshold as matching objects.

[0126] In one exemplary embodiment, the object features of each reference object in a set of reference objects are object features pre-stored according to the object type, corresponding to the object type of each reference object.

[0127] In one exemplary embodiment, the first determining module includes:

[0128] The first determination submodule is used to determine the similarity between the object features of the target object and the object features of the associated objects when there are associated objects in a set of reference objects that are related to the object position of the target object.

[0129] The second determination submodule is used to determine the associated object as the matching object when the similarity between the object features of the associated object and the object features of the target object is greater than or equal to a preset similarity threshold.

[0130] In one exemplary embodiment, the above-described apparatus further includes:

[0131] The second determining unit is used to determine the reference objects in a set of reference objects whose distance between the associated position and the object position of the target object is less than or equal to a preset distance threshold as associated objects.

[0132] In one exemplary embodiment, the first determining module includes:

[0133] The third determination submodule is used to determine the similarity between the object features of each reference object in a set of reference objects and the object features of the target object when there are no associated objects in a set of reference objects that are related to the object position of the target object. This similarity is used as the similarity between each reference object and the target object.

[0134] The fourth determination submodule is used to determine the reference object with the highest similarity among a set of reference objects as the matching object.

[0135] In one exemplary embodiment, the above-described apparatus further includes at least one of the following:

[0136] The first fusion unit is used to fuse sensor data collected by a set of sensors on the moving body before determining the matching object that matches the target object in a preset set of reference objects, so as to obtain the object features of each reference object in the set of reference objects, wherein the set of sensors includes the target sensor.

[0137] The second fusion unit is used to fuse the sensor data collected by the target sensor through multiple data acquisitions to obtain the object features of each reference object in a set of reference objects.

[0138] In one exemplary embodiment, the completion unit includes:

[0139] The first completion module is used to complete the missing edge of the first leg of the target object by using the leg features of the matching object that match the first leg.

[0140] The second completion module is used to complete the second leg when the second leg of the target object is missing, by using the leg features in the matching object that match the second leg and the leg position of the second leg in the target object.

[0141] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0142] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of any of the door frame detection methods described above in the embodiments of this application.

[0143] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0144] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0145] S1, perform object recognition on the target data collected by the target sensor on the moving body to obtain the target object;

[0146] S2, determine the matching object that matches the target object from a preset set of reference objects;

[0147] S3, if it is determined that the object features of the target object are missing based on the object features of the matched object, the object features of the matched object are used to complete the object features of the target object, which is then used as the obstacle recognition result.

[0148] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0149] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0150] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described door frame detection method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0151] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 5As shown, it includes a processor 502, a communication interface 504, a memory 506, and a communication bus 508. The processor 502, communication interface 504, and memory 506 communicate with each other via the communication bus 508.

[0152] Memory 506 is used to store computer programs;

[0153] When processor 502 executes a computer program stored in memory 506, it performs the following steps:

[0154] S1, perform object recognition on the target data collected by the target sensor on the moving body to obtain the target object;

[0155] S2, determine the matching object that matches the target object from a preset set of reference objects;

[0156] S3, if it is determined that the object features of the target object are missing based on the object features of the matched object, the object features of the matched object are used to complete the object features of the target object, which is then used as the obstacle recognition result.

[0157] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 5 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0158] The aforementioned memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0159] As an example, the memory 506 described above may include, but is not limited to, the identification unit 402, the first determination unit 404, and the completion unit 406 in the door frame detection device. Furthermore, it may include, but is not limited to, other module units in the door frame detection device described above, which will not be elaborated upon in this example.

[0160] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0161] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0162] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. The device that implements the above door frame detection method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, handheld computer, mobile internet device (MID), PAD, etc. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0163] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0164] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0165] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0166] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0169] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An obstacle recognition method characterized by, include: The target object is identified by performing object recognition on the target sensor collected on the moving body. Identify the matching object that matches the target object from a preset set of reference objects; If, based on the object features of the matched object, it is determined that the object features of the target object are missing, the object features of the matched object are used to complete the object features of the target object, and this completeness is used as the obstacle recognition result. The matching objects that match the target object from the predetermined set of reference objects include: Among the set of reference objects, those whose object features are similar to the object features of the target object to a preset similarity threshold are identified as the matching objects. The step of determining the reference objects in the set of reference objects whose similarity to the object features of the target object is greater than or equal to a preset similarity threshold as the matching objects includes: If there is an associated object in the set of reference objects that is related to the object position of the target object, determine the similarity between the object features of the target object and the object features of the associated object. If the similarity between the object features of the associated object and the object features of the target object is greater than or equal to the similarity threshold, the associated object is identified as the matching object.

2. The method of claim 1, wherein, The object features of each reference object in the set of reference objects are object features that are pre-saved according to the object type and correspond to the object type of each reference object.

3. The method according to claim 1, characterized in that, The method further includes: Among the set of reference objects, the reference object whose distance between its associated position and the object position of the target object is less than or equal to a preset distance threshold is identified as the associated object.

4. The method according to claim 1, characterized in that, The step of determining the reference objects in the set of reference objects whose similarity to the object features of the target object is greater than or equal to a preset similarity threshold as the matching objects includes: If there are no associated objects in the set of reference objects that are related to the object position of the target object, the similarity between the object features of each reference object in the set of reference objects and the object features of the target object is determined as the similarity to each reference object. Among the set of reference objects, the reference object with the highest similarity that is greater than or equal to the preset similarity threshold is identified as the matching object.

5. The method according to any one of claims 1 to 4, characterized in that, Before determining the matching object that matches the target object from a preset set of reference objects, the method further includes at least one of the following: The sensor data collected by a set of sensors on the moving body are fused to obtain the object features of each reference object in the set of reference objects, wherein the set of sensors includes the target sensor; The sensor data collected from the target sensor through multiple data acquisitions are fused to obtain the object features of each reference object in the set of reference objects.

6. The method according to any one of claims 1 to 4, characterized in that, The step of using the object features of the matched object to complete the object features of the target object includes: In the case where the first leg of the target object is missing an edge, the missing edge of the first leg is filled in using the leg features in the matching object that match the first leg. In the case where the second leg of the target object is missing, the second leg is completed using the leg features in the matching object that match the second leg and the leg position of the second leg in the target object.

7. An obstacle recognition device, characterized in that, include: The identification unit is used to identify the target object by analyzing the data collected by the target sensor on the moving body. The first determining unit is used to determine the matching object that matches the target object from a preset set of reference objects; The completion unit is used to complete the object features of the target object using the object features of the matching object when it is determined from the object features of the matching object that the object features of the target object are missing, and this completion serves as the obstacle recognition result. The first determining unit is further configured to: determine the reference objects in the set of reference objects whose object features are similar to the object features of the target object to a preset similarity threshold as the matching objects. The first determining unit is further configured to: determine the similarity between the object features of the target object and the object features of the associated object when there is an associated object in the set of reference objects that is associated with the object position of the target object; The similarity between the object features of the associated object and the object features of the target object is high.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 6 through the computer program.

Images