Sweeping robot positioning and navigation system based on vision and laser fusion

Abstract

The invention provides a sweeping robot positioning and navigation system based on vision and laser fusion, and the system comprises a first collection module which carries out the space information acquisition of a home space through LDS laser, and obtains first collection information; the second acquisition module adopts a camera device to perform visual information acquisition on the home space to obtain second acquisition information; performing fusion analysis on the first collection information and the second collection information through a fusion analysis module, determining a home space initial model, and performing region condition setting to obtain a home space target model; and the positioning navigation module determines a cleaning scheme based on the home space target model, performs cleaning control and navigation prompt, performs identification and positioning according to the first acquisition information and the second acquisition information, and performs cleaning control adjustment and navigation prompt update in combination with an identification and positioning result. According to the invention, hybrid positioning navigation in two modes is realized, errors of positioning navigation are reduced, cleaning efficiency of the sweeping robot is improved, and more convenience is brought to users.

Description

Technical Field

[0001] The present invention relates to the technical field of positioning and navigation, and particularly relates to a positioning and navigation system for a floor cleaning robot based on the fusion of vision and laser. Background Art

[0002] With the development of social productivity and the improvement of economic level, the social demand for robot products is increasing day by day, and various robot technologies emerge as the times require. As one of the most commonly used household appliances, the floor cleaning robot, also known as the lazy floor sweeper, is an intelligent household appliance that can automatically vacuum the floor, and can automatically locate and navigate in the home space, so as to realize the intelligent cleaning of the home space.

[0003] At present, the positioning and navigation system for the floor cleaning robot usually uses a single method for positioning and navigation, resulting in a large positioning and navigation error, which affects the cleaning efficiency of the floor cleaning robot. Therefore, the present invention proposes a positioning and navigation system for a floor cleaning robot based on the fusion of vision and laser, which combines vision and laser to achieve hybrid positioning and navigation, reduces the positioning and navigation error, enables the floor cleaning robot to better clean the home space, improves the cleaning efficiency of the floor cleaning robot, and brings more convenience to users. Summary of the Invention

[0004] The purpose of the present invention is to provide a positioning and navigation system for a floor cleaning robot based on the fusion of vision and laser to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A positioning and navigation system for a floor cleaning robot based on the fusion of vision and laser, comprising: a first acquisition module, a second acquisition module, a fusion analysis module, and a positioning and navigation module;

[0006] The first acquisition module is used to obtain spatial information of the home space by using LDS laser to obtain the first acquisition information;

[0007] The second acquisition module is used to collect visual information of the home space by using a camera device to obtain the second acquisition information;

[0008] The fusion analysis module is used to fuse and analyze the first acquisition information and the second acquisition information, determine the initial model of the home space, and set regional conditions based on the initial model of the home space to obtain the target model of the home space;

[0009] The positioning and navigation module is used to determine the cleaning plan based on the target model of the home space, perform cleaning control and navigation prompts for the floor cleaning robot according to the cleaning plan, and at the same time identify and locate according to the first acquisition information and the second acquisition information, and then adjust the cleaning control and update the navigation prompts in combination with the recognition and positioning results.

[0010] Further, the first acquisition module includes: a first acquisition and control unit and a first monitoring unit;

[0011] The first acquisition and control unit is used to control the LDS laser to perform comprehensive information acquisition on the home space, obtain a first acquisition control signal, the LDS laser emits a first laser pulse according to the first acquisition control signal, and based on the first laser pulse scattered in all directions, a laser acquisition signal is obtained, and the first laser acquisition information is obtained according to the laser acquisition signal;

[0012] The first monitoring unit is used to determine a target monitoring area based on the current position of the sweeping robot, combine the target monitoring area to control the LDS laser to emit a second laser pulse, and use the second laser pulse to perform real-time information acquisition on the target monitoring area to obtain laser real-time acquisition information.

[0013] Further, the second acquisition module includes: a second acquisition and control unit and a second monitoring unit;

[0014] The second acquisition and control unit is used to obtain a second acquisition control signal, and according to the second acquisition control signal, control the imaging device to perform omnidirectional visual information acquisition on the home space to obtain the first visual acquisition information;

[0015] The second monitoring unit is used to determine a target monitoring direction based on the current position of the sweeping robot, and perform visual real-time acquisition using the control imaging device based on the target monitoring direction to obtain a target monitoring image, and obtain visual real-time acquisition information according to the target monitoring image.

[0016] Further, the fusion analysis module includes: a first processing unit, a second processing unit, a fusion analysis unit, and a condition setting unit;

[0017] The first processing unit is used to perform signal parsing and optimization on the first laser acquisition information to obtain the first acquisition optimization information;

[0018] The second processing unit is used to perform image optimization on the first visual acquisition information to obtain the second acquisition optimization information;

[0019] The fusion analysis unit is used to perform information fusion analysis according to the first acquisition optimization information and the second acquisition optimization information, construct a home space diagram, and obtain an initial home space model;

[0020] The condition setting unit is used to perform limitation in the initial home space model according to the area condition setting information to obtain a target home space model.

[0021] Further, the fusion analysis unit performs information fusion analysis according to the first acquisition optimization information and the second acquisition optimization information, including:

[0022] Perform home space distribution analysis based on the first acquisition optimization information, determine the items and item distribution information in the home space, create models for the items to obtain item models, then build models for the item models according to the item distribution to obtain a home space diagram, and obtain the first initial model of the home space after confirming the home space diagram;

[0023] Perform image acquisition information analysis for the second acquisition optimization information, determine the acquisition orientation of the image, perform image stitching according to the acquisition orientation of the image, determine the image combination information, perform three-dimensional image conversion based on the image combination information, determine the three-dimensional image information of the home space, and build a model according to the three-dimensional image information of the home space to obtain the second initial model of the home space;

[0024] Use a feature analysis model to perform feature analysis on the first initial model of the home space and the second initial model of the home space to obtain the first model features and the second model features, and determine the third initial model of the home space according to the first model features and the second model features.

[0025] Furthermore, determining the third initial model of the home space according to the first model features and the second model features includes:

[0026] Perform feature matching on the first model features and the second model features, analyze the feature similarity between the first model features and the second model features based on the same feature points, and determine whether the model features match according to the feature similarity to obtain a feature matching result;

[0027] Screen out the matching model features according to the feature matching result, and determine the first model based on the matching model features to obtain the architecture of the third initial model of the home space;

[0028] Analyze the advantageous features of the LDS laser acquisition method according to the first model features in combination with the matching model features to determine the first distinguishing feature;

[0029] Supplement the feature model in the architecture of the third initial model of the home space according to the first distinguishing feature to obtain the first processed third initial model of the home space;

[0030] Analyze the advantageous features of the camera device acquisition method according to the second model features in combination with the matching model features to determine the second distinguishing feature;

[0031] Supplement the feature model in the first processed third initial model of the home space according to the second distinguishing feature to obtain the second processed third initial model of the home space.

[0032] Further, the condition setting unit performs limitation in the initial model of the home space according to the area condition setting information, including:

[0033] Conduct no-go zone and boundary analysis for the home space to determine the no-go zone and boundary of the home space;

[0034] Obtain additional condition information of the user through the interaction terminal;

[0035] Perform limitation in the initial model of the home space according to the no-go zone and boundary of the home space and the attachment condition information to obtain the target model of the home space.

[0036] Further, the positioning and navigation module includes: a scheme analysis unit, an identification and positioning unit, and an updated navigation unit;

[0037] The scheme analysis unit is used to determine a cleaning scheme for the target model of the home space in combination with the cleaning attributes of the floor cleaning robot, and perform cleaning path planning for the cleaning scheme to obtain the cleaning control information of the floor cleaning robot;

[0038] The identification and positioning unit is used to perform preliminary analysis based on the laser real-time acquisition information and the visual real-time acquisition information to determine the position of the floor cleaning robot in the home space, obtain the current position of the floor cleaning robot, and further analyze the laser real-time acquisition information and the visual real-time acquisition information in combination with the current position of the floor cleaning robot to determine whether there are obstacles in the forward direction of the floor cleaning robot, and identify the obstacles in the case of obstacles to obtain the forward identification information;

[0039] The updated navigation unit is used to perform cleaning control and navigation prompts for the floor cleaning robot according to the cleaning control information of the floor cleaning robot, and at the same time, after the identification and positioning unit obtains the current position and the forward identification information of the floor cleaning robot, adjust and update the cleaning control and navigation prompts.

[0040] Further, the scheme analysis unit determines a cleaning scheme for the target model of the home space in combination with the cleaning attributes of the floor cleaning robot, and performs cleaning path planning for the cleaning scheme, including:

[0041] Determine whether there are specific area restrictions. When there are specific area restrictions, perform static space analysis for the specific area in the target model of the home space to determine the area to be cleaned; when there are no specific area restrictions, perform overall static space analysis for the target model of the home space to determine the area to be cleaned;

[0042] Obtain the cleaning attribute information of the floor cleaning robot to determine the effective cleaning area size and the cleaning power range;

[0043] Analyze the items in the target area in the home space target model according to the area to be cleaned, determine the items existing in the area to be cleaned, and obtain the cleaning characteristics of the items;

[0044] Combine the cleaning characteristics of the items with the cleaning power range to conduct an effective cleaning intensity analysis, and determine the cleaning strategy for the area to be cleaned based on the effective cleaning intensity;

[0045] Perform a full-coverage path planning for the area to be cleaned in combination with the size of the effective cleaning area, and determine the cleaning path of the sweeping robot;

[0046] Obtain the cleaning control information of the sweeping robot according to the cleaning strategy of the area to be cleaned and the cleaning path of the sweeping robot.

[0047] Furthermore, the updated navigation unit adjusts and updates for cleaning control and navigation prompts, including:

[0048] Determine the navigation prompt information according to the current position of the sweeping robot in combination with the cleaning path of the sweeping robot, and conduct navigation prompts according to the navigation prompt information;

[0049] Conduct an obstacle analysis for the forward recognition information to determine whether the obstacle is an item that the sweeping robot can directly clean;

[0050] When the obstacle is not an item that the sweeping robot can directly clean, obtain the size information of the obstacle, conduct an avoidance path analysis according to the size information of the obstacle in combination with the cleaning path of the sweeping robot, determine the avoidance navigation based on the avoidance path, and update the navigation prompt according to the avoidance navigation;

[0051] When the obstacle is an item that the sweeping robot can directly clean, determine the mode adjustment information for the sweeping robot according to the characteristics of the obstacle to obtain the mode adjustment information, then combine the obstacle with the cleaning strategy of the area to be cleaned to determine the target cleaning intensity, conduct a cleaning intensity adjustment analysis according to the target cleaning intensity in combination with the current cleaning intensity to determine the cleaning intensity adjustment information, and at the same time obtain the size information of the obstacle, predict the cleaning time of the obstacle according to the size information of the obstacle in combination with the cleaning path of the sweeping robot to determine the predicted cleaning time of the obstacle, and then conduct a cleaning control adjustment for the sweeping robot according to the mode adjustment information, the cleaning intensity adjustment information, and the predicted cleaning time of the obstacle.

[0052] The present invention realizes the synchronous information collection of LDS laser and camera devices for the home space through the first collection module and the second collection module. When determining the target model of the home space in the fusion analysis module, the first collection information and the second collection information are fused and analyzed, so that the initial model of the home space can be obtained from the information of the home space collected by two methods. Therefore, in the positioning and navigation module, hybrid positioning and navigation in two ways can be realized, eliminating the disadvantages of a single method, improving the accuracy of positioning and navigation, and at the same time enabling the sweeping robot to clean the home space more efficiently, bringing more convenience to users.

[0053] Other features and advantages of the present invention will be described in the following specification, and part of them will be obvious from the specification or understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structure specifically pointed out in the application documents.

[0054] The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments. Description of the Drawings

[0055] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the accompanying drawings:

[0056] Figure 1 is a schematic diagram of the positioning and navigation system of the sweeping robot described in the present invention;

[0057] Figure 2 is a schematic diagram of the first collection module in the positioning and navigation system of the sweeping robot described in the present invention;

[0058] Figure 3 is a schematic diagram of the second collection module in the positioning and navigation system of the sweeping robot described in the present invention;

[0059] Figure 4 is a schematic diagram of the fusion analysis module in the positioning and navigation system of the sweeping robot described in the present invention;

[0060] Figure 5 is a schematic diagram of the positioning and navigation module in the positioning and navigation system of the sweeping robot described in the present invention. Detailed Embodiments

[0061] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0062] As Figure 1As shown in the figure, an embodiment of the present invention provides a positioning and navigation system for a sweeping robot based on vision and laser fusion, including: a first acquisition module, a second acquisition module, a fusion analysis module, and a positioning and navigation module;

[0063] The first acquisition module is used to obtain spatial information of the home space by using an LDS laser, and obtain the first acquisition information;

[0064] The second acquisition module is used to collect visual information of the home space by using a camera device, and obtain the second acquisition information;

[0065] The fusion analysis module is used to fuse and analyze the first acquisition information and the second acquisition information, determine the initial model of the home space, and perform regional condition setting based on the initial model of the home space to obtain the target model of the home space;

[0066] The positioning and navigation module is used to determine the cleaning plan based on the target model of the home space, perform cleaning control and navigation prompts for the sweeping robot according to the cleaning plan, simultaneously identify and locate according to the first acquisition information and the second acquisition information, and then adjust the cleaning control and update the navigation prompts in combination with the identification and positioning results.

[0067] In the above technical solution, the first acquisition module and the second acquisition module are connected to the fusion analysis module, and at the same time, the first acquisition module and the second acquisition module are also connected to the positioning and navigation module.

[0068] In the above technical solution, the regional condition setting includes: restricted area definition, boundary setting, cleaning preference, etc.

[0069] In the above technical solution, the camera device is usually a camera with a small body volume and high pixels, or other visual acquisition devices.

[0070] In the above technical solution, the first acquisition module and the second acquisition module are used to synchronously collect information of the home space by the LDS laser and the camera device, so that when the fusion analysis module determines the target model of the home space, the first acquisition information and the second acquisition information are fused and analyzed, so that the initial model of the home space can be obtained from the information of the home space collected by the two methods, so that a hybrid positioning and navigation of the two methods can be realized in the positioning and navigation module, eliminating the disadvantages of a single method, improving the accuracy of positioning and navigation, and at the same time enabling the sweeping robot to clean the home space more efficiently, bringing more convenience to users.

[0071] As Figure 2 shown, in an embodiment provided by the present invention, the first acquisition module includes: a first acquisition control unit and a first monitoring unit;

[0072] The first acquisition and control unit is used to control the LDS laser to comprehensively collect information about the home space, obtain the first acquisition control signal. The LDS laser emits the first laser pulse according to the first acquisition control signal, and based on the first laser pulse scattered in all directions, obtains the laser acquisition signal, and obtains the first laser acquisition information according to the laser acquisition signal;

[0073] The first monitoring unit is used to determine the target monitoring area based on the current position of the sweeping robot, combine the target monitoring area to control the LDS laser to emit the second laser pulse, and use the second laser pulse to perform real-time information collection on the target monitoring area to obtain the laser real-time acquisition information.

[0074] In the above technical solution, the first acquisition information includes: the first laser acquisition information and the laser real-time acquisition information.

[0075] In the above technical solution, the first laser pulse realizes global information collection, and the second laser pulse realizes local area information collection.

[0076] In the above technical solution, the target monitoring area is dynamically changing.

[0077] The above technical solution realizes information collection based on the LDS laser acquisition method through the first acquisition module. It can not only make the collected information accurate and ensure the accuracy of the first laser acquisition information and the laser real-time acquisition information, but also obtain the first laser acquisition information and the laser real-time acquisition information in a short time, improve the efficiency of the first acquisition module for information collection. Moreover, through the first acquisition and control unit and the first monitoring unit, different controls of the LDS laser can be realized, so that different laser signal collections can be carried out based on different situations, improve the performance of the first acquisition module, and through the first monitoring unit, real-time local area monitoring of the current situation of the sweeping robot can be realized, understand the current situation of the sweeping robot, and provide convenience for subsequent positioning and navigation.

[0078] As Figure 3 shown, in an embodiment provided by the present invention, the second acquisition module includes: a second acquisition and control unit and a second monitoring unit;

[0079] The second acquisition and control unit is used to obtain the second acquisition control signal, and according to the second acquisition control signal, control the camera device to perform omnidirectional visual information collection on the home space to obtain the first visual acquisition information;

[0080] The second monitoring unit is used to determine the target monitoring direction based on the current position of the sweeping robot, and based on the target monitoring direction, use the control camera device to perform visual real-time collection to obtain the target monitoring image, and obtain the visual real-time acquisition information according to the target monitoring image.

[0081] In the above technical solution, the second acquisition information includes: the first visual acquisition information and the real-time visual acquisition information.

[0082] In the above technical solution, the target monitoring direction is dynamically changing, which is related to the forward direction of the sweeping robot.

[0083] In the above technical solution, the visual information acquisition of the second acquisition module is realized through the imaging device, so that the information acquisition of the home space based on vision can collect all items in the home space, avoid missing acquisition data, ensure the comprehensiveness of the second acquisition data, and the second monitoring unit can realize the real-time monitoring of the target direction of the current situation of the sweeping robot, so that the positioning and navigation can be adjusted according to the current situation of the sweeping robot, and avoid anomalies during the cleaning process of the sweeping robot.

[0084] As Figure 4 shown, in an embodiment provided by the present invention, the fusion analysis module includes: a first processing unit, a second processing unit, a fusion analysis unit, and a condition setting unit;

[0085] The first processing unit is used to perform signal parsing and optimization on the first laser acquisition information to obtain the first acquisition optimization information;

[0086] The second processing unit is used to perform image optimization on the first visual acquisition information to obtain the second acquisition optimization information;

[0087] The fusion analysis unit is used to perform information fusion analysis according to the first acquisition optimization information and the second acquisition optimization information, construct a home space diagram, and obtain an initial home space model;

[0088] The condition setting unit is used to perform limitation in the initial home space model according to the area condition setting information to obtain the target home space model.

[0089] In the above technical solution, performing signal parsing and optimization on the first laser acquisition information includes: performing signal parsing on the first laser acquisition information, decomposing the first laser acquisition information into multiple acquisition signals, and then performing signal optimization on the acquisition signals, and using the acquisition signals after signal optimization as the first acquisition optimization information, wherein the signal optimization includes: noise reduction, enhancement, filtering, etc.

[0090] In the above technical solution, the image optimization includes: grayscale conversion, binarization, noise elimination, image enhancement, etc.

[0091] In the above technical solution, the area condition setting information can be originally defined in the home space or pre-set by the user.

[0092] The above technical solution optimizes the first laser acquisition information and the first visual acquisition information through the first processing unit and the second processing unit respectively, reduces the interference of irrelevant information in the first laser acquisition information and the first visual acquisition information, improves the accuracy of the first laser acquisition information and the first visual acquisition information, thereby providing guarantee for the fusion analysis unit to perform fusion analysis, reducing the error of the initial model of the home space, and moreover, through the condition setting unit, it can be limited in the initial model of the home space, so that the target model of the home space can better conform to the user's needs and the actual situation of the home space, and further enables the sweeping robot to better perform cleaning control.

[0093] In an embodiment provided by the present invention, the fusion analysis unit performs information fusion analysis according to the first acquisition optimization information and the second acquisition optimization information, including:

[0094] Perform home space distribution analysis according to the first acquisition optimization information, determine the items and item distribution information in the home space, create a model for the items to obtain an item model, then build a model for the item model according to the item distribution to obtain a home space diagram, and obtain the first initial model of the home space after confirming the home space diagram;

[0095] Perform image acquisition information analysis on the second acquisition optimization information, determine the acquisition orientation of the image, splice the images according to the acquisition orientation of the image to determine the image combination information, perform three-dimensional image conversion based on the image combination information to determine the three-dimensional image information of the home space, and establish a model according to the three-dimensional image information of the home space to obtain the second initial model of the home space;

[0096] Use the feature analysis model to perform feature analysis on the first initial model of the home space and the second initial model of the home space to obtain the first model feature and the second model feature, and determine the third initial model of the home space according to the first model feature and the second model feature.

[0097] In the above technical solution, the third initial model of the home space is the final initial model of the home space.

[0098] In the above technical solution, after obtaining the first initial model of the home space and the second initial model of the home space, the first initial model of the home space and the second initial model of the home space are respectively verified, where,

[0099] When validating the first initial model of the home space, analyze it in combination with the first acquisition optimization information, including: obtaining the first model information by acquiring model information for the first initial model of the home space; performing preliminary validation analysis by combining the first model information with the first acquisition optimization information to determine whether the amount of information is the same, and obtaining the first validation analysis result; when the first validation analysis result indicates that the amount of information is the same, perform correspondence analysis on the first model information in combination with the first acquisition optimization information to determine the matching result between the first model information and the first acquisition optimization information. If all the first model information and the first acquisition optimization information are successfully matched, the first initial model of the home space is verified and no correction is required. If there is information that cannot be matched between the first model information and the first acquisition optimization information, screen out the information that cannot be matched in the first acquisition optimization information to determine the screened information, and then perform information correction on the first initial model of the home space according to the screened information to obtain the corrected first initial model of the home space. When the first validation analysis result indicates that the amount of information is different, perform difference set analysis on the first model information and the first acquisition optimization information. Perform difference set analysis on the first model information according to the first acquisition optimization information to obtain the first difference set data, perform difference set analysis on the first acquisition optimization information according to the first model information to obtain the second difference set data, determine the target position in the first initial model of the home space for the second difference set data, and perform information correction on the target position in combination with the first difference set data to obtain the corrected first initial model of the home space.

[0100] When validating the second initial model of the home space, analyze it in combination with the second acquisition optimization information, including: obtaining the second model information by acquiring model information for the second initial model of the home space; performing a preliminary validation analysis by combining the second model information with the second acquisition optimization information to determine whether the amount of information is the same, and obtaining the second validation analysis result; when the second validation analysis result indicates that the amount of information is the same, perform a correspondence analysis on the second model information in combination with the second acquisition optimization information to determine the matching result between the second model information and the second acquisition optimization information. If all the second model information and the second acquisition optimization information are successfully matched, the second initial model of the home space is verified and no correction is required. If there is information that cannot be matched between the second model information and the second acquisition optimization information, screen out the information that cannot be matched in the second acquisition optimization information to determine the screened information, and then perform information correction on the second initial model of the home space according to the screened information to obtain the corrected second initial model of the home space. When the second validation analysis result indicates that the amount of information is different, perform a difference set analysis on the second model information and the second acquisition optimization information. Perform a difference set analysis on the second model information according to the second acquisition optimization information to obtain the third difference set data. Perform a difference set analysis on the second acquisition optimization information according to the second model information to obtain the fourth difference set data. Determine the target position in the second initial model of the home space for the fourth difference set data, and perform information correction on the target position in combination with the third difference set data to obtain the corrected second initial model of the home space.

[0101] The above technical solution realizes the establishment of a model based on the first acquisition optimization information, so as to obtain a home space model under the LDS laser acquisition method. At the same time, it also realizes the establishment of a model based on the second acquisition optimization information, so as to obtain a home space model under the visual acquisition method. The two acquisition methods are modeled, so that the first initial model of the home space and the second initial model of the home space are fused and analyzed in a model way, which is convenient for model mapping correspondence, provides a reference correspondence relationship for the fusion analysis, makes the fusion analysis more well-organized, and enables in the third initial model of the home space, the disadvantages of the first initial model of the home space are overcome through the second initial model of the home space, and the disadvantages of the second initial model of the home space are overcome through the first initial model of the home space, realizing the optimization of the two acquisition methods, reducing the error caused by a single acquisition method, and improving the comprehensiveness and accuracy of the third initial model of the home space. In addition, by respectively verifying the first initial model of the home space and the second initial model of the home space, the consistency between the first initial model of the home space and the second initial model of the home space and the first acquisition optimization information and the second acquisition optimization information is ensured, the accuracy of the first initial model of the home space and the second initial model of the home space is ensured, and the errors of the first initial model of the home space and the second initial model of the home space caused by information loss or information error during the process of obtaining the first initial model of the home space and the second initial model of the home space according to the first acquisition optimization information and the second acquisition optimization information are avoided.

[0102] In an embodiment provided by the present invention, determining the third initial model of the home space according to the first model feature and the second model feature includes:

[0103] Perform feature matching on the first model feature and the second model feature, analyze the feature similarity between the first model feature and the second model feature based on the same feature point position, and determine whether the model features match according to the feature similarity to obtain a feature matching result;

[0104] Screen out the matching model features according to the feature matching result, and determine the first model based on the matching model features to obtain the architecture of the third initial model of the home space;

[0105] Analyze the advantageous features of the LDS laser acquisition method according to the first model feature in combination with the matching model features to determine the first distinguishing feature;

[0106] Supplement the feature model in the architecture of the third initial model of the home space according to the first distinguishing feature to obtain the first processed third initial model of the home space;

[0107] Analyze the advantageous features of the camera device acquisition method according to the second model feature in combination with the matching model features to determine the second distinguishing feature;

[0108] Supplement the feature model in the third initial model of the first processed home space according to the second distinguishing feature to obtain the third initial model of the second processed home space.

[0109] In the above technical solution, when analyzing the feature similarity between the first model feature and the second model feature based on the same feature point position, it includes: retrieving the model features corresponding to the feature point in the first model feature and the second model feature respectively according to the feature point to obtain the first analysis information and the second analysis information of the feature point; analyzing whether the first analysis information and the second analysis information are the same. If the first analysis information and the second analysis information are the same, the model features of the feature point match. If the first analysis information and the second analysis information are different, semantic analysis is performed on the first analysis information and the second analysis information, the similarity between the first analysis information and the second analysis information is calculated to obtain the semantic information similarity of the feature point, and then the semantic information similarity of the feature point is judged in combination with the semantic similarity judgment threshold. If the semantic information similarity of the feature point is greater than the semantic similarity judgment threshold, the model features of the feature point match. If the semantic information similarity of the feature point is not greater than the semantic similarity judgment threshold, the model features of the feature point do not match.

[0110] In the above technical solution, the dominant feature is the other model features in the first model feature except the matching model features or the other model features in the second model feature except the matching model features.

[0111] In the above technical solution, the third initial model of the second processed home space is the final third initial model of the home space.

[0112] The above technical solution realizes the fusion of the first initial model of the home space and the second initial model of the home space, enables the two models to jointly build a model as the final initial model of the home space, realizes the mutual correction of the first initial model of the home space and the second initial model of the home space, reduces the data error caused by the disadvantages of the first initial model of the home space or the second initial model of the home space, improves the comprehensiveness and accuracy of the third initial model of the home space. Moreover, analyzing the feature similarity between the first model feature and the second model feature based on the same feature point position enables the first model feature and the second model feature to perform feature matching in an orderly manner, avoids the similarity calculation of non-corresponding model features, improves the effectiveness of the similarity calculation, reduces the waste of unnecessary time, and realizes the superposition of dominant features by supplementing the feature model in the architecture of the third initial model of the home space according to the first distinguishing feature and supplementing the feature model in the third initial model of the first processed home space according to the second distinguishing feature, making the third initial model of the second processed home space more comprehensive.

[0113] In an embodiment provided by the present invention, the condition setting unit performs limitation in the initial model of the home space according to the area condition setting information, including:

[0114] Perform restricted area and boundary analysis on the home space to determine the restricted area and boundary of the home space;

[0115] Obtain additional condition information of the user through the interaction terminal;

[0116] Perform limitation in the initial model of the home space according to the restricted area and boundary of the home space and the additional condition information to obtain the target model of the home space.

[0117] In the above technical solution, the additional condition information of the user may or may not exist. When there is no additional condition information, the limitation is directly performed in the initial model of the home space according to the restricted area and boundary of the home space.

[0118] Through the condition setting unit, the above technical solution can not only endow the initial model of the home space with the limited features of the home space, but also add and limit the additional condition information of the user in the initial model of the home space, enrich the characteristics of the target model of the home space, make the target model of the home space more in line with the actual situation of the home space, and at the same time, can also make the target model of the home space meet the needs of the user, so that the sweeping robot can better clean in the home space.

[0119] As Figure 5 shown, in an embodiment provided by the present invention, the positioning and navigation module includes: a scheme analysis unit, an identification and positioning unit, and an updated navigation unit;

[0120] The scheme analysis unit is used to determine the cleaning scheme in combination with the cleaning attributes of the sweeping robot for the target model of the home space, and plan the cleaning path for the cleaning scheme to obtain the cleaning control information of the sweeping robot;

[0121] The identification and positioning unit is used to perform preliminary analysis according to the laser real-time acquisition information and the visual real-time acquisition information to determine the position of the sweeping robot in the home space, obtain the current position of the sweeping robot, and further analyze the laser real-time acquisition information and the visual real-time acquisition information in combination with the current position of the sweeping robot to determine whether there are obstacles in the forward direction of the sweeping robot, and identify the obstacles in the case of obstacles to obtain the forward identification information;

[0122] The updated navigation unit is used to perform cleaning control and navigation prompts for the sweeping robot according to the cleaning control information of the sweeping robot, and at the same time, after the identification and positioning unit obtains the current position and the forward identification information of the sweeping robot, adjust and update the cleaning control and navigation prompts.

[0123] In the above technical solution, the solution analysis unit and the recognition and positioning unit are both connected to the update navigation unit.

[0124] In the above technical solution, when the recognition and positioning unit performs a preliminary analysis based on the laser real-time acquisition information and the visual real-time acquisition information, it only needs to determine the position in the home space according to some features in the laser real-time acquisition information and the visual real-time acquisition information, and there is no need to perform a comprehensive analysis on all the laser real-time acquisition information and the visual real-time acquisition information.

[0125] In the above technical solution, further analysis is performed on the laser real-time acquisition information and the visual real-time acquisition information in combination with the current position of the sweeping robot, including:

[0126] Perform an obstacle analysis on the laser real-time acquisition information in combination with the current position of the sweeping robot. If there is an obstacle, identify the obstacle to determine what the obstacle is to obtain a first analysis result;

[0127] Perform an obstacle analysis on the visual real-time acquisition information in combination with the current position of the sweeping robot. If there is an obstacle, identify the obstacle to determine what the obstacle is to obtain a second analysis result;

[0128] Comprehensively analyze the first analysis result and the second analysis result. If both the first analysis result and the second analysis result indicate that there is no obstacle, there is no obstacle in the forward direction of the sweeping robot. Otherwise, there is an obstacle in the forward direction of the sweeping robot. At this time, retrieve what the obstacle is identified in the first analysis result and / or the second analysis result, and use what the obstacle is as the front recognition information.

[0129] The above technical solution realizes the analysis and control of the cleaning solution of the sweeping robot through the positioning and navigation module, enabling the sweeping robot to better clean the home space. Moreover, through the recognition and positioning module, real-time positioning and navigation analysis are performed on the sweeping robot based on the laser real-time acquisition information and the visual real-time acquisition information. Not only can the current position of the sweeping robot be understood in real time, enabling the first acquisition module and the second acquisition module to perform information acquisition based on the current position of the sweeping robot, improving the effectiveness of real-time information acquisition and enhancing the efficiency of information acquisition, but also it can be promptly determined whether there is an obstacle in the forward direction of the sweeping robot, so as to make adjustments in a timely manner according to the obstacle, avoid the sweeping robot performing ineffective cleaning at the same position, reduce the loss of the sweeping robot caused by useless work, and at the same time improve the flexibility of the sweeping robot positioning and navigation system, enabling the sweeping robot positioning and navigation system to more efficiently and accurately perform control adjustment and navigation for the sweeping robot, thereby improving the performance of the sweeping robot and providing more convenience for users.

[0130] In an embodiment provided by the present invention, the solution analysis unit determines a cleaning solution for the target model of the home space in combination with the cleaning attributes of the floor sweeping robot, and performs a cleaning path planning for the cleaning solution, including:

[0131] Determine whether there are specific area restrictions. When there are specific area restrictions, perform a static space analysis for the specific area in the target model of the home space to determine the area to be cleaned; when there are no specific area restrictions, perform an overall static space analysis for the target model of the home space to determine the area to be cleaned;

[0132] Obtain the cleaning attribute information of the floor sweeping robot, and determine the size of the effective cleaning area and the cleaning power range;

[0133] Perform an object analysis for the target area in the target model of the home space according to the area to be cleaned to determine the objects existing in the area to be cleaned, and obtain the cleaning characteristics of the objects;

[0134] Perform an effective cleaning intensity analysis by combining the cleaning characteristics of the objects with the cleaning power range, and determine the cleaning strategy for the area to be cleaned based on the effective cleaning intensity;

[0135] Perform a full-coverage path planning for the area to be cleaned in combination with the size of the effective cleaning area, and determine the cleaning path of the floor sweeping robot;

[0136] Obtain the cleaning control information of the floor sweeping robot according to the cleaning strategy of the area to be cleaned and the cleaning path of the floor sweeping robot.

[0137] In the above technical solution, the cleaning attribute information of the floor sweeping robot includes: the size of the effective cleaning area and the cleaning power range.

[0138] In the above technical solution, the cleaning intensities of different objects are different. For example, the cleaning intensity of the carpet is greater, and the cleaning intensity of the floor is smaller.

[0139] In the above technical solution, by determining the cleaning solution and performing the cleaning path planning, not only can the floor sweeping robot automatically achieve intelligent cleaning of the area to be cleaned, adjust the cleaning intensities for different objects, enable the floor sweeping robot to clean in the area to be cleaned according to the cleaning strategy, improve the cleaning effect of the floor sweeping robot, make the user more satisfied with the floor sweeping robot, but also can realize the cleaning path planning for the floor sweeping robot, enable the floor sweeping robot to perform automatic route control according to the cleaning path, and intelligently perform comprehensive cleaning of the area to be cleaned.

[0140] In an embodiment provided by the present invention, the update navigation unit adjusts and updates the cleaning control and navigation prompts, including:

[0141] Determine navigation prompt information based on the current position of the floor cleaning robot in combination with the cleaning path of the floor cleaning robot, and perform navigation prompts according to the navigation prompt information;

[0142] Perform obstacle analysis on the front recognition information to determine whether the obstacle is an item that the floor cleaning robot can directly clean;

[0143] When the obstacle is not an item that the floor cleaning robot can directly clean, obtain the size information of the obstacle, perform avoidance path analysis based on the size information of the obstacle in combination with the cleaning path of the floor cleaning robot, determine avoidance navigation based on the avoidance path, and update the navigation prompt according to the avoidance navigation;

[0144] When the obstacle is an item that the floor cleaning robot can directly clean, determine mode adjustment information for the floor cleaning robot according to the characteristics of the obstacle to obtain the mode adjustment information, then determine the target cleaning intensity by combining the obstacle with the cleaning strategy of the area to be cleaned, perform cleaning intensity adjustment analysis according to the target cleaning intensity in combination with the current cleaning intensity to determine the cleaning intensity adjustment information, and at the same time obtain the size information of the obstacle, predict the cleaning time of the obstacle according to the size information of the obstacle in combination with the cleaning path of the floor cleaning robot to determine the predicted cleaning time of the obstacle, and then perform cleaning control adjustment on the floor cleaning robot according to the mode adjustment information, the cleaning intensity adjustment information, and the predicted cleaning time of the obstacle.

[0145] In the above technical solution, items that the floor cleaning robot can directly clean include: carpets, floor mats, etc., and items that the floor cleaning robot cannot directly clean include: furniture items, toys, etc.

[0146] In the above technical solution, after avoiding the obstacle according to the avoidance path, return to the original cleaning path and continue cleaning.

[0147] In the above technical solution, when determining the mode adjustment information for the floor cleaning robot according to the characteristics of the obstacle, adjust the floor cleaning robot to the climbing mode, and perform climbing ability analysis and adjustment of the floor cleaning robot in combination with the thickness of the obstacle.

[0148] The above technical solution enables a more intuitive understanding of the movement direction of the floor sweeping robot through navigation prompts, and can also implement the cleaning path control of the floor sweeping robot based on the navigation prompts, enabling the floor sweeping robot to perform a comprehensive cleaning in the area to be cleaned better. Moreover, through obstacle analysis, when the obstacle is not an item that the floor sweeping robot can directly clean, it can timely confirm the avoidance path and then avoid based on the avoidance path, preventing the floor sweeping robot from being blocked by obstacles and unable to perform subsequent cleaning, which affects the cleaning efficiency of the floor sweeping robot and reduces the damage to the performance of the floor sweeping robot. At the same time, when the obstacle is an item that the floor sweeping robot can directly clean, it can adjust the cleaning intensity and mode control of the floor sweeping robot in combination with the characteristics of the obstacle, enabling cleaning according to different obstacles and ensuring the cleaning effect of the floor sweeping robot.

[0149] Those skilled in the art should understand that the first and second in the present invention only refer to different application stages.

[0150] After considering the specification and practicing the disclosure herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0151] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A sweeping robot positioning and navigation system based on vision and laser fusion, characterized in that: The sweeping robot positioning and navigation system comprises: a first acquisition module, a second acquisition module, a fusion analysis module and a positioning and navigation module; The first acquisition module is used to acquire spatial information of the home space using LDS laser to obtain first acquisition information; The second acquisition module is used to use a camera device to collect visual information of the home space to obtain second collected information; The fusion analysis module is used to fuse and analyze the first collected information and the second collected information to determine the initial model of the home space, and to set regional conditions based on the initial model of the home space to obtain a target model of the home space; The positioning and navigation module is used to determine a cleaning plan based on a home space target model, and to perform cleaning control and navigation prompts for the sweeping robot according to the cleaning plan. At the same time, it performs identification and positioning based on the first collected information and the second collected information, and then adjusts the cleaning control and updates the navigation prompts based on the identification and positioning results.

2. The sweeping robot positioning and navigation system according to claim 1, characterized in that: The first acquisition module includes: a first acquisition control unit and a first monitoring unit; The first acquisition control unit is used to control the LDS laser to perform comprehensive information acquisition on the home space, obtain a first acquisition control signal, the LDS laser emits a first laser pulse according to the first acquisition control signal, obtains a laser acquisition signal after the first laser pulse is scattered in all directions, and obtains the laser first acquisition information according to the laser acquisition signal; The first monitoring unit is used to determine the target monitoring area based on the current position of the sweeping robot, control the LDS laser to emit a second laser pulse in combination with the target monitoring area, and use the second laser pulse to collect real-time information on the target monitoring area to obtain laser real-time collection information.

3. The sweeping robot positioning and navigation system according to claim 2, characterized in that: The second acquisition module includes: a second acquisition control unit and a second monitoring unit; The second acquisition control unit is used to obtain a second acquisition control signal, and control the camera device to collect all-round visual information of the home space according to the second acquisition control signal to obtain the visual first acquisition information; The second monitoring unit is used to determine the target monitoring position based on the current position of the sweeping robot, control the camera device to perform real-time visual acquisition based on the target monitoring position to obtain a target monitoring image, and obtain real-time visual acquisition information based on the target monitoring image.

4. The sweeping robot positioning and navigation system according to claim 3, characterized in that: The fusion analysis module includes: a first processing unit, a second processing unit, a fusion analysis unit and a condition setting unit; The first processing unit is used to perform signal analysis and optimization on the laser first acquisition information to obtain first acquisition optimization information; The second processing unit is used to perform image optimization on the visual first acquisition information to obtain second acquisition optimization information; The fusion analysis unit is used to perform information fusion analysis based on the first collection optimization information and the second collection optimization information, construct a home space diagram, and obtain an initial model of the home space; The condition setting unit is used to set limits in the home space initial model according to the regional condition setting information to obtain the home space target model.

5. The cleaning robot positioning and navigation system according to claim 4, characterized in that: The fusion analysis unit performs information fusion analysis according to the first acquisition optimization information and the second acquisition optimization information, including: Performing a home space distribution analysis based on the first collected optimization information, determining items in the home space and item distribution information, creating models for the items to obtain item models, and then building models for the item models according to the item distribution to obtain a home space diagram, and confirming the home space diagram to obtain a first initial model of the home space; Perform image acquisition information analysis on the second acquisition optimization information, determine the image acquisition orientation, perform image stitching according to the image acquisition orientation, determine image combination information, perform three-dimensional image conversion based on the image combination information, determine the three-dimensional image information of the home space, establish a model according to the three-dimensional image information of the home space, and obtain a second initial model of the home space; A feature analysis model is used to perform feature analysis on the first initial model of the home space and the second initial model of the home space to obtain first model features and second model features, and a third initial model of the home space is determined based on the first model features and the second model features.

6. The cleaning robot positioning and navigation system according to claim 5, characterized in that: Determining a third initial model of the home space according to the first model feature and the second model feature includes: Perform feature matching on the first model feature and the second model feature, analyze the feature similarity between the first model feature and the second model feature based on the same feature point, and determine whether the model features match according to the feature similarity to obtain a feature matching result; According to the feature matching result, the matched model features are screened out, and the first model is determined based on the matched model features to obtain the architecture of the third initial model of the home space; Analyze the advantageous features of the LDS laser acquisition method according to the first model features in combination with the matched model features, and determine the first distinguishing feature; Supplementing the feature model in the framework of the third initial model of the home space according to the first distinguishing feature, to obtain the third initial model of the home space after the first processing; Analyzing the advantageous features of the acquisition method of the camera device according to the second model features in combination with the matched model features to determine the second distinguishing feature; The feature model is supplemented in the third initial model of the home space after the first processing according to the second distinguishing feature to obtain the third initial model of the home space after the second processing.

7. The cleaning robot positioning and navigation system according to claim 4, characterized in that: The condition setting unit is defined in the home space initial model according to the regional condition setting information, including: Conduct restricted areas and boundary analysis for home space and determine restricted areas and boundaries of home space; Obtaining additional condition information of the user through the interactive terminal; According to the restricted areas and boundaries of the home space and the accessory condition information, the home space initial model is defined to obtain the home space target model.

8. The cleaning robot positioning and navigation system according to claim 3, characterized in that: The positioning and navigation module includes: a solution analysis unit, an identification and positioning unit, and an update and navigation unit; The solution analysis unit is used to determine a cleaning solution based on the home space target model and the cleaning properties of the sweeping robot, and to plan a cleaning path based on the cleaning solution to obtain cleaning control information of the sweeping robot; The identification and positioning unit is used to perform preliminary analysis based on the laser real-time collection information and the visual real-time collection information to determine the position of the sweeping robot in the home space, obtain the current position of the sweeping robot, and further analyze the laser real-time collection information and the visual real-time collection information in combination with the current position of the sweeping robot to determine whether there are obstacles in the forward direction of the sweeping robot, and identify the obstacles if there are obstacles to obtain front recognition information; The updated navigation unit is used to perform cleaning control and navigation prompts for the sweeping robot according to the cleaning control information of the sweeping robot, and at the same time, after the identification and positioning unit obtains the current position and front identification information of the sweeping robot, the cleaning control and navigation prompts are adjusted and updated.

9. The cleaning robot positioning and navigation system according to claim 8, characterized in that: The solution analysis unit determines a cleaning solution based on the home space target model and the cleaning properties of the sweeping robot, and plans a cleaning path based on the cleaning solution, including: Determine whether there are specific area restrictions. When there are specific area restrictions, perform static space analysis on the specific area in the home space target model to determine the area to be cleaned; when there are no specific area restrictions, perform overall static space analysis on the home space target model to determine the area to be cleaned; Obtain the cleaning property information of the sweeping robot to determine the effective cleaning area size and cleaning strength range; Analyze the items in the target area according to the area to be cleaned in the home space target model, determine the items in the area to be cleaned, and obtain the cleaning features of the items; The cleaning characteristics of the items are combined with the cleaning strength range to perform effective cleaning intensity analysis, and the cleaning strategy for the area to be cleaned is determined based on the effective cleaning intensity; Combine the size of the effective cleaning area to plan a full coverage path for the area to be cleaned and determine the cleaning path of the sweeping robot; The cleaning control information of the sweeping robot is obtained according to the cleaning strategy of the area to be cleaned and the cleaning path of the sweeping robot.

10. The cleaning robot positioning and navigation system according to claim 9, characterized in that: The updating navigation unit adjusts and updates the cleaning control and navigation prompts, including: Determine navigation prompt information according to the current position of the sweeping robot and the cleaning path of the sweeping robot, and provide navigation prompt according to the navigation prompt information; Perform obstacle analysis based on the front recognition information to determine whether the obstacle is something that the sweeping robot can clean directly; When the obstacle is not an object that can be directly cleaned by the sweeping robot, obtain the size information of the obstacle, perform avoidance path analysis based on the size information of the obstacle and the cleaning path of the sweeping robot, determine avoidance navigation based on the avoidance path, and update the navigation prompt according to the avoidance navigation; When the obstacle is an object that can be directly cleaned by the sweeping robot, the mode adjustment information is determined for the sweeping robot according to the characteristics of the obstacle, and the mode adjustment information is obtained. Then, the target cleaning intensity is determined based on the obstacle and the cleaning strategy of the area to be cleaned. The cleaning intensity adjustment analysis is performed based on the target cleaning intensity and the current cleaning intensity to confirm the cleaning intensity adjustment information. At the same time, the size information of the obstacle is obtained. The cleaning time of the obstacle is predicted based on the size information of the obstacle and the cleaning path of the sweeping robot. The predicted cleaning time of the obstacle is determined, and then the cleaning control adjustment is performed for the sweeping robot according to the mode adjustment information, the cleaning intensity adjustment information and the predicted cleaning time of the obstacle.

Images