House plan generation method and device

Abstract

The invention discloses a house type image generation method and device, and the method comprises the steps: obtaining the mapping data of fitting lines, image information, point cloud information and the like of a target region through sensors with different principles, such as a radar sensor, an image sensor, a three-dimensional information sensor and the like, and providing comprehensive basic data for building a target house type image. Therefore, multi-sensor information fusion is carried out, and the target house type image of the target area is established, so that the house type image generation accuracy can be improved, and the use experience of related intelligent equipment such as a sweeper, a weeding machine and a scrubber can be improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a method and device for generating a floor plan. Background Art

[0002] For indoor cleaning equipment such as sweepers, or other smart devices used in indoor scenarios, it is usually necessary to establish an indoor floor plan and perform subsequent automated cleaning functions based on the pre-established indoor floor plan.

[0003] However, the current method of establishing floor plans based on smart devices usually uses sensors with a single principle, such as image sensors such as cameras, to obtain indoor space information, which leads to poor accuracy in the generation of floor plans in some scenarios, resulting in low intelligence and accuracy of indoor cleaning equipment. Therefore, it is particularly important to improve the accuracy of floor plan generation. Summary of the invention

[0004] The technical problem to be solved by the present invention is to obtain a target floor plan of the target area by fusing multi-sensor information through the relevant data of the target area collected by each sensor, thereby improving the comprehensiveness of the basic data used to establish the target floor plan, thereby improving the accuracy of floor plan generation, and further facilitating improving the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a method for generating a floor plan, the method comprising:

[0006] Determine a first floor plan of the target area according to the data collected by the first sensor;

[0007] Determining object category information and two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor;

[0008] Determine a second floor plan according to the first floor plan and the two-dimensional coordinate information of each of the objects;

[0009] Determine the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determine the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object;

[0010] A target floor plan is determined according to the second floor plan and the target object information.

[0011] As an optional implementation manner, the first sensor includes a radar sensor, and determining the first floor plan of the target area according to data collected by the first sensor includes:

[0012] Determine a plurality of fitting lines in a first direction and a second direction respectively by using the radar sensor;

[0013] Performing line preprocessing on the fitting lines in the first direction and the fitting lines in the second direction, and determining a slam map according to the results of the line preprocessing; the line preprocessing includes: one or more of correction, deletion, and filling;

[0014] According to the slam map, wall information and door information are determined, and the slam map is divided according to the wall information and the door information to obtain a first floor plan corresponding to each room.

[0015] As an optional implementation manner, dividing the SLAM map according to the wall information and the door information to obtain a first floor plan corresponding to each room includes:

[0016] According to the wall information and the door information, the slam map is morphologically processed according to a preset width to obtain initial contour information;

[0017] Determine sub-profile information corresponding to each room according to the initial profile information;

[0018] The room area indicated by each sub-profile information is determined, and according to the room area, each sub-profile information is screened and / or merged to obtain a first floor plan corresponding to each room.

[0019] As an optional implementation manner, the second sensor includes an image sensor, and determining the object category information and the two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor includes:

[0020] Performing calibration processing on the image sensor to determine a preset coordinate system;

[0021] Acquiring image information of each object in the target area through an image sensor;

[0022] Performing target detection processing on the image information to obtain object category information corresponding to each object;

[0023] The image information is processed for ranging to obtain distance information and orientation information corresponding to each object, and the distance information and the orientation information are transformed according to the preset coordinate system to obtain two-dimensional coordinate information corresponding to each object.

[0024] As an optional implementation manner, the third sensor includes a three-dimensional information sensor, and determining the three-dimensional point cloud information of each object according to data collected by the third sensor includes:

[0025] Acquiring initial three-dimensional point cloud information corresponding to each object through the three-dimensional information sensor;

[0026] Performing point cloud preprocessing on the initial three-dimensional point cloud information to obtain three-dimensional point cloud information corresponding to each object; the point cloud preprocessing includes filtering processing and matching processing;

[0027] And, determining the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object includes:

[0028] Determining current position information of the three-dimensional information sensor;

[0029] Determine the three-dimensional coordinate information corresponding to each object according to the posture information, the three-dimensional point cloud information and the two-dimensional coordinate information;

[0030] Target object information of each object is determined according to the three-dimensional coordinate information and the object category information.

[0031] As an optional implementation, the method is applied to a cleaning device, which includes: a sweeper, a scrubber or a mop;

[0032] After determining the target floor plan according to the second floor plan and the target object information, the method further includes:

[0033] According to the target floor plan, determine the target room and cleaning parameters;

[0034] A cleaning process is performed according to the target room and the cleaning parameters, wherein the cleaning parameters include: equipment parameters and cleaning degree parameters.

[0035] As an optional embodiment, the device parameters include: a cleaning trajectory parameter and a cleaning mode parameter; wherein the cleaning trajectory parameter is used to indicate the cleaning trajectory of the cleaning device when performing the cleaning process, and the cleaning mode parameter is used to indicate the cleaning mode of the cleaning device when performing the cleaning process;

[0036] Determining the target room and cleaning parameters according to the target floor plan includes:

[0037] Determine at least one target room according to the target floor plan, and determine a cleanliness parameter of each target room;

[0038] Determining the cleaning trajectory parameter and the cleaning mode parameter according to the cleaning degree parameter;

[0039] And, performing cleaning processing according to the target room and the cleaning parameters includes:

[0040] According to the cleaning trajectory parameters and the cleaning mode parameters, a corresponding cleaning process is performed on each of the target rooms until the cleaning degree parameters of each of the target rooms meet a preset termination condition.

[0041] A second aspect of the present invention discloses a device for generating a floor plan, the device comprising:

[0042] A floor plan determination module, used to determine a first floor plan of a target area according to data collected by the first sensor;

[0043] An object information determination module, used to determine object category information and two-dimensional coordinate information of each object in the target area according to data collected by the second sensor;

[0044] The floor plan determination module is further used to determine a second floor plan according to the first floor plan and the two-dimensional coordinate information of each object;

[0045] The object information determination module is further used to determine the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determine the target object information of each object according to the object category information of each object, the two-dimensional coordinate information and the three-dimensional point cloud information;

[0046] The floor plan determination module is further used to determine a target floor plan based on the second floor plan and the target object information.

[0047] As an optional implementation, the first sensor includes a radar sensor, and the specific manner in which the floor plan determination module determines the first floor plan of the target area according to the data collected by the first sensor includes:

[0048] Determine a plurality of fitting lines in a first direction and a second direction respectively by using the radar sensor;

[0049] Performing line preprocessing on the fitting lines in the first direction and the fitting lines in the second direction, and determining a slam map according to the results of the line preprocessing; the line preprocessing includes: one or more of correction, deletion, and filling;

[0050] According to the slam map, wall information and door information are determined, and the slam map is divided according to the wall information and the door information to obtain a first floor plan corresponding to each room.

[0051] As an optional implementation manner, the floor plan determination module divides the SLAM map according to the wall information and the door information to obtain a specific manner of the first floor plan corresponding to each room, including:

[0052] According to the wall information and the door information, the slam map is morphologically processed according to a preset width to obtain initial contour information;

[0053] Determine sub-profile information corresponding to each room according to the initial profile information;

[0054] The room area indicated by each sub-profile information is determined, and according to the room area, each sub-profile information is screened and / or merged to obtain a first floor plan corresponding to each room.

[0055] As an optional implementation, the second sensor includes an image sensor, and the specific manner in which the object information determination module determines the object category information and the two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor includes:

[0056] Performing calibration processing on the image sensor to determine a preset coordinate system;

[0057] Acquiring image information of each object in the target area through an image sensor;

[0058] Performing target detection processing on the image information to obtain object category information corresponding to each object;

[0059] The image information is processed for ranging to obtain distance information and orientation information corresponding to each object, and the distance information and the orientation information are transformed according to the preset coordinate system to obtain two-dimensional coordinate information corresponding to each object.

[0060] As an optional implementation, the third sensor includes a three-dimensional information sensor, and the object information determination module determines the three-dimensional point cloud information of each object according to the data collected by the third sensor, including:

[0061] Acquiring initial three-dimensional point cloud information corresponding to each object through the three-dimensional information sensor;

[0062] Performing point cloud preprocessing on the initial three-dimensional point cloud information to obtain three-dimensional point cloud information corresponding to each object; the point cloud preprocessing includes filtering processing and matching processing;

[0063] And, the object information determination module determines the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object, including:

[0064] Determining current position information of the three-dimensional information sensor;

[0065] Determine the three-dimensional coordinate information corresponding to each object according to the posture information, the three-dimensional point cloud information and the two-dimensional coordinate information;

[0066] Target object information of each object is determined according to the three-dimensional coordinate information and the object category information.

[0067] As an optional implementation, the method is applied to a cleaning device, which includes: a sweeper, a scrubber or a mop;

[0068] The device also includes:

[0069] A cleaning module, used after the floor plan determination module determines the target floor plan according to the second floor plan and the target object information;

[0070] According to the target floor plan, determine the target room and cleaning parameters;

[0071] A cleaning process is performed according to the target room and the cleaning parameters, wherein the cleaning parameters include: equipment parameters and cleaning degree parameters.

[0072] As an optional embodiment, the device parameters include: a cleaning trajectory parameter and a cleaning mode parameter; wherein the cleaning trajectory parameter is used to indicate the cleaning trajectory of the cleaning device when performing the cleaning process, and the cleaning mode parameter is used to indicate the cleaning mode of the cleaning device when performing the cleaning process;

[0073] The cleaning module determines the specific manner of the target room and cleaning parameters according to the target floor plan, including:

[0074] Determine at least one target room according to the target floor plan, and determine a cleanliness parameter of each target room;

[0075] Determining the cleaning trajectory parameter and the cleaning mode parameter according to the cleaning degree parameter;

[0076] And, the specific manner in which the cleaning module performs the cleaning process according to the target room and the cleaning parameters includes:

[0077] According to the cleaning trajectory parameters and the cleaning mode parameters, a corresponding cleaning process is performed on each of the target rooms until the cleaning degree parameters of each of the target rooms meet a preset termination condition.

[0078] The third aspect of the present invention discloses another device for generating a floor plan, the device comprising:

[0079] A memory storing executable program code;

[0080] a processor coupled to the memory;

[0081] The processor calls the executable program code stored in the memory to execute the floor plan generation method disclosed in the first aspect of the present invention.

[0082] The fourth aspect of the present invention discloses a computer storable medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the floor plan generation method disclosed in the first aspect of the present invention.

[0083] In an embodiment of the present invention, sensors with different principles such as radar sensors, image sensors, and three-dimensional information sensors are used to obtain mapping data such as fitting lines, image information, and point cloud information of the target area, thereby providing comprehensive basic data for establishing a target floor plan. Multi-sensor information fusion is then performed to establish a target floor plan for the target area, thereby improving the accuracy of floor plan generation, and further facilitating the user experience of related smart devices such as sweepers, lawn mowers, scrubbers, and other equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0085] Figure 1 It is a schematic diagram of an application scenario of a floor plan generation method disclosed in an embodiment of the present invention;

[0086] Figure 2 It is a flow chart of a method for generating a floor plan disclosed in an embodiment of the present invention;

[0087] Figure 3 It is a flowchart of another method for generating a floor plan disclosed in an embodiment of the present invention;

[0088] Figure 4 It is a flowchart of another method for generating a floor plan disclosed in an embodiment of the present invention;

[0089] Figure 5 It is a structural schematic diagram of a floor plan generating device disclosed in an embodiment of the present invention;

[0090] Figure 6 It is a structural schematic diagram of another device for generating a floor plan disclosed in an embodiment of the present invention;

[0091] Figure 7 It is a structural schematic diagram of another floor plan generating device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0094] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0095] For indoor cleaning equipment such as sweepers, or other smart devices used in indoor scenarios, it is usually necessary to establish an indoor floor plan and perform subsequent automated cleaning functions based on the pre-established indoor floor plan.

[0096] However, the current method of establishing floor plans based on smart devices usually uses sensors with a single principle, such as image sensors such as cameras, to obtain indoor space information, which leads to poor accuracy in the generation of floor plans in some scenarios, resulting in low intelligence and accuracy of indoor cleaning equipment. Therefore, it is particularly important to improve the accuracy of floor plan generation.

[0097] See also Figure 1 , Figure 1 Schematic diagram of an application scenario of a floor plan generation method disclosed in an embodiment of the present invention. Figure 1 As shown, the smart device in the figure can be a smart cleaning device such as a sweeper, a scrubber, or a lawn mower. It is necessary to establish a floor plan of the corresponding target area in advance to realize the cleaning function. For these devices, a single-principle information acquisition module A is usually used to obtain information about the target area. For example, the information acquisition module A can be a radar sensor, an image sensor, a three-dimensional information sensor, etc. However, the information obtained by a sensor based on a single principle has limitations and cannot fully reflect the status of the current area, resulting in inaccurate floor plans. As a result, in the application of smart devices, there are problems such as hitting walls, repeated cleaning, or incorrectly finding the target room, which are not good user experience.

[0098] The floor plan generation method provided in this application may involve sensors based on different principles, including but not limited to lidar sensors, infrared sensors, RGB cameras, and time of flight (TOF) sensors. A SLAM map is constructed based on a radar sensor, and an image algorithm is used to construct the overall floor plan and divide the room; an RGB camera is used to obtain an obstacle or furniture image, and an object is identified in the RGB image and the position of the object is determined at a medium and long distance, and the final positioning can be performed in combination with an inertial measurement unit (IMU) sensor; at the same time, a three-dimensional information sensor such as a structured light sensor or TOF is used to construct the 3D information of the object at a close distance; finally, the close-range information of the object is fused with the medium and long-range information, and placed on the floor plan constructed based on the SLAM map.

[0099] The present invention discloses a method and device for generating a floor plan. By using sensors with different principles such as radar sensors, image sensors and three-dimensional information sensors, mapping data such as fitting lines, image information and point cloud information of a target area are obtained, and comprehensive basic data for establishing a target floor plan is provided. Multi-sensor information fusion is then performed to establish a target floor plan of the target area, thereby improving the accuracy of floor plan generation, and further facilitating the user experience of related intelligent devices such as sweepers, lawn mowers, floor scrubbers and the like.

[0100] Embodiment 1

[0101] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a method for generating a floor plan disclosed in an embodiment of the present invention. Figure 2 As shown, the floor plan generation method may include the following operations:

[0102] S101, determining a first floor plan of a target area according to data collected by a first sensor;

[0103] Specifically, the first floor plan may be floor plan information in a two-dimensional direction, and an image sensor or a radar sensor may be used to obtain corresponding information to establish the first floor plan. The target area may be an indoor environment or an outdoor environment constrained by certain conditions or ranges, such as a yard surrounded by a wall, or an outdoor area defined by predetermined coordinates.

[0104] S102, determining object category information and two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor;

[0105] Specifically, the second sensor can be various image sensors, such as an RGB camera or an infrared camera, which obtains corresponding data based on a monocular vision method, and then determines the object category information and two-dimensional coordinate information of each object in the target area through data processing. Furthermore, it is also feasible to use a more expensive binocular or multi-eye vision solution to implement the corresponding method of the present application.

[0106] S103, determining a second floor plan according to the first floor plan and the two-dimensional coordinate information of each of the objects;

[0107] Specifically, the second floor plan can be a two-dimensional floor plan or a three-dimensional floor plan. The three-dimensional floor plan can be constructed and processed on the application terminal based on the data collected by the aforementioned sensors, such as the APP end or the page end. Obviously, the reference for construction is the two-dimensional floor plan, which provides the wall height and the plan view of the target area, and then constructs a three-dimensional floor plan from a bird's-eye view or a horizontal view. In other words, according to the solution of the present application, the two-dimensional basic data for establishing a three-dimensional floor plan can be obtained, so that technicians can think of a solution to further obtain a three-dimensional floor plan without any creative labor.

[0108] S104, determining the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determining the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object;

[0109] S105: Determine a target floor plan according to the second floor plan and the target object information.

[0110] Specifically, the target object information is a type of three-dimensional information. The three-dimensional target object information is integrated into the two-dimensional floor plan, thereby further establishing a three-dimensional target floor plan, modeling each object according to the corresponding coordinate position, and placing the whole object at the coordinate point to improve the effect of smart device application, achieve a higher level of visualization, and enhance the user experience.

[0111] In the present application, an intelligent device coordinate system can be established based on the intelligent device where the sensor is located. Other sensors can be calibrated during initialization. For details, please refer to other implementation methods. By calibrating the data and constructing a transformation matrix, coordinate transformations of different systems can be performed, which can be used to convert data from different sensors into the same coordinate system for fusion processing.

[0112] This embodiment uses sensors with different principles such as radar sensors, image sensors, and three-dimensional information sensors to obtain mapping data such as fitting lines, image information, and point cloud information of the target area, and provides comprehensive basic data for establishing a target floor plan. Multi-sensor information fusion is then performed to establish a target floor plan for the target area, thereby improving the accuracy of floor plan generation, and thus helping to improve the user experience of related smart devices such as sweepers, lawn mowers, scrubbers, and other equipment.

[0113] In an optional embodiment, the first sensor includes a radar sensor, and determining the first floor plan of the target area according to data collected by the first sensor includes:

[0114] Determine a plurality of fitting lines in a first direction and a second direction respectively by using the radar sensor;

[0115] The first direction and the second direction may generally be a horizontal direction and a vertical direction based on a specific world coordinate system, and the corresponding coordinate system may also be established by a corresponding sensor.

[0116] Performing line preprocessing on the fitting lines in the first direction and the fitting lines in the second direction, and determining a slam map according to the results of the line preprocessing; the line preprocessing includes: one or more of correction, deletion, and filling;

[0117] Specifically, the correction process includes removing overlapping straight lines. For the fitted straight lines, there may be overlaps with each other. For example, the vertical distance between two horizontal straight lines may deviate by several pixels, but the middle area overlaps. The overlapping part needs to be removed to reduce the amount of data processed.

[0118] The deletion criteria include clustering the lines, removing outliers, and merging the lines that are clustered into a set. The merging basis is to score the clustered lines according to their length and distance to each other, and merge them according to the scores.

[0119] The filling standard includes connecting and filling all straight lines based on the distance between them and the overall direction of the room.

[0120] According to the slam map, wall information and door information are determined, and the slam map is divided according to the wall information and the door information to obtain a first floor plan corresponding to each room.

[0121] It should be noted that the first sensor may also be an image sensor, which acquires contour information of the target area by means of monocular vision.

[0122] It can be seen that this optional embodiment determines multiple fitting lines through the radar sensor, and processes the fitting lines to obtain a slam map, and then further determines the wall information and door information according to the slam map, thereby determining the first floor plan corresponding to each room, which can be used for data fusion with data obtained by other sensors, thereby improving the accuracy of floor plan generation, and further helping to improve the user experience of related smart devices, such as sweepers, lawn mowers, scrubbers and other equipment.

[0123] In an optional embodiment, dividing the SLAM map according to the wall information and the door information to obtain a first floor plan corresponding to each room includes:

[0124] According to the wall information and the door information, the slam map is morphologically processed according to a preset width to obtain initial contour information;

[0125] Determine sub-profile information corresponding to each room according to the initial profile information;

[0126] The room area indicated by each sub-profile information is determined, and according to the room area, each sub-profile information is screened and / or merged to obtain a first floor plan corresponding to each room.

[0127] Specifically, the screening process and the merging process can be used to reduce the probability of rooms being divided incorrectly, and the first floor plan corresponding to each room can be obtained according to the sub-profile information. In the process of establishing the first floor plan, there may still be certain misdivision problems only through the preprocessing of the fitting lines. The solution provided by this embodiment, combined with the line preprocessing, can reduce the probability of rooms being divided incorrectly, thereby improving the effectiveness of establishing the first floor plan.

[0128] It can be seen that this optional embodiment obtains the initial contour information of the slam map through morphological processing, and further determines the sub-contour information of each room, and performs further screening and / or merging processing according to the corresponding room area to obtain the first floor plan corresponding to each room, thereby improving the accuracy of the first floor plan corresponding to each room, and then improving the accuracy of floor plan generation. Furthermore, it is beneficial to improve the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0129] In an optional embodiment, the second sensor includes an image sensor, and determining the object category information and two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor includes:

[0130] Performing calibration processing on the image sensor to determine a preset coordinate system;

[0131] The calibration process includes chessboard image acquisition, internal parameter calculation, external parameter calibration image acquisition, external parameter calculation, coordinate mapping matrix calculation and other processes. A specific calibration process can refer to Zhang Zhengyou calibration method.

[0132] Acquiring image information of each object in the target area through an image sensor;

[0133] Specifically, the image information can be used for position recognition and semantic recognition of an object, that is, to obtain two-dimensional coordinate information and / or object category information of the corresponding object. Please continue to refer to the following steps.

[0134] Performing target detection processing on the image information to obtain object category information corresponding to each object;

[0135] The image information is processed for ranging to obtain distance information and orientation information corresponding to each object, and the distance information and the orientation information are transformed according to the preset coordinate system to obtain two-dimensional coordinate information corresponding to each object.

[0136] The image sensor described in this optional embodiment can also be used in the aforementioned slam map generation process. However, through the combined use of the two types of sensors, the image sensor can be used to obtain low obstacles that are invisible to the radar laser in this step, so as to further improve the accuracy of floor plan generation and the user experience of smart devices in applications.

[0137] It can be seen that this optional embodiment calibrates the image sensor to determine a preset coordinate system based on the image sensor, and obtains image information of each object in the target area based on the calibrated image sensor, and obtains object category information and two-dimensional coordinate information of each object based on the image information, thereby improving the accuracy of obtaining object category information and object two-dimensional coordinate information, thereby improving the accuracy of floor plan generation, and further helping to improve the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0138] In an optional embodiment, the third sensor includes a three-dimensional information sensor, and determining the three-dimensional point cloud information of each object according to the data collected by the third sensor includes:

[0139] Acquiring initial three-dimensional point cloud information corresponding to each object through the three-dimensional information sensor;

[0140] Performing point cloud preprocessing on the initial three-dimensional point cloud information to obtain three-dimensional point cloud information corresponding to each object; the point cloud preprocessing includes filtering processing and matching processing;

[0141] Specifically, the 3D point cloud information can be used to identify objects and obtain the 3D coordinate information of the objects, which can further improve the probability of object uniqueness and improve the accuracy of floor plan generation. Point cloud preprocessing requires removing point clouds with a discrete degree higher than a preset threshold, where filtering can be achieved through a pass filter, a voxel filter, a statistical filter or a radius filter.

[0142] And, determining the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object includes:

[0143] Determining current position information of the three-dimensional information sensor;

[0144] Determine the three-dimensional coordinate information corresponding to each object according to the posture information, the three-dimensional point cloud information and the two-dimensional coordinate information;

[0145] Target object information of each object is determined according to the three-dimensional coordinate information and the object category information.

[0146] It can be seen that this optional embodiment determines the three-dimensional point cloud information of each object through a three-dimensional information sensor, and determines the three-dimensional coordinate information of the object by acquiring the sensor's posture information and combining the three-dimensional point cloud information and the two-dimensional coordinate information, and finally combines the object category information of the object to obtain the target object information of each object, and then determine the three-dimensional target floor plan, obtain relevant data of the target area through multiple data sources based on different principles, and perform multi-sensor data fusion, thereby improving the accuracy of floor plan generation, which is beneficial to improving the user experience of related smart devices, such as sweepers, lawn mowers, scrubbers and other equipment.

[0147] In an optional embodiment, before determining the current position and posture information of the three-dimensional information sensor, the method further includes: performing calibration processing on the three-dimensional information sensor to determine a second preset coordinate system;

[0148] The step of determining target object information of each object according to the three-dimensional coordinate information and the object category information includes:

[0149] Converting the three-dimensional coordinate information according to the second preset coordinate system to obtain standard coordinate information;

[0150] According to the standard coordinate information and the object category information, the coordinate position and object posture of each object in the standard coordinate information are determined.

[0151] It can be seen that this optional embodiment can improve the accuracy of determining the specific position and shape of each object in the target area by performing corresponding calibration processing on the three-dimensional information sensor, thereby improving the accuracy of floor plan generation. Furthermore, it is beneficial to improve the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0152] See also Figure 3 , Figure 3 It is a flowchart of another method for generating a floor plan disclosed in an embodiment of the present invention, and is used to specifically illustrate the solution when the aforementioned implementation modes are combined for application.

[0153] like Figure 3As shown, in the process of constructing the slam map, the information of the IMU sensor can also be integrated, combined with the data of the radar sensor, so as to obtain a more accurate multi-layer slam map containing the contour information of each height layer, and then through segmentation, corrosion, expansion and other methods, the specific location of the walls and doors in the current target area can be determined to obtain the slam map, that is, the aforementioned first floor plan. According to the aforementioned scheme, the image sensor is used to determine the category and location of the object, and combined with the relevant information of the first floor plan, a two-dimensional or three-dimensional second floor plan can be obtained. The figure provides only one type of scheme. After the point cloud information of each object is subsequently obtained, a three-dimensional target floor plan is established based on the second floor plan and the point cloud information. In actual applications, both the second floor plan and the first floor plan can include three-dimensional information.

[0154] Embodiment 2

[0155] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of another method for generating a floor plan disclosed in an embodiment of the present invention. Figure 4 As shown, the method is applied to a cleaning device, the cleaning device includes: a sweeper, a scrubber or a mop, and the method includes:

[0156] S201, determining a first floor plan of a target area according to data collected by a first sensor;

[0157] S202, determining object category information and two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor;

[0158] S203, determining a second floor plan according to the first floor plan and the two-dimensional coordinate information of each of the objects;

[0159] S204, determining the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determining the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object;

[0160] S205: Determine a target floor plan according to the second floor plan and the target object information.

[0161] For the detailed description of S201 to S205, please refer to the relevant description of S101 to S105 in the first embodiment, which will not be repeated here.

[0162] S206, determining the target room and cleaning parameters according to the target floor plan;

[0163] S207. Perform cleaning processing according to the target room and the cleaning parameters; wherein the cleaning parameters include: equipment parameters and cleaning degree parameters.

[0164] The aforementioned method for determining the target floor plan can be applied to intelligent cleaning equipment. Specifically, during the operation of the intelligent cleaning equipment, it is necessary to determine the target room and cleaning parameters based on the target floor plan, and then determine the corresponding cleaning plan, the number of cleaning times for each room and the corresponding cleaning intensity, cleaning cloth humidity and other equipment parameters. The degree of dirtiness of each target room can be determined manually by a remote terminal device, or by an image sensor integrated in the cleaning device, or by a separately set image sensor.

[0165] It can be seen that this optional embodiment can determine the target room to be cleaned and the cleaning parameters through the determined target floor plan, thereby obtaining a cleaning plan and executing the cleaning process. By improving the accuracy of floor plan generation, the accuracy of floor plans in the application of smart cleaning equipment is improved, thereby improving the user experience of smart cleaning equipment.

[0166] In an optional embodiment, the device parameters include: a cleaning trajectory parameter and a cleaning mode parameter; wherein the cleaning trajectory parameter is used to indicate the cleaning trajectory of the cleaning device when performing the cleaning process, and the cleaning mode parameter is used to indicate the cleaning mode of the cleaning device when performing the cleaning process;

[0167] Determining the target room and cleaning parameters according to the target floor plan includes:

[0168] Determine at least one target room according to the target floor plan, and determine a cleanliness parameter of each target room;

[0169] Determining the cleaning trajectory parameter and the cleaning mode parameter according to the cleaning degree parameter;

[0170] As described in the previous optional embodiment, the cleaning degree parameter can be used to determine the cleaning trajectory parameter and the cleaning method parameter. According to the cleaning trajectory parameter and the cleaning method parameter, the corresponding specific cleaning scheme can be determined, and the cleaning process can be performed according to the corresponding cleaning scheme.

[0171] And, performing cleaning processing according to the target room and the cleaning parameters includes:

[0172] According to the cleaning trajectory parameters and the cleaning mode parameters, a corresponding cleaning process is performed on each of the target rooms until the cleaning degree parameters of each of the target rooms meet a preset termination condition.

[0173] It can be seen that this optional embodiment can determine the cleanliness parameters of each room by determining the target floor plan, and then determine the cleaning trajectory parameters and cleaning method parameters, thereby generating and executing the corresponding cleaning plan to clean each target room. Therefore, by improving the accuracy of floor plan generation, the accuracy and cleaning effect of floor plans in the application of smart cleaning equipment are improved, thereby improving the user experience of smart cleaning equipment.

[0174] Embodiment 3

[0175] The embodiment of the present invention also provides a floor plan generating device to implement the above method, please refer to Figure 5 , Figure 5 Schematic diagram of the structure of a floor plan generating device disclosed in an embodiment of the present invention. Figure 5 As shown, based on any other embodiment, the device includes:

[0176] A floor plan determination module 31 is used to determine a first floor plan of a target area according to data collected by the first sensor;

[0177] An object information determination module 32, configured to determine object category information and two-dimensional coordinate information of each object in the target area according to data collected by the second sensor;

[0178] The floor plan determination module 31 is further used to determine a second floor plan according to the first floor plan and the two-dimensional coordinate information of each of the objects;

[0179] The object information determination module 32 is further used to determine the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determine the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object;

[0180] The floor plan determination module 31 is further configured to determine a target floor plan according to the second floor plan and the target object information.

[0181] This embodiment uses sensors with different principles such as radar sensors, image sensors, and three-dimensional information sensors to obtain mapping data such as fitting lines, image information, and point cloud information of the target area, and provides comprehensive basic data for establishing a target floor plan. Multi-sensor information fusion is then performed to establish a target floor plan for the target area, thereby improving the accuracy of floor plan generation, and thus helping to improve the user experience of related smart devices such as sweepers, lawn mowers, scrubbers, and other equipment.

[0182] In an optional embodiment, the first sensor includes a radar sensor, and the specific manner in which the floor plan determination module 31 determines the first floor plan of the target area according to the data collected by the first sensor includes:

[0183] Determine a plurality of fitting lines in a first direction and a second direction respectively by using the radar sensor;

[0184] Performing line preprocessing on the fitting lines in the first direction and the fitting lines in the second direction, and determining a slam map according to the results of the line preprocessing; the line preprocessing includes: one or more of correction, deletion, and filling;

[0185] According to the slam map, wall information and door information are determined, and the slam map is divided according to the wall information and the door information to obtain a first floor plan corresponding to each room.

[0186] It can be seen that this optional embodiment determines multiple fitting lines through the radar sensor, and processes the fitting lines to obtain a slam map, and then further determines the wall information and door information according to the slam map, thereby determining the first floor plan corresponding to each room, which can be used for data fusion with data obtained by other sensors, thereby improving the accuracy of floor plan generation, and further helping to improve the user experience of related smart devices, such as sweepers, lawn mowers, scrubbers and other equipment.

[0187] In an optional embodiment, the floor plan determination module 31 divides the SLAM map according to the wall information and the door information to obtain a first floor plan corresponding to each room, including:

[0188] According to the wall information and the door information, the slam map is morphologically processed according to a preset width to obtain initial contour information;

[0189] Determine sub-profile information corresponding to each room according to the initial profile information;

[0190] The room area indicated by each sub-profile information is determined, and according to the room area, each sub-profile information is screened and / or merged to obtain a first floor plan corresponding to each room.

[0191] It can be seen that this optional embodiment obtains the initial contour information of the slam map through morphological processing, and further determines the sub-contour information of each room, and performs further screening and / or merging processing according to the corresponding room area to obtain the first floor plan corresponding to each room, thereby improving the accuracy of the first floor plan corresponding to each room, and then improving the accuracy of floor plan generation. Furthermore, it is beneficial to improve the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0192] In an optional embodiment, the second sensor includes an image sensor, and the object information determination module 32 determines the object category information and the two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor, including:

[0193] Performing calibration processing on the image sensor to determine a preset coordinate system;

[0194] Acquiring image information of each object in the target area through an image sensor;

[0195] Performing target detection processing on the image information to obtain object category information corresponding to each object;

[0196] The image information is processed for ranging to obtain distance information and orientation information corresponding to each object, and the distance information and the orientation information are transformed according to the preset coordinate system to obtain two-dimensional coordinate information corresponding to each object.

[0197] It can be seen that this optional embodiment calibrates the image sensor to determine a preset coordinate system based on the image sensor, and obtains image information of each object in the target area based on the calibrated image sensor, and obtains object category information and two-dimensional coordinate information of each object based on the image information, thereby improving the accuracy of obtaining object category information and object two-dimensional coordinate information, thereby improving the accuracy of floor plan generation, and further helping to improve the user experience of related smart devices, such as sweepers, lawn mowers, floor scrubbers and other equipment.

[0198] In an optional embodiment, the third sensor includes a three-dimensional information sensor, and the object information determination module 32 determines the three-dimensional point cloud information of each object according to the data collected by the third sensor, including:

[0199] Acquiring initial three-dimensional point cloud information corresponding to each object through the three-dimensional information sensor;

[0200] Performing point cloud preprocessing on the initial three-dimensional point cloud information to obtain three-dimensional point cloud information corresponding to each object; the point cloud preprocessing includes filtering processing and matching processing;

[0201] And, the object information determination module determines the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object, including:

[0202] Determining current position information of the three-dimensional information sensor;

[0203] Determine the three-dimensional coordinate information corresponding to each object according to the posture information, the three-dimensional point cloud information and the two-dimensional coordinate information;

[0204] Target object information of each object is determined according to the three-dimensional coordinate information and the object category information.

[0205] It can be seen that this optional embodiment determines the three-dimensional point cloud information of each object through a three-dimensional information sensor, and determines the three-dimensional coordinate information of the object by acquiring the sensor's posture information and combining the three-dimensional point cloud information and the two-dimensional coordinate information, and finally combines the object category information of the object to obtain the target object information of each object, and then determine the three-dimensional target floor plan, obtain relevant data of the target area through multiple data sources based on different principles, and perform multi-sensor data fusion, thereby improving the accuracy of floor plan generation, which is beneficial to improving the user experience of related smart devices, such as sweepers, lawn mowers, scrubbers and other equipment.

[0206] In an optional embodiment, the method is applied to a cleaning device, and the cleaning device includes: a sweeper, a scrubber or a mop;

[0207] See also Figure 6 , Figure 6 Schematic diagram of the structure of a floor plan generating device disclosed in an embodiment of the present invention. Figure 6 As shown, the device also includes:

[0208] A cleaning module 33 is used for determining a target floor plan according to the second floor plan and the target object information by the floor plan determination module 31;

[0209] According to the target floor plan, determine the target room and cleaning parameters;

[0210] A cleaning process is performed according to the target room and the cleaning parameters, wherein the cleaning parameters include: equipment parameters and cleaning degree parameters.

[0211] It can be seen that this optional embodiment can determine the target room to be cleaned and the cleaning parameters through the determined target floor plan, thereby obtaining a cleaning plan and executing the cleaning process. By improving the accuracy of floor plan generation, the accuracy of floor plans in the application of smart cleaning equipment is improved, thereby improving the user experience of smart cleaning equipment.

[0212] In an optional embodiment, the device parameters include: a cleaning trajectory parameter and a cleaning mode parameter; wherein the cleaning trajectory parameter is used to indicate the cleaning trajectory of the cleaning device when performing the cleaning process, and the cleaning mode parameter is used to indicate the cleaning mode of the cleaning device when performing the cleaning process;

[0213] The cleaning module 33 determines the specific method of the target room and cleaning parameters according to the target floor plan, including:

[0214] Determine at least one target room according to the target floor plan, and determine a cleanliness parameter of each target room;

[0215] Determining the cleaning trajectory parameter and the cleaning mode parameter according to the cleaning degree parameter;

[0216] And, the specific manner in which the cleaning module performs the cleaning process according to the target room and the cleaning parameters includes:

[0217] According to the cleaning trajectory parameters and the cleaning mode parameters, a corresponding cleaning process is performed on each of the target rooms until the cleaning degree parameters of each of the target rooms meet a preset termination condition.

[0218] It can be seen that this optional embodiment can determine the cleanliness parameters of each room by determining the target floor plan, and then determine the cleaning trajectory parameters and cleaning method parameters, thereby generating and executing the corresponding cleaning plan to clean each target room. Therefore, by improving the accuracy of floor plan generation, the accuracy and cleaning effect of floor plans in the application of smart cleaning equipment are improved, thereby improving the user experience of smart cleaning equipment.

[0219] Embodiment 4

[0220] See also Figure 7 , Figure 7 FIG. 1 is a schematic diagram of the structure of another device for generating a floor plan disclosed in an embodiment of the present invention. Figure 7 As shown, the floor plan generating device may include:

[0221] The device further includes a processor (Processor) 291, a memory (Memory) 292 storing executable program code; and may also include a communication interface (Communication Interface) 293 and a bus 294. The processor 291, the memory 292, and the communication interface 293 may communicate with each other through the bus 294. The communication interface 293 may be used for information transmission. The processor 291 is coupled to the memory 292, and the processor 291 may call the logic instructions (executable program code) in the memory 292 to execute the floor plan generation method described in any of the above embodiments.

[0222] In addition, the logic instructions in the above-mentioned memory 292 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0223] The memory 292 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, that is, implementing the methods in the above method embodiments.

[0224] The memory 292 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 292 may include a high-speed random access memory and may also include a non-volatile memory.

[0225] An embodiment of the present invention further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are called, they are used to implement the method described in any one of the embodiments.

[0226] An embodiment of the present invention also discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the floor plan generation method described in any embodiment.

[0227] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0228] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0229] Finally, it should be noted that the floor plan generation method and device disclosed in the embodiments of the present invention only disclose preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a floor plan, characterized in that: The method comprises: Determine a first floor plan of the target area according to the data collected by the first sensor; Determining object category information and two-dimensional coordinate information of each object in the target area according to the data collected by the second sensor; Determine a second floor plan according to the first floor plan and the two-dimensional coordinate information of each of the objects; Determine the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determine the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object; A target floor plan is determined according to the second floor plan and the target object information.

2. The method according to claim 1, characterized in that The first sensor includes a radar sensor, and determining a first floor plan of the target area according to data collected by the first sensor includes: Determine a plurality of fitting lines in a first direction and a second direction respectively by using the radar sensor; Performing line preprocessing on the fitting lines in the first direction and the fitting lines in the second direction, and determining a slam map according to the results of the line preprocessing; the line preprocessing includes: one or more of correction, deletion, and filling; According to the slam map, wall information and door information are determined, and the slam map is divided according to the wall information and the door information to obtain a first floor plan corresponding to each room.

3. The method according to claim 2, characterized in that The step of dividing the SLAM map according to the wall information and the door information to obtain a first floor plan corresponding to each room includes: According to the wall information and the door information, the slam map is morphologically processed according to a preset width to obtain initial contour information; Determine sub-profile information corresponding to each room according to the initial profile information; The room area indicated by each sub-profile information is determined, and according to the room area, each sub-profile information is screened and / or merged to obtain a first floor plan corresponding to each room.

4. The method according to claim 1, characterized in that The second sensor includes an image sensor, and determining object category information and two-dimensional coordinate information of each object in the target area according to data collected by the second sensor includes: Performing calibration processing on the image sensor to determine a preset coordinate system; Acquiring image information of each object in the target area through an image sensor; Performing target detection processing on the image information to obtain object category information corresponding to each object; The image information is processed for ranging to obtain distance information and orientation information corresponding to each object, and the distance information and the orientation information are transformed according to the preset coordinate system to obtain two-dimensional coordinate information corresponding to each object.

5. The method according to claim 1, characterized in that The third sensor includes a three-dimensional information sensor, and determining the three-dimensional point cloud information of each object according to the data collected by the third sensor includes: Acquiring initial three-dimensional point cloud information corresponding to each object through the three-dimensional information sensor; Performing point cloud preprocessing on the initial three-dimensional point cloud information to obtain three-dimensional point cloud information corresponding to each object; the point cloud preprocessing includes filtering processing and matching processing; And, determining the target object information of each object according to the object category information, the two-dimensional coordinate information and the three-dimensional point cloud information of each object includes: Determining current position information of the three-dimensional information sensor; Determine the three-dimensional coordinate information corresponding to each object according to the posture information, the three-dimensional point cloud information and the two-dimensional coordinate information; Target object information of each object is determined according to the three-dimensional coordinate information and the object category information.

6. The method according to any one of claims 1 to 5, characterized in that: The method is applied to cleaning equipment, which includes: a sweeper, a scrubber or a mop; After determining the target floor plan according to the second floor plan and the target object information, the method further includes: According to the target floor plan, determine the target room and cleaning parameters; A cleaning process is performed according to the target room and the cleaning parameters, wherein the cleaning parameters include: equipment parameters and cleaning degree parameters.

7. The method according to claim 6, characterized in that The equipment parameters include: cleaning trajectory parameters and cleaning mode parameters; wherein the cleaning trajectory parameters are used to indicate the cleaning trajectory of the cleaning equipment when performing the cleaning process, and the cleaning mode parameters are used to indicate the cleaning mode of the cleaning equipment when performing the cleaning process; Determining the target room and cleaning parameters according to the target floor plan includes: Determine at least one target room according to the target floor plan, and determine a cleanliness parameter of each target room; Determining the cleaning trajectory parameter and the cleaning mode parameter according to the cleaning degree parameter; And, performing cleaning processing according to the target room and the cleaning parameters includes: According to the cleaning trajectory parameters and the cleaning mode parameters, a corresponding cleaning process is performed on each of the target rooms until the cleaning degree parameters of each of the target rooms meet a preset termination condition.

8. A floor plan generating device, characterized in that: The device comprises: A floor plan determination module, used to determine a first floor plan of a target area according to data collected by the first sensor; An object information determination module, used to determine object category information and two-dimensional coordinate information of each object in the target area according to data collected by the second sensor; The floor plan determination module is further used to determine a second floor plan according to the first floor plan and the two-dimensional coordinate information of each object; The object information determination module is further used to determine the three-dimensional point cloud information of each object according to the data collected by the third sensor, and determine the target object information of each object according to the object category information of each object, the two-dimensional coordinate information and the three-dimensional point cloud information; The floor plan determination module is further used to determine a target floor plan based on the second floor plan and the target object information.

9. A floor plan generating device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the floor plan generation method as described in any one of claims 1-7.

10. A computer storable medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the floor plan generation method as described in any one of claims 1-7.

Images