Mobile equipment control method and device, mobile equipment and storage medium

Abstract

The invention provides a control method and device of mobile equipment, the mobile equipment and a storage medium, and relates to the field of smart home. The mobile equipment comprises an equipment main body and a mechanical arm arranged on the equipment main body, the equipment main body is provided with a first acquisition device, the mechanical arm is provided with a second acquisition device, and the method comprises the following steps: acquiring first environment information acquired by the first acquisition device; performing first identification on the to-be-identified object based on the first environment information to obtain a first category identification result of the to-be-identified object; and controlling the position of the equipment main body according to the first category identification result so as to control the collection view of the second collection device.

Description

Technical Field

[0001] The present application relates to the field of smart home, and in particular to a control method and device for a movable device, a movable device, and a readable storage medium. Background Art

[0002] As a member of smart home, the ability of sweeping robots to complete cleaning tasks efficiently and intelligently depends largely on their accurate recognition of indoor environments. In related technologies, the camera of sweeping robots is installed on the main body of the device. Since the main body of the device is relatively low, the camera is installed at a relatively low height, resulting in a poor shooting angle, which in turn affects the accuracy of indoor environment recognition. Summary of the invention

[0003] In view of this, the present application provides a control method and device for a movable device, a movable device and a readable storage medium, which solve the problem of the accuracy of the indoor environment recognition of the sweeping robot in the related art.

[0004] In a first aspect, an embodiment of the present application provides a control method for a movable device, wherein the movable device includes a device body and a mechanical arm disposed on the device body, the device body is provided with a first acquisition device, and the mechanical arm is provided with a second acquisition device, the method comprising:

[0005] Acquire first environmental information collected by the first collecting device;

[0006] Performing a first identification of the object to be identified based on the first environmental information to obtain a first category identification result of the object to be identified;

[0007] According to the first category recognition result, the position of the device body is controlled to control the collection field of view of the second collection device.

[0008] In a second aspect, an embodiment of the present application provides a control device for a movable device, wherein the movable device includes a device body and a mechanical arm disposed on the device body, the device body is provided with a first acquisition device, the mechanical arm is provided with a second acquisition device, and the device includes:

[0009] An acquisition module, used to acquire the first environmental information collected by the first acquisition device;

[0010] A first identification module, configured to perform a first identification of an object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified;

[0011] A control module is used to control the position of the device body according to the first category recognition result, so as to control the collection field of view of the second collection device.

[0012] In a third aspect, an embodiment of the present application provides a removable device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be executed on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0014] In an embodiment of the present application, a first acquisition device is controlled to acquire first environmental information of the current spatial environment where the movable device is located, and an object to be identified in the current spatial environment is identified based on the first environmental information to obtain a first category identification result including the existence of the object to be identified, the category of the object to be identified, the first confidence level corresponding to the category, and the location information of the object to be identified. The first category identification result is judged to determine whether the category identification based on the first environmental information is accurate based on the first category identification result. Based on the first category identification result, the position of the device body and the posture of the robotic arm are controlled to control the collection field of view of the second acquisition device, so that information collection and secondary identification of the object to be identified are performed through the second acquisition device.

[0015] In an embodiment of the present application, a first identification is performed on the first environmental information collected by the first acquisition device to obtain a first category identification result. According to the first category identification result, the position of the device body is controlled to control the collection field of view of the second acquisition device, so that the second acquisition device can be adjusted to a state where the shooting environmental information is more suitable, thereby collecting environmental information with a better viewing angle and a larger range, thereby improving the recognition accuracy when performing a second identification through the second acquisition device.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram showing the structure of a mobile device according to an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram showing a flow chart of a control method for a movable device according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram showing an angle comparison between a first acquisition device and a second acquisition device in an embodiment of the present application is shown;

[0021] Figure 4 A structural block diagram of a control device for a movable device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0024] The control method, device, removable device and readable storage medium of the mobile device provided in the embodiments of the present application are described in detail below in conjunction with the accompanying drawings through specific embodiments and their application scenarios. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0025] The present application embodiment provides a movable device, which may be a cleaning robot and a household robot. The cleaning robot includes a sweeping robot, a mopping robot, a sweeping and mopping all-in-one machine, etc. Figure 1As shown, the movable device 100 includes a device body 101, a mechanical arm 102, a first collection device 103 arranged on the device body 101, and a second collection device 104 arranged on the mechanical arm 102. The mechanical arm 102 is a multi-degree-of-freedom mechanical arm. The mechanical arm 102 can include two states, one is a lifting state, and the other is a falling state. The mechanical arm 102 can realize the clamping and carrying of objects. For example, when the movable device is cleaning the floor, if there is an obstacle on the cleaning route, the mechanical arm 102 is controlled to extend, and the obstacle is clamped and moved by the clamping claw of the mechanical arm 102, so that the movable device can clean the bottom of the obstacle position and the surrounding ground, thereby improving the floor cleaning coverage rate; when tidying up the room, the movable device automatically identifies the obstacle on the ground and marks the position, clamps the obstacle by the clamping claw of the mechanical arm 102, and moves the obstacle to the storage position, thereby realizing room tidying.

[0026] The device body 101 is provided with a first acquisition device 103, and the mechanical arm 102 is provided with a second acquisition device 104. The first acquisition device 103 and the second acquisition device 104 may be RGB cameras, depth sensors, or three-dimensional imaging sensors.

[0027] When the mechanical arm 102 is in the raised state, the second acquisition device 104 is located at a higher height than the first acquisition device 103, and the acquisition field of view of the second acquisition device 104 is higher than that of the first acquisition device 103, so that information with a wider viewing angle can be acquired. In addition, in addition to the higher height of the second acquisition device 104, since it is installed on the mechanical arm 102, the position and posture of the second acquisition device 104 can be adjusted by adjusting the mechanical arm 102, that is, the second acquisition device 104 can be located at different positions, and the acquisition angle can also be adjusted, so as to acquire information from multiple angles and directions.

[0028] The present application embodiment provides a control method for a movable device, such as Figure 2 As shown, the method includes:

[0029] Step 201: Acquire first environmental information collected by a first collecting device.

[0030] In this step, the first acquisition device is controlled to acquire first environmental information of the current spatial environment where the movable device is located. If the first acquisition device is an RGB camera, the first environmental information is image information; if the first acquisition device is a depth sensor or a three-dimensional imaging sensor, the first environmental information is three-dimensional information.

[0031] Step 202: Perform a first identification of the object to be identified based on the first environmental information to obtain a first category identification result of the object to be identified.

[0032] In this step, the object to be identified in the current spatial environment is identified based on the first environmental information, and a first category identification result is obtained, including the existence of the object to be identified, the category of the object to be identified, the first confidence corresponding to the category, and the location information of the object to be identified.

[0033] Step 203: Control the position of the device body according to the first category recognition result to control the collection field of view of the second collection device.

[0034] In one embodiment, the first category recognition result is judged, so as to determine whether the category recognition based on the first environmental information is accurate according to the first category recognition result. If accurate, the first category recognition result is used as the final recognition result, and the position of the device body is controlled without further recognition; if inaccurate, the position of the device body is controlled to be adjusted to control the collection field of view of the second collection device, so as to collect information and perform secondary recognition of the object to be identified through the second collection device.

[0035] In another embodiment, it is also possible not to determine whether the category recognition based on the first environmental information is accurate, but to default to adjusting the position of the device body after the first recognition to control the collection field of view of the second collection device, thereby performing information collection and secondary recognition of the object to be identified through the second collection device.

[0036] The acquisition field of view at least includes the acquisition angle (or acquisition viewing angle) and acquisition position of the second acquisition device, and the acquisition position includes height, distance, and the like.

[0037] In one embodiment, after the second recognition, a second category recognition result is obtained, and the second category recognition result can be used as the final recognition result, or the first category recognition result and the second category recognition result are combined to obtain the final recognition result.

[0038] In an embodiment of the present application, a first identification is performed on the first environmental information collected by the first acquisition device to obtain a first category identification result. According to the first category identification result, the position of the device body is controlled to control the collection field of view of the second acquisition device, so that the second acquisition device can be adjusted to a state where the shooting environmental information is more suitable, thereby collecting environmental information with a better viewing angle and a larger range, thereby improving the recognition accuracy when performing a second identification through the second acquisition device.

[0039] In one embodiment of the present application, a first category recognition result includes a category of an object to be recognized and a first confidence level corresponding to the category; based on the first category recognition result, the position of the device body is controlled to control the collection field of view of the second collection device, including: if the first confidence level is within a preset confidence interval, the position of the device body is controlled to control the collection field of view of the second collection device.

[0040] In this embodiment, it is determined whether the first confidence is in a preset confidence interval. If the first confidence is less than the lower limit of the preset confidence interval (for example, the lower limit is 0.1, 0.2, 0.3 or 0.4, etc.), it indicates that it is highly likely not in this category, and the reliability of not being in this category is high; if the first confidence is greater than the upper limit of the preset confidence interval (for example, the upper limit is 0.8, 0.9, 0.95 or 0.98, etc.), it indicates that it is highly likely in this category, and the reliability of being in this category is high; the first confidence being in the preset confidence interval indicates that it may be in this category, or it may not be, the reliability is not high, the identification is not accurate, and further determination is required. Therefore, the position of the device body is controlled to control the collection field of view of the second collection device, for example, to control the collection viewing angle of the second collection device, to collect information through the second collection device, and to perform secondary identification of the object to be identified.

[0041] In one embodiment of the present application, according to the first category recognition result, the position of the robot arm can also be controlled to control the collection field of view of the second collection device, for example, to control the collection viewing angle of the second collection device. That is to say, in one embodiment, the collection field of view of the second collection device can be controlled by controlling the position of the device body alone. In another embodiment, it is necessary to control the position of the device body and the position of the robot arm to jointly control the collection field of view of the second collection device, and the position of the device body and the position of the robot arm can be controlled when the first confidence level is in a preset confidence interval.

[0042] In one embodiment of the present application, according to the first category recognition result, controlling the position of the device body and the posture of the robot arm to control the collection field of view of the second collection device includes:

[0043] According to the category of the object to be identified, determine the acquisition field of view corresponding to the category;

[0044] Calculate the reachable position of the device body and the reachable optimal posture of the robot arm according to the position information of the object to be identified included in the first category recognition result, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view;

[0045] The device body is controlled to move to a reachable position, and the robotic arm is controlled to move to an optimal reachable position, so as to control the collection field of view of the second collection device.

[0046] In this embodiment, according to the category of the object to be identified in the first category recognition result, the collection field of view of the desired verification recognition is confirmed, and then the collection field of view of the second collection device is controlled based on the collection field of view to achieve secondary recognition. In this embodiment of the application, based on the collection field of view corresponding to the category of the object to be identified, the second collection device is controlled to achieve data collection at a more suitable angle and orientation to ensure the accuracy of the secondary recognition.

[0047] Specifically, according to the position information of the object to be identified, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view, the reachable position of the device body and the reachable optimal posture of the robot arm are calculated. Further, the device body is controlled to move to the reachable position, and the robot arm is controlled to move to the reachable optimal posture, so as to control the acquisition field of view of the second acquisition device.

[0048] The method for calculating the reachable position of the device body and the reachable optimal posture of the robot arm is: based on the position information of the object to be identified and the position information of the device body, the distance and direction between the two are calculated. Combined with the mobility of the device body and the path planning algorithm, it is determined whether the device body can reach the position of the object to be identified. If the device body can reach the position of the object to be identified, the position of the object to be identified is determined to be a reachable position; if the device body cannot directly reach the position of the object to be identified, a nearest reachable position is calculated based on the path planning algorithm and obstacle avoidance strategy.

[0049] According to the structural parameters of the robot arm (such as joint length, angle limit, etc.), kinematic solution is performed. Combined with the position information of the object to be identified, the collection field of view of the second acquisition device and the kinematic solution result of the robot arm, an optimal posture is calculated so that the end of the robot arm can accurately reach the position of the object to be identified, and the second acquisition device can obtain clear images or data with a larger angle range.

[0050] Through the above-mentioned method, the present application enables the second acquisition device to observe the object to be identified at a more reasonable angle and orientation.

[0051] In one embodiment of the present application, according to the category of the object to be identified, determining the acquisition field of view corresponding to the category includes:

[0052] According to the correspondence between the category and the acquisition field of view, the acquisition field of view corresponding to the category of the object to be identified is determined; or, the size information of the object to be identified is determined according to the category of the object to be identified, and the corresponding acquisition field of view is determined according to the size information.

[0053] In one embodiment, the second acquisition device has different acquisition fields corresponding to different categories, that is, the corresponding relationship between different categories and different acquisition fields is pre-stored, and the categories are mapped one by one to the acquisition fields. After the category of the object to be identified is determined, the acquisition field corresponding to the category can be determined based on the corresponding relationship.

[0054] In another embodiment, the size information of the object to be identified is determined according to the category. For example, if the object to be identified is a pet bed, and the pet bed in the room is an object of fixed size, and the size information of the fixed size objects in the room has been stored, after the object to be identified is identified as a pet bed, its size information, that is, the size, height and other information of the object to be identified can be determined. Furthermore, the corresponding acquisition field of view is determined according to the size information.

[0055] In the embodiment of the present application, after the category of the object to be identified is identified, the collection field of view of the second collection device for data collection is determined according to the category, so that data is collected at a more suitable angle and orientation to ensure the accuracy of the secondary identification.

[0056] In one embodiment of the present application, before determining whether the first confidence level is within a preset confidence level interval, the method includes:

[0057] According to the category of the object to be identified, a preset confidence interval corresponding to the category is determined.

[0058] In this embodiment, different categories correspond to different preset confidence intervals, that is, the correspondence between different categories and different preset confidence intervals is pre-stored, and the categories and preset confidence intervals are mapped one by one. After determining the category of the object to be identified, the preset confidence interval corresponding to the category can be determined based on the correspondence.

[0059] Since objects of different categories have different characteristics, the corresponding confidence intervals are also different. A more reasonable confidence threshold can be set according to the characteristics of the specific category, which helps to more accurately judge the reliability of recognition, thereby avoiding unnecessary misjudgments and improving recognition accuracy.

[0060] In one embodiment of the present application, the method further includes:

[0061] Controlling the second acquisition device to acquire the second environmental information in the acquisition field of view;

[0062] A second identification of the object to be identified is performed based on the second environmental information to obtain a second category identification result of the object to be identified.

[0063] In this embodiment, after controlling the position of the device body and the posture of the robot arm to control the collection field of view of the second collection device, the second environment information is collected by the second collection device under the collection field of view, for example, the second environment information is collected according to a determined collection angle of view. If the second collection device is an RGB camera, the second environment information is image information; if the second collection device is a depth sensor or a three-dimensional imaging sensor, the second environment information is three-dimensional information. Then, based on the second environment information, the object to be identified in the current spatial environment is identified to obtain a second category identification result including the existence of the object to be identified, the category of the object to be identified, the second confidence corresponding to the category, and the location information of the object to be identified.

[0064] like Figure 3 As shown, when the mechanical arm is in the raised state, the height of the second acquisition device 104 is higher than the height of the first acquisition device 103, and the acquisition field of view of the second acquisition device 104 is larger than that of the first acquisition device 103, so that data with a larger viewing angle range can be acquired, that is, for the shooting of the object to be identified, the field of view B is larger than the field of view C. In addition, in addition to the higher height of the second acquisition device, since it is installed on the mechanical arm, the position and posture of the second acquisition device can be adjusted by adjusting the mechanical arm, that is, the second acquisition device can be located at different positions, and the acquisition angle can also be adjusted, so as to acquire data from multiple angles and directions.

[0065] In the embodiment of the present application, when the recognition result corresponding to the first acquisition device installed on the equipment body is unreliable, the second acquisition device installed on the robotic arm is used to provide observation from a more reasonable perspective to improve the recognition accuracy.

[0066] In the embodiment of the present application, the second acquisition device is called for secondary recognition only when the confidence level of the first category recognition result falls within the unreliable interval. This on-demand calling method can save computing resources and time and avoid unnecessary waste of resources.

[0067] In one embodiment of the present application, the method also includes: if it is determined based on the second category recognition result that there is an object to be identified, coordinate system one processing and information fusion processing are performed on the first environmental information and the second environmental information to obtain fused three-dimensional information of the object to be identified, and the fused three-dimensional information includes size, height, and coverage area position.

[0068] In this embodiment, since both the first acquisition device and the second acquisition device have blind spots in the field of vision, in order to improve the accuracy of recognition, after determining that there is an object to be identified in the spatial environment, the first environment information and the second environment information are processed by the coordinate system and information fusion to obtain the fused three-dimensional information such as the size, coverage area position, etc. of the object to be identified.

[0069] Specifically, if the first acquisition device and the second acquisition device are RGB cameras, the first environmental information and the second environmental information are both image information, and the depth information or point cloud information of the object to be identified is estimated based on the optical flow and image segmentation; if the first acquisition device and the second acquisition device are depth sensors or three-dimensional imaging sensors, the first environmental information and the second environmental information are both depth information or point cloud information. Thus, the depth / point cloud information of the object to be identified relative to the first acquisition device is obtained through the first acquisition device, and the depth / point cloud information relative to the second acquisition device is obtained through the second acquisition device. However, since their respective coordinate systems are inconsistent, the observation surfaces that can observe the object to be identified may also be inconsistent, so it is necessary to unify the coordinate system and perform data fusion.

[0070] In one embodiment, the coordinate system processing may include: selecting a global reference coordinate system, which may be a device body, to establish a three-dimensional space coordinate system. Calibrate the first acquisition device and the second acquisition device to determine their respective positions and postures in the global reference coordinate system. The calibration process generally includes determining the sensor installation position, angle, focal length and other parameters of the acquisition device, and calculating the relationship between these parameters and the global reference coordinate system. According to the calibration results of the acquisition device, the coordinate points of the depth information or point cloud information corresponding to the first acquisition device and the depth information or point cloud information corresponding to the second acquisition device are transformed from their respective local coordinate systems to the global reference coordinate system.

[0071] In one embodiment, the information fusion process may include: extracting feature information related to the object to be identified from the depth information or point cloud information correspondingly collected by the first acquisition device and the depth information or point cloud information correspondingly collected by the second acquisition device, and the feature information may include edges, corners, textures, colors, etc. Using a stereo matching algorithm, the feature points in the images collected by the first acquisition device and the second acquisition device are matched to find the corresponding relationship between them. Based on the matching results, a disparity map is generated, which is a two-dimensional image in which the value of each pixel represents the disparity value of the corresponding point in the images of the two acquisition devices. Finally, the results of feature extraction and disparity calculation are combined to obtain three-dimensional information of the object to be identified.

[0072] In the embodiment of the present application, there is no need for the movable device to observe around the object to be identified. Instead, based on the fusion of the first acquisition device and the second acquisition device, more accurate three-dimensional information of the object to be identified is obtained, and the principle of binoculars is used to assist in improving the depth calculation accuracy.

[0073] In one embodiment of the present application, the method further includes: controlling the movable device to perform a preset cleaning action according to the category of the object to be identified included in the second category identification result and fusing the three-dimensional information.

[0074] In this embodiment, the movable device is controlled to perform a preset cleaning action based on the fusion of three-dimensional information such as the type, size, and location of the object to be identified, such as a focused cleaning strategy for flat wet stains and an avoidance cleaning strategy for pet nests.

[0075] In the embodiment of the present application, the recognition accuracy of the object to be recognized and the size estimation accuracy of the object to be recognized are improved through two recognitions, so that the corresponding cleaning action is implemented to improve the cleaning effect.

[0076] As a specific implementation of the control method of the above-mentioned movable device, the embodiment of the present application provides a control device for a movable device, wherein the movable device includes a device body and a mechanical arm arranged on the device body, wherein the device body is provided with a first acquisition device, and the mechanical arm is provided with a second acquisition device. Figure 4 As shown, the control device 400 of the movable device includes: an acquisition module 401 , a first identification module 402 and a control module 403 .

[0077] The acquisition module 401 is used to acquire the first environment information collected by the first collection device;

[0078] A first identification module 402, configured to perform a first identification of an object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified;

[0079] The control module 403 is used to control the position of the device body according to the first category recognition result, so as to control the collection field of view of the second collection device.

[0080] Furthermore, the first category recognition result includes the category of the object to be recognized and the first confidence level corresponding to the category. The control module 403 is specifically configured to:

[0081] If the first confidence level is within a preset confidence level interval, the position of the device body is controlled to control the collection field of view of the second collection device.

[0082] Furthermore, the control module can also control the posture of the robot arm according to the first category recognition result to control the collection field of view of the second collection device;

[0083] The device also includes:

[0084] A viewing angle range determination module is used to determine the acquisition viewing angle range corresponding to the category according to the category of the object to be identified;

[0085] A calculation module, used to calculate the reachable position of the device body and the reachable optimal posture of the robot arm according to the position information of the object to be identified included in the first category recognition result, as well as the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view;

[0086] The control module 403 is specifically used to control the device body to move to a reachable position, and control the robot arm to move to a reachable optimal posture, so as to control the collection field of view of the second collection device.

[0087] Furthermore, the viewing angle range determination module is specifically used to:

[0088] Determine the acquisition field of view corresponding to the category of the object to be identified according to the correspondence between the category and the acquisition field of view; or

[0089] The size information of the object to be identified is determined according to the category of the object to be identified, and the corresponding acquisition field of view range is determined according to the size information.

[0090] Furthermore, the device also includes:

[0091] The confidence interval determination module is used to determine a preset confidence interval corresponding to the category according to the category of the object to be identified.

[0092] Furthermore, the control module 403 is also used to control the second collection device to collect the second environmental information in the collection field of view;

[0093] The device also includes:

[0094] The second identification module is used to perform a second identification of the object to be identified based on the second environmental information to obtain a second category identification result of the object to be identified.

[0095] Furthermore, the device also includes:

[0096] An information fusion module, for performing coordinate system one processing and information fusion processing on the first environment information and the second environment information if it is determined that there is an object to be identified according to the second category identification result, so as to obtain fused three-dimensional information of the object to be identified, wherein the fused three-dimensional information includes size, height, and coverage area position;

[0097] The control module 403 is further used to control the movable device to perform a preset cleaning action according to the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

[0098] The control device 400 of the mobile device in the embodiment of the present application can be a mobile device, or a component in the mobile device, such as an integrated circuit or a chip. The control device 400 of the mobile device provided in the embodiment of the present application can realize Figure 1 To avoid repetition, the various processes implemented in the embodiment of the control method for a movable device are not described here.

[0099] An embodiment of the present application also provides a removable device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the various steps of the control method embodiment of the above-mentioned removable device are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0100] The memory can be used to store software programs and various data. The memory may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an environmental information playback function, etc.), etc. In addition, the memory may include a volatile memory or a non-volatile memory, or the memory may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0101] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor.

[0102] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the control method embodiment of the above-mentioned movable device is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0103] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0104] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A control method for a movable device, characterized in that: The movable device comprises a device body and a mechanical arm arranged on the device body, the device body is provided with a first acquisition device, and the mechanical arm is provided with a second acquisition device, and the method comprises: Acquire first environmental information collected by the first collecting device; Performing a first identification of the object to be identified based on the first environmental information to obtain a first category identification result of the object to be identified; According to the first category recognition result, the position of the device body is controlled to control the collection field of view of the second collection device.

2. The method according to claim 1, characterized in that The first category recognition result includes the category of the object to be recognized and a first confidence level corresponding to the category; The step of controlling the position of the device body according to the first category recognition result to control the collection field of view of the second collection device includes: If the first confidence level is within a preset confidence level interval, the position of the device body is controlled to control the collection field of view of the second collection device.

3. The method according to claim 2, characterized in that According to the first category recognition result, the position and posture of the robotic arm can also be controlled to control the collection field of view of the second collection device; According to the first category recognition result, controlling the position of the device body and the posture of the robot arm to control the collection field of view of the second collection device includes: According to the category of the object to be identified, determining a collection field of view corresponding to the category; Calculate the reachable position of the device body and the reachable optimal posture of the robot arm according to the position information of the object to be identified included in the first category recognition result, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view; The device body is controlled to move to the reachable position, and the robotic arm is controlled to move to the reachable optimal posture, so as to control the collection field of view of the second collection device.

4. The method according to claim 3, characterized in that The step of determining, according to the category of the object to be identified, a collection field of view corresponding to the category includes: Determine the acquisition field of view corresponding to the category of the object to be identified according to the correspondence between the category and the acquisition field of view; or The size information of the object to be identified is determined according to the category of the object to be identified, and the corresponding acquisition field of view range is determined according to the size information.

5. The method according to claim 3, characterized in that: Before determining whether the first confidence level is within a preset confidence level interval, the method includes: According to the category of the object to be identified, the preset confidence interval corresponding to the category is determined.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Controlling the second acquisition device to acquire second environmental information in the acquisition field of view; A second identification of the object to be identified is performed based on the second environmental information to obtain a second category identification result of the object to be identified.

7. The method according to claim 6, characterized in that The method further comprises: If it is determined that the object to be identified exists according to the second category identification result, coordinate system one processing and information fusion processing are performed on the first environment information and the second environment information to obtain fused three-dimensional information of the object to be identified, wherein the fused three-dimensional information includes size, height, and coverage area position; The movable device is controlled to execute a preset cleaning action according to the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

8. A control device for a movable device, characterized in that: The movable device comprises a device body and a mechanical arm arranged on the device body, the device body is provided with a first acquisition device, the mechanical arm is provided with a second acquisition device, and the device comprises: An acquisition module, used to acquire the first environmental information collected by the first acquisition device; A first identification module, configured to perform a first identification of an object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified; A control module is used to control the position of the device body according to the first category recognition result, so as to control the collection field of view of the second collection device.

9. The device according to claim 8, characterized in that The first category recognition result includes the category of the object to be recognized and a first confidence level corresponding to the category. The control module is specifically configured to: If the first confidence level is within a preset confidence level interval, the position of the device body is controlled to control the collection field of view of the second collection device.

10. The device according to claim 9, characterized in that The control module can also control the posture of the robotic arm according to the first category recognition result to control the collection field of view of the second collection device; The device also includes: A viewing angle range determination module, used to determine, according to the category of the object to be identified, a collection viewing angle range corresponding to the category; a calculation module, configured to calculate the reachable position of the device body and the reachable optimal posture of the robot arm according to the position information of the object to be identified included in the first category recognition result, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view; The control module is specifically used to control the device body to move to the reachable position, and control the robotic arm to move to the reachable optimal posture, so as to control the collection field of view of the second collection device.

11. The device according to claim 10, characterized in that The viewing angle range determination module is specifically used for: Determine the acquisition field of view corresponding to the category of the object to be identified according to the correspondence between the category and the acquisition field of view; or The size information of the object to be identified is determined according to the category of the object to be identified, and the corresponding acquisition field of view range is determined according to the size information.

12. The device according to claim 10, characterized in that The device also includes: The confidence interval determination module is used to determine the preset confidence interval corresponding to the category according to the category of the object to be identified.

13. The device according to any one of claims 8 to 12, characterized in that The control module is further used to control the second acquisition device to collect second environmental information in the acquisition field of view; The device also includes: The second identification module is used to perform a second identification of the object to be identified based on the second environmental information to obtain a second category identification result of the object to be identified.

14. The device according to claim 13, characterized in that The device also includes: an information fusion module, configured to, if it is determined according to the second category recognition result that the object to be identified exists, perform coordinate system one processing and information fusion processing on the first environment information and the second environment information to obtain fused three-dimensional information of the object to be identified, wherein the fused three-dimensional information includes size, height, and coverage area position; The control module is further used to control the movable device to perform a preset cleaning action according to the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

15. A movable device, characterized in that: The device comprises a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the movable device according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the control method of the mobile device according to any one of claims 1 to 7 are implemented.

Images