Control method and apparatus for movable device, and movable device and storage medium

Abstract

The present application relates to the field of smart home. Provided are a control method and apparatus for a movable device, and a movable device and a storage medium. The movable device comprises a device main body and a mechanical arm provided on the device main body. The device main body is provided with a first acquisition apparatus, and the mechanical arm is provided with a second acquisition apparatus. The method comprises: acquiring first environment information acquired by the first acquisition apparatus; on the basis of the first environment information, performing first recognition on an object to be recognized, so as to obtain a first category recognition result of said object; and, on the basis of the first category recognition result, controlling the position of the device main body, so as to control an acquisition field of view of the second acquisition apparatus.

Description

Control methods, devices, mobile devices, and storage media for mobile devices

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510009676.5, filed on January 2, 2025, entitled "Control Method, Apparatus, Mobile Device and Storage Medium for Mobile Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of smart homes, and in particular to a method, apparatus, mobile device, and readable storage medium for controlling a mobile device. Background Technology

[0004] As a member of smart home technology, the ability of robotic vacuum cleaners to efficiently and intelligently complete cleaning tasks largely depends on their accurate identification of the indoor environment. In related technologies, the camera of a robotic vacuum cleaner is mounted on the main body of the device. Because the main body of the device is relatively low, the camera's mounting height is also low, resulting in a poor shooting angle and consequently affecting the accuracy of indoor environment identification. Summary of the Invention

[0005] In view of this, this application provides a control method, device, mobile device, and readable storage medium for a mobile device, which solves the problem of the accuracy of indoor environment recognition by sweeping robots in related technologies.

[0006] In a first aspect, embodiments of this application provide a control method for a mobile device, the mobile device including a device body and a robotic arm disposed on the device body, the device body being provided with a first data acquisition device, and the robotic arm being provided with a second data acquisition device, the method comprising:

[0007] Obtain the first environmental information collected by the first acquisition device;

[0008] Based on the first environmental information, the first identification of the object to be identified is performed to obtain the first category identification result of the object to be identified;

[0009] Based on the first category identification result, the position of the main body of the device is controlled to control the field of view of the second acquisition device.

[0010] Secondly, embodiments of this application provide a control device for a mobile device, the mobile device including a device body and a robotic arm disposed on the device body, the device body being provided with a first data acquisition device, and the robotic arm being provided with a second data acquisition device, the device comprising:

[0011] The acquisition module is used to acquire the first environmental information collected by the first acquisition device;

[0012] The first identification module is used to perform a first identification of the object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified.

[0013] The control module is used to control the position of the main body of the device according to the first category identification result, so as to control the acquisition field of the second acquisition device.

[0014] Thirdly, embodiments of this application provide a mobile device including a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions implementing the steps of the method as described in the first aspect when executed by the processor.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0016] In this embodiment, a first acquisition device is controlled to acquire first environmental information of the current spatial environment where the mobile device is located. Based on the first environmental information, an object to be identified within the current spatial environment is identified, resulting in a first category identification result, which includes the presence of the object to be identified, the category of the object to be identified, the first confidence level corresponding to the category, and the location information of the object to be identified. The first category identification result is judged to determine whether the category identification based on the first environmental information is accurate. Based on the first category identification result, the position of the main body of the device and the pose of the robotic arm are controlled to control the acquisition field of view of the second acquisition device, thereby enabling information acquisition and secondary identification of the object to be identified through the second acquisition device.

[0017] In this embodiment, the first environmental information collected by the first acquisition device is first identified to obtain a first category identification result. Based on the first category identification result, the position of the main body of the device is controlled to control the acquisition field of view of the second acquisition device, so that the second acquisition device can be adjusted to a more suitable state for capturing environmental information, thereby acquiring environmental information with a better perspective and a wider range, and thus improving the accuracy of identification when performing secondary identification through the second acquisition device.

[0018] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 shows a schematic diagram of the structure of a mobile device according to an embodiment of this application;

[0021] Figure 2 shows a flowchart illustrating the control method of a mobile device according to an embodiment of this application;

[0022] Figure 3 shows a schematic diagram comparing the angles of the first and second acquisition devices according to an embodiment of this application;

[0023] Figure 4 shows a structural block diagram of the control device for a mobile device according to an embodiment of this application. Embodiments of the present invention

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The control method, apparatus, mobile device, and readable storage medium of the present application provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] This application provides a mobile device, which can be a cleaning robot or a household robot. Cleaning robots include sweeping robots, mopping robots, and combined sweeping and mopping robots. As shown in Figure 1, the mobile device 100 includes a main body 101, a robotic arm 102, a first acquisition device 103 disposed on the main body 101, and a second acquisition device 104 disposed on the robotic arm 102. The robotic arm 102 is a multi-degree-of-freedom robotic arm, which can include two states: a raised state and a lowered state. The robotic arm 102 can grasp and transport objects. For example, when cleaning the floor, if there is an obstacle in the cleaning path, the mobile device extends the robotic arm 102, grasps and moves the obstacle using its grippers, enabling the mobile device to clean the bottom and surrounding area of ​​the obstacle, improving floor cleaning coverage. When tidying up a room, the mobile device automatically identifies obstacles on the floor and marks their locations. It then grasps the obstacle using the grippers of the robotic arm 102 and moves it to a storage location, thus tidying up the room.

[0028] A first acquisition device 103 is installed on the main body 101 of the device, and a second acquisition device 104 is installed on the robotic arm 102. The first acquisition device 103 and the second acquisition device 104 can be RGB cameras, depth sensors, or 3D imaging sensors, etc.

[0029] When the robotic arm 102 is in the raised state, the height of the second acquisition device 104 is higher than that of the first acquisition device 103, and the field of view of the second acquisition device 104 is also higher than that of the first acquisition device 103, enabling it to acquire information with a wider field of view. Furthermore, in addition to the higher height of the second acquisition device 104, because it is mounted on the robotic arm 102, the position and orientation of the second acquisition device 104 can be adjusted by adjusting the robotic arm 102. That is, the second acquisition device 104 can be located in different positions, and the acquisition angle can also be adjusted, thereby acquiring information from multiple angles and directions.

[0030] This application provides a control method for a mobile device, as shown in Figure 2. The method includes:

[0031] Step 201: Obtain the first environmental information collected by the first acquisition device.

[0032] In this step, the first acquisition device is controlled to acquire first environmental information of the current spatial environment where the mobile device is located. If the first acquisition device is an RGB camera, the first environmental information is image information; if the first acquisition device is a depth sensor or a 3D imaging sensor, the first environmental information is 3D information.

[0033] Step 202: Based on the first environmental information, perform the first identification of the object to be identified to obtain the first category identification result of the object to be identified.

[0034] In this step, the object to be identified within the current spatial environment is identified based on the first environmental information, resulting in a first category identification result that includes the existence of the object to be identified, the category of the object to be identified, the first confidence level corresponding to the category, and the location information of the object to be identified.

[0035] Step 203: Based on the first category identification result, control the position of the main body of the device to control the acquisition field of view of the second acquisition device.

[0036] In one implementation, a first category identification result is determined, thereby determining whether the category identification based on the first environmental information is accurate. If accurate, the first category identification result is taken as the final identification result, and the position of the main body of the device is controlled without further identification; if inaccurate, the position of the main body of the device is adjusted to control the field of view of the second acquisition device, thereby enabling information acquisition and secondary identification of the object to be identified through the second acquisition device.

[0037] In another implementation, instead of judging whether the category identification based on the first environmental information is accurate, the position of the main body of the control device is adjusted after the first identification is performed to control the field of view of the second acquisition device, so as to collect information and perform secondary identification of the object to be identified through the second acquisition device.

[0038] The field of view includes at least the acquisition angle (or acquisition perspective) and acquisition position of the second acquisition device, and the acquisition position includes height, distance, etc.

[0039] In one embodiment, a second category identification result is obtained after secondary identification. The second category identification result can be used as the final identification result, or the first category identification result and the second category identification result can be combined to obtain the final identification result.

[0040] In this embodiment, the first environmental information collected by the first acquisition device is first identified to obtain a first category identification result. Based on the first category identification result, the position of the main body of the device is controlled to control the acquisition field of view of the second acquisition device, so that the second acquisition device can be adjusted to a more suitable state for capturing environmental information, thereby acquiring environmental information with a better perspective and a wider range, and thus improving the accuracy of identification when performing secondary identification through the second acquisition device.

[0041] In one embodiment of this application, the first category identification result includes the category of the object to be identified and the first confidence level corresponding to the category; according to the first category identification result, controlling the position of the main body of the device to control the field of view of the second acquisition device includes: if the first confidence level is within a preset confidence interval, then controlling the position of the main body of the device to control the field of view of the second acquisition device.

[0042] In this embodiment, it is determined whether the first confidence level is within a preset confidence interval. If the first confidence level is less than the lower limit of the preset confidence interval (e.g., the lower limit is 0.1, 0.2, 0.3, or 0.4), it indicates a high probability that the object is not in that category, and the reliability of not being in that category is relatively high. If the first confidence level is greater than the upper limit of the preset confidence interval (e.g., the upper limit is 0.8, 0.9, 0.95, or 0.98), it indicates a high probability that the object is in that category, and the reliability of being in that category is relatively high. If the first confidence level is within the preset confidence interval, it indicates that the object may or may not be in that category, the reliability is not high, and the identification is inaccurate, so further determination is needed. Therefore, the position of the main body of the control device is used to control the field of view of the second acquisition device, such as controlling the acquisition angle of the second acquisition device, to collect information and perform secondary identification of the object to be identified.

[0043] In one embodiment of this application, based on the first category identification result, the pose of the robotic arm can also be controlled to control the field of view of the second acquisition device, such as controlling the acquisition angle of the second acquisition device. That is, in one embodiment, controlling the position of the device body alone is sufficient to control the field of view of the second acquisition device. In another embodiment, controlling both the position of the device body and the pose of the robotic arm is required to control the field of view of the second acquisition device, and the position of the device body and the pose of the robotic arm can be controlled when the first confidence level is within a preset confidence interval.

[0044] In one embodiment of this application, based on the first category identification result, the position of the main body of the device and the pose of the robotic arm are controlled to control the field of view of the second acquisition device, including:

[0045] Determine the field of view corresponding to the category of the object to be identified;

[0046] Based on the location information of the object to be identified included in the first category of identification results, as well as the location information of the main body of the device, the installation angle of the second acquisition device, and the acquisition field of view, the reachable position of the main body of the device and the reachable optimal pose of the robotic arm are calculated.

[0047] The control device moves its main body to a reachable position and the control robot moves to an optimal reachable pose in order to control the field of view of the second acquisition device.

[0048] In this embodiment, based on the category of the object to be identified in the first category identification result, the desired field of view for verification is determined. Then, the field of view of the second acquisition device is controlled based on this field of view to achieve secondary identification. In this embodiment, the second acquisition device is controlled based on the field of view corresponding to the category of the object to be identified, enabling data acquisition at a more suitable angle and orientation, thus ensuring the accuracy of the secondary identification.

[0049] Specifically, based on the location information of the object to be identified, the location information of the main body of the device, and the installation angle and field of view of the second acquisition device, the reachable position of the main body of the device and the optimal pose of the robotic arm are calculated. Further, the main body of the device is controlled to move to the reachable position, and the robotic arm is controlled to move to the optimal pose, thereby controlling the field of view of the second acquisition device.

[0050] The method for calculating the reachable position of the main body of the device and the optimal reachable pose of the robotic arm is as follows: Based on the position information of the object to be identified and the position information of the main body of the device, the distance and direction between them are calculated. Combining the mobility of the main body of the device and the path planning algorithm, it is determined whether the main body of the device can reach the position of the object to be identified. If the main body of the device can reach the position of the object to be identified, the position of the object to be identified is determined as a reachable position; if the main body of the device cannot directly reach the position of the object to be identified, the nearest reachable position is calculated based on the path planning algorithm and obstacle avoidance strategy.

[0051] Based on the structural parameters of the robotic arm (such as joint length and angle limitations), kinematic calculations are performed. Combining the position information of the object to be identified, the field of view of the second acquisition device, and the kinematic calculation results of the robotic arm, an optimal pose is calculated, enabling the end effector of the robotic arm to accurately reach the position of the object to be identified, and allowing the second acquisition device to acquire clear images or data with a wider angle range.

[0052] This application enables the second acquisition device to observe the object to be identified from a more reasonable angle and orientation through the above-mentioned method.

[0053] In one embodiment of this application, determining the acquisition field of view corresponding to the category of the object to be identified includes:

[0054] Based on the correspondence between categories and acquisition field of view, determine the acquisition field of view corresponding to the category of the object to be identified; or, determine the size information of the object to be identified based on its category, and determine the corresponding acquisition field of view based on its size information.

[0055] In one embodiment, the field of view of the second acquisition device varies depending on the category. That is, the correspondence between different categories and different field of view is pre-stored, with a one-to-one mapping between categories and field of view. After determining the category of the object to be identified, the field of view corresponding to that category can be determined based on the correspondence.

[0056] In another embodiment, the size information of the object to be identified is determined based on its category. For example, if the object to be identified is a pet bed, and pet beds in a room are fixed-size items, and the size information of fixed-size items in a room is already stored, then after identifying the object to be identified as a pet bed, its size information, such as the size and height of the object to be identified, can be determined. Further, the corresponding field of view is determined based on the size information.

[0057] In this embodiment of the application, after identifying the category of the object to be identified, the field of view for controlling the second acquisition device to acquire data is determined according to the category, so as to acquire data at a more suitable angle and orientation, and ensure the accuracy of secondary identification.

[0058] In one embodiment of this application, before determining whether the first confidence level is within a preset confidence interval, the method includes:

[0059] Based on the category of the object to be identified, determine the pre-set confidence interval corresponding to the category.

[0060] In this embodiment, different categories correspond to different preset confidence intervals; that is, the correspondence between different categories and different preset confidence intervals is stored in advance, with a one-to-one mapping between categories and preset confidence intervals. After determining the category of the object to be identified, the preset confidence interval corresponding to the category can be determined based on the correspondence.

[0061] Since different categories of objects have different characteristics, their corresponding confidence intervals also differ. Setting a more reasonable confidence threshold based on the specific characteristics of each category helps to more accurately determine the reliability of the identification, thereby avoiding unnecessary misjudgments and improving recognition accuracy.

[0062] In one embodiment of this application, the method further includes:

[0063] Control the second acquisition device to acquire second environmental information within the acquisition field of view;

[0064] Based on the second environmental information, a second identification of the object to be identified is performed to obtain the second category identification result of the object to be identified.

[0065] In this embodiment, after controlling the position of the main body of the control device and the pose of the robotic arm to control the field of view of the second acquisition device, the second acquisition device acquires second environmental information within this field of view, for example, acquiring second environmental information according to a predetermined acquisition angle. Wherein, if the second acquisition device is an RGB camera, the second environmental information is image information; if the second acquisition device is a depth sensor or a 3D imaging sensor, the second environmental information is 3D information. Furthermore, based on the second environmental information, the object to be identified within the current spatial environment is identified, resulting in a second category identification result that includes the presence of the object to be identified, the category of the object to be identified, the second confidence level corresponding to that category, and the location information of the object to be identified.

[0066] As shown in Figure 3, when the robotic arm is in the raised state, the height of the second acquisition device 104 is higher than that of the first acquisition device 103. The field of view of the second acquisition device 104 is larger than that of the first acquisition device 103, enabling it to acquire data with a wider viewing angle. That is, for capturing images of the object to be identified, the field of view B is larger than the field of view C. Furthermore, besides the higher height of the second acquisition device, because it is mounted on the robotic arm, its position and orientation can be adjusted by adjusting the robotic arm. This means the second acquisition device can be positioned in different locations, and the acquisition angle can also be adjusted, thereby acquiring data from multiple angles and directions.

[0067] In this embodiment of the application, when the recognition result corresponding to the first acquisition device installed on the main body of the device is unreliable, the second acquisition device installed on the robotic arm is used to provide a more reasonable perspective for observation and improve the recognition accuracy.

[0068] In this embodiment of the application, the second acquisition device will only be invoked for secondary identification when the confidence level of the first category identification result falls within the unreliable range. This on-demand invocation method can save computing resources and time and avoid unnecessary waste of resources.

[0069] In one embodiment of this application, the method further includes: if it is determined that there is an object to be identified based on the second category identification result, then coordinate system processing and information fusion processing are performed on the first environmental information and the second environmental information to obtain fused three-dimensional information of the object to be identified, the fused three-dimensional information including size, height, and coverage area location.

[0070] In this embodiment, since both the first and second acquisition devices have blind spots, in order to improve the accuracy of recognition, after determining that there is an object to be identified in the spatial environment, the first environmental information and the second environmental information are processed by coordinate system integration and information fusion to obtain fused three-dimensional information such as the size and coverage area of ​​the object to be identified.

[0071] Specifically, if the first and second acquisition devices are RGB cameras, then both the first and second environmental information are image information. The depth or point cloud information of the object to be identified is estimated based on optical flow and image segmentation. If the first and second acquisition devices are depth sensors or 3D imaging sensors, then both the first and second environmental information are depth or point cloud information. Thus, the depth / point cloud information of the object to be identified relative to the first acquisition device is obtained through the first acquisition device, and the depth / point cloud information relative to the second acquisition device is obtained through the second acquisition device. However, because their coordinate systems are inconsistent, the observation surfaces of the object to be identified that they can observe may also be inconsistent. Therefore, it is necessary to unify the coordinate systems and perform data fusion.

[0072] In one embodiment, coordinate system processing may include: selecting a global reference coordinate system, which may be a three-dimensional spatial coordinate system based on the device body; calibrating the first and second acquisition devices to determine their respective positions and orientations in the global reference coordinate system; the calibration process typically includes determining parameters such as the sensor mounting position, angle, and focal length of the acquisition devices, and calculating the relationship between these parameters and the global reference coordinate system; and, based on the calibration results of the acquisition devices, transforming the coordinate points of the depth information or point cloud information acquired by the first and second acquisition devices from their respective local coordinate systems to the global reference coordinate system.

[0073] In one embodiment, the information fusion process may include: extracting feature information related to the object to be identified from the depth information or point cloud information acquired by the first acquisition device and the depth information or point cloud information acquired by the second acquisition device. This feature information may include edges, corners, textures, colors, etc. A stereo matching algorithm is used to match feature points in the images acquired by the first and second acquisition devices to find their correspondences. Based on the matching results, a disparity map is generated. The disparity map is a two-dimensional image where the value of each pixel represents the disparity value of the corresponding point in the images from the two acquisition devices. Finally, the results of feature extraction and disparity calculation are combined to obtain the three-dimensional information of the object to be identified.

[0074] In this embodiment, there is no need for the mobile device to observe the object around its perimeter. Instead, it obtains more accurate three-dimensional information of the object by fusing the first and second acquisition devices, and improves the accuracy of depth calculation based on the principle of binocular vision.

[0075] In one embodiment of this application, the method further includes: controlling the mobile device to perform a preset cleaning action based on the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

[0076] In this embodiment, 3D information, such as the category, size, and location of the object to be identified, is fused to control the mobile device to perform preset cleaning actions. Examples include targeted cleaning strategies for flat, wet stains and avoidance cleaning strategies for pet beds.

[0077] In this embodiment, by performing two identifications, the accuracy of identifying the object to be identified and the accuracy of estimating the size of the object to be identified are improved, thereby implementing the corresponding cleaning action and improving the cleaning effect.

[0078] As a specific implementation of the control method for the aforementioned mobile device, this application provides a control device for a mobile device. The mobile device includes a device body and a robotic arm disposed on the device body. The device body is provided with a first acquisition device, and the robotic arm is provided with a second acquisition device. As shown in FIG4, the control device 400 for the mobile device includes: an acquisition module 401, a first identification module 402, and a control module 403.

[0079] The acquisition module 401 is used to acquire the first environmental information collected by the first acquisition device.

[0080] The first identification module 402 is used to perform a first identification of the object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified.

[0081] The control module 403 is used to control the position of the main body of the device according to the first category identification result, so as to control the acquisition field of the second acquisition device.

[0082] Furthermore, the first category identification result includes the category of the object to be identified and the first confidence level corresponding to the category. The control module 403 is specifically used for:

[0083] If the first confidence level is within the preset confidence level range, the position of the main body of the control device is adjusted to control the field of view of the second acquisition device.

[0084] Furthermore, the control module can also control the pose of the robotic arm based on the first category recognition result, so as to control the acquisition field of view of the second acquisition device;

[0085] The device also includes:

[0086] The field of view determination module is used to determine the acquisition field of view corresponding to the category of the object to be identified.

[0087] The calculation module is used to calculate the reachable position of the device body and the optimal reachable pose of the robotic arm based on the position information of the object to be identified included in the first category of identification results, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view.

[0088] The control module 403 is specifically used to control the main body of the device to move to an accessible position and to control the robotic arm to move to an optimal position, so as to control the field of view of the second acquisition device.

[0089] Furthermore, the viewpoint range determination module is specifically used for:

[0090] Based on the correspondence between categories and the field of view, determine the field of view corresponding to the category of the object to be identified; or,

[0091] The size information of the object to be identified is determined based on its category, and the corresponding field of view is determined based on the size information.

[0092] Furthermore, the device also includes:

[0093] The confidence interval determination module is used to determine the pre-set confidence interval corresponding to the category of the object to be identified.

[0094] Furthermore, the control module 403 is also used to control the second acquisition device to acquire second environmental information within the acquisition field of view;

[0095] The device also includes:

[0096] The second identification module is used to perform a second identification of the object to be identified based on the second environmental information, and to obtain a second category identification result of the object to be identified.

[0097] Furthermore, the device also includes:

[0098] The information fusion module is used to perform coordinate system processing and information fusion processing on the first environmental information and the second environmental information if it is determined that there is an object to be identified based on the second category identification result, so as to obtain the fused three-dimensional information of the object to be identified. The fused three-dimensional information includes size, height, and coverage area location.

[0099] The control module 403 is also used to control the mobile device to perform preset cleaning actions based on the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

[0100] The control device 400 for the mobile device in this embodiment can be the mobile device itself, or a component within the mobile device, such as an integrated circuit or a chip. The control device 400 for the mobile device provided in this embodiment can implement the various processes implemented in the control method embodiment of the mobile device in FIG1; to avoid repetition, these processes will not be described again here.

[0101] This application also provides a mobile device, which includes a processor and a memory. The memory stores a program or instructions that can run on the processor. When the program or instructions are executed by the processor, they implement the various steps of the control method embodiment of the mobile device described above and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0102] Memory can be used to store software programs and various data. Memory can primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area can store the operating system, application programs or instructions required for at least one function (such as sound playback, environmental information playback, etc.). Furthermore, memory can include volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0103] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and applications, while the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor.

[0104] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described control method embodiments of the mobile device and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0105] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0106] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A control method for a mobile device, wherein, The mobile device includes a device body and a robotic arm disposed on the device body. The device body is provided with a first data acquisition device, and the robotic arm is provided with a second data acquisition device. The method includes: Obtain the first environmental information collected by the first acquisition device; Based on the first environmental information, the first identification of the object to be identified is performed to obtain the first category identification result of the object to be identified; Based on the first category identification result, the position of the main body of the device is controlled to control the field of view of the second acquisition device.

2. The method according to claim 1, wherein, The first category identification result includes the category of the object to be identified and the first confidence level corresponding to the category; The step of controlling the position of the main body of the device according to the first category identification result, so as to control the field of view of the second acquisition device, includes: If the first confidence level is within a preset confidence level range, the position of the main body of the device is controlled to control the field of view of the second acquisition device.

3. The method according to claim 2, wherein, Based on the first category identification result, the pose of the robotic arm can also be controlled to control the field of view of the second acquisition device; Based on the first category identification result, the position of the main body of the device and the pose of the robotic arm are controlled to control the field of view of the second acquisition device, including: Based on the category of the object to be identified, determine the acquisition field of view corresponding to the category; Based on the location information of the object to be identified included in the first category identification result, the location information of the main body of the device, the installation angle of the second acquisition device, and the acquisition field of view, the reachable position of the main body of the device and the reachable optimal pose of the robotic arm are calculated. The device body is controlled to move to the reachable position, and the robotic arm is controlled to move to the reachable optimal pose, so as to control the field of view of the second acquisition device.

4. The method according to claim 3, wherein, The step of determining the acquisition field of view corresponding to the category of the object to be identified includes: Based on the correspondence between categories and the field of view, determine the field of view corresponding to the category of the object to be identified; or... The size information of the object to be identified is determined based on the category of the object to be identified, and the corresponding field of view is determined based on the size information.

5. The method according to claim 3, wherein, Before determining whether the first confidence level is within a preset confidence interval, the method includes: Based on the category of the object to be identified, the preset confidence interval corresponding to the category is determined.

6. The method according to any one of claims 1 to 5, wherein, The method further includes: Control the second acquisition device to acquire second environmental information within the acquisition field of view; Based on the second environmental information, a second identification of the object to be identified is performed to obtain a second category identification result for the object to be identified.

7. The method according to claim 6, wherein, The method further includes: If the existence of the object to be identified is determined based on the second category identification result, then coordinate system processing and information fusion processing are performed on the first environmental information and the second environmental information to obtain the fused three-dimensional information of the object to be identified. The fused three-dimensional information includes size, height, and coverage area location. Based on the category of the object to be identified included in the second category identification result, and the fused three-dimensional information, the mobile device is controlled to perform a preset cleaning action.

8. A control device for a mobile device, wherein, The mobile device includes a device body and a robotic arm disposed on the device body. The device body is provided with a first data acquisition device, and the robotic arm is provided with a second data acquisition device. The device includes: The acquisition module is used to acquire the first environmental information collected by the first acquisition device; The first identification module is used to perform a first identification of the object to be identified based on the first environmental information, and obtain a first category identification result of the object to be identified. The control module is used to control the position of the main body of the device according to the first category identification result, so as to control the acquisition field of the second acquisition device.

9. The apparatus according to claim 8, wherein, The first category identification result includes the category of the object to be identified and the first confidence level corresponding to the category. The control module is used for: If the first confidence level is within a preset confidence level range, the position of the main body of the device is controlled to control the field of view of the second acquisition device.

10. The apparatus according to claim 9, wherein, The control module can also control the pose of the robotic arm based on the first category recognition result, so as to control the acquisition field of view of the second acquisition device; The device further includes: The viewing angle range determination module is used to determine the acquisition field of view range corresponding to the category of the object to be identified; The calculation module is used to calculate the reachable position of the device body and the reachable optimal pose of the robotic arm based on the position information of the object to be identified included in the first category identification result, the position information of the device body, the installation angle of the second acquisition device, and the acquisition field of view. The control module is used to control the main body of the device to move to the reachable position and to control the robotic arm to move to the reachable optimal pose, so as to control the field of view of the second acquisition device.

11. The apparatus according to claim 10, wherein, The view range determination module is used for: Based on the correspondence between categories and the field of view, determine the field of view corresponding to the category of the object to be identified; or... The size information of the object to be identified is determined based on the category of the object to be identified, and the corresponding field of view is determined based on the size information.

12. The apparatus according to claim 10, wherein, The device further includes: The confidence interval determination module is used to determine the preset confidence interval corresponding to the category of the object to be identified.

13. The apparatus according to any one of claims 8 to 12, wherein, The control module is also used to control the second acquisition device to acquire second environmental information under the acquisition field of view; The device further includes: The second identification module is used to perform a second identification of the object to be identified based on the second environmental information, and obtain a second category identification result for the object to be identified.

14. The apparatus according to claim 13, wherein, The device further includes: The information fusion module is used to perform coordinate system processing and information fusion processing on the first environmental information and the second environmental information if it is determined that the object to be identified exists according to the second category identification result, so as to obtain the fused three-dimensional information of the object to be identified. The fused three-dimensional information includes size, height, and coverage area location. The control module is further configured to control the mobile device to perform a preset cleaning action based on the category of the object to be identified included in the second category identification result and the fused three-dimensional information.

15. A mobile device, wherein, It includes a processor and a memory, the memory storing a program or instructions that run on the processor, the program or instructions being executed by the processor to implement the steps of the control method for the mobile device as described in any one of claims 1 to 7.

16. A computer-readable storage medium having a program or instructions stored thereon, wherein, When the program or instructions are executed by the processor, they implement the steps of the control method for the mobile device as described in any one of claims 1 to 7.

Images