Cleaning device control method and apparatus, electronic device, storage medium, and cleaning device

Abstract

A cleaning device control method and apparatus, a cleaning device, and a related device. The method comprises: controlling to turn on a first image acquisition apparatus (211) and / or a second image acquisition apparatus (221); when one of the first image acquisition apparatus (211) and the second image acquisition apparatus (221) is in a valid state and the other is in an invalid state, in response to a switching instruction of a user, a cleaning device switching the image acquisition apparatus in the valid state to the invalid state, and switching the image acquisition apparatus in the invalid state to the valid state (S120); and the cleaning device adjusting pose information of a mechanical arm module (210) and / or adjusting position information of a chassis (220) to update a first image and / or a second image (S130).

Description

Cleaning device control method, control apparatus, electronic device, storage medium, and cleaning device

[0001] Cross Reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 2024113225219, filed on September 20, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of smart home, and more particularly, to a cleaning device control method, a cleaning device control apparatus, an electronic device, a storage medium, and a cleaning device. BACKGROUND

[0004] At present, a camera is arranged on a cleaning device, so that a user can remotely view environmental information around the cleaning device through the camera during cleaning and other movements of the cleaning device by using an application software (APP) of a smart device, and even remotely communicate. However, how to see a larger range of environmental information or better remotely communicate is a technical problem to be solved. SUMMARY

[0005] A series of simplified concepts are introduced in the summary section, which will be further described in detail in the specific embodiment section. The summary section of the present disclosure does not mean to attempt to limit the key features and necessary technical features of the claimed technical solutions, and even less to determine the protection scope of the claimed technical solutions.

[0006] The present disclosure provides a cleaning device control method, a cleaning device control apparatus, an electronic device, a storage medium, and a cleaning device, which are intended to at least be able to solve the technical problem of how to see a larger range of environmental information or better remotely communicate.

[0007] In a first aspect, the disclosure provides a cleaning device control method. The cleaning device includes a mechanical arm module and a chassis. The mechanical arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device. The method includes: the cleaning device controlling the first image acquisition device and / or the second image acquisition device to be turned on in response to a video instruction of a user, wherein the first image acquisition device is used to acquire a first image within a corresponding field of view, and the second image acquisition device is used to acquire a second image within a corresponding field of view; in a case where one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, the cleaning device switches the image acquisition device in the active state to the inactive state and the image acquisition device in the inactive state to the active state in response to a switching instruction of the user; and the cleaning device adjusts pose information of the mechanical arm module and / or adjusts position information of the chassis to update the first image and / or the second image.

[0008] In some embodiments, adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis includes:

[0009] Adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis in response to a state adjustment instruction of the user.

[0010] In some embodiments, adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis includes:

[0011] Performing target detection on the first image and / or the second image.

[0012] Positioning a target object based on a target detection result.

[0013] Adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis based on positioning information of the target object.

[0014] In some embodiments, performing target detection on the first image and / or the second image includes:

[0015] Obtaining a bounding box of the target object in the first image and / or the second image.

[0016] Adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis based on the positioning information of the target object includes:

[0017] Obtaining a center point of the bounding box of the target object and a preset center point offset, and adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis based on the offset to position the target object at a center position of the first image and / or the second image.

[0018] In some embodiments, the method further includes:

[0019] In a case where the first image acquisition device is in an active state, the mechanical arm module is controlled to extend in an initial posture to obtain a first image, wherein the first image is a fixed-angle image of an environment in which the cleaning device is located;

[0020] Adjusting the posture information of the mechanical arm module and / or adjusting the position information of the chassis includes:

[0021] Performing target detection on the first image;

[0022] Based on the target detection result, the posture information of the mechanical arm module and / or the position information of the chassis are adjusted to update the first image, and the adjusting action is stopped when the target object appears in the updated first image.

[0023] In some embodiments, the method further includes:

[0024] Displaying a user interface, wherein the user interface includes a map display area;

[0025] In response to a target point setting instruction of a user, the cleaning device is controlled to move to a position where the target point is located to obtain a first image and / or a second image at the position where the target point is located by using the first image acquisition device and / or the second image acquisition device, wherein the user interface further includes an image display area, the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

[0026] In some embodiments, the method further includes:

[0027] Comparing the target detection result with a pre-stored whitelist image to obtain a comparison result;

[0028] Based on the comparison result, identity information of the detected target object is determined to perform an alarm in a case where the identity information indicates that the target object is a stranger.

[0029] In some embodiments, the method further includes:

[0030] Based on the identity information of the target object, current state information of the target object is detected;

[0031] The current state information is compared with pre-stored state information of the target object to obtain a state monitoring result.

[0032] In a second aspect, a cleaning device control apparatus is provided. The cleaning device includes a robot arm module and a chassis. The robot arm module is provided with a first image acquisition device. The chassis is provided with a second image acquisition device. The control apparatus includes: an image acquisition unit configured to control the cleaning device to turn on the first image acquisition device and / or the second image acquisition device in response to a video instruction from a user. The first image acquisition device is configured to acquire a first image within a corresponding field of view. The second image acquisition device is configured to acquire a second image within a corresponding field of view. The control apparatus further includes: a switching unit configured to control the cleaning device to switch the image acquisition device in an active state to an inactive state and switch the image acquisition device in an inactive state to an active state in response to a switching instruction, when one of the first image acquisition device and the second image acquisition device is in the active state and the other is in the inactive state. The control apparatus further includes: an adjustment control unit configured to control the cleaning device to adjust pose information of the robot arm module and / or adjust position information of the chassis to update the first image and / or the second image.

[0033] In a third aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory stores computer program instructions executable by the processor. The computer program instructions, when executed by the processor, implement the cleaning device control method as described above.

[0034] In a fourth aspect, a storage medium is provided. The storage medium stores computer program instructions. The computer program instructions, when executed by a processor, implement the cleaning device control method as described above.

[0035] In a fifth aspect, a cleaning device is provided. The cleaning device includes: an electronic device as described above; and a robot arm module and a chassis. The robot arm module is provided with a first image acquisition device. The chassis is provided with a second image acquisition device.

[0036] In some embodiments, the first image acquisition device is disposed at a central position of a robot arm end of the robot arm module.

[0037] According to the technical solution, the cleaning device outputs the first image and the second image to the user through the first image acquisition device and the second image acquisition device, and in the case that one of the first image acquisition device and the second image acquisition device is in an effective state, the cleaning device can switch the states of the two image acquisition devices in response to the switching instruction of the user, so as to provide the image collected by the image acquisition device in the effective state to the user, and in addition, the content of the first image or the second image can be updated by adjusting the pose of the mechanical arm module or adjusting the position of the chassis. Since the mechanical arm module has high degrees of freedom, the first image can be viewed from multiple angles and multiple directions, and richer environmental information is provided to the user. The above switching function can provide more choices for the user, meet different needs of various users, and improve the user experience.

[0038] The cleaning device control method of the present disclosure, other advantages, objects and features of the present disclosure will be embodied in part by the following description, and will be understood by those skilled in the art through research and practice of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0041] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of exemplary embodiments. The accompanying drawings are included to provide a description of exemplary embodiments and are not meant to limit the present application. Moreover, the same reference numerals are used throughout the accompanying drawings to represent the same elements. In the drawings:

[0042] FIG. 1 is a schematic flowchart of a cleaning device control method according to some embodiments of the present disclosure;

[0043] FIG. 2 is a schematic structural block diagram of a cleaning device according to some embodiments of the present disclosure;

[0044] FIG. 3 is a schematic diagram of a user interface according to some embodiments of the present disclosure;

[0045] FIG. 4 is a schematic block diagram of a cleaning device control device according to some embodiments of the present disclosure;

[0046] Fig. 5 is a schematic block diagram of an electronic device according to some embodiments of the present disclosure; and

[0047] Fig. 6 is a schematic structural block diagram of another cleaning device according to some embodiments of the present disclosure. Embodiments of the present application

[0048] In order to enable a more complete understanding of the above-mentioned objects, features and advantages of the present application, the following will further describe the solutions of the present application. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0049] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and the above-mentioned drawings (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be described clearly and completely in the following description of the embodiments of the present application in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.

[0050] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein; obviously, the embodiments in the description are only a part of the embodiments of the present application, not all the embodiments.

[0051] Generally, the height of the main body of the cleaning device such as the sweeping robot is about 10 centimeters or so, and the camera arranged on the chassis will result in a relatively low viewing angle of the camera, and the user can see limited environmental information, and cannot view the overall picture of the home information; especially when making a remote call, through the camera on the chassis, only the legs of the standing person at a very short distance of, for example, 1 meter in front of the device can be seen, and the face of the close-range person cannot be seen, which brings a very poor video call experience to the user; and since the camera is fixedly installed, the video angle cannot be adjusted according to the face position of the long-distance person.

[0052] To solve the above technical problems, according to a first aspect of the present application, a cleaning device control method is provided. FIG. 1 is a schematic flowchart of a cleaning device control method 100 according to some embodiments of the present disclosure. FIG. 2 is a schematic structural block diagram of a cleaning device according to some embodiments of the present disclosure. As shown in FIG. 2, the cleaning device 200 can include a mechanical arm module 210 and a chassis 220, and the mechanical arm module 210 can be arranged on the chassis 220. The mechanical arm module 210 can have multiple degrees of freedom, and can be capable of telescopic folding, rotating around corresponding joints or the chassis, and the like. The mechanical arm module 210 has a gripper at the end, which can grasp objects. The chassis 220 can include driving wheels, cleaning components, and the like. The cleaning device 200 can be driven by the chassis 220 to perform actions such as turning, accelerating, or decelerating. For example, under the drive of the driving wheels, the cleaning device 200 can move on the surface to be cleaned, and use the cleaning components to clean the surface to be cleaned. The chassis 220 and the mechanical arm module 210 can move individually in response to control signals, and can also move cooperatively to complete corresponding operations. For example, the first image acquisition device 211 is arranged on the mechanical arm module 210, and the second image acquisition device 221 is arranged on the chassis 220. Both the first image acquisition device 211 and the second image acquisition device 221 can be image acquisition devices such as cameras. In some embodiments, the first image acquisition device 211 can be arranged at the center of the end of the mechanical arm module, which can better ensure the image acquisition effect of the first image acquisition device 211. As shown in FIG. 1, the method 100 can include the following steps S110 to S130.

[0053] In step S110, the cleaning device controls the first image acquisition device and / or the second image acquisition device to be turned on in response to a video instruction of a user; the first image acquisition device is used to acquire a first image within a corresponding field of view, and the second image acquisition device is used to acquire a second image within a corresponding field of view.

[0054] Exemplarily, the user can establish a communication connection with the cleaning device through an application software (APP), and send a video instruction to the cleaning device to start the remote video function of the cleaning device. After receiving the video instruction of the user, the cleaning device can control the first image acquisition device on the mechanical arm module to start, or control the second image acquisition device on the chassis to start, or control the first image acquisition device and the second image acquisition device to start at the same time. It should be noted that, since the installation positions of the first image acquisition device and the second image acquisition device are different; therefore, in the case that the first image acquisition device and the second image acquisition device start at the same time, the first image acquired by the first image acquisition device within the corresponding field of view and the second image acquired by the second image acquisition device within the corresponding field of view can contain different environmental information or different parts of a person. Exemplarily, in response to the operation of the user, a user interface can be displayed on the APP, and the user can remotely interact with the cleaning device through the user interface. The user interface can include a video display area for displaying the first image or the second image; when the first image acquisition device and the second image acquisition device start at the same time, the video display area can be divided into two parts to display the first image and the second image respectively.

[0055] In step S120, in the case that one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, the cleaning device switches the image acquisition device in the active state to the inactive state and switches the image acquisition device in the inactive state to the active state in response to the switching instruction of the user.

[0056] Exemplarily, the following description takes the first image acquisition device as an example. In the case that the first image acquisition device is turned on and the corresponding first image is displayed on the user interface, it can be considered that the first image acquisition device is in an active state at this time; in addition to this case, it can be considered that the first image acquisition device is in an inactive state. When one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state; for example, the first image acquisition device is in an active state and the second image acquisition device is in an inactive state, the user can send a switching instruction to the cleaning device through the APP, for example, by clicking the icon of the first image acquisition device or the second image acquisition device on the user interface to generate the switching instruction. After receiving the switching instruction, the cleaning device can switch the first image acquisition device to the inactive state and switch the second image acquisition device to the active state; at this time, the second image acquired by the second image acquisition device is displayed on the user interface of the APP.

[0057] In step S130, the cleaning device adjusts the pose information of the mechanical arm module and / or adjusts the position information of the chassis to update the first image and / or the second image.

[0058] For example, the cleaning device can move on the chassis to "walk" in the whole house under the control of the cruising instruction. During the movement of the chassis, the second image acquisition device on the chassis is in an effective state, and the second image can be acquired by using the second image acquisition device. With the movement of the chassis, the image content of the acquired second image is constantly updated. For example, the first image acquisition device is switched to the effective state in response to the switching instruction. At this time, the first image is acquired by using the first image acquisition device, and the chassis can also move in the whole house. During the movement, the image content of the first image is constantly updated. In other embodiments, during the movement of the chassis, the mechanical arm module can be extended, rotated, or the like in a preset manner. During the movement, the image content of the first image is also constantly updated. For the respective movements of the mechanical arm module and the chassis, the two can be performed simultaneously, or in time division, or one of them can be performed. Regardless of which group of actions is performed, the image content of the first image is constantly updated with the corresponding action.

[0059] According to the above technical solution, the cleaning device outputs the first image and the second image to the user by using the first image acquisition device and the second image acquisition device. In the case where one of the first image acquisition device and the second image acquisition device is in an effective state, the cleaning device can switch the states of the two image acquisition devices in response to the switching instruction of the user, so as to provide the image acquired by the image acquisition device in the effective state to the user. In addition, the content of the first image or the second image can be updated by adjusting the pose of the mechanical arm module or adjusting the position of the chassis. Since the mechanical arm module has a high degree of freedom, the first image can be viewed from multiple angles and multiple directions, and more abundant environmental information can be provided to the user. The above switching function can provide more choices for the user to meet the different needs of various users, thereby improving the user experience.

[0060] In some embodiments, the step S130 of adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis can include: adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis in response to the state adjustment instruction of the user.

[0061] FIG. 3 is a schematic diagram of a user interface according to some embodiments of the present disclosure. In the user interface shown in FIG. 3, only one image acquisition device is in an active state, and thus only one image is displayed in the video display area of the user interface. The video display area is the area in which the scene image is displayed in FIG. 3. As shown in FIG. 3, the user interface can be provided with position adjustment components 311 and 312, so that the user can send state adjustment instructions to the cleaning device through the position adjustment component 311 and the position adjustment component 312. For example, in the case where the second image acquisition device is in an active state and the video display area of the user interface displays only the second image, the user can use the position adjustment component 311 to adjust the position of the chassis; for example, by clicking the downward arrow on the left side of the position adjustment component 311, the user can control the chassis to move backward, and by clicking the upward arrow on the right side of the position adjustment component 311, the user can control the chassis to move forward; the step size of the chassis moving forward or backward controlled by a single or continuous click of the position adjustment component 311 can be reasonably set in the underlying logic building, which is not limited herein. For example, in the case where the first image acquisition device is in an active state and the video display area of the user interface displays only the first image, the user can use the position adjustment component 311 and the position adjustment component 312 to jointly control the pose of the robotic arm module. For example, by clicking the downward arrow on the left side of the position adjustment component 311, the user can control the robotic arm end of the robotic arm module to descend; by clicking the upward arrow on the right side of the position adjustment component 311, the user can control the robotic arm end to ascend; by clicking the counterclockwise rotation component on the left side of the position adjustment component 312, the user can control the robotic arm module to rotate counterclockwise relative to the chassis or control the robotic arm end of the robotic arm module to rotate counterclockwise relative to other joints, components, etc. in the robotic arm module; by clicking the clockwise rotation component on the right side of the position adjustment component 312, the user can control the robotic arm module to rotate clockwise relative to the chassis or control the robotic arm end of the robotic arm module to rotate clockwise relative to other joints, components, etc. in the robotic arm module; the robotic arm end of the robotic arm module is provided with the first image acquisition device. After adjusting the pose of the robotic arm module or the position of the chassis, the image acquisition area of the first image acquisition device or the second image acquisition device also changes accordingly; correspondingly, the content of the acquired first image or second image also changes, thereby achieving the updating of the image content by adjusting the pose of the robotic arm module and the position of the chassis.

[0062] It should be noted that the above position adjustment of the robotic arm module and the chassis by the position adjustment component is only exemplary and does not mean a limitation on the adjustment method of the robotic arm module and the chassis. In actual applications, any reasonable method can be used to adjust the position of the robotic arm module and the chassis based on the state adjustment instructions of the user.

[0063] Thus, the pose of the mechanical arm module and the position of the chassis can be controlled through the above-mentioned interactive manner, and the first image and the second image can be updated. The updated first image and second image include the content information that the user expects to obtain, so as to better meet the user's needs and improve the user's experience.

[0064] For example, the cleaning device can be provided with a sound playing device such as a loudspeaker, and can also be provided with a sound collecting device such as a microphone. The user can make a remote video call with the person in front of the cleaning device through the APP.

[0065] In some embodiments, the method further comprises:

[0066] When the first image collecting device is in an active state, the mechanical arm module is controlled to extend in an initial pose to obtain a first image. The first image is a fixed angle image of the environment in which the cleaning device is located.

[0067] For example, the initial pose can be a pre-set pose before leaving the factory. When the first image collecting device is in an active state, the mechanical arm module extends in the pre-set pose to obtain a fixed angle image of the environment in the pose.

[0068] In this embodiment, the step S130 of adjusting the pose information of the mechanical arm module and / or adjusting the position information of the chassis can include steps S134-S135.

[0069] In step S134, target detection is performed on the first image.

[0070] For example, any existing or future target detection algorithm can be used to perform target detection on the first image. The target includes static or dynamic living or non-living things such as people, pets, potted plants, or furniture.

[0071] In step S135, based on the target detection result, the pose information of the mechanical arm module and / or the position information of the chassis are adjusted to update the first image, and the adjustment action is stopped when the target appears in the updated first image.

[0072] For example, the target detection result can represent the probability of the target detected in the current first image. If the probability is less than or equal to the preset probability, it can be considered that the target does not appear in the first image. At this time, the pose information of the robotic arm module or the position information of the chassis can be adjusted separately or jointly. Based on the adjusted robotic arm module and chassis, a new first image can be obtained using the first image acquisition device set on the robotic arm module, and target detection can be performed on the new first image until the probability of the target detected in the new first image is greater than the preset probability, such as the aforementioned 80%. Then it can be determined that a target has appeared in the field of view of the current first image acquisition device, so as to realize the subsequent tracking of the target.

[0073] Therefore, when using the first image acquisition device for remote video viewing, the robotic arm module or chassis can be adjusted by target positioning so that the target object can appear in the first image, that is, the target object is found before target tracking begins.

[0074] In some implementations, step S130, which adjusts the pose information of the robotic arm module and / or the position information of the chassis, may include steps S131-S133.

[0075] In step S131, target detection is performed on the first image and / or the second image.

[0076] For ease of description and understanding, the following detailed explanation uses the first image as an example. Based on the description of the first image, the relevant scheme for the second image can be understood. For example, after acquiring the first image using the first image acquisition device, object detection can be performed on the first image. The target here includes the aforementioned person conducting a video call locally, and can also include static or dynamic living or non-living things appearing in the current scene, such as pets, potted plants, or furniture. Any existing or future object detection algorithm can be used to perform object detection on the first image to obtain the object detection result.

[0077] In step S132, the target object is located based on the target detection results.

[0078] For example, the target detection result can represent the probability of the target detected in the current first image. When the detected target is a person and the probability is greater than a preset probability, such as 80%, the target can be identified. In this embodiment, the target is the person making a video call locally, who appears in the first image. At this time, based on the first image, the current position information of the cleaning equipment, and the pose information of the robotic arm module, the position information of the target in the room can be determined, thereby achieving the positioning of the target.

[0079] In step S133, the pose information of the mechanical arm module and / or the position information of the chassis is adjusted based on the positioning information of the target object.

[0080] For example, after the positioning information of the target object corresponding to the current first image is determined, a new first image can be obtained by adjusting the pose of the mechanical arm module. For example, when the moving range of the target object exceeds the action range of the mechanical arm module or the target object moves out of the current room, a new first image can be obtained by combining the movement of the chassis. The target object is ensured to be in the new first image, and the positioning tracking of the target object is realized. The adjustment method of the pose information of the mechanical arm module and the position information of the chassis has been described in detail above, and will not be described here. For example, the above scheme can be executed after the user clicks the "one-key pet searching" component in the user interface shown in FIG. 3. It should be noted that the "one-key pet searching" function can be realized by automatically adjusting the mechanical arm module and the chassis through the above target detection and target object positioning tracking. It can also be realized by manually adjusting the mechanical arm module and the chassis. Alternatively, human intervention can be made at any time during the automatic adjustment, and the position information of the chassis or the pose information of the mechanical arm module can be manually adjusted to supplement the operation of the target object tracking which cannot be effectively realized by the automatic adjustment method. Thus, the mechanical arm module and the chassis can be controlled in various ways to realize the positioning tracking of the target object.

[0081] In some embodiments, the target detection on the first image and / or the second image in step S131 can include obtaining a bounding box of the target object in the first image and / or the second image.

[0082] For example, the target detection can include obtaining a bounding box of the target object in the first image. The bounding box can be a minimum outer envelope rectangular box of the target object.

[0083] In step S133, the pose information of the mechanical arm module and / or the position information of the chassis is adjusted based on the positioning information of the target object.

[0084] Exemplarily, after the bounding box of the target object is acquired, a center point of the bounding box can be determined, for example, in the case where the bounding box is a rectangular box, the intersection of the diagonal lines of the rectangular box can be determined as the center point of the rectangular box. Exemplarily, the preset center point can be the center point of the first image, in the case where the first image is a rectangle, the preset center point is the intersection of the diagonal lines of the first image. After the center point of the bounding box is determined, the offset between the center point of the bounding box and the preset center point can be calculated; the offset can be expressed in the form of coordinates, or can be expressed by using an offset angle and a straight-line distance between the center point of the bounding box and the preset center point, which is not limited herein. Based on the determined offset, the pose information of the mechanical arm module is adjusted correspondingly, the first image acquisition device on the mechanical arm module after the pose adjustment is used to obtain a new first image again, and a new center point of the bounding box of the target object in the new first image is acquired again, so that the new center point coincides with the preset center point, that is, the target object is located at the center position of the first image.

[0085] Therefore, it can be ensured that the target object is located in the middle of the image, and excellent video experience is brought to the user.

[0086] It should be noted that, in combination with the foregoing description, the first image acquisition device and the second image acquisition device can acquire images simultaneously, or can acquire images alternatively in response to a switching instruction. By reading the above-mentioned related technical solutions for processing the first image, the specific implementation manners of the two schemes of simultaneously acquiring the first image and the second image and only acquiring the second image can be understood, and for the sake of brevity, will not be described in detail. In addition, the above steps S134-S135 can be executed before the steps S131-S133 in the embodiment in which the first image acquisition device is in an effective state.

[0087] In some embodiments, the method can further include:

[0088] displaying a user interface, the user interface including a map display area.

[0089] Exemplarily, as described above, in response to the operation of the user, the user interface can be displayed on the APP, and the user can remotely interact with the cleaning device through the user interface. As shown in FIG. 3, the user interface can include a map display area 320; the map displayed by the map display area can be a global image acquired by the cleaning device, and can represent the distribution of each room in the entire home.

[0090] In response to a target point setting instruction of the user, the cleaning device is controlled to move to the position of the target point, so as to acquire the first image and / or the second image at the position of the target point by using the first image acquisition device and / or the second image acquisition device; the user interface further includes an image display area, the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

[0091] For example, the user can set a target point by first clicking a "point-to-point" component in the user interface and then clicking any point in the map display area 320. Then, according to the position of the target point in the map display area 320, the actual position of the target point is determined, and the cleaning device is controlled to move to the actual position in the real environment scene. At this position, the first image can be acquired by using the first image acquisition device, the second image can be acquired by using the second image acquisition device, or the first image and the second image can be acquired by using the two image acquisition devices at the same time.

[0092] Thus, the environmental information image of the target point can be acquired more targetedly in response to the operation of the user, and the self-defined needs of the user can be met.

[0093] In some embodiments, the method can further include:

[0094] The target detection result is compared with the pre-stored whitelist image to obtain a comparison result.

[0095] For example, the cleaning device can acquire an image by using the first image acquisition device or the second image acquisition device, and then send the acquired image to the APP. The user can add the corresponding image to the whitelist on the user interface of the APP. For example, the whitelist image can also be acquired directly by using a smart device such as a mobile phone, and the acquired whitelist image can be input into the APP. The whitelist can include images of family members of the user, or images of identity-recognized personnel added by the user. For example, after the target detection result is acquired, the target detection result can be compared with the pre-stored whitelist image to obtain a comparison result. In some embodiments, the comparison result can be represented by the similarity between the target detection result and any image in the whitelist.

[0096] Based on the comparison result, the identity information of the detected target object is determined to perform an alarm in a case where the identity information indicates that the target object is a stranger.

[0097] Exemplarily, the plurality of similarity results in the comparison result can be sorted, and the target detection result with the largest similarity is taken as the detected target object; when the value of the largest similarity is greater than a preset value, it can be considered that the corresponding target object is the person in the white list, so as to determine that the identity information of the target object is safe; when the value of the largest similarity is less than the preset value, if the value of the largest similarity is less than or equal to the preset value, it can be considered that the person matching the target object is not found in the white list, that is, it is determined that the identity of the target object is a stranger. At this time, the cleaning device can send an alarm signal to the APP to prompt the user that a stranger has entered the home. Alternatively, the cleaning device can also use the sound playing device arranged thereon to alarm, so as to prompt the person in the white list that a stranger has entered the home.

[0098] Therefore, the monitoring function can be realized by using the image acquisition device arranged on the cleaning device, the personal safety of the user and the family members is ensured, and since the first image acquisition device can acquire images from multiple angles, the image acquisition range is expanded, and further guarantee is provided for the smooth realization of the monitoring function.

[0099] In some embodiments, the method can further include:

[0100] Based on the identity information of the target object, detecting current state information of the target object.

[0101] Exemplarily, the image of the pet can also be pre-stored in the white list. Based on the foregoing scheme, when it is determined that the identity information of the target object is safe, the current state information of the target object can be detected; the current state information can include the current action of the person or the pet, such as running, jumping, walking, reading, eating, etc.

[0102] Comparing the current state information with the pre-stored state information of the target object to obtain a state monitoring result.

[0103] Exemplarily, in the case where the target object is a person, the obtained current state information is compared with the pre-stored state information of the person in the white list; if the state information matching the current state information is not found in the pre-stored state information, it can be considered that the person is in an abnormal state at this time, for example, an old person falls down; if the state information matching the current state information is found in the pre-stored state information, it can be considered that the person is in a normal state at this time, and the output state detection result is the current state information, for example, a child reads a book, a pet eats, etc.; “matching” is the same as determining whether to match according to the similarity sorting result as described above, and is not described herein again for brevity.

[0104] Therefore, the state of the members in the white list can be monitored, and the dynamic of the members in the family can be more effectively understood. Moreover, since the first image acquisition device can acquire images from multiple angles, the image acquisition range is expanded, and further guarantee is provided for the smooth implementation of the state monitoring function.

[0105] According to a second aspect of the present application, a cleaning device control apparatus is further provided. The cleaning device includes a mechanical arm module and a chassis. The mechanical arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device. FIG. 4 is a schematic block diagram of a cleaning device control apparatus 400 according to some embodiments of the present disclosure. As shown in FIG. 4, the control apparatus can include:

[0106] An image acquisition unit 410 is configured to control the cleaning device to turn on the first image acquisition device and / or the second image acquisition device in response to a video instruction of a user. In some embodiments, the first image acquisition device is configured to acquire a first image within a corresponding field of view, and the second image acquisition device is configured to acquire a second image within a corresponding field of view.

[0107] A switching unit 420 is configured to control the cleaning device to switch the image acquisition device in the active state to the inactive state and switch the image acquisition device in the inactive state to the active state in response to a switching instruction, when one of the first image acquisition device and the second image acquisition device is in the active state and the other is in the inactive state.

[0108] An adjustment control unit 430 is configured to control the cleaning device to adjust the pose information of the mechanical arm module and / or adjust the position information of the chassis, so as to update the first image and / or the second image.

[0109] According to a third aspect of the present application, an electronic device is further provided. FIG. 5 is a schematic block diagram of an electronic device 500 according to some embodiments of the present disclosure. As shown in FIG. 5, the electronic device 500 can include a processor 510 and a memory 520. The memory 520 stores computer program instructions executable by the memory 520, and the computer program instructions are executed by the processor 510 to implement the cleaning device control method as described above.

[0110] According to a fourth aspect of the present application, a storage medium is further provided, which includes program instructions stored thereon. When the program instructions are executed by a processor, the cleaning device control method as described above is implemented. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer readable storage medium can be any combination of one or more computer readable storage media.

[0111] According to a fifth aspect of the present application, a cleaning device is further provided. FIG. 6 is a schematic structural block diagram of another cleaning device 600 according to some embodiments of the present disclosure. The cleaning device 600 can include:

[0112] The electronic device 500 as described above;

[0113] The mechanical arm module 610 is provided with a first image acquisition device 611, and the chassis 620 is provided with a second image acquisition device 621. The mechanical arm module is connected with the chassis, and the electronic device is configured to adjust pose information of the mechanical arm module and / or adjust position information of the chassis.

[0114] In some embodiments, the first image acquisition device 611 is arranged at a central position of a mechanical arm end of the mechanical arm module 610.

[0115] The specific details and beneficial effects of the cleaning device control apparatus, the electronic device, the storage medium, and the cleaning device can be understood by those of ordinary skill in the art through reading the above description of the cleaning device control method, and are not repeated here for brevity.

[0116] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and / or device can be implemented in other manners. For example, the apparatus embodiments described above are only schematic; the division of units is only a logical function division; an actual implementation can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the displayed or discussed mutual couplings or direct couplings or communication connections, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0118] In addition, each functional unit in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0119] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the prior art that essentially contributes or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0120] The above, the above embodiments are only to illustrate the technical solutions of the present application, rather than limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cleaning device control method, the cleaning device comprising a mechanical arm module and a chassis, the mechanical arm module being provided with a first image acquisition device, and the chassis being provided with a second image acquisition device, the method comprising: controlling, by the cleaning device, to turn on the first image acquisition device and / or the second image acquisition device in response to a video instruction of a user, wherein the first image acquisition device is used to acquire a first image within a corresponding field of view, and the second image acquisition device is used to acquire a second image within a corresponding field of view; in a case where one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, switching, by the cleaning device, the image acquisition device in the active state to the inactive state and the image acquisition device in the inactive state to the active state in response to a switching instruction of the user; adjusting, by the cleaning device, pose information of the mechanical arm module and / or position information of the chassis to update the first image and / or the second image.

2. The cleaning device control method of claim 1, wherein, The adjusting of the pose information of the mechanical arm module and / or the position information of the chassis comprises: adjusting, by the cleaning device, the pose information of the mechanical arm module and / or the position information of the chassis in response to a state adjustment instruction of the user.

3. The cleaning device control method of claim 1, wherein, The adjusting of the pose information of the mechanical arm module and / or the position information of the chassis comprises: detecting a target in the first image and / or the second image; positioning the target based on a target detection result; adjusting the pose information of the mechanical arm module and / or the position information of the chassis based on positioning information of the target.

4. The cleaning device control method of claim 3, wherein, The detecting of the target in the first image and / or the second image comprises: obtaining a bounding box of the target in the first image and / or the second image. The adjusting of the pose information of the mechanical arm module and / or the position information of the chassis based on the positioning information of the target comprises: obtaining a center point of the bounding box of the target and a preset center point offset, and adjusting the pose information of the mechanical arm module and / or the position information of the chassis based on the offset to make the target located at a center position of the first image and / or the second image.

5. The cleaning device control method of claim 2, further comprising: controlling, by the cleaning device, the mechanical arm module to extend out in an initial pose to acquire the first image when the first image acquisition device is turned on in the active state, wherein the first image is a fixed angle image in an environment where the cleaning device is located; The adjusting of the pose information of the mechanical arm module and / or the position information of the chassis comprises: detecting a target in the first image; adjusting the pose information of the mechanical arm module and / or the position information of the chassis based on a target detection result to update the first image until the updated first image appears the target.

6. The cleaning device control method of claim 1, further comprising: display a user interface, wherein the user interface comprises a map display area; in response to a target point setting instruction of a user, control the cleaning device to move to a position where the target point is located, to acquire the first image and / or the second image at the position where the target point is located by using the first image acquisition device and / or the second image acquisition device, wherein the user interface further comprises an image display area, and the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

7. The cleaning device control method of claim 3, further comprising: comparing the target detection result with a pre-stored whitelist image to obtain a comparison result; based on the comparison result, determining identity information of the detected target object, and in a case where the identity information indicates that the target object is a stranger, performing an alarm.

8. The cleaning device control method of claim 7, further comprising: based on the identity information of the target object, detecting current state information of the target object; comparing the current state information with pre-stored state information of the target object to obtain a state monitoring result.

9. A cleaning device control apparatus, the cleaning device comprising a mechanical arm module and a chassis, the mechanical arm module being provided with a first image acquisition device, and the chassis being provided with a second image acquisition device, the control apparatus comprising: an image acquisition unit configured to control the cleaning device to start the first image acquisition device and / or the second image acquisition device in response to a video instruction of a user, wherein the first image acquisition device is configured to acquire a first image within a corresponding field of view, and the second image acquisition device is configured to acquire a second image within a corresponding field of view; a switching unit configured to control the cleaning device to switch the image acquisition device in an active state to an inactive state and switch the image acquisition device in an inactive state to an active state in response to a switching instruction in a case where one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state; an adjustment control unit configured to control the cleaning device to adjust pose information of the mechanical arm module and / or adjust position information of the chassis to update the first image and / or the second image.

10. An electronic device comprising a processor and a memory, wherein, The memory stores computer program instructions, and the computer program instructions are executed by the processor to implement the cleaning device control method of any one of claims 1 to 8.

11. A storage medium comprising computer program instructions stored thereon, wherein, The computer program instructions are executed by the processor to implement the cleaning device control method of any one of claims 1 to 8.

12. A cleaning device, comprising: the electronic device of claim 10; a mechanical arm module and a chassis, the mechanical arm module being provided with a first image acquisition device, and the chassis being provided with a second image acquisition device; the mechanical arm module being connected to the chassis; and the electronic device being configured to adjust pose information of the mechanical arm module and / or adjust position information of the chassis.

13. The cleaning apparatus of claim 12, wherein, The first image acquisition device is arranged at a central position of a mechanical arm end of the mechanical arm module.

Images