Cleaning equipment control method and device, cleaning equipment and related equipment

Abstract

The invention provides a cleaning equipment control method and device, cleaning equipment and related equipment, and relates to the technical field of smart home, and the method comprises the steps: controlling to start a first image collection device and / or a second image collection device; when one of the first image acquisition device and the second image acquisition device is in a valid state and the other one is in an invalid state, the cleaning equipment responds to a switching instruction of a user, the image acquisition device in the valid state is switched to the invalid state, and the image acquisition device in the invalid state is switched to the valid state; and the cleaning equipment adjusts the pose information of the mechanical arm module and / or the position information of the chassis so as to update the first image and / or the second image. Due to the fact that the mechanical arm module has the high degree of freedom, the multi-angle and multi-direction first image can be viewed, richer environment information is provided for a user, more choices are provided for the user through the switching function, different requirements of various users are met, and the use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and more specifically, to a cleaning device control method, a cleaning device control device, an electronic device, a storage medium, and a cleaning device. Background Art

[0002] Currently, cleaning equipment is equipped with cameras. Users can use the application software (APP) of smart devices to remotely view the environmental information around the cleaning equipment through the camera during the cleaning and other movements of the cleaning equipment, and even make remote calls. However, how to see a larger range of environmental information or make better remote calls is a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The Summary of the Invention introduces a series of simplified concepts that will be further described in the Detailed Description of the Invention. The Summary of the Invention is not intended to limit the key features and essential features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0004] In a first aspect, the present invention provides a method for controlling a cleaning device, wherein the cleaning device includes a robotic arm module and a chassis, wherein the robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device, and the method includes:

[0005] The cleaning device controls the activation of the first image acquisition device and / or the second image acquisition device in response to a video instruction from the user, wherein the first image acquisition device is used to acquire a first image within a corresponding field of view, and the second image acquisition device is used to acquire a second image within the corresponding field of view;

[0006] In a case where one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, the cleaning device switches the image acquisition device in the active state to the inactive state and switches the image acquisition device in the inactive state to the active state in response to a switching instruction of the user;

[0007] The cleaning device adjusts the posture information of the robot arm module and / or adjusts the position information of the chassis to update the first image and / or the second image.

[0008] In some embodiments, adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis includes:

[0009] In response to the user's state adjustment instruction, the posture information of the robotic arm module and / or the position information of the chassis are adjusted.

[0010] In some embodiments, adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis includes:

[0011] performing object detection on the first image and / or the second image;

[0012] Based on the target detection results, locate the target object;

[0013] Based on the positioning information of the target object, the posture information of the robotic arm module and / or the position information of the chassis are adjusted.

[0014] In some embodiments, performing object detection on the first image and / or the second image includes:

[0015] Acquire a bounding box of the target object in the first image and / or the second image;

[0016] Based on the positioning information of the target object, adjust the posture information of the robot arm module and / or adjust the position information of the chassis, including:

[0017] Obtain the offset between the center point of the target object's bounding box and the preset center point, and adjust the posture information of the robotic arm module and / or the position information of the chassis based on the offset so that the target object is located at the center position of the first image and / or the second image.

[0018] In some embodiments, the method further comprises:

[0019] When the first image acquisition device is turned on and is in an effective state, the robotic arm module is controlled to extend in an initial posture to obtain a first image, wherein the first image is a fixed-angle image of the environment in which the cleaning device is located;

[0020] Adjust the posture information of the robot arm module and / or adjust the position information of the chassis, including:

[0021] performing target detection on the first image;

[0022] Based on the target detection result, the posture information of the robotic arm module and / or the position information of the chassis are adjusted to update the first image, and the adjustment action is stopped when the target object appears in the updated first image.

[0023] In some embodiments, the method further comprises:

[0024] displaying a user interface, wherein the user interface includes a map display area;

[0025] In response to the user's target point setting instruction, the cleaning equipment is controlled to move to the location of the target point, so as to use the first image acquisition device and / or the second image acquisition device to obtain the first image and / or the second image at the location of the target point, wherein the user interface also includes an image display area, the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

[0026] In some embodiments, the method further comprises:

[0027] Compare the target detection result with the pre-stored whitelist image to obtain the comparison result;

[0028] Based on the comparison result, the identity information of the detected target object is determined, so as to generate an alarm if the identity information indicates that the target object is a stranger.

[0029] In some embodiments, the method further comprises:

[0030] Based on the identity information of the target object, detect the current state information of the target object;

[0031] Compare the current status information with the pre-stored status information of the target object to obtain the status monitoring results.

[0032] In the second aspect, a cleaning equipment control device is also proposed. The cleaning equipment includes a robotic arm module and a chassis. The robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device. The control device includes:

[0033] An image acquisition unit, configured to control the cleaning device to activate a first image acquisition device and / or a second image acquisition device in response to a user's video instruction, wherein the first image acquisition device is configured to acquire a first image within a corresponding field of view, and the second image acquisition device is configured to acquire a second image within the corresponding field of view;

[0034] a switching unit for controlling the cleaning device to switch the image acquisition device in the valid state to the invalid state and to switch the image acquisition device in the invalid state to the valid state in response to a switching instruction when one of the first image acquisition device and the second image acquisition device is in the valid state and the other is in the invalid state;

[0035] The adjustment control unit is used to control the cleaning device to adjust the posture information of the robot arm module and / or adjust the position information of the chassis to update the first image and / or the second image.

[0036] In a third aspect, an electronic device is also proposed, comprising a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the cleaning device control method described above when the processor is running.

[0037] In a fourth aspect, a storage medium is also proposed, on which program instructions are stored. The program instructions are used to execute the cleaning device control method described above when running.

[0038] In a fifth aspect, a cleaning device is also proposed, the cleaning device comprising:

[0039] electronic devices as described above;

[0040] A robotic arm module and a chassis, wherein the robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device.

[0041] In some embodiments, the first image acquisition device is disposed at the center of the end of the robotic arm of the robotic arm module.

[0042] According to the above technical solution, the cleaning equipment outputs the first image and the second image to the user through the first image acquisition device and the second image acquisition device. When one of the first image acquisition device and the second image acquisition device is in a valid state, the cleaning equipment can switch the states of the two image acquisition devices in response to the user's switching instruction, thereby providing the image acquired by the image acquisition device in the valid state to the user. In addition, the content of the first image or the second image can be updated by adjusting the posture of the robotic arm module or adjusting the position of the chassis. Since the robotic arm module has a high degree of freedom, it can view the first image from multiple angles and directions, providing the user with richer environmental information. The above switching function can provide users with more choices, meet the different needs of various users, and enhance the user experience.

[0043] The cleaning equipment control method of the present invention, and other advantages, objectives and features of the present invention will be reflected in part through the following description, and will also be understood by those skilled in the art through research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the exemplary embodiments below. The accompanying drawings are for illustration purposes only and are not to be considered limiting of the present application. Throughout the accompanying drawings, the same reference symbols denote the same components. In the accompanying drawings:

[0047] Figure 1 A schematic flow chart of a cleaning equipment control method provided in an embodiment of the present application;

[0048] Figure 2 A schematic structural block diagram of a cleaning device provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of a user interface provided in an embodiment of the present application;

[0050] Figure 4 A schematic block diagram of a cleaning equipment control device provided in an embodiment of the present application;

[0051] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application; and

[0052] Figure 6 A schematic structural block diagram of another cleaning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0054] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0056] Generally, the height of the main body of cleaning equipment such as a sweeping robot is about 10 cm. Setting the camera on the chassis will result in a relatively low camera viewing angle. The user can only see limited environmental information and cannot view the full picture of home information. Especially when making a long-distance call, the camera on the chassis can only see the legs of a standing person at a very short distance in front of the device, such as 1 meter away, but cannot see the face of a close person, which brings a very poor video call experience to the user. In addition, since the camera is installed in a fixed manner, the video angle cannot be adjusted according to the face position of the distant person.

[0057] In order to solve the above technical problems, according to the first aspect of the present application, a cleaning equipment control method is proposed. Figure 1 A schematic flow chart of a cleaning equipment control method 100 provided in an embodiment of the present application. Figure 2 This is a schematic structural diagram of a cleaning device provided in an embodiment of the present application. Figure 2 As shown, the cleaning device 200 may include a robotic arm module 210 and a chassis 220. The robotic arm module 210 may be mounted on the chassis 220. The robotic arm module 210 may have multiple degrees of freedom, enabling it to extend and fold, rotate about corresponding joints or the chassis, and so on. The robotic arm module 210 may have a gripper at the end that can grasp objects. The chassis may include components such as drive wheels and cleaning elements. Driven by the chassis 220, the cleaning device 200 may perform actions such as turning, accelerating, and decelerating. For example, driven by the drive wheels, the cleaning device 200 may move over a surface to be cleaned and use the cleaning elements to clean the surface. The chassis 220 and the robotic arm module 210 may move independently in response to control signals, or they may coordinate their movements to perform corresponding operations. For example, the robotic arm module 210 may be provided with a first image acquisition device 211, and the chassis 220 may be provided with a second image acquisition device 221. Both the first image acquisition device 211 and the second image acquisition device 221 may be image acquisition devices such as cameras. The first image acquisition device 211 can be set at the center of the end of the robot arm in the robot arm module, which can better ensure the image acquisition effect of the first image acquisition device 211. Figure 1 As shown, the method 100 may include the following steps.

[0058] In step S110, the cleaning device controls to start the first image acquisition device and / or the second image acquisition device in response to the user's video instruction, wherein the first image acquisition device is used to acquire a first image within the corresponding field of view, and the second image acquisition device is used to acquire a second image within the corresponding field of view.

[0059] For example, a user can establish a communication connection with the cleaning device through an application software (APP) and send a video command to the cleaning device to activate the remote video function of the cleaning device. After receiving the user's video command, the cleaning device can control the first image acquisition device on the robotic arm module to turn on, or control the second image acquisition device on the chassis to turn on, or control the first image acquisition device and the second image acquisition device to turn on at the same time. It should be noted that since the installation positions of the first image acquisition device and the second image acquisition device are different, when the first image acquisition device and the second image acquisition device are turned on at the same time, the first image within the corresponding field of view acquired by the first image acquisition device and the second image within the corresponding field of view acquired by the second image acquisition device may contain different environmental information or different parts of the person. For example, in response to the user's operation, a user interface can be displayed on the APP, and the user can remotely interact with the cleaning device through the user interface. The user interface may include a video display area for displaying the first image or the second image. When the first image acquisition device and the second image acquisition device are turned on at the same time, the video display area can be divided into two, displaying the first image and the second image respectively.

[0060] In step S120, when one of the first image acquisition device and the second image acquisition device is in a valid state and the other is in an invalid state, the cleaning device switches the image acquisition device in the valid state to the invalid state and switches the image acquisition device in the invalid state to the valid state in response to a switching instruction from the user.

[0061] For example, the following description uses the first image acquisition device as an example. When the first image acquisition device is turned on and displays the corresponding first image on the user interface, the first image acquisition device can be considered to be in an active state. In all other cases, the first image acquisition device can be considered to be in an inactive state. When one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, for example, the first image acquisition device is in an active state and the second image acquisition device is in an inactive state, the user can send a switching instruction to the cleaning device through the app, such as by clicking on the icon of the first image acquisition device or the second image acquisition device on the user interface to generate the switching instruction. After receiving the switching instruction, the cleaning device can switch the first image acquisition device to an inactive state and the second image acquisition device to an active state. At this time, the second image captured by the second image acquisition device is displayed on the app's user interface.

[0062] In step S130 , the cleaning device adjusts the posture information of the robot arm module and / or adjusts the position information of the chassis to update the first image and / or the second image.

[0063] Exemplarily, the cleaning device can "walk" throughout the house by moving the chassis under the control of a cruise command. During the movement of the chassis, the second image acquisition device on the chassis is in an active state. At this time, the second image acquisition device can be used to acquire a second image. As the chassis moves, the image content of the acquired second image is continuously updated. Exemplarily, if the first image acquisition device is switched to an active state in response to a switching command, the first image is acquired by the first image acquisition device. The chassis can also move throughout the house. During the movement, the image content of the first image is continuously updated. Furthermore, during the movement of the chassis, the robotic arm module can extend, rotate, and perform other actions in a preset manner. During the actions, the image content of the first image is also continuously updated. The actions performed by the robotic arm module and the chassis can be performed simultaneously, in a time-sharing manner, or one of them can be performed. Regardless of which set of actions is performed, the image content of the first image is continuously updated with the corresponding action.

[0064] According to the above technical solution, the cleaning equipment outputs the first image and the second image to the user through the first image acquisition device and the second image acquisition device. When one of the first image acquisition device and the second image acquisition device is in a valid state, the cleaning equipment can switch the states of the two image acquisition devices in response to the user's switching instruction, thereby providing the image acquired by the image acquisition device in the valid state to the user. In addition, the content of the first image or the second image can be updated by adjusting the posture of the robotic arm module or adjusting the position of the chassis. Since the robotic arm module has a high degree of freedom, it can view the first image from multiple angles and directions, providing the user with richer environmental information. The above switching function can provide users with more choices, meet the different needs of various users, and enhance the user experience.

[0065] In some embodiments, step S130 of adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis may include: adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis in response to a user's state adjustment instruction.

[0066] Figure 3 A schematic diagram of a user interface provided in an embodiment of the present application. Figure 3 In the embodiment shown, only one image acquisition device is in an active state, so only one image is displayed in the video display area on the user interface. Figure 3 The area where the scene image is displayed. Figure 3As shown, the user interface can be provided with position adjustment components 311 and 312, and the user can send status adjustment instructions to the cleaning device through the position adjustment components 311 and 312. For example, when the second image acquisition device is in an effective state and the video display area of the user interface only displays the second image, the user can use the position adjustment component 311 to adjust the position of the chassis. For example, by clicking the downward arrow on the left side of the position adjustment component 311, the chassis can be controlled to move backward, and by clicking the upward arrow on the right side of the position adjustment component 311, the chassis can be controlled to move forward. The step length of the forward or backward movement of the chassis controlled by a single or continuous click of the position adjustment component 311 can be reasonably set in the underlying logic construction and is not limited here. For example, when the first image acquisition device is in an effective state and the video display area of the user interface only displays the first image, the user can use the position adjustment component 311 and the position adjustment component 312 to jointly control and adjust the posture of the robotic arm module. For example, clicking the downward arrow on the left side of the position adjustment component 311 can control the end of the manipulator arm module to descend, clicking the upward arrow on the right side of the position adjustment component 311 can control the end of the manipulator arm to ascend, clicking the counterclockwise rotation component on the left side of the position adjustment component 312 can control the manipulator arm module to rotate counterclockwise relative to the chassis or control the end of the manipulator arm module to rotate counterclockwise relative to other joints, components, etc. in the manipulator arm module, and clicking the clockwise rotation component on the right side of the position adjustment component 312 can control the manipulator arm module to rotate clockwise relative to the chassis or control the end of the manipulator arm module to rotate clockwise relative to other joints, components, etc. in the manipulator arm module, wherein the end of the manipulator arm module is equipped with a first image acquisition device. After the posture of the manipulator arm module or the position of the chassis is adjusted, the image acquisition area of the first image acquisition device or the second image acquisition device also changes accordingly, and accordingly, the content of the acquired first image or second image also changes. Thus, by adjusting the posture of the manipulator arm module and the position of the chassis, the image content is updated.

[0067] It should be noted that the above-mentioned use of the position adjustment component to adjust the position of the robotic arm module and the chassis is merely exemplary and does not mean a limitation on the adjustment method of the robotic arm module and the chassis. In actual applications, any reasonable method can be used to adjust the position of the robotic arm module and the chassis based on the user's status adjustment instructions.

[0068] Therefore, the posture of the robotic arm module and the position of the chassis can be controlled through the above-mentioned interactive method to achieve the update of the first image and the second image. The updated first image and the second image include the content information that the user expects to obtain, which better meets the user's needs and improves the user experience.

[0069] For example, the cleaning device may be provided with a sound playing device such as a speaker, and may also be provided with a sound collecting device such as a microphone, so that the user can make a remote video call with the person in front of the cleaning device through the APP.

[0070] In some embodiments, the method further comprises:

[0071] When the first image acquisition device is in an effective state, the robot arm module is controlled to extend in an initial posture to obtain a first image, wherein the first image is a fixed-angle image of the environment in which the cleaning device is located.

[0072] Exemplarily, the initial posture may be a posture preset before leaving the factory. When the first image acquisition device is controlled to be in an effective state, the robotic arm module extends in a preset posture, thereby obtaining a fixed-angle image of the environment in the posture.

[0073] In this embodiment, step S130 of adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis may include:

[0074] Step S134: performing target detection on the first image.

[0075] Exemplarily, any existing or future target detection algorithm may be used to perform target detection on the first image, where the targets include static or dynamic living or non-living objects such as people, pets, potted plants, or furniture.

[0076] Step S135 , based on the target detection result, adjusting the posture information of the robot arm module and / or adjusting the position information of the chassis to update the first image, and stopping the adjustment action when the target object appears in the updated first image.

[0077] Exemplarily, the target detection result can represent the probability corresponding to the target detected in the current first image. If the probability is less than or equal to the preset probability, it can be considered that the target does not appear in the first image. At this time, the posture information of the robotic arm module or the position information of the chassis are adjusted separately or jointly. Based on the adjusted robotic arm module and chassis, a new first image can be obtained using the first image acquisition device set on the robotic arm module, and target detection can be performed on the new first image until the probability corresponding to the target detected in the new first image is greater than the preset probability, for example, when it is 80% as mentioned above, it can be determined that the target appears within the field of view of the current first image acquisition device, so that subsequent tracking of the target can be achieved.

[0078] Therefore, when using the first image acquisition device for remote video viewing, the robotic arm module or chassis can be adjusted by target positioning so that the target object can appear in the first image, that is, the target object is first found before starting target tracking.

[0079] In some embodiments, step S130 of adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis may include:

[0080] Step S131: performing object detection on the first image and / or the second image.

[0081] For ease of description and understanding, the following detailed description is given using the first image as an example. Based on the relevant description of the first image, the relevant scheme of the second image can be understood. For example, after the first image is acquired using the first image acquisition device, target detection can be performed on the first image. The targets here include the people who are making video calls locally as mentioned above, and can also include static or dynamic living or non-living things that appear in the current scene, such as pets, potted plants, or furniture. Any existing or future target detection algorithm can be used to perform target detection on the first image to obtain a target detection result.

[0082] Step S132: Position the target object based on the target detection result.

[0083] Exemplarily, the target detection result can represent the probability corresponding to the target detected in the current first image. When the detected target is a person and this probability is greater than a preset probability, such as 80%, it can be determined that the target object, in this embodiment, is the person making a video call locally, and appears in the first image. At this time, based on the first image, the position information of the current cleaning equipment and the posture information of the robotic arm module, the position information of the target object in the room can be determined to achieve the positioning of the target object.

[0084] Step S133: Based on the positioning information of the target object, adjust the posture information of the robot arm module and / or adjust the position information of the chassis.

[0085] Exemplarily, after determining the positioning information of the target object corresponding to the current first image, a new first image can be obtained by adjusting the posture of the robotic arm module. Exemplarily, when the target object's movement range exceeds the range of motion of the robotic arm module or the target object walks out of the current room, a new first image can be obtained in combination with the movement of the chassis. Ensure that the target object is still within the new first image to achieve positioning tracking of the target object. The method for adjusting the posture information of the robotic arm module and the position information of the chassis has been described in detail above and will not be repeated here. Exemplarily, the above scheme can be used when the user clicks Figure 3It should be noted that the "one-click pet search" function can be achieved by automatically adjusting the robotic arm module and chassis through the above-mentioned target detection and target positioning and tracking. It can also be achieved by the user manually adjusting the robotic arm module and chassis. Alternatively, human intervention can be performed at any time during the automatic adjustment process, and the user can manually adjust the position information of the chassis or the posture information of the robotic arm module to perform manual supplementary operations for situations where the target object cannot be effectively tracked using the automatic adjustment method. Thus, the robotic arm module and chassis can be controlled in a variety of ways to achieve positioning and tracking of the target object.

[0086] In some embodiments, step S131 of performing target detection on the first image and / or the second image may include: obtaining a bounding box of the target object in the first image and / or the second image.

[0087] Exemplarily, the target detection method may include obtaining a bounding box of the target in the first image, wherein the bounding box may be a minimum outer envelope rectangular box of the target.

[0088] Step S133 may include adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis based on the positioning information of the target object, which may include: obtaining the offset between the center point of the bounding box of the target object and the preset center point, and adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis based on the offset, so that the target object is located at the center position of the first image and / or the second image.

[0089] Exemplarily, after obtaining the bounding box of the target object, the center point of the bounding box can be determined. For example, when the bounding box is a rectangular box, the intersection of the diagonals of the rectangular box can be determined as the center point of the rectangular box. Exemplarily, the preset center point can be the center point of the first image. When the first image is rectangular, the preset center point is the intersection of the diagonals of the first image. After determining the center point of the bounding box, the offset between the center point of the bounding box and the preset center point can be calculated, wherein the offset can be expressed in coordinate form, or can be expressed by an offset angle and a straight-line distance between the center point of the bounding box and the preset center point, which is not limited here. Based on the determined offset, the posture information of the robotic arm module is adjusted accordingly, and a new first image is obtained again using the first image acquisition device on the robotic arm module after the posture adjustment, and the new center point of the bounding box of the target object in the new first image is obtained again, so that the new center point coincides with the preset center point, that is, the target object is located at the center of the first image.

[0090] In this way, the target object can be ensured to be located in the center of the image, giving users an excellent video experience.

[0091] It should be noted that, in conjunction with the foregoing, the first image acquisition device and the second image acquisition device can simultaneously acquire images, or alternatively, acquire images in response to a switching instruction. By reading the above-mentioned technical solutions for processing the first image, one can understand the specific implementations of the two solutions of simultaneously acquiring the first and second images and acquiring only the second image. For the sake of brevity, a detailed description is omitted. Furthermore, steps S134-S135 can be executed before steps S131-S133 in embodiments where the first image acquisition device is in an active state.

[0092] In some embodiments, the method may further include:

[0093] A user interface is displayed, wherein the user interface includes a map display area.

[0094] For example, as mentioned above, in response to the user's operation, a user interface can be displayed on the APP, and the user can remotely interact with the cleaning device through the user interface. Figure 3 As shown, the user interface may include a map display area 320. The map displayed in the map display area may be a global image obtained by the cleaning device, and may represent the distribution of various rooms in the entire home.

[0095] In response to the user's target point setting instruction, the cleaning equipment is controlled to move to the location of the target point, so as to use the first image acquisition device and / or the second image acquisition device to obtain the first image and / or the second image at the location of the target point, wherein the user interface also includes an image display area, the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

[0096] For example, a user can first click the "Point to Go" component in the user interface, then click any point within the map display area 320 to set it as the target point. The actual location of the target point is then determined based on the location of the target point within the map display area 320, and the cleaning device is controlled to move to the actual location within the real environment. At this location, a first image can be acquired using the first image acquisition device, a second image can be acquired using the second image acquisition device, or both can be used simultaneously.

[0097] Therefore, the environmental information image of the target point can be obtained more specifically in response to the user's operation to meet the user's customized needs.

[0098] In some embodiments, the method may further include:

[0099] Compare the target detection results with the pre-stored whitelist images to obtain the comparison results.

[0100] Exemplarily, the cleaning device can use the first image acquisition device or the second image acquisition device to acquire images, and then send the acquired images to the APP, and the user can add the corresponding images to the whitelist on the user interface of the APP. Exemplarily, the whitelist image can also be directly acquired using a smart device such as a mobile phone, and the acquired whitelist image can be input into the APP. The whitelist can include images of the user's family members, and can also include images of people whose identities the user has added. Exemplarily, after obtaining the target detection result, it can be compared with the above-mentioned pre-stored whitelist image to obtain a comparison result, wherein the comparison result can be represented by the similarity between the target detection result and any image in the whitelist.

[0101] Based on the comparison result, the identity information of the detected target object is determined, so as to generate an alarm if the identity information indicates that the target object is a stranger.

[0102] For example, the multiple similarity results in the comparison results can be sorted, and the target detection result with the greatest similarity can be used as the detected target. When the maximum similarity value is greater than a preset value, it can be considered that the corresponding target is a person on the whitelist, thereby determining that the identity information of the target is safe. Conversely, if the maximum similarity value is less than or equal to the preset value, it can be considered that no matching person is found in the whitelist, that is, the identity of the target is determined to be a stranger. At this time, the cleaning device can send an alarm signal to the APP to remind the user that a stranger has entered the home. Alternatively, the cleaning device can also use its own sound playback device to sound an alarm, so as to remind the user that a stranger has entered when there is a person on the whitelist at home.

[0103] Therefore, the image acquisition device provided on the cleaning equipment can be used to realize the monitoring function to ensure the personal safety of the user and family members. Moreover, since the first image acquisition device can acquire images from multiple angles, the image acquisition range is expanded, providing further guarantee for the smooth implementation of the monitoring function.

[0104] In some embodiments, the method may further include:

[0105] Based on the identity information of the target object, the current state information of the target object is detected.

[0106] For example, pet images can be pre-stored in the whitelist. Based on the above solution, after confirming the security of the target's identity information, the current state information of the target can be detected, where the current state information can include the current action of the person or pet, such as running, jumping, walking, reading, eating, etc.

[0107] Compare the current status information with the pre-stored status information of the target object to obtain the status monitoring results.

[0108] For example, when the target object is a person, the current state information obtained is compared with the state information of the person pre-stored in the whitelist. If no state information matching the current state information is found in the pre-stored state information, it can be considered that the person is in an abnormal state, such as an elderly person falling. If state information matching the current state information is found in the pre-stored state information, it can be considered that the person is in a normal state, and the output state detection result is the current state information, such as a child reading, a pet eating, etc. The "match" is the same as the determination of whether a match is found based on the similarity sorting results mentioned above, and for the sake of brevity, it will not be repeated here.

[0109] In this way, the status of the members in the whitelist can be monitored, and the dynamics of the family members can be understood more effectively. In addition, since the first image acquisition device can acquire images from multiple angles, the image acquisition range is expanded, providing further guarantee for the smooth implementation of the status monitoring function.

[0110] According to a second aspect of the present application, a cleaning device control device is also provided. The cleaning device includes a robotic arm module and a chassis, wherein the robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device. Figure 4 This is a schematic block diagram of a cleaning equipment control device 400 provided in an embodiment of the present application. Figure 4 As shown, the control device may include:

[0111] An image acquisition unit 410 is configured to control the cleaning device to activate a first image acquisition device and / or a second image acquisition device in response to a user's video instruction, wherein the first image acquisition device is configured to acquire a first image within a corresponding field of view, and the second image acquisition device is configured to acquire a second image within the corresponding field of view;

[0112] a switching unit 420 configured to control the cleaning device to switch the image acquisition device in the active state to the inactive state and to switch the image acquisition device in the inactive state to the active state in response to a switching instruction when one of the first image acquisition device and the second image acquisition device is in the active state and the other is in the inactive state;

[0113] The adjustment control unit 430 is used to control the cleaning device to adjust the posture information of the robot arm module and / or adjust the position information of the chassis to update the first image and / or the second image.

[0114] According to a third aspect of the present application, an electronic device is also provided. Figure 5 Schematic block diagram of an electronic device 500 provided in an embodiment of the present application. Figure 5As shown, the electronic device 500 may include a processor 510 and a memory 520 , wherein the memory 520 stores computer program instructions, and the computer program instructions are used by the processor 510 to execute the cleaning device control method described above when the processor 510 is running.

[0115] According to a fourth aspect of the present application, a storage medium is further provided, on which program instructions are stored, which are used to execute the cleaning device control method described above when the program instructions are executed. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0116] According to a fifth aspect of the present application, a cleaning device is also proposed. Figure 6 This is a schematic structural block diagram of another cleaning device 600 provided in an embodiment of the present application. The cleaning device 600 may include:

[0117] The electronic device 500 as described above;

[0118] A robotic arm module 610 and a chassis 620 , wherein the robotic arm module 610 is provided with a first image acquisition device 611 , and the chassis 620 is provided with a second image acquisition device 621 .

[0119] In some embodiments, the first image acquisition device 611 is disposed at the center of the end of the robotic arm of the robotic arm module 610 .

[0120] A person skilled in the art can understand the specific details and beneficial effects of the cleaning equipment control device, electronic device, storage medium and cleaning equipment by reading the above description of the cleaning equipment control method, and will not be repeated here for the sake of brevity.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed devices and / or equipment can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0125] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cleaning equipment control method, characterized in that: The cleaning device includes a robotic arm module and a chassis, the robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device, and the method includes: The cleaning device controls to start the first image acquisition device and / or the second image acquisition device in response to a video instruction from a user, wherein the first image acquisition device is used to acquire a first image within a corresponding field of view, and the second image acquisition device is used to acquire a second image within the corresponding field of view; In a case where one of the first image acquisition device and the second image acquisition device is in an active state and the other is in an inactive state, the cleaning device switches the active image acquisition device to an inactive state and switches the inactive image acquisition device to an active state in response to a switching instruction from a user; The cleaning device adjusts the posture information of the robotic arm module and / or adjusts the position information of the chassis to update the first image and / or the second image.

2. The cleaning equipment control method according to claim 1, wherein: The adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis includes: In response to a state adjustment instruction from a user, the posture information of the robotic arm module and / or the position information of the chassis are adjusted.

3. The cleaning equipment control method according to claim 1, wherein: The adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis includes: performing object detection on the first image and / or the second image; Based on the target detection results, locate the target object; Based on the positioning information of the target object, the posture information of the robotic arm module and / or the position information of the chassis are adjusted.

4. The cleaning equipment control method according to claim 3, wherein: The performing target detection on the first image and / or the second image includes: Acquire a bounding box of the target object in the first image and / or the second image; The adjusting the posture information of the robotic arm module and / or the position information of the chassis based on the positioning information of the target object includes: Obtain the offset between the center point of the bounding box of the target object and the preset center point, and adjust the posture information of the robotic arm module and / or the position information of the chassis based on the offset so that the target object is located at the center position of the first image and / or the second image.

5. The cleaning equipment control method according to claim 2, wherein: The method further comprises: When the first image acquisition device is turned on and is in the effective state, the robotic arm module is controlled to extend in an initial posture to obtain the first image, wherein the first image is a fixed-angle image of the environment in which the cleaning device is located; The adjusting the posture information of the robotic arm module and / or adjusting the position information of the chassis includes: performing target detection on the first image; Based on the target detection result, the posture information of the robotic arm module and / or the position information of the chassis are adjusted to update the first image, and the adjustment action is stopped until the target object appears in the updated first image.

6. The cleaning equipment control method according to claim 1, wherein: The method further comprises: displaying a user interface, wherein the user interface includes a map display area; In response to the user's target point setting instruction, the cleaning device is controlled to move to the location of the target point, so as to use the first image acquisition device and / or the second image acquisition device to obtain the first image and / or the second image at the location of the target point, wherein the user interface also includes an image display area, the image display area is used to display the first image and / or the second image, and the target point is located in the map display area.

7. The cleaning equipment control method according to claim 3, wherein: The method further comprises: Comparing the target detection result with the pre-stored whitelist image to obtain a comparison result; Based on the comparison result, the identity information of the detected target object is determined, so as to generate an alarm if the identity information indicates that the target object is a stranger.

8. The cleaning equipment control method according to claim 7, wherein: The method further comprises: Detecting current status information of the target object based on the identity information of the target object; The current status information is compared with the pre-stored status information of the target object to obtain a status monitoring result.

9. A cleaning equipment control device, characterized in that: The cleaning device includes a robotic arm module and a chassis, the robotic arm module is provided with a first image acquisition device, the chassis is provided with a second image acquisition device, and the control device includes: an image acquisition unit, configured to control the cleaning device to activate the first image acquisition device and / or the second image acquisition device in response to a user's video instruction, wherein the first image acquisition device is configured to acquire a first image within a corresponding field of view, and the second image acquisition device is configured to acquire a second image within the corresponding field of view; a switching unit for controlling the cleaning device to switch the image acquisition device in the valid state to the invalid state and to switch the image acquisition device in the invalid state to the valid state in response to a switching instruction when one of the first image acquisition device and the second image acquisition device is in the valid state and the other is in the invalid state; An adjustment control unit is used to control the cleaning device to adjust the posture information of the robotic arm module and / or adjust the position information of the chassis to update the first image and / or the second image.

10. An electronic device, characterized in that: The cleaning device comprises a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the cleaning device control method according to any one of claims 1 to 8 when the processor is running.

11. A storage medium having program instructions stored thereon, wherein the program instructions are used to execute the cleaning device control method according to any one of claims 1 to 8 when run.

12. A cleaning device, characterized in that: The cleaning equipment comprises: The electronic device according to claim 10; A robotic arm module and a chassis, wherein the robotic arm module is provided with a first image acquisition device, and the chassis is provided with a second image acquisition device.

13. The cleaning device according to claim 12, wherein The first image acquisition device is arranged at the center position of the end of the robotic arm of the robotic arm module.

Images