Object state detection method and device and self-moving equipment

Abstract

The invention discloses an object state detection method and device and self-moving equipment. The object state detection method is applied to the self-moving equipment, the self-moving equipment comprises a self-moving chassis and a mechanical arm connected with the self-moving chassis, the mechanical arm comprises a working arm and a manipulator, the working arm and the manipulator are connected through a mechanical joint, and the mechanical joint comprises a joint elbow driving motor used for driving the working arm to rotate. The method comprises the steps that when the self-moving equipment is in a carrying state, detection information is obtained, and the detection information comprises at least one of first point cloud data of the manipulator in a preset range when the manipulator is in a carrying posture and current data of a joint elbow driving motor; and the processing state of the manipulator for the object is determined based on the detection information, and the processing state comprises a clamping state or a falling state. And falling of an object can be detected in time.

Description

Technical Field

[0001] This application belongs to the field of automatic control, and particularly relates to a method and device for detecting the state of an object, as well as a self-mobile device. Background Art

[0002] With the continuous development of science and technology and the continuous improvement of people's living standards, self-mobile devices have increasingly entered our daily lives. Currently, self-mobile devices can use robotic arms to grasp or move objects.

[0003] During the process of the robotic arm performing grasping and transportation tasks, the object may accidentally fall due to reasons such as mechanical vibration, insufficient grasping force, or item deformation, resulting in task failure or system errors.

[0004] Therefore, it is necessary to provide a method for detecting the state of an object to achieve real-time detection of the object's falling situation. Summary of the Invention

[0005] Embodiments of this application provide a method and device for detecting the state of an object, as well as a self-mobile device, which can at least to some extent detect whether an object has fallen when the self-mobile device is in a carrying state.

[0006] Other features and advantages of this application will become apparent through the following detailed description, or will be partially learned through the practice of this application.

[0007] According to a first aspect of the embodiments of this application, a method for detecting the state of an object is provided, which is applied to a self-mobile device. The self-mobile device includes a self-mobile chassis and a robotic arm connected to the self-mobile chassis. The robotic arm includes a working arm and a manipulator. The working arm and the manipulator are connected by a mechanical joint. The mechanical joint includes a joint elbow drive motor for driving the working arm to rotate. The method includes:

[0008] When the self-mobile device is in a carrying state, obtain detection information, where the detection information includes at least one of the first point cloud data within a preset range of the manipulator in the carrying posture and the current data of the joint elbow drive motor;

[0009] Based on the detection information, determine the processing state of the manipulator with respect to the object, where the processing state includes a clamping state or a falling state.

[0010] In some possible implementation manners, the detection information includes the first point cloud data. Determining the processing state of the manipulator with respect to the object based on the detection information includes:

[0011] Based on the first point cloud data, determine the occlusion value within a preset range of the manipulator in the carrying posture;

[0012] If the occlusion value meets the first condition, the processing state is the dropping state.

[0013] In some possible implementation manners, the first condition includes any one of the following:

[0014] The occlusion value is not greater than a first threshold value, where the first threshold value is determined by a first occlusion value within a preset range when the self-moving device is in an unloaded state and the mechanical arm is in a carrying posture;

[0015] The occlusion value is less than a second threshold value and the duration exceeds a first preset duration, where the second threshold value is determined by a second occlusion value within a preset range when the self-moving device is in a carrying state and the mechanical arm is in a carrying posture, and the second threshold value is greater than the first threshold value;

[0016] The occlusion value is less than the occlusion value corresponding to the previous moment and the duration exceeds a second preset duration.

[0017] In some possible implementation manners, the self-moving device further includes a first sensor disposed on the top of the self-moving chassis, and the data acquisition range of the first sensor covers the preset range when the mechanical arm is in a carrying posture;

[0018] The obtaining of the detection information includes:

[0019] Collecting the first point cloud data through the first sensor.

[0020] In some possible implementation manners, the robotic arm further includes a support structure, and the working arm includes a first working arm and a second working arm; the mechanical hand and the first working arm are connected through a first mechanical joint; the first working arm and the second working arm are connected through a second mechanical joint, and the second mechanical joint drives the first working arm to rotate; the second working arm and the support structure are connected through a third mechanical joint, and the third mechanical joint drives the second working arm to rotate; the second working arm is connected to the self-moving chassis through the support structure;

[0021] The joint elbow drive motor includes a first drive motor of the second mechanical joint and a second drive motor of the third mechanical joint.

[0022] In some possible implementation manners, the detection information includes current data, and the current data includes a first current of the first drive motor and a second current of the second drive motor;

[0023] Determining the processing state of the mechanical hand for an object based on the detection information includes:

[0024] If the first current and the second current meet the second condition, the processing state is the dropping state.

[0025] In some possible implementation manners, the second condition includes any one of the following:

[0026] The first current is not greater than a first preset current, and the second current is not greater than a second preset current, where the first preset current is determined based on the first current of the first drive motor when the self-moving device is in an unloaded state, and the second preset current is determined based on the second current of the second drive motor when the self-moving device is in an unloaded state;

[0027] The first current is less than a third preset current, the second current is less than a fourth preset current, and the duration exceeds a third preset duration, where the third preset current is determined based on the third current of the first drive motor when the self-moving device is in a carrying state, the fourth preset current is determined based on the fourth current of the second drive motor when the self-moving device is in a carrying state, the third current is greater than the first preset current, and the fourth current is greater than the second preset current;

[0028] The first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and the duration exceeds a fourth preset duration.

[0029] In some possible implementation manners, the detection information further includes first point cloud data, and determining the processing state of the manipulator for an object based on the detection information includes:

[0030] Determining an occlusion value within a preset range of the manipulator based on the first point cloud data;

[0031] If the occlusion value meets the first condition, and the first current and the second current meet the second condition, the processing state is the dropping state.

[0032] According to a second aspect of the embodiments of the present application, there is provided a detection device for an object state, configured in a self-moving device, the self-moving device including a self-moving chassis and a robotic arm connected to the self-moving chassis, the robotic arm including a working arm and a manipulator, the working arm and the manipulator being connected by a mechanical joint, the mechanical joint including a joint elbow drive motor for driving the working arm to rotate, the device including:

[0033] An information acquisition unit, configured to acquire detection information when the self-moving device is in a carrying state, where the detection information includes at least one of first point cloud data within a preset range in a carrying posture of the manipulator and current data of the joint elbow drive motor;

[0034] A state determination unit, configured to determine a processing state of the manipulator with respect to an object based on the detection information, where the processing state includes a gripping state or a dropping state.

[0035] In some possible implementation manners, the detection information includes the first point cloud data, and the state determination unit is specifically configured to:

[0036] Determine an occlusion value within a preset range when the manipulator is in a carrying posture based on the first point cloud data;

[0037] If the occlusion value meets a first condition, the processing state is a dropping state.

[0038] In some possible implementation manners, the first condition includes any one of the following:

[0039] The occlusion value is not greater than a first threshold, where the first threshold is determined based on a first occlusion value within a preset range when the self - moving device is in an unloaded state and the manipulator is in a carrying posture;

[0040] The occlusion value is less than a second threshold and the duration exceeds a first preset duration, where the second threshold is determined based on a second occlusion value within a preset range when the self - moving device is in a carrying state and the manipulator is in a carrying posture, and the second occlusion value is greater than the first threshold;

[0041] The occlusion value is less than the occlusion value corresponding to the previous moment and the duration exceeds a second preset duration.

[0042] In some possible implementation manners, the self - moving device further includes a first sensor disposed on the top of the self - moving chassis, and a data acquisition range of the first sensor covers the preset range when the manipulator is in a carrying posture;

[0043] The state acquisition unit is specifically configured to:

[0044] Collect the first point cloud data through the first sensor.

[0045] In some possible implementation manners, the robotic arm further includes a support structure, and the working arm includes a first working arm and a second working arm; the manipulator and the first working arm are connected through a first mechanical joint; the first working arm and the second working arm are connected through a second mechanical joint, and the second mechanical joint drives the first working arm to rotate; the second working arm and the support structure are connected through a third mechanical joint, and the third mechanical joint drives the second working arm to rotate; the second working arm is connected to the self - moving chassis through the support structure;

[0046] The elbow joint driving motor includes a first driving motor of the second mechanical joint and a second driving motor of the third mechanical joint.

[0047] In some possible implementation manners, the detection information includes current data, and the current data includes a first current of the first driving motor and a second current of the second driving motor;

[0048] The state determination unit is specifically configured to:

[0049] If the first current and the second current meet a second condition, the processing state is a falling state.

[0050] In some possible implementation manners, the second condition includes any one of the following:

[0051] The first current is not greater than a first preset current, and the second current is not greater than a second preset current, where the first preset current is determined based on the first current of the first driving motor when the self - moving device is in an unloaded state, and the second preset current is determined based on the second current of the second driving motor when the self - moving device is in an unloaded state;

[0052] The first current is less than a third preset current, the second current is less than a fourth preset current, and the duration exceeds a third preset duration, where the third preset current is determined based on a third current of the first driving motor when the self - moving device is in a carrying state, the fourth preset current is determined based on a fourth current of the second driving motor when the self - moving device is in a carrying state, the third current is greater than the first preset current, and the fourth current is greater than the second preset current;

[0053] The first current is less than the current of the first driving motor at the previous moment, the second current is less than the current of the second driving motor at the previous moment, and the duration exceeds a fourth preset duration.

[0054] In some possible implementation manners, the detection information further includes first point cloud data, and the state determination unit is specifically configured to:

[0055] Determine an occlusion value within a preset range of the manipulator based on the first point cloud data;

[0056] If the occlusion value meets a first condition, and the first current and the second current meet the second condition, the processing state is a falling state.

[0057] According to a third aspect of the embodiments of the present application, a self - moving device is provided, which is characterized by including a controller, a self - moving chassis, and a robotic arm connected to the self - moving chassis. The robotic arm includes a working arm and a mechanical hand. The working arm and the mechanical hand are connected by a mechanical joint. The mechanical joint includes an elbow joint drive motor for driving the working arm to rotate. The controller is configured to execute the above - mentioned method for detecting the object state.

[0058] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory. It is characterized in that the processor executes the computer program to implement the above - mentioned method for detecting the object state.

[0059] According to a fifth aspect of the embodiments of the present application, a computer - readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above - mentioned embodiments are implemented.

[0060] According to a sixth aspect of the embodiments of the present application, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the method in the above - mentioned embodiments are implemented.

[0061] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application.

[0062] The beneficial effects brought by the technical solutions provided in the embodiments of the present application are as follows:

[0063] When the self - moving device is in the carrying state, the mechanical hand is in the carrying posture. By obtaining at least one of the first point cloud data within a preset range when the mechanical hand is in the carrying posture and the current data of the elbow joint drive motor for driving the working arm to rotate, it is possible to timely detect whether an object has fallen through the visual dimension and / or the physical dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings here are incorporated into the description and form a part of this description, showing the embodiments consistent with the present application, and are used together with the description to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0065] Figure 1 is a schematic structural diagram of the self - moving device provided in the embodiments of the present application;

[0066] Figure 2 is a flowchart of a method for detecting the object state provided in the embodiments of the present application;

[0067] Figure 3 Schematic diagram of the carrying posture of the self - moving device provided by the embodiment of the present application;

[0068] Figure 4 Schematic diagram of the clamping posture of the self - moving device provided by the embodiment of the present application;

[0069] Figure 5 Schematic diagram of the structure of a robotic arm provided by the embodiment of the present application;

[0070] Figure 6 Schematic diagram of the structure of a robotic arm provided by an example of the present application;

[0071] Figure 7 Schematic diagram of the field of view of the first sensor provided by an example of the present application;

[0072] Figure 8 Top view of the self - moving device when the robotic arm is in the clamping posture in an example of the present application;

[0073] Figure 9 Schematic diagram of a scheme for detecting the state of an object provided by the embodiment of the present application;

[0074] Figure 10 Front view of the self - moving device in an example of the present application;

[0075] Figure 11 Schematic diagram of the field of view of the second sensor provided by an example of the present application;

[0076] Figure 12 Schematic diagram of the structure of a device for detecting the state of an object provided by the embodiment of the present application;

[0077] Figure 13 Schematic diagram of the structure of an electronic device for detecting the state of an object provided by the embodiment of the present application. Detailed implementation manners

[0078] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0079] In addition, the described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0080] The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0081] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0082] It should also be noted that the terms "first", "second", etc. in the specification, claims, and above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the objects so used can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described.

[0083] The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below by describing several exemplary embodiments. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0084] First, the structure of the self-mobile device of the present application will be described. As Figure 1 shown, the self-mobile device 100 includes a self-mobile chassis 110 and a robotic arm 120 connected to the self-mobile chassis. The robotic arm 120 includes a robot hand 121 and a working arm 122. The working arm 122 and the robot hand 121 are connected by a mechanical joint 123. The mechanical joint 123 includes a joint elbow drive motor 1231 for driving the working arm 122 to rotate.

[0085] Specifically, the manipulator 121 is used to grasp an object; the joint elbow drive motor 1231 is used to drive the working arm 122 to rotate to cooperate with the manipulator 121 to grasp the object. After grasping the object, the working arm 122 is used to bear the gravity of the grasped object.

[0086] It can be understood that in this application, the manipulator 121 can fix the object in various forms, such as clamping, grasping, holding, lifting, etc. For the convenience of description, it is uniformly referred to as clamping in this application.

[0087] The embodiment of this application provides a method for detecting the state of an object, which is applied to a self-mobile device. Specifically, it can be applied to the controller of the self-mobile device. The controller can be set in the self-mobile chassis or in the robotic arm, and this is not limited.

[0088] As Figure 2 shown, in some possible implementation manners, taking the execution entity as a self-mobile device as an example, the method for detecting the state of an object may include the following steps:

[0089] Step S201, when the self-mobile device is in the carrying state, obtain detection information.

[0090] Among them, the self-mobile device refers to a robot or intelligent device with the ability of autonomous movement. The self-mobile device may also have the ability of cleaning. For example, it may be a floor cleaning robot.

[0091] Among them, the carrying state is the fixed posture of the self-mobile device after grasping an object.

[0092] Specifically, the carrying state may include the fixed posture of the self-mobile device during transporting the object after grasping the object, or may also include the posture of the self-mobile device after grasping the object but before starting to transport the object.

[0093] Combined with Figure 3 and Figure 4 shown, Figure 3 is the carrying posture of the robotic arm 120, Figure 4 is the grasping posture of the robotic arm 120. The angle of the working arm in the carrying posture is different from the angle of the working arm in the grasping posture. Specifically, the working arm 122 can be driven by the joint elbow drive motor to rotate different angles, so as to realize the switching between the carrying posture and the grasping posture.

[0094] Among them, the detection information includes at least one of the first point cloud data within a preset range when the manipulator is in the carrying posture and the current data of the joint elbow drive motor.

[0095] Specifically, the first point cloud data is three-dimensional spatial data collected by a sensor, consisting of a large number of discrete points, and each point contains three-dimensional coordinate information.

[0096] In the specific implementation process, when the manipulator grabs an object, the first point cloud data within the preset range of the manipulator is inconsistent with the first point cloud data when the manipulator is unloaded. For example, when the manipulator grabs an object, there are more dense points corresponding to the first point cloud data. The specific method for obtaining the first point cloud data will be further elaborated in detail below.

[0097] Specifically, the current data can reflect the load current required by the joint elbow drive motor to resist gravity. That is to say, when the manipulator grabs an object, the load current corresponding to the joint elbow drive motor to resist gravity will be greater than the load current corresponding to the case when the manipulator is unloaded.

[0098] Step S202, determining the processing state of the manipulator for the object based on the detection information.

[0099] Among them, the processing state includes a gripping state or a dropping state.

[0100] Specifically, the gripping state is the state where the object is grabbed by the manipulator when the manipulator is in the gripping posture; the dropping state is the state where the object drops from the manipulator when the manipulator is in the gripping posture.

[0101] In the above embodiments, when the self-mobile device is in the carrying state, the manipulator is in the carrying posture. By obtaining at least one of the first point cloud data within the preset range when the manipulator is in the carrying posture and the current data of the joint elbow drive motor for driving the working arm to rotate, it is possible to detect in a timely manner whether the object has dropped through the visual dimension and / or the physical dimension.

[0102] The following will elaborate in detail the process of specifically determining the processing state when detecting the first point cloud data in combination with embodiments.

[0103] In some possible implementation manners, the detection information includes the first point cloud data. Step S202 determining the processing state of the manipulator for the object based on the detection information may include:

[0104] (1) Determining the occlusion value within the preset range of the manipulator in the carrying posture based on the first point cloud data.

[0105] Specifically, the first point cloud data can be filtered, for example, removing outliers and background noise to obtain the number of valid point clouds within the preset range; then, based on the number of valid point clouds within the preset range, the occlusion value is determined.

[0106] In the specific implementation process, the occlusion value is positively correlated with the number of valid point clouds within a preset range.

[0107] For example, the number of valid point clouds within the preset range can be directly used as the occlusion value.

[0108] It should be noted that the occlusion value is determined when the manipulator is in the carrying posture. If it is not in the carrying posture, it may exceed the data acquisition range of the sensor for obtaining the first point cloud data. Moreover, the first point cloud data needs to be obtained in a fixed posture to have a comparison value.

[0109] (2) If the occlusion value meets the first condition, the processing state is the dropping state.

[0110] Specifically, the first condition includes any one of the following:

[0111] ① The occlusion value is not greater than the first threshold.

[0112] Among them, the first threshold is determined by the first occlusion value within the preset range when the manipulator is in the carrying posture when the self-mobile device is in the no-load state.

[0113] Specifically, the first threshold can be understood as the no-load threshold. By comparing the occlusion value and the no-load threshold, it can be determined whether the manipulator has grasped an object in the carrying posture.

[0114] For example, the preset range includes the range within 0.15 meters centered on the manipulator, and the first threshold is 20 points; when the occlusion value exceeds 20 points, it can be considered that the object is in the grasped state; when the occlusion value is 20 points, it can be considered that the object is in the dropping state.

[0115] ② The occlusion value is less than the second threshold and the duration exceeds the first preset duration.

[0116] Among them, the second threshold is determined by the second occlusion value within the preset range when the manipulator is in the carrying posture when the self-mobile device is in the carrying state, and the second occlusion value is greater than the first threshold.

[0117] Specifically, the second threshold can be understood as the carrying threshold corresponding to the currently clamped object. By comparing the occlusion value and the carrying threshold corresponding to the object, it can be determined whether the object has dropped.

[0118] It should be noted that for different objects, the corresponding carrying thresholds are different. When determining whether an object has dropped, it must be compared with the carrying threshold corresponding to this object.

[0119] For example, the preset range includes the range within 0.15 meters centered on the manipulator. The first threshold (no-load threshold) is 20 points. After the clamping action is performed, when the manipulator is in the clamping posture, if the detected occlusion value is 50 points, then 50 points can be used as the second threshold. When in the transportation state, if the occlusion value within the preset range when the manipulator is in the transportation posture is less than 50 points and lasts for 3 s, it can be considered that the object has fallen.

[0120] In this embodiment, it is considered that the object has fallen only when the occlusion value is less than the second threshold and the duration exceeds the first preset duration, which can avoid misjudgment caused by fluctuations in the first point cloud data when the self-mobile device is in the transportation state.

[0121] ③ The occlusion value is less than the occlusion value corresponding to the previous moment, and the duration exceeds the second preset duration.

[0122] Specifically, if the occlusion value within the preset range suddenly decreases when the manipulator is in the transportation posture and the duration exceeds the second preset duration, it can be considered that the object has fallen.

[0123] For example, the preset range includes the range within 0.15 meters centered on the manipulator. If the occlusion value suddenly changes from 50 points to 20 points and 20 points lasts for 2 s, it can be considered that the object has fallen.

[0124] It can be understood that the first preset duration and the second preset duration can be the same or different, and the present application does not limit this.

[0125] Similarly, in this embodiment, it is considered that the object has fallen only when the occlusion value is less than the occlusion value corresponding to the previous moment and the duration exceeds the second preset duration, which can avoid misjudgment caused by fluctuations in the first point cloud data when the self-mobile device is in the transportation state.

[0126] In the above embodiment, when the manipulator is in the transportation posture, by comparing the occlusion value with the no-load threshold, or by comparing the transportation threshold corresponding to the currently clamped object, or by comparing the occlusion value with the occlusion value corresponding to the previous moment, it can be determined whether the object has fallen, and misjudgment caused by fluctuations in the first point cloud data when the self-mobile device is in the transportation state can also be avoided.

[0127] The following will describe the specific structure of the robotic arm of the present application in combination with embodiments.

[0128] In some possible implementation manners, such as Figure 5As shown, the robotic arm 120 further includes a support structure 124. The working arm 122 includes a first working arm 1221 and a second working arm 1222. The robotic hand 121 and the first working arm 1221 are connected through a first mechanical joint 1231. The first working arm 1221 and the second working arm 1222 are connected through a second mechanical joint 1232, and the second mechanical joint 1232 drives the first working arm 1221 to rotate. The second working arm 1222 and the support structure 124 are connected through a third mechanical joint 1233, and the third mechanical joint 1233 drives the second working arm 1222 to rotate. The second working arm 1222 is connected to the self-moving chassis 110 through the support structure 124.

[0129] In one example, as Figure 6 shown, the support structure 124 includes a support arm 1241, a rotating base 1242, and a base 1243. Specifically, the second working arm 1222 and the support arm 1241 are connected through a third mechanical joint 1233, and the third mechanical joint 1233 drives the second working arm 1222 to rotate. The other end of the support arm is connected to the rotating base 1242 through a fourth mechanical joint 1234, so that the support arm 1241 can be folded or unfolded relative to the rotating base 1242, such as the support arm 1241 can be lifted or lowered relative to the rotating base 1242. The rotating base 1242 is connected to the base 1243 through a fifth mechanical joint 1235, so that the rotating base 1242 can rotate relative to the base 1243. The entire robotic arm 120 is fixed to the self-moving chassis through the base 1243 and the mounting structure. The mounting structure can be a mounting seat, a mounting hole, a card slot, or other structures.

[0130] Specifically, the support structure can also have other forms. For example, there can be at least one connecting arm and mechanical joint in the support structure. This application does not limit this.

[0131] That is to say, the robotic arm provided by the embodiment of the present invention is a five-degree-of-freedom, three-arm-segment foldable design. Thus, the movement range of the robotic arm can be increased, and then the object clamping range of the self-moving device can be improved, and the use range of the product can be expanded.

[0132] The following will describe the specific acquisition process of the first point cloud data in combination with embodiments.

[0133] In some possible implementation manners, the self-moving device further includes a first sensor disposed on the top of the self-moving chassis, and the data acquisition range of the first sensor covers the preset range when the robotic hand is in the carrying posture.

[0134] The obtaining of the detection information includes:

[0135] Collect the first point cloud data through the first sensor.

[0136] Among them, the first sensor may include a solid-state LiDAR (Light Detection and Ranging) or a ToF (Time-of-Flight) depth camera, which is installed on the top of the self-moving chassis.

[0137] Such as Figure 7 As shown, in the example of this application, the first sensor is a TOF sensor. The field of view of the first sensor 701 is shown in the figure. It can be seen that when the manipulator is in the clamping posture, the field of view of the first sensor covers the preset range of the manipulator, so that the first point cloud data can be collected.

[0138] Such as Figure 8 As shown, Figure 8 This is a top view of the self-moving device 100 when the robotic arm 120 is in the clamping posture in an example of this application. When the robotic arm 120 is in the clamping posture, from a top view, the clamping direction of the robotic arm 120 is the first direction, and the straight line where the first direction is located passes through the center of the self-moving chassis 110; the first sensor 701 is arranged along the second direction at the edge of the top wall of the self-moving chassis 110, and the second direction is the opposite direction of the first direction.

[0139] In other embodiments, the first sensor may also be arranged at other positions on the top of the self-moving chassis, as long as the field of view of the first sensor covers the preset range of the manipulator, and this is not limited.

[0140] The following will further elaborate on the specific process of determining the processing state based on the current data in this application in combination with specific embodiments.

[0141] In the embodiments of this application, the joint elbow drive motor includes the first drive motor of the second mechanical joint and the second drive motor of the third mechanical joint.

[0142] Specifically, the second mechanical joint drives the first working arm to rotate, and the third mechanical joint drives the second working arm to rotate. The first mechanical arm and the second working arm are used to bear the gravity of the object clamped by the manipulator. Therefore, the current data of the first drive motor and the current data of the second drive motor can reflect the load current required by the joint elbow drive motor to resist gravity.

[0143] In some possible implementation manners, the detection information includes current data, and the current data includes the first current of the first drive motor and the second current of the second drive motor.

[0144] Step S202 of determining the processing state of the manipulator for the object based on the detection information may include:

[0145] If the first current and the second current meet the second condition, the processing state is the dropping state.

[0146] In this embodiment, by combining the first current and the second current to determine the processing state, the accuracy of the processing state determination can be improved.

[0147] In some possible implementation manners, the second condition includes any one of the following:

[0148] ① The first current is not greater than a first preset current, and the second current is not greater than a second preset current.

[0149] Wherein, the first preset current is determined based on a first current of the first drive motor when the self - moving device is in an unloaded state, and the second preset current is determined based on a second current of the second drive motor when the self - moving device is in an unloaded state.

[0150] Specifically, the first preset current can be understood as the no - load current. By comparing the currents of the first drive motor and the second drive motor with their respective no - load currents, it can be determined whether the manipulator has grasped an object.

[0151] For example, the first preset current is 0.08 A, that is, when the robotic arm is in the unloaded state, the current of the first drive motor is 0.08 A, and the second preset current is 0.08 A, that is, when the robotic arm is in the unloaded state, the current of the second drive motor is 0.08 A. When the first current is greater than 0.08 A, or the second current is greater than 0.08 A, it can be considered that the object is in the grasped state; when the first current is 0.08 A and the second current is 0.08 A, it can be considered that the object is in the dropping state.

[0152] ② The first current is less than a third preset current, the second current is less than a fourth preset current, and the duration exceeds a third preset duration.

[0153] Wherein, the third preset current is determined based on a third current of the first drive motor when the self - moving device is in a carrying state, the fourth preset current is determined based on a fourth current of the second drive motor when the self - moving device is in a carrying state, the third current is greater than the first preset current, and the fourth current is greater than the second preset current.

[0154] Specifically, the third preset current and the fourth preset current can be understood as the carrying currents corresponding to the currently clamped object. By comparing the first current with the third preset current and comparing the second current with the fourth preset current, it can be determined whether the object has dropped.

[0155] It should be noted that for different objects, the corresponding carrying current is different. When determining whether an object has fallen, it is necessary to compare it with the carrying current corresponding to this object for judgment.

[0156] For example, in the no-load state of the robotic arm, the current of the first drive motor is 0.08 A, and the current of the second drive motor is 0.08 A; after the gripping action, the current of the first drive motor is 0.15 A, and the current of the second drive motor is 0.2 A. At this time, 0.15 A can be used as the third preset current, and 0.2 A can be used as the fourth preset current. If in the carrying state, the first current is less than 0.15 A, and the second current is less than 0.2 A, and it lasts for 3 s, it can be considered that the processing state is the falling state.

[0157] In this embodiment, only when the first current is less than the third preset current, the second current is less than the fourth preset current, and the duration exceeds the third preset duration, is it considered that the object has fallen, which can avoid misjudgment caused by fluctuations in current data during the carrying state of the self-mobile device.

[0158] ③ The first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and the duration exceeds the fourth preset duration.

[0159] Specifically, if the current data of the robotic arm suddenly decreases during the carrying state and the duration exceeds the fourth preset duration, it can be considered that the processing state is the falling state.

[0160] For example, when the robotic arm is in the carrying state, the first current data suddenly changes from 0.15 A to 0.08 A, and the second current data suddenly changes from 0.2 A to 0.08 A, and it lasts for 2 s, then it can be considered that the object has fallen.

[0161] Similarly, in this embodiment, only when the first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and it lasts for the fourth preset duration, is it considered that the object has fallen, which can avoid misjudgment caused by fluctuations in current data during the carrying state of the self-mobile device.

[0162] It can be understood that the third preset duration and the fourth preset duration can be the same or different, and the present application does not limit this.

[0163] In the above embodiments, by comparing the current data with the no-load current, or by comparing the carrying current corresponding to the currently gripped object, or by comparing the current data with the current data corresponding to the previous moment, it is possible to determine whether the object has fallen, and it is also possible to avoid misjudgment caused by fluctuations in current data during the carrying state of the self-mobile device.

[0164] In some possible embodiments, the detection information further includes first point cloud data, and determining whether an object has fallen based on the detection information includes:

[0165] Determining an occlusion value within a preset range of the robotic arm based on the first point cloud data;

[0166] If the occlusion value meets the first condition and the first current and the second current meet the second condition, the processing state is the fallen state.

[0167] In this embodiment, the first point cloud data and the current data can be combined for simultaneous judgment. Only when the first point cloud data meets the first condition and the current data simultaneously meets the second condition is it considered that the object has fallen. At this time, misjudgment caused by instability of the current data or the first point cloud data can be avoided, effectively improving the accuracy of object fall detection.

[0168] The following will illustrate the object state detection method of the present application with examples.

[0169] As Figure 9 shown, in one example, the object state detection method of the present application may include the following steps:

[0170] When the self - moving device is in the carrying state, obtaining detection information; wherein, the detection information includes first point cloud data within a preset range of the robotic arm in the carrying posture and current data of the joint elbow drive motor;

[0171] Determining an occlusion value within a preset range of the robotic arm based on the first point cloud data;

[0172] If the occlusion value meets the first condition and the current data meets the second condition, the processing state of the object is the fallen state.

[0173] The following will illustrate the compensation mechanism after detecting that an object has fallen with embodiments.

[0174] In some possible embodiments, the method further includes:

[0175] If it is determined based on the detection information that the processing state of the robotic arm for the object is the fallen state, controlling a prompting device to issue an object - fallen reminder.

[0176] Specifically, if it is detected that the processing state of the object is the fallen state, the object - fallen can be prompted first.

[0177] In some possible embodiments, as Figure 10 shown, Figure 10Fig. 0 is a front view of the self - moving device 100 in an example. A second sensor 1001 may also be provided along the side wall of the self - moving chassis 110 in a first direction. The second sensor 1001 may also include a solid - state LiDAR or ToF.

[0178] As Figure 11 shown, Figure 11 Fig. 7 shows the field of view of the second sensor in an example. In this example, the second sensor is a TOF, which may also be referred to as a front - mounted TOF. If an object drops, the second sensor can collect second point - cloud data in front of the self - moving device to determine whether there is an object.

[0179] In some possible implementation manners, the method further includes:

[0180] If it is determined based on the detection information that the processing state of the manipulator for the object is a dropped state, determine the object drop position through the second point - cloud data, and plan an object clamping path based on the object drop position;

[0181] Re - clamp the object based on the object clamping path.

[0182] Specifically, after clamping the object, if the detection information can be obtained again, and it is determined based on the new detection information that the object is in a clamped state, the transportation task can be continued.

[0183] In the above - mentioned object state detection method, when the self - moving device is in the transportation state, the manipulator is in the transportation posture. By obtaining at least one of the first point - cloud data within a preset range when the manipulator is in the transportation posture and the current data of the joint elbow drive motor for driving the working arm to rotate, it is possible to detect in a timely manner whether the object has dropped through the visual dimension and / or the physical dimension.

[0184] Furthermore, when the manipulator is in the transportation posture, by comparing the occlusion value and the no - load threshold, or by comparing the transportation threshold corresponding to the currently clamped object, or by comparing the occlusion value and the occlusion value corresponding to the previous moment, it is possible to determine whether the object has dropped, and it is also possible to avoid misjudgment caused by fluctuations in the first point - cloud data during the transportation state of the self - moving device.

[0185] Furthermore, by comparing the current data and the no - load current, or by comparing the transportation current corresponding to the currently clamped object, or by comparing the current data and the current data corresponding to the previous moment, it is possible to determine whether the object has dropped, and it is also possible to avoid misjudgment caused by fluctuations in the current data during the transportation state of the self - moving device.

[0186] Further, by jointly judging the first point cloud data and the current data, it is considered that the object has fallen only when the first point cloud data meets the first condition and the current data meets the second condition at the same time. In this way, misjudgment caused by the instability of the current data or the first point cloud data can be avoided, and the accuracy of object fall detection can be effectively improved.

[0187] In some possible implementation manners, as Figure 12 shown, a detection device 1200 for the state of an object is provided, which is configured on a self-moving device. The self-moving device includes a self-moving chassis and a robotic arm connected to the self-moving chassis. The robotic arm includes a working arm and a mechanical hand. The working arm and the mechanical hand are connected by a mechanical joint. The mechanical joint includes a joint elbow drive motor for driving the working arm to rotate. The device includes:

[0188] An information acquisition unit 1201, configured to acquire detection information when the self-moving device is in a carrying state. The detection information includes at least one of the first point cloud data within a preset range in the carrying posture of the mechanical hand and the current data of the joint elbow drive motor;

[0189] A state determination unit 1202, configured to determine the processing state of the mechanical hand for the object based on the detection information. The processing state includes a clamping state or a falling state.

[0190] In some possible implementation manners, the detection information includes the first point cloud data. The state determination unit 1202 is specifically configured to:

[0191] Determine an occlusion value within a preset range in the carrying posture of the mechanical hand based on the first point cloud data;

[0192] If the occlusion value meets the first condition, the processing state is the falling state.

[0193] In some possible implementation manners, the first condition includes any one of the following:

[0194] The occlusion value is not greater than a first threshold, where the first threshold is determined by a first occlusion value within a preset range in the carrying posture of the mechanical hand when the self-moving device is in an unloaded state;

[0195] The occlusion value is less than a second threshold and the duration exceeds a first preset duration, where the second threshold is determined by a second occlusion value within a preset range in the carrying posture of the mechanical hand when the self-moving device is in a carrying state, and the second occlusion value is greater than the first threshold;

[0196] The occlusion value is less than the occlusion value corresponding to the previous moment and the duration exceeds a second preset duration.

[0197] In some possible embodiments, the self - moving device further includes a first sensor disposed on top of the self - moving chassis, and the data acquisition range of the first sensor covers the preset range when the manipulator is in the carrying posture;

[0198] The state acquisition unit 1202 is specifically configured to:

[0199] Collect the first point cloud data through the first sensor.

[0200] In some possible embodiments, the robotic arm further includes a support structure, and the working arm includes a first working arm and a second working arm; the manipulator and the first working arm are connected by a first mechanical joint; the first working arm and the second working arm are connected by a second mechanical joint, and the second mechanical joint drives the first working arm to rotate; the second working arm and the support structure are connected by a third mechanical joint, and the third mechanical joint drives the second working arm to rotate; the second working arm is connected to the self - moving chassis through the support structure;

[0201] The joint elbow drive motor includes a first drive motor of the second mechanical joint and a second drive motor of the third mechanical joint.

[0202] In some possible embodiments, the detection information includes current data, and the current data includes a first current of the first drive motor and a second current of the second drive motor;

[0203] The state determination unit 1202 is specifically configured to:

[0204] If the first current and the second current meet the second condition, the processing state is the falling state.

[0205] In some possible embodiments, the second condition includes any one of the following:

[0206] The first current is not greater than a first preset current, and the second current is not greater than a second preset current, where the first preset current is determined based on the first current of the first drive motor when the self - moving device is in the no - load state, and the second preset current is determined based on the second current of the second drive motor when the self - moving device is in the no - load state;

[0207] The first current is less than a third preset current, the second current is less than a fourth preset current, and the duration exceeds a third preset duration, where the third preset current is determined based on a third current of the first drive motor when the self - moving device is in a carrying state, the fourth preset current is determined based on a fourth current of the second drive motor when the self - moving device is in a carrying state, the third current is greater than a first preset current, and the fourth current is greater than a second preset current;

[0208] The first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and the duration exceeds a fourth preset duration.

[0209] In some possible implementation manners, the detection information further includes first point cloud data, and the state determination unit 1202 is specifically configured to:

[0210] Determine an occlusion value within a preset range of the manipulator based on the first point cloud data;

[0211] If the occlusion value meets a first condition, and the first current and the second current meet a second condition, then the processing state is a dropping state.

[0212] When the self - moving device is in a carrying state, the manipulator is in a carrying posture. By acquiring at least one of the first point cloud data within a preset range when the manipulator is in the carrying posture and the current data of the joint elbow drive motor for driving the working arm to rotate, it is possible to timely detect whether an object has dropped through the visual dimension and / or the physical dimension.

[0213] Furthermore, when the manipulator is in the carrying posture, by comparing the occlusion value with an unloaded threshold, or by comparing the carrying threshold corresponding to the currently held object, or by comparing the occlusion value with the occlusion value corresponding to the previous moment, it is possible to determine whether the object has dropped, and it is also possible to avoid misjudgment caused by fluctuations in the first point cloud data during the carrying state of the self - moving device.

[0214] Furthermore, by comparing the current data with the no - load current, or by comparing the carrying current corresponding to the currently held object, or by comparing the current data with the current data corresponding to the previous moment, it is possible to determine whether the object has dropped, and it is also possible to avoid misjudgment caused by fluctuations in the current data during the carrying state of the self - moving device.

[0215] Further, by simultaneously judging in combination with the first point cloud data and the current data, it is considered that an object has fallen only when the first point cloud data meets the first condition and the current data simultaneously meets the second condition. At this time, misjudgment caused by the instability of the current data or the first point cloud data can be avoided, effectively improving the accuracy of object fall detection.

[0216] In an alternative embodiment, an electronic device is provided, such as Figure 13 shown. Figure 13 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as connected through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 may be used for data interaction between this electronic device and other electronic devices, such as sending and / or receiving data, etc. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0217] The processor 4001 may be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 4001 may also be a combination that implements computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0218] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard structure) bus, etc. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 13 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0219] The memory 4003 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited here.

[0220] The memory 4003 is used to store the computer program for implementing the embodiments of this application and is controlled by the processor 4001 for execution. The processor 4001 is configured to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0221] Exemplarily, the electronic device can be a control terminal or a cleaning device. When the electronic device is a control terminal, the processor 4001 is configured to execute step S301 and step S302. When the electronic device is a cleaning device, the processor 4001 is configured to execute step S901 and step S902.

[0222] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement the steps and corresponding contents of the foregoing method embodiments.

[0223] The embodiments of this application also provide a computer program product, including a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding contents of the foregoing method embodiments.

[0224] It should be understood that although the flowchart of the embodiments of the present application indicates each operation step by arrows, the execution order of these steps is not limited to the order indicated by the arrows. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.

[0225] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. A method for detecting the state of an object, characterized in that, Applied to a self - moving device, the self - moving device includes a self - moving chassis and a robotic arm connected to the self - moving chassis. The robotic arm includes a working arm and a manipulator. The working arm and the manipulator are connected by a mechanical joint. The mechanical joint includes a joint elbow drive motor for driving the working arm to rotate. The method includes: When the self - moving device is in a carrying state, obtain detection information. Among them, the detection information includes at least one of the first point cloud data within a preset range when the manipulator is in a carrying posture and the current data of the joint elbow drive motor; Based on the detection information, determine the processing state of the manipulator for an object. Among them, the processing state includes a clamping state or a dropping state.

2. The method according to claim 1, characterized in that The detection information includes the first point cloud data. Determining the processing state of the manipulator for an object based on the detection information includes: Based on the first point cloud data, determine the occlusion value within a preset range when the manipulator is in a carrying posture; If the occlusion value meets the first condition, the processing state is the dropping state.

3. The method according to claim 2, characterized in that, The first condition includes any one of the following: The occlusion value is not greater than a first threshold value, where the first threshold value is determined by the first occlusion value within a preset range when the manipulator is in a carrying posture when the self - moving device is in an unloaded state; The occlusion value is less than a second threshold value and the duration exceeds a first preset duration, where the second threshold value is determined by the second occlusion value within a preset range when the manipulator is in a carrying posture when the self - moving device is in a carrying state, and the second occlusion value is greater than the first threshold value; The occlusion value is less than the occlusion value corresponding to the previous moment and the duration exceeds a second preset duration.

4. The method according to claim 2, wherein The self - moving device further includes a first sensor disposed on the top of the self - moving chassis. The data acquisition range of the first sensor covers the preset range when the manipulator is in a carrying posture; Obtaining the detection information includes: Collect the first point cloud data through the first sensor.

5. The method according to claim 1, characterized in that, The robotic arm further includes a support structure. The working arm includes a first working arm and a second working arm; the manipulator and the first working arm are connected by a first mechanical joint; the first working arm and the second working arm are connected by a second mechanical joint, and the second mechanical joint drives the first working arm to rotate; The second working arm and the support structure are connected by a third mechanical joint, and the third mechanical joint drives the second working arm to rotate; The second working arm is connected to the self - moving chassis through the support structure; The joint elbow drive motor includes a first drive motor of the second mechanical joint and a second drive motor of the third mechanical joint.

6. The method according to claim 5, wherein The detection information includes current data, and the current data includes a first current of the first drive motor and a second current of the second drive motor; Determining the processing state of the manipulator for an object based on the detection information includes: If the first current and the second current meet the second condition, the processing state is the dropping state.

7. The method according to claim 6, wherein The second condition includes any one of the following: The first current is not greater than a first preset current, and the second current is not greater than a second preset current. Wherein, the first preset current is determined based on a first current of the first drive motor when the self - moving device is in an unloaded state, and the second preset current is determined based on a second current of the second drive motor when the self - moving device is in an unloaded state; the first current is less than a third preset current, the second current is less than a fourth preset current, and the duration exceeds a third preset duration. Wherein, the third preset current is determined based on a third current of the first drive motor when the self - moving device is in a carrying state, the fourth preset current is determined based on a fourth current of the second drive motor when the self - moving device is in a carrying state, the third current is greater than the first preset current, and the fourth current is greater than the second preset current; The first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and the duration exceeds a fourth preset duration.

8. The method according to claim 6, characterized in that, The detection information further includes first point cloud data. Determining the processing state of the manipulator for an object based on the detection information includes: Determining an occlusion value within a preset range of the manipulator based on the first point cloud data; If the occlusion value meets a first condition and the first current and the second current meet a second condition, then the processing state is a dropping state.

9. A detection device for the state of an object, characterized in that, Configured on a self - moving device, the self - moving device includes a self - moving chassis and a robotic arm connected to the self - moving chassis. The robotic arm includes a working arm and a manipulator. The working arm and the manipulator are connected by a mechanical joint. The mechanical joint includes an elbow joint drive motor for driving the working arm to rotate. The detection device includes: An information acquisition unit for acquiring detection information when the self - moving device is in a carrying state, where the detection information includes at least one of first point cloud data within a preset range of the manipulator in a carrying posture and current data of the elbow joint drive motor; A state determination unit for determining the processing state of the manipulator for an object based on the detection information, where the processing state includes a clamping state or a dropping state.

10. The device according to claim 9, wherein, The detection information includes the first point cloud data, and the state determination unit is specifically used for: Determining an occlusion value within a preset range of the manipulator in a carrying posture based on the first point cloud data; If the occlusion value meets the first condition, then the processing state is a dropping state.

11. The device according to claim 10, wherein The first condition includes any one of the following: The occlusion value is not greater than a first threshold value, where the first threshold value is determined based on a first occlusion value within a preset range of the manipulator in a carrying posture when the self - moving device is in an unloaded state; The occlusion value is less than a second threshold value, and the duration exceeds a first preset duration, where the second threshold value is determined based on a second occlusion value within a preset range of the manipulator in a carrying posture when the self - moving device is in a carrying state, and the second occlusion value is greater than the first threshold value; The occlusion value is less than the occlusion value corresponding to the previous moment and lasts for more than a second preset duration.

12. The device according to claim 10, characterized in that, The self-moving device further includes a first sensor disposed on top of the self-moving chassis, and the data acquisition range of the first sensor covers the preset range of the manipulator in the carrying posture. The state acquisition unit is specifically configured to: Collect the first point cloud data through the first sensor.

13. The device according to claim 9, characterized in that, The robotic arm further includes a support structure. The working arm includes a first working arm and a second working arm. The manipulator and the first working arm are connected through a first mechanical joint. The first working arm and the second working arm are connected through a second mechanical joint, and the second mechanical joint drives the first working arm to rotate. The second working arm and the support structure are connected through a third mechanical joint, and the third mechanical joint drives the second working arm to rotate. The second working arm is connected to the self-moving chassis through the support structure. The joint elbow drive motor includes a first drive motor of the second mechanical joint and a second drive motor of the third mechanical joint.

14. The device according to claim 13, characterized in that, The detection information includes current data, and the current data includes a first current of the first drive motor and a second current of the second drive motor. The state determination unit is specifically configured to: If the first current and the second current meet a second condition, the processing state is the falling state.

15. The device according to claim 14, characterized in that, The second condition includes any one of the following: The first current is not greater than a first preset current, and the second current is not greater than a second preset current, where the first preset current is determined based on the first current of the first drive motor when the self-moving device is in the no-load state, and the second preset current is determined based on the second current of the second drive motor when the self-moving device is in the no-load state. The first current is less than a third preset current, the second current is less than a fourth preset current, and it lasts for more than a third preset duration, where the third preset current is determined based on a third current of the first drive motor when the self-moving device is in the carrying state, the fourth preset current is determined based on a fourth current of the second drive motor when the self-moving device is in the carrying state, the third current is greater than the first preset current, and the fourth current is greater than the second preset current. The first current is less than the current of the first drive motor at the previous moment, the second current is less than the current of the second drive motor at the previous moment, and it lasts for more than a fourth preset duration.

16. The device according to claim 14, characterized in that, The detection information further includes first point cloud data. The state determination unit is specifically configured to: Determine the occlusion value within the preset range of the manipulator based on the first point cloud data. If the occlusion value meets a first condition and the first current and the second current meet the second condition, the processing state is the falling state.

17. A self - moving device, characterized in that, It includes a controller, a self-moving chassis, and a robotic arm connected to the self-moving chassis. The robotic arm includes a working arm and a manipulator. The working arm and the manipulator are connected by a mechanical joint. The mechanical joint includes a joint elbow drive motor for driving the rotation of the working arm. The controller is used to execute the method for detecting the state of an object according to any one of claims 1-8.

18. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.

Images