Machine device control method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a machine device control method, device, electronic device and computer-readable medium. A specific implementation of the method includes: in response to determining that the execution result of the first instruction does not reach the preset execution result, obtaining a first point cloud set sequence and a first depth image set sequence; verifying the execution behavior of the target machine device to generate behavior verification information; in response to determining that the behavior verification information represents that the execution behavior of the target machine device is abnormal, generating a movement trajectory set; filtering out the movement trajectory that meets the preset trajectory conditions from the movement trajectory set as the target movement trajectory; generating device adjustment information; generating device action instructions; and controlling the target machine device to move along the target movement trajectory according to the device action instructions. This implementation can accurately and efficiently handle the abnormal behavior of the target machine device.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a machine device control method, device, electronic device, and computer-readable medium. Background Art

[0002] At present, with the continuous development of intelligent robots, intelligent robots are increasingly used in people's daily lives. For the abnormal behavior adjustment of machine devices, the method usually adopted is: to adjust the machine device with abnormal behavior through human assistance.

[0003] However, when the above method is used to adjust a machine device with abnormal behavior, the following technical problems often occur:

[0004] The device needs human intervention to be adjusted, which leads to low adjustment efficiency.

[0005] In the process of adopting technical solutions to solve the above technical problems, the following technical problems often occur: the generation process of the target movement trajectory is also crucial, which greatly affects how the subsequent target machine device moves accurately. In response to the above technical problems, the conventional solution is generally to achieve the generation of the target movement trajectory by continuously adjusting the generation method corresponding to the movement trajectory set. However, the above conventional solution still has the following problem: despite the continuous adjustment of the generation method of the movement trajectory set, the target movement trajectory obtained is difficult and not accurate enough in the absence of data accuracy.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a machine device control method, device, electronic device and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a machine device control method, comprising: in response to determining that a first instruction execution result corresponding to a target machine device does not reach a preset execution result, using a laser radar device and a depth camera device configured by the target machine device to obtain a first point cloud set sequence and a first depth image set sequence corresponding to the target machine device; verifying the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information; in response to determining that the behavior verification information indicates that the execution behavior of the target machine device is abnormal, generating a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence; selecting a movement trajectory that meets a preset trajectory condition from the movement trajectory set as a target movement trajectory; generating device adjustment information corresponding to the target movement trajectory; generating a device action instruction for the device adjustment information; and controlling the target machine device to move along the target movement trajectory according to the device action instruction.

[0010] In a second aspect, some embodiments of the present disclosure provide a machine device control device, including: an acquisition unit, configured to, in response to determining that the execution result of the first instruction corresponding to the target machine device does not reach a preset execution result, acquire a first point cloud set sequence and a first depth image set sequence corresponding to the target machine device using a laser radar device and a depth camera device configured by the target machine device; a verification unit, configured to verify the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information; a first generation unit, configured to, in response to determining that the behavior verification information represents that the execution behavior of the target machine device is abnormal, generate a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence; a screening unit, configured to screen out a movement trajectory that meets a preset trajectory condition from the movement trajectory set as a target movement trajectory; a second generation unit, configured to generate device adjustment information corresponding to the target movement trajectory; a third generation unit, configured to generate a device action instruction for the device adjustment information; and a control unit, configured to control the target machine device to move along the target movement trajectory according to the device action instruction.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0013] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the machine device control method of some embodiments of the present disclosure can accurately and efficiently implement the abnormal behavior processing of the target machine device. Specifically, the reason why the relevant device adjustment for abnormal behavior is not efficient is that the device adjustment needs to be performed in a human-involved manner, resulting in the problem of low adjustment efficiency. Based on this, the machine device control method of some embodiments of the present disclosure, first, in response to determining that the first instruction execution result corresponding to the target machine device does not reach the preset execution result, the laser radar device and the depth camera device configured by the above-mentioned target machine device are used to obtain the first point cloud set sequence and the first depth image set sequence corresponding to the above-mentioned target machine device. Here, the first point cloud set sequence generated by the laser radar device and the first depth image set sequence generated by the depth camera device facilitate the subsequent clear understanding of the movement of the target machine device within a predetermined time period and the surrounding environment corresponding to the target machine device, so as to facilitate the subsequent execution of abnormal behavior verification for the target machine device. Then, according to the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence, the execution behavior of the above-mentioned target machine device can be accurately verified to generate behavior verification information. Here, by performing a behavior verification on the target machine device, it is determined whether the target machine device is abnormal. Next, in response to determining that the above-mentioned behavior verification information represents that the execution behavior of the above-mentioned target machine device is abnormal, the movement trajectory set corresponding to the above-mentioned target machine device can be accurately generated according to the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence. Here, through the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence, not only the position information and device posture information corresponding to the target machine device can be grasped, but also the environmental conditions of the surrounding environment corresponding to the target machine device can be understood, thereby accurately generating a movement trajectory set. Then, the movement trajectory that meets the preset trajectory conditions is selected from the above-mentioned movement trajectory set as the target movement trajectory, so as to serve as the subsequent target machine device to be moved. Secondly, the device adjustment information corresponding to the above-mentioned target movement trajectory is generated to facilitate the target machine device to move the device. Furthermore, the device action instruction for the above-mentioned device adjustment information is generated to facilitate the control of the target machine device to execute the corresponding instruction content. Finally, according to the above-mentioned device action instruction, the above-mentioned target machine device is controlled to move along the above-mentioned target movement trajectory to get rid of the dilemma of the target machine device being in abnormal behavior. In summary, based on the first point cloud set sequence and the first depth image set sequence, a movement trajectory for the target machine device can be accurately generated to achieve device processing of the target machine device under abnormal execution behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flow chart of some embodiments of a machine device control method according to the present disclosure;

[0016] Figure 2 is a schematic diagram of the structure of some embodiments of the machine device control device according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] refer to Figure 1, shows a process 100 of some embodiments of the machine device control method according to the present disclosure. The machine device control method comprises the following steps:

[0025] Step 101, in response to determining that the execution result of the first instruction corresponding to the target machine device does not reach the preset execution result, using the laser radar device and the depth camera device configured by the above-mentioned target machine device to obtain the first point cloud set sequence and the first depth image set sequence corresponding to the above-mentioned target machine device.

[0026] In some embodiments, in response to determining that the first instruction execution result corresponding to the target machine device does not reach the preset execution result, the execution subject (for example, an electronic device) of the machine device control method can use the laser radar device and the depth camera device configured by the target machine device through a wired connection method or a wireless connection method to obtain the first point cloud set sequence and the first depth image set sequence corresponding to the target machine device. Among them, the target machine device may be a machine device that is performing an ongoing execution behavior. The ongoing execution behavior may be the execution behavior corresponding to the first instruction execution result. That is, the instruction content of the behavior instruction corresponding to the first instruction execution result is the ongoing execution behavior. The first instruction execution result may be the last execution result of the behavior instruction. For example, the ongoing execution behavior may be a mopping behavior. The target machine device may be a mopping robot that is performing a mopping behavior. The first instruction execution result may be the last mopping result of the target machine device performing the mopping behavior in the target area. For example, the first instruction execution result may include: a result representing that the mopping behavior has not been completed in the target area and a result representing that the mopping behavior has been completed in the target area. The preset execution result may be a pre-set expected execution result set for the ongoing execution behavior. For example, the preset execution result may be a result characterizing that the mopping behavior has been completed in the target area. The laser radar device may be a device for collecting point cloud sets. The depth camera device may be a device for generating depth images. There is a one-to-one correspondence between the first point cloud set in the first point cloud set sequence and the historical time in the historical time period. The historical time period may be the first 5 minutes before the current time. There is a one-to-one correspondence between the first depth image set in the first depth image set sequence and the historical time in the historical time period. The first depth image set may be a depth image set taken by at least one depth camera device configured for the target machine device at the historical time.

[0027] Step 102 : verifying the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information.

[0028] In some embodiments, the execution subject may verify the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information. The verification information includes: verification information indicating that the execution behavior of the target machine device is not currently in an abnormal execution state, and verification information indicating that the execution behavior of the target machine device is currently in an abnormal execution state.

[0029] As an example, first, the execution subject may determine the actual motion trajectory corresponding to the target machine device using the first point cloud set sequence and the first depth image set sequence. Then, in response to determining that the actual motion trajectory is the same as the preset behavior trajectory corresponding to the execution behavior of the target machine device, verification information is generated indicating that the execution behavior of the target machine device is not currently in an abnormal execution state. Finally, in response to determining that the actual motion trajectory is not the same as the preset behavior trajectory corresponding to the execution behavior of the target machine device, verification information is generated indicating that the execution behavior of the target machine device is currently in an abnormal execution state.

[0030] In some optional implementations of some embodiments, the step of verifying the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information may include the following steps:

[0031] The first step is to generate a three-dimensional space information sequence for the predetermined space of the target machine device according to the first point cloud set sequence and the first depth image set sequence. The predetermined space may be a spherical space with the target machine device as the center and a radius of a preset distance. The three-dimensional space information may characterize the spatial arrangement of the target machine device in the predetermined space at a historical time. In practice, the spatial arrangement may include: the object arrangement position of each object in the space, and the spatial structure of each object in the space.

[0032] The second step is to determine the device posture information sequence corresponding to the target machine device according to the three-dimensional space information sequence. The device posture information in the device posture information sequence has a one-to-one correspondence with the historical time in the historical time period. The device posture information can be the device posture of the target machine device.

[0033] As an example, first, the execution subject may extract the device space information corresponding to the target machine device from each three-dimensional space information in the three-dimensional space information sequence to obtain the device space information sequence. The device space information may be the space information of the target machine device in a predetermined space. Then, the device posture information corresponding to each device space information in the device space information sequence is determined to obtain the device posture information sequence.

[0034] The third step is to generate the above-mentioned behavior verification information according to the above-mentioned device posture information sequence.

[0035] As an example, first, the execution subject may generate a device movement behavior corresponding to the device posture information sequence. Then, the behavior similarity between the device movement behavior and the execution behavior being executed is determined. Finally, in response to determining that the behavior similarity is greater than a target value, verification information is generated indicating that the execution behavior of the target machine device is not currently in an abnormal execution state. In response to determining that the behavior similarity is less than or equal to the target value, verification information is generated indicating that the execution behavior of the target machine device is currently in an abnormal execution state.

[0036] Optionally, the generating of the three-dimensional space information sequence for the predetermined space of the target machine device according to the first point cloud set sequence and the first depth image set sequence may include the following steps:

[0037] The first step is to determine the time information corresponding to each first point cloud set in the first point cloud set sequence to obtain a time information sequence, wherein the time information may be the time content corresponding to the historical time.

[0038] In the second step, for each time information in the above time information sequence, perform the following generation steps:

[0039] Sub-step 1, determining the first point cloud set in the above-mentioned first point cloud set sequence and corresponding to the above-mentioned time information as the first target point cloud set, and determining the first depth image set in the above-mentioned first depth image set sequence and corresponding to the above-mentioned time information as the first target depth image.

[0040] Sub-step 2: Generate spatial information corresponding to the predetermined space according to the device position corresponding to the target machine device. The spatial information may include but is not limited to at least one of the following: the size of the predetermined space, the internal point cloud set of the predetermined space, and the depth image set corresponding to the predetermined space.

[0041] As an example, the execution subject may determine that a spherical space with a device position corresponding to the target machine device as the space center and a radius of a preset distance is the predetermined space, so as to obtain space information corresponding to the predetermined space.

[0042] Sub-step 3, filtering out a first target point cloud subset corresponding to the above-mentioned spatial information from the above-mentioned first target point cloud set, and filtering out a first target depth image subset corresponding to the above-mentioned spatial information from the above-mentioned first target depth image set. The first target point cloud subset may be a point cloud subset located inside the above-mentioned predetermined space. The first target depth image subset may be a depth image subset that has a photographic association relationship with the predetermined space.

[0043] Sub-step 4, generating a distance information set for the first target depth image subset. There is a one-to-one correspondence between the first target depth image in the first target depth image subset and the distance information in the distance information set. The distance information may be the actual distance between the depth shooting device and an object near the target machine device.

[0044] Sub-step 5: generating three-dimensional spatial information corresponding to the time information based on the first target point cloud subset and the distance information set.

[0045] As an example, the execution entity may combine the first target point cloud subset and the distance information set to generate three-dimensional space information.

[0046] Step 103 , in response to determining that the behavior verification information indicates that the execution behavior of the target machine device is abnormal, generating a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence.

[0047] In some embodiments, in response to determining that the above-mentioned behavior verification information represents that the execution behavior of the above-mentioned target machine device is abnormal, the above-mentioned execution subject may generate a set of movement trajectories corresponding to the above-mentioned target machine device according to the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence. Among them, the abnormal execution behavior may include but is not limited to at least one of the following: the target machine device is stuck and cannot move, and the target machine device moves randomly. The movement trajectories in the movement trajectory set may be various movement trajectories that can be executed by the target machine device to solve the problem of abnormal execution behavior. That is, the movement trajectories in the movement trajectory set can theoretically help the target machine device get rid of the problem of abnormal execution behavior.

[0048] In some optional implementations of some embodiments, the target machine device is equipped with a plurality of movable device telescopic brackets. The plurality of movable device telescopic brackets support the target machine device to perform at least one of the following actions: reversing action, erecting action, moving action. The plurality of movable device telescopic brackets can be placed on brackets at various predetermined positions in the target machine device. The device telescopic bracket can support a telescopic function. The brackets of the plurality of movable device telescopic brackets cooperate to realize the diversified movements of the target machine device.

[0049] Optionally, the generating of the movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence may include the following steps:

[0050] The first step is to generate movable space information for the target machine device according to the first point cloud set sequence and the first depth image set sequence. The movable space information may be space information of a space in which the target machine device can move. In practice, the movable space information may be the three-dimensional coordinates of the space in which the target machine device can move.

[0051] As an example, for each first point cloud set in the above-mentioned first point cloud set sequence, first, determine the first depth image set that has a time correspondence relationship with the above-mentioned first point cloud set as the target depth image set. Then, based on the above-mentioned first point cloud set and the target depth image set, determine the spatial information corresponding to the above-mentioned target machine device as the device spatial information, and determine the spatial information corresponding to at least one object around the target machine device as the object spatial information, and obtain at least one object spatial information. Next, remove at least one space corresponding to the object spatial information and the space corresponding to the device spatial information from the predetermined space to obtain the spatial information corresponding to the remaining space as the remaining spatial information. Finally, determine the union of the respective remaining spatial information in the obtained at least one remaining spatial information sequence as the movable spatial information.

[0052] The second step is to determine whether there is storage space information in the pre-stored space information database whose spatial information similarity with the above movable space information is greater than the target value. The target value may be a preset value. The spatial information similarity may be spatial size similarity and spatial shape similarity. Spatial size similarity may characterize the difference in spatial size between two spaces. Spatial shape similarity may characterize the difference in spatial shape between two spaces. The spatial information database stores storage space information sets of various spatial sizes and various spatial shapes. The storage space information whose spatial information similarity is greater than the target value may be storage space information whose spatial size similarity is greater than the target value and whose spatial shape similarity is greater than the target value.

[0053] The third step is to determine at least one target storage space information in response to the determination of existence, wherein the spatial information similarity between the target storage space information and the movable space information is greater than a target value.

[0054] The fourth step is to determine the execution action information sequence corresponding to each target storage space information in the at least one target storage space information, and obtain at least one execution action information sequence. The execution action information in the execution action information sequence is the action information of the action to be executed by the target machine device. The execution action information sequence is the action information sequence used to move in the space corresponding to the target storage space information.

[0055] As an example, the execution subject may determine the execution action information sequence corresponding to each target storage space information in the at least one target storage space information through a pre-prepared execution action information table to obtain at least one execution action information sequence, wherein the execution action information table represents the correspondence between the target storage space information and the execution action information sequence.

[0056] Step 5: Based on the movable space information, each execution action information sequence in the at least one execution action information sequence is simulated to generate simulation information and obtain at least one simulation information. The simulation information represents whether the execution action information sequence corresponding to the execution action sequence in the space corresponding to the movable space information is successful. The simulation information includes: information representing that the movement is unsuccessful and information representing that the movement is successful.

[0057] Step 6: Filter out simulation information that meets a preset condition from the at least one simulation information as target simulation information to obtain at least one target simulation information. The preset condition may be that the simulation information is information that indicates that the movement has been successful.

[0058] In the seventh step, at least one moving trajectory corresponding to the at least one target simulation information is determined as a moving trajectory set. There is a one-to-one correspondence between the target simulation information in the at least one target simulation information and the moving trajectory in the at least one moving trajectory. The moving trajectory can be a moving trajectory composed of the moving trajectories corresponding to the target execution action information sequences in the at least one target execution action information sequence. There is a one-to-one correspondence between the target execution action information sequence in the at least one target execution action information sequence and the target simulation information in the at least one target simulation information.

[0059] Optionally, the steps further include:

[0060] The first step is, in response to determining that it does not exist, using a search-based path planning algorithm or a sampling-based path planning algorithm to determine at least one candidate movement trajectory corresponding to the above movable space information. In practice, the search-based path planning algorithm can be but is not limited to at least one of the following: A* algorithm, Dijkstra algorithm. The sampling-based path planning algorithm can be but is not limited to at least one of the following: PRM algorithm. The candidate movement trajectory can be an alternative movement trajectory for the target machine device to move in the space corresponding to the movable space information.

[0061] The second step is to determine the movement action sequence corresponding to each candidate movement trajectory information in the at least one candidate movement trajectory and relative to the target machine device, and obtain at least one movement action sequence. The movement action sequence in the at least one movement action sequence corresponds to the candidate movement trajectory in the at least one candidate movement trajectory. The movement action sequence can represent how the target machine device moves, so that the movement trajectory is a candidate movement trajectory.

[0062] The third step is to select the mobile action sequences whose actions are in the at least one action from the at least one mobile action sequence to obtain a mobile action sequence group. Wherein, each mobile action in each mobile action sequence in the mobile action sequence group is a subsequence of at least one action. That is, there cannot be an action in the mobile action sequence in the mobile action sequence group that is not in at least one action.

[0063] In step 4, in response to determining that the number of movement action sequences included in the movement action sequence group is greater than the target value, a target movement action sequence whose corresponding number of movement actions meets a preset number condition is selected from the movement action sequence group to obtain a target movement action sequence group. The target value may be a value "1". The preset number condition may be a movement action sequence with the smallest number of movement actions.

[0064] In step 5, in response to determining that the number of target mobile action sequences included in the target mobile action sequence group is greater than the target value, the estimated movement time and estimated movement difficulty corresponding to each target mobile action sequence in the target mobile action sequence group are determined. The estimated movement time may be the time wasted for the target machine device to execute the target mobile action sequence. The estimated movement difficulty may be the difficulty of the target machine device to execute the target mobile action sequence.

[0065] In practice, the estimated difficulty of moving can be generated by the following steps:

[0066] Sub-step 1, obtaining a preset action difficulty corresponding to each action in at least one action.

[0067] Sub-step 2, determining the action difficulty corresponding to each target movement action in the target movement action sequence, and obtaining an action difficulty set.

[0068] Sub-step 3, adding up the difficulty of each action in the above action difficulty set to generate the estimated movement difficulty.

[0069] Step 6: Filter out the target moving action sequences whose estimated moving time meets the preset time condition and whose estimated moving difficulty meets the preset difficulty condition from the above target moving action sequence group as the actual moving action sequence, and obtain the actual moving action sequence group. The preset time condition may be the target moving action sequence in the target moving action sequence group whose corresponding time length is less than the first preset value. The preset difficulty condition may be the target moving action sequence in the target moving action sequence group whose corresponding estimated moving difficulty is less than the second preset value.

[0070] In the seventh step, the candidate movement trajectory group corresponding to the above actual movement action sequence group is determined as a movement trajectory set.

[0071] Step 104 , selecting a movement trajectory that meets a preset trajectory condition from the movement trajectory set as a target movement trajectory.

[0072] In some embodiments, the execution subject may select a moving trajectory that satisfies a preset trajectory condition from the moving trajectory set as a target moving trajectory. The target moving trajectory may be a moving trajectory to be executed by a target machine device. The preset trajectory condition may be a trajectory screening condition pre-set for the moving trajectory. For example, the preset trajectory condition may be a target moving trajectory whose curvature of the corresponding trajectory curve is less than a predetermined curvature value, or a target moving trajectory whose space occupied by the movement of the corresponding trajectory curve is less than a predetermined space size.

[0073] Considering the problems of the above conventional solutions, facing the above technical problems: despite the continuous adjustment of the generation method of the mobile trajectory set, the target mobile trajectory obtained is difficult and not accurate enough in the absence of data accuracy. Combining the advantages / technical status, we can decide to adopt the following solution.

[0074] In some optional implementations of some embodiments, after step 104, the steps further include:

[0075] In the first step, in response to determining that there is no moving trajectory that meets the preset trajectory condition in the moving trajectory set, the point cloud sampling frequency corresponding to the first point cloud set sequence is determined as the first point cloud sampling frequency. The point cloud sampling frequency can be a sampling frequency for sampling the point cloud. For example, the first point cloud sampling frequency samples a point cloud every 1 millisecond.

[0076] In the second step, perform the following sending steps:

[0077] Sub-step 1: Increase the point cloud sampling frequency of the laser radar device to obtain a second point cloud sampling frequency, wherein the second point cloud sampling frequency is greater than the first point cloud sampling frequency. For example, the second point cloud sampling frequency samples a point cloud every 0.5 milliseconds.

[0078] Sub-step 2: generating a third point cloud set sequence for the second point cloud sampling frequency.

[0079] Sub-step 3: Generate a candidate movement trajectory set according to the third point cloud set sequence and the first depth image set sequence. The generation of the candidate movement trajectory set may refer to the generation of the movement trajectory set corresponding to the target machine device.

[0080] Sub-step 4: determining whether there is a movement trajectory that meets a preset trajectory condition in the above candidate movement trajectory set.

[0081] Sub-step 5, in response to determining that it does not exist, determining whether the frequency corresponding to the second point cloud sampling frequency reaches a maximum value.

[0082] Sub-step 6, in response to determining that the maximum value is reached, generating information indicating that there is no target movement track, and sending device abnormal behavior information for the above-mentioned target machine device to the target user terminal.

[0083] In the third step, in response to determining the existence, determining the movement trajectory that meets the preset trajectory condition as the target movement trajectory.

[0084] The above-mentioned content of "In some optional implementation methods of some embodiments", as an inventive point of the present disclosure, solves the technical problem mentioned in the background technology: "Although the generation method of the mobile trajectory set is continuously adjusted, the target mobile trajectory obtained is difficult and not accurate enough in the case of lack of data accuracy." Based on this, the present disclosure continuously adjusts the sampling frequency corresponding to the point cloud sampling frequency of the laser radar device to obtain more and more accurate point cloud data, so that the subsequent candidate mobile trajectory set is more accurate. On the premise of obtaining accurate alternative mobile trajectories, the target mobile trajectory can be determined more quickly and accurately.

[0085] Step 105: Generate device adjustment information corresponding to the target movement trajectory.

[0086] In some embodiments, the execution subject may generate device adjustment information corresponding to the target movement trajectory. The device adjustment information may be adjustment information required to control the target machine device to move according to the target movement trajectory. In practice, the device adjustment information may include: adjustment information of multiple bracket adjustment angles corresponding to multiple movable device telescopic brackets and a moving device of the target machine device. The moving device of the target machine device may be a device used for daily movement of the target machine device.

[0087] Step 106: Generate a device action instruction for the device adjustment information.

[0088] In some embodiments, the execution subject may generate a device action instruction for the device adjustment information, wherein the device action instruction may be an instruction for instructing the target machine device to perform an action according to the device adjustment information.

[0089] As an example, the execution entity may add device adjustment information to an indicator template to generate a device action instruction.

[0090] Step 107, controlling the target machine device to move along the target moving trajectory according to the device action instruction.

[0091] In some embodiments, the execution subject may control the target machine device to move along the target moving trajectory according to the device action instruction.

[0092] In some optional implementations of some embodiments, after step 107, the steps further include:

[0093] The first step is to generate a second instruction execution result corresponding to the device action instruction, wherein the second instruction execution result can represent the movement result after the target machine device executes the device action instruction.

[0094] The second step is to perform the following control steps according to the device adjustment information:

[0095] Sub-step 1, in response to determining that the execution result of the second instruction indicates that the execution behavior corresponding to the target machine device is abnormal, obtaining a second point cloud set sequence and a second depth image set sequence corresponding to the target machine device. The second point cloud set sequence and the second depth image set sequence are obtained based on the process in which the target machine device moves along the target moving trajectory.

[0096] Sub-step 2: adjusting the device adjustment information according to the second point cloud set sequence and the second depth image set sequence to obtain adjusted device adjustment information as the device adjustment information.

[0097] As an example, first, the execution subject may generate the current three-dimensional space information of the target machine device at the current moment according to the second point cloud set sequence and the second depth image set sequence. Then, according to the current three-dimensional space information, the device motion simulation of the target machine device for the device adjustment information is performed to obtain a motion simulation scene. Finally, according to the motion collision situation in the motion simulation scene, the device adjustment information is adjusted to obtain the adjusted device adjustment information as the device adjustment information.

[0098] Sub-step 3, continue to execute the above control steps according to the device adjustment information.

[0099] In a third step, in response to determining that the second instruction execution result indicates that the execution behavior corresponding to the target machine device does not have abnormality, the target machine device is controlled to continue to execute the instruction corresponding to the first instruction execution result.

[0100] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the machine device control method of some embodiments of the present disclosure can accurately and efficiently implement the abnormal behavior processing of the target machine device. Specifically, the reason why the relevant device adjustment for abnormal behavior is not efficient is that the device adjustment needs to be performed in a human-involved manner, resulting in the problem of low adjustment efficiency. Based on this, the machine device control method of some embodiments of the present disclosure, first, in response to determining that the first instruction execution result corresponding to the target machine device does not reach the preset execution result, the laser radar device and the depth camera device configured by the above-mentioned target machine device are used to obtain the first point cloud set sequence and the first depth image set sequence corresponding to the above-mentioned target machine device. Here, the first point cloud set sequence generated by the laser radar device and the first depth image set sequence generated by the depth camera device facilitate the subsequent clear understanding of the movement of the target machine device within a predetermined time period and the surrounding environment corresponding to the target machine device, so as to facilitate the subsequent execution of abnormal behavior verification for the target machine device. Then, according to the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence, the execution behavior of the above-mentioned target machine device can be accurately verified to generate behavior verification information. Here, by performing a behavior verification on the target machine device, it is determined whether the target machine device is abnormally executed. Next, in response to determining that the above-mentioned behavior verification information represents that the execution behavior of the above-mentioned target machine device is abnormal, the movement trajectory set corresponding to the above-mentioned target machine device can be accurately generated according to the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence. Here, through the above-mentioned first point cloud set sequence and the above-mentioned first depth image set sequence, not only the position information and device posture information corresponding to the target machine device can be grasped, but also the environmental conditions of the surrounding environment corresponding to the target machine device can be understood, thereby accurately generating a movement trajectory set. Then, the movement trajectory that meets the preset trajectory conditions is selected from the above-mentioned movement trajectory set as the target movement trajectory, so as to serve as the subsequent target machine device to be moved. Secondly, the device adjustment information corresponding to the above-mentioned target movement trajectory is generated to facilitate the target machine device to move the device. Furthermore, the device action instruction for the above-mentioned device adjustment information is generated to facilitate the control of the target machine device to execute the corresponding instruction content. Finally, according to the above-mentioned device action instruction, the above-mentioned target machine device is controlled to move along the above-mentioned target movement trajectory to get rid of the dilemma of the target machine device being in abnormal behavior. In summary, based on the first point cloud set sequence and the first depth image set sequence, a movement trajectory for the target machine device can be accurately generated to achieve device processing of the target machine device under abnormal execution behavior.

[0101] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a machine device control device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the machine device control device can be specifically applied to various electronic devices.

[0102] like Figure 2 As shown, a machine device control device 200 includes: an acquisition unit 201, a verification unit 202, a first generation unit 203, a screening unit 204, a second generation unit 205, a third generation unit 206 and a control unit 207. The acquisition unit 201 is configured to, in response to determining that the execution result of the first instruction corresponding to the target machine device does not reach the preset execution result, obtain the first point cloud set sequence and the first depth image set sequence corresponding to the target machine device using the laser radar device and the depth camera device configured by the target machine device; the verification unit 202 is configured to verify the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information; the first generation unit 203 is configured to, in response to determining that the behavior verification information characterizes the execution behavior of the target machine device The execution behavior is abnormal. According to the first point cloud set sequence and the first depth image set sequence, a moving trajectory set corresponding to the target machine device is generated; the screening unit 204 is configured to screen out a moving trajectory that meets the preset trajectory condition from the moving trajectory set as the target moving trajectory; the second generating unit 205 is configured to generate device adjustment information corresponding to the target moving trajectory; the third generating unit 206 is configured to generate a device action instruction for the device adjustment information; the control unit 207 is configured to control the target machine device to move along the target moving trajectory according to the device action instruction.

[0103] It is understood that the units described in the machine control device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the machine device control device 200 and the units included therein, and will not be described in detail here.

[0104] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0105] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0106] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0107] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0108] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0109] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0110] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to determining that the execution result of the first instruction corresponding to the target machine device does not reach the preset execution result, the laser radar device and the depth camera device configured by the target machine device are used to obtain the first point cloud set sequence and the first depth image set sequence corresponding to the target machine device; according to the first point cloud set sequence and the first depth image set sequence, the execution behavior of the target machine device is verified to generate behavior verification information; in response to determining that the behavior verification information indicates that the execution behavior of the target machine device is abnormal, according to the first point cloud set sequence and the first depth image set sequence, a movement trajectory set corresponding to the target machine device is generated; from the movement trajectory set, a movement trajectory that meets the preset trajectory condition is selected as the target movement trajectory; the device adjustment information corresponding to the target movement trajectory is generated; the device action instruction for the device adjustment information is generated; according to the device action instruction, the target machine device is controlled to move along the target movement trajectory.

[0111] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a verification unit, a first generation unit, a screening unit, a second generation unit, a third generation unit and a control unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the second generation unit may also be described as a "unit for generating device adjustment information corresponding to the above-mentioned target movement trajectory".

[0114] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0115] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for controlling a machine device, comprising: In response to determining that the execution result of the first instruction corresponding to the target machine device does not reach the preset execution result, using the laser radar device and the depth camera device configured by the target machine device to obtain a first point cloud set sequence and a first depth image set sequence corresponding to the target machine device; Verifying the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information; In response to determining that the behavior verification information indicates that the execution behavior of the target machine device is abnormal, generating a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence, wherein the target machine device is configured with a plurality of movable device telescopic brackets, and the plurality of movable device telescopic brackets support the target machine device to perform at least one of the following actions: a reversing action, an erecting action, and a moving action; and The step of generating a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence includes: Generate movable space information for the target machine device according to the first point cloud set sequence and the first depth image set sequence; Determine whether there is storage space information in a pre-stored space information database whose space information similarity with the movable space information is greater than a target value; In response to determining the existence, determining at least one target storage space information; Determine an execution action information sequence corresponding to each target storage space information in the at least one target storage space information to obtain at least one execution action information sequence; According to the movable space information, performing execution simulation on each execution action information sequence in the at least one execution action information sequence to generate simulation information, thereby obtaining at least one simulation information; Filtering out simulation information that meets a preset condition from the at least one simulation information as target simulation information, and obtaining at least one target simulation information; Determining at least one movement trajectory corresponding to the at least one target simulation information as a movement trajectory set; In response to determining that there is no such information, determining at least one candidate movement trajectory corresponding to the movable space information by using a search-based path planning algorithm or a sampling-based path planning algorithm; Determine a movement action sequence corresponding to each candidate movement trajectory information in the at least one candidate movement trajectory and relative to the target machine device to obtain at least one movement action sequence; Filter out the mobile action sequences whose actions are in the at least one action from the at least one mobile action sequence to obtain a mobile action sequence group; In response to determining that the number of movement action sequences included in the movement action sequence group is greater than the target value, a target movement action sequence whose corresponding number of movement actions meets a preset number condition is selected from the movement action sequence group to obtain a target movement action sequence group; In response to determining that the number of target movement action sequences included in the target movement action sequence group is greater than the target value, determining the movement estimated time and movement estimated difficulty corresponding to each target movement action sequence in the target movement action sequence group; Filter out target movement action sequences whose estimated movement time meets a preset time condition and whose estimated movement difficulty meets a preset difficulty condition from the target movement action sequence group as actual movement action sequences to obtain an actual movement action sequence group; Determine a candidate movement trajectory group corresponding to the actual movement action sequence group as a movement trajectory set; Selecting a moving trajectory that meets a preset trajectory condition from the moving trajectory set as a target moving trajectory; Generating device adjustment information corresponding to the target movement trajectory, wherein the generating device adjustment information corresponding to the target movement trajectory includes: Determine a movement action sequence corresponding to the target movement trajectory as the movement action sequence to be executed; Determine the bracket information corresponding to each to-be-performed movement action in the to-be-performed movement action sequence, wherein the bracket information includes: bracket movement sub-information corresponding to each device telescopic bracket of a plurality of movable device telescopic brackets; determining the obtained bracket information sequence as the device adjustment information; generating a device action instruction for the device adjustment information; According to the device action instruction, the target machine device is controlled to move along the target movement trajectory.

2. The method according to claim 1, wherein: The method further comprises: generating a second instruction execution result corresponding to the device action instruction; According to the device adjustment information, perform the following control steps: In response to determining that the execution result of the second instruction indicates that the execution behavior corresponding to the target machine device is abnormal, obtaining a second point cloud set sequence and a second depth image set sequence corresponding to the target machine device, wherein the second point cloud set sequence and the second depth image set sequence are obtained based on the process in which the target machine device moves along the target moving trajectory; According to the second point cloud set sequence and the second depth image set sequence, the device adjustment information is adjusted to obtain adjusted device adjustment information as the device adjustment information; Continue to execute the control step according to the device adjustment information; In response to determining that the second instruction execution result indicates that the corresponding execution behavior of the target machine device is not abnormal, the target machine device is controlled to continue to execute the instruction corresponding to the first instruction execution result.

3. The method according to claim 1, wherein: The verifying the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information includes: Generate a three-dimensional space information sequence of a predetermined space for the target machine device according to the first point cloud set sequence and the first depth image set sequence; Determining a device posture information sequence corresponding to the target machine device according to the three-dimensional space information sequence; The behavior verification information is generated according to the device posture information sequence.

4. The method according to claim 3, wherein: The step of generating a three-dimensional space information sequence for a predetermined space of the target machine device according to the first point cloud set sequence and the first depth image set sequence comprises: Determine the time information corresponding to each first point cloud set in the first point cloud set sequence to obtain a time information sequence; For each time information in the time information sequence, the following generation steps are performed: Determine a first point cloud set in the first point cloud set sequence and corresponding to the time information as a first target point cloud set, and determine a first depth image set in the first depth image set sequence and corresponding to the time information as a first target depth image set; Generating space information corresponding to a predetermined space according to a device position corresponding to the target machine device; Filtering out a first target point cloud subset corresponding to the spatial information from the first target point cloud set, and filtering out a first target depth image subset corresponding to the spatial information from the first target depth image set; generating a distance information set for the first target depth image subset; The three-dimensional space information corresponding to the time information is generated according to the first target point cloud subset and the distance information set.

5. A machine device control device, comprising: an acquisition unit, configured to, in response to determining that the execution result of the first instruction corresponding to the target machine device does not reach a preset execution result, acquire a first point cloud set sequence and a first depth image set sequence corresponding to the target machine device by using a laser radar device and a depth camera device configured by the target machine device; a verification unit, configured to verify the execution behavior of the target machine device according to the first point cloud set sequence and the first depth image set sequence to generate behavior verification information; The first generating unit is configured to generate a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence in response to determining that the behavior verification information represents an abnormal execution behavior of the target machine device, wherein the target machine device is configured with a plurality of movable device telescopic brackets, and the plurality of movable device telescopic brackets support the target machine device to perform at least one of the following actions: a reversing action, an erecting action, and a moving action; and the generating the movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence includes: generating a movement trajectory set corresponding to the target machine device according to the first point cloud set sequence and the first depth image set sequence The method comprises the steps of: determining whether there is storage space information in a pre-stored space information database whose spatial information similarity with the movable space information is greater than a target value; determining at least one target storage space information in response to the determination of existence; determining an execution action information sequence corresponding to each target storage space information in the at least one target storage space information, and obtaining at least one execution action information sequence; performing execution simulation on each execution action information sequence in the at least one execution action information sequence according to the movable space information to generate simulation information, and obtaining at least one simulation information; filtering out simulation information that meets a preset condition from the at least one simulation information as the target simulation information, and obtaining to at least one target simulation information; determining at least one movement trajectory corresponding to the at least one target simulation information as a movement trajectory set; in response to determining that there is no such movement trajectory, using a search-based path planning algorithm or a sampling-based path planning algorithm, determining at least one candidate movement trajectory corresponding to the movable space information; determining a movement action sequence corresponding to each candidate movement trajectory information in the at least one candidate movement trajectory relative to the target machine device, to obtain at least one movement action sequence; filtering out movement action sequences whose actions are in the at least one action from the at least one movement action sequence, to obtain a movement action sequence group; in response to determining that the number of movement action sequences included in the movement action sequence group is greater than a target value, selecting a target mobile action sequence whose corresponding number of mobile actions meets a preset number condition from the mobile action sequence group to obtain a target mobile action sequence group; in response to determining that the number of target mobile action sequences included in the target mobile action sequence group is greater than the target value, determining a mobile estimated time and a mobile estimated difficulty corresponding to each target mobile action sequence in the target mobile action sequence group; selecting a target mobile action sequence whose mobile estimated time meets a preset time condition and whose mobile estimated difficulty meets a preset difficulty condition from the target mobile action sequence group as an actual mobile action sequence to obtain an actual mobile action sequence group; and determining a candidate mobile trajectory group corresponding to the actual mobile action sequence group as a mobile trajectory set; a screening unit, configured to screen out a movement trajectory that meets a preset trajectory condition from the movement trajectory set as a target movement trajectory; The second generating unit is configured to generate device adjustment information corresponding to the target movement trajectory, wherein the generating device adjustment information corresponding to the target movement trajectory includes: determining a movement action sequence corresponding to the target movement trajectory as a movement action sequence to be executed; determining support information corresponding to each movement action to be executed in the movement action sequence to be executed, wherein the support information includes: support movement sub-information corresponding to each device telescopic support of a plurality of movable device telescopic supports; and determining the obtained support information sequence as the device adjustment information; a third generating unit, configured to generate a device action instruction for the device adjustment information; The control unit is configured to control the target machine device to move along the target movement trajectory according to the device action instruction.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images