Method, device and system for determining moving track of operation machine and storage medium

Abstract

The invention relates to the technical field of operation machinery, in particular to an operation machinery moving track determination method, device and system and a storage medium. The method comprises the following steps: acquiring state parameters of an operation machine in an operation scene, wherein the state parameters comprise an operation device parameter, an obstacle parameter and a target point parameter; based on the operation device parameters and the obstacle parameters, the obstacle state is determined, and the obstacle state is used for indicating the space-time distance between the obstacle and the operation device; determining a target point state based on the operation device parameters and the target point parameters, the target point state being used for indicating a space-time distance between the target point and the operation device; and based on the obstacle state and the target point state, determining a moving track from the working device to the target point. And through space-time distance positioning, the state determination accuracy is effectively improved, and the movement track is better determined.

Description

Technical Field

[0001] The present invention relates to the technical field of working machinery, and in particular to a method, device, system and storage medium for determining a moving trajectory of a working machinery. Background Art

[0002] As a key technology, reinforcement learning has important practical significance and broad market prospects when applied to automatic obstacle avoidance of tower cranes and other operating equipment.

[0003] Taking tower cranes as an example, one of the current difficulties of reinforcement learning lies in the distance measurement standard. Since tower cranes have different motion rules and speeds in rotation and amplitude change, the relevant distance determination method is often not suitable for the distance measurement standard in the current scenario, resulting in low tower crane recognition accuracy, which is not conducive to ensuring the safety of operation and affects the overall operation efficiency. Summary of the invention

[0004] The present application provides a method, device, system and storage medium for determining the moving trajectory of an operating machine, so as to provide a more efficient and accurate positioning method for the operating machine and effectively improve the positioning perception performance.

[0005] In a first aspect, an embodiment of the present application provides a method for determining a moving trajectory of a working machine, the method comprising:

[0006] Acquire state parameters of the operating machinery in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, and target point parameters that the operating device needs to reach in the operating scenario; determine the obstacle state based on the operating device parameters and the obstacle parameters, wherein the obstacle state is used to indicate the space-time distance between the obstacle and the operating device, and the space-time distance between the obstacle and the operating device is used to indicate the shortest time required for the obstacle to reach the operating device; determine the target point state based on the operating device parameters and the target point parameters, wherein the target point state is used to indicate the space-time distance between the target point and the operating device, and the space-time distance between the target point and the operating device is used to indicate the shortest time required for the operating device to reach the target point; determine the movement trajectory of the operating device to the target point in the operating scenario based on the obstacle state and the target point state.

[0007] The above method, the embodiment of the present application introduces the way of determining the state by time and space distance, so as to better locate the state and thus more accurately determine the movement trajectory, solve the problem of inaccurate determination of the movement trajectory of the current working machinery, and effectively improve the accuracy of the identification and positioning of the working machinery and the determination of the movement trajectory.

[0008] In some optional embodiments, the space-time distance includes a variable amplitude space-time distance in a variable amplitude dimension, and a rotation space-time distance in a rotation dimension. The rotation space-time distance in the rotation dimension is determined based on the rotation dimension distance and the rotation dimension speed, and the rotation space-time distance is used to indicate the shortest time required to reach the target rotation position; the variable amplitude space-time distance in the variable amplitude dimension is determined based on the variable amplitude dimension distance and the variable amplitude dimension speed, and the variable amplitude space-time distance is used to indicate the shortest time required to reach the target variable amplitude position.

[0009] The above method, the embodiment of the present application, introduces the way of confirming the state of objects by using the time-space distance, and fully considers the influence of the rotation dimension and the amplitude dimension of the working machinery in the working scene on the state and speed of objects in the working scene, thereby making the calculated object state and the moving trajectory planned based on the object state more accurate, more adaptable and more convenient.

[0010] In some optional implementations, determining the obstacle state based on the working device parameter and the obstacle parameter includes:

[0011] According to the polar coordinates of at least one key point of the bounding box corresponding to the obstacle in the obstacle parameters, the rotational space-time distance between each key point in the at least one key point and the working device is determined to obtain at least one rotational space-time distance between the obstacle and the working device; the variable amplitude space-time distance between the polar coordinates of each key point in the at least one key point and the working device is determined to obtain at least one variable amplitude space-time distance between the obstacle and the working device; based on the at least one rotational space-time distance and the at least one variable amplitude space-time distance, the obstacle state is determined.

[0012] Exemplarily, in the description of the reinforcement learning task, when describing the information of an obstacle, the obstacle can be regarded as a cube, and the absolute position of the obstacle can be represented by the polar coordinates of the key points of the bounding box corresponding to the obstacle. For example, in order to facilitate the description and calculation, the two-dimensional view of the obstacle in the polar coordinate system is introduced as an example, assuming that the two-dimensional view of the obstacle is a rectangle, wherein, in order to better describe the positional relationship between the obstacle and the working device, it is necessary to calculate the space-time distance between the four vertices of the obstacle and the working device respectively, wherein the space-time distance calculation between each vertex and the working device will obtain two values ​​of the rotation space-time distance and the amplitude-varying space-time distance. Therefore, when the space-time distance calculation is performed based on the four vertices and the working device, eight values ​​will be obtained, including four rotation space-time distances and four amplitude-varying space-time distances, which are used to describe the information of this obstacle.

[0013] It should be noted that the process of calculating the space-time distance based on the polar coordinates of the four vertices of the rectangle corresponding to the obstacle in the embodiment of the present application is only an example and does not constitute a limitation on the embodiment of the present application. For example, in order to calculate the obstacle state more finely, the embodiment of the present application can select multiple vertices (for example, more than four vertices or less than four vertices) for calculation in combination with the actual shape of the obstacle.

[0014] In some optional implementations, determining the variable amplitude temporal and spatial distance between the polar coordinates of each vertex and the working device includes:

[0015] Determine the amplitude change position of each key point according to the polar coordinates of each key point in the at least one key point; determine the first amplitude change difference between each key point and the working device according to the amplitude change position of each key point and the amplitude change position of the working device in the working device parameters; determine the amplitude change time-space distance between each key point and the working device according to the maximum amplitude change speed of the working device in the working device parameters and the first amplitude change difference between each key point and the working device, and obtain at least one amplitude change time-space distance.

[0016] In some optional implementations, determining the rotational space-time distance between the polar coordinates of each vertex and the working device includes:

[0017] According to the polar coordinates of each key point in the at least one key point, the arc difference between each key point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters, and the arc difference between each key point and the working device, the rotational space-time distance between each key point and the working device is determined to obtain at least one rotational space-time distance.

[0018] Illustratively, when determining the radian difference from the vertex to the working device in the rotational dimension, the embodiment of the present application can confirm the radian difference more carefully based on the movement direction of the working device. For example, when the working device moves in a clockwise direction, the radian difference in the rotational dimension can be directly determined by the difference between the polar coordinates of the vertex and the polar coordinates of the working device. For another example, when the working device moves in a counterclockwise direction, the radian difference in the rotational dimension can be determined by subtracting the difference in the polar coordinates from 2π.

[0019] In some optional implementations, determining the target point state based on the operating device parameters and the target point parameters includes:

[0020] According to the rotation position of the target point in the target point parameters, the arc difference between the target point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters and the arc difference between the target point and the working device, the rotation time and space distance between the target point and the working device is determined; according to the amplitude change position of the target point in the target point parameters and the amplitude change position of the working device in the working device parameters, the second amplitude change difference between the working device and the target point is determined;

[0021] According to the maximum amplitude change speed of the working device in the working device parameters and the second amplitude change difference between the target point and the working device, the amplitude change time-space distance between the target point and the working device is determined; based on the rotation time-space distance between the target point and the working device and the amplitude change time-space distance between the target point and the working device, the state of the target point is determined.

[0022] In a second aspect, an embodiment of the present application provides a method for determining a moving trajectory of a working machine, the method comprising:

[0023] The state parameters of the working machine in the working scene are obtained, and the state parameters include working device parameters, obstacle parameters, working machine parameters and target point parameters that the working device needs to reach in the working scene; the activity range area corresponding to the working machine is cut into a plurality of fan-shaped radial grid areas according to the working machine parameters, and the activity range area is obtained based on the range covered by the circular polar coordinate system established with the supporting body of the working machine as the origin and the boom of the working machine as the positive semi-axis; based on the working device parameters and the obstacle parameters, a first space-time distance vector between the obstacle and the working device in the grid area is determined, and the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; based on the working device parameters and the target point parameters, a second space-time distance vector between the target point and the working device in the grid area is determined, and the second space-time distance vector is used to represent the shortest time required for the working device to reach the target point; based on the first space-time distance vector and the second space-time distance vector, a moving trajectory of the working device to the target point in the working scene is determined.

[0024] The above method, an embodiment of the present application provides a method of dividing the activity range of the working machinery into a plurality of sector-shaped radial grid areas, and performing state positioning based on the grid areas, which effectively improves the accuracy of state recognition and is more helpful for determining the movement trajectory.

[0025] In some optional implementations, the step of cutting the activity range area corresponding to the working machine into a plurality of sector-shaped radial grid areas according to the working machine parameters includes:

[0026] According to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters, the number of divisions of the working machine based on the amplitude change direction in the activity range area is determined; according to the amplitude change speed of the working machine, the number of divisions of the working machine based on the rotation direction in the activity range area is determined; according to the number of divisions in the amplitude change direction and the number of divisions in the rotation direction, the activity range area is cut into a plurality of sector-shaped radial grid areas.

[0027] In some optional implementations, determining the number of divisions of the amplitude change direction of the operating machine according to the rotation speed and the amplitude change speed of the operating machine includes:

[0028] The maximum amplitude variation difference of the operating machine is determined according to the maximum amplitude variation of the operating machine and the minimum amplitude variation of the operating machine; the number of divisions of the amplitude variation direction of the operating machine is determined based on the maximum amplitude variation difference of the operating machine and the maximum rotation speed of the operating machine.

[0029] In some optional implementations, determining the number of cuts for the rotation direction of the working machine includes:

[0030] The number of divisions of the rotation direction of the working machine is determined based on the ratio of 2π to the maximum rotation speed of the working machine.

[0031] In some optional implementations, determining a first spatiotemporal distance vector between the obstacle and the working device in the grid area based on the working device parameter and the obstacle parameter includes:

[0032] The first position of the working device in the grid area is determined according to the working device parameters; the second position of the obstacle in the grid area is determined according to the obstacle parameters; the first amplitude variation space-time distance between the obstacle and the working device in the amplitude variation dimension is determined according to the number of sections in the amplitude variation direction between the first position and the second position in the grid area; the first rotation space-time distance between the obstacle and the working device in the rotation dimension is determined according to the number of sections in the rotation direction between the first position and the second position in the grid area; the first space-time distance vector is obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

[0033] In some optional implementations, determining a second spatiotemporal distance vector between the target point and the operating device in the grid area based on the operating device parameters and the target point parameters includes:

[0034] The first position of the working device within the grid area is determined according to the working device parameters; the third position of the target point within the grid area is determined according to the target point parameters; the second amplitude variation space-time distance between the target point and the working device in the amplitude variation dimension is determined according to the number of portions in the amplitude variation direction between the first position and the third position within the grid area; the second rotation space-time distance between the target point and the working device in the rotation dimension is determined according to the number of portions in the rotation direction between the first position and the third position within the grid area; the second space-time distance vector is obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

[0035] In a third aspect, an embodiment of the present application provides a device for determining a moving trajectory of a working machine, the device comprising:

[0036] An acquisition module, used to acquire state parameters of the operating machine in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, and target point parameters that the operating device needs to reach in the operating scenario;

[0037] A processing module is used to determine the obstacle state based on the working device parameters and the obstacle parameters, the obstacle state is used to indicate the space-time distance between the obstacle and the working device, and the space-time distance between the obstacle and the working device is used to indicate the shortest time required for the obstacle to reach the working device; determine the target point state based on the working device parameters and the target point parameters, the target point state is used to indicate the space-time distance between the target point and the working device, and the space-time distance between the target point and the working device is used to indicate the shortest time required for the working device to reach the target point; determine the movement trajectory of the working device to the target point in the working scene based on the obstacle state and the target point state.

[0038] In some optional implementation manners, the processing module is specifically used for:

[0039] According to the polar coordinates of at least one key point of the bounding box corresponding to the obstacle in the obstacle parameters, the rotational space-time distance between each key point in the at least one key point and the working device is determined to obtain at least one rotational space-time distance between the obstacle and the working device; the variable amplitude space-time distance between the polar coordinates of each key point in the at least one key point and the working device is determined to obtain at least one variable amplitude space-time distance between the obstacle and the working device; based on the at least one rotational space-time distance and the at least one variable amplitude space-time distance, the obstacle state is determined.

[0040] In some optional implementation manners, the processing module is specifically used for:

[0041] Determine the amplitude change position of each key point according to the polar coordinates of each key point in the at least one key point; determine the first amplitude change difference between each key point and the working device according to the amplitude change position of each key point and the amplitude change position of the working device in the working device parameters; determine the amplitude change time-space distance between each key point and the working device according to the maximum amplitude change speed of the working device in the working device parameters and the first amplitude change difference between each key point and the working device, and obtain at least one amplitude change time-space distance.

[0042] In some optional implementation manners, the processing module is specifically used for:

[0043] According to the polar coordinates of each key point in the at least one key point, the arc difference between each key point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters, and the arc difference between each key point and the working device, the rotational space-time distance between each key point and the working device is determined to obtain at least one rotational space-time distance.

[0044] In some optional implementation manners, the processing module is specifically used for:

[0045] According to the rotation position of the target point in the target point parameters, the arc difference between the target point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters and the arc difference between the target point and the working device, the rotation time and space distance between the target point and the working device is determined; according to the amplitude change position of the target point in the target point parameters and the amplitude change position of the working device in the working device parameters, the second amplitude change difference between the working device and the target point is determined;

[0046] According to the maximum amplitude change speed of the working device in the working device parameters and the second amplitude change difference between the target point and the working device, the amplitude change time-space distance between the target point and the working device is determined; based on the rotation time-space distance between the target point and the working device and the amplitude change time-space distance between the target point and the working device, the state of the target point is determined.

[0047] In a fourth aspect, an embodiment of the present application provides a device for determining a moving trajectory of a working machine, the device comprising:

[0048] An acquisition module is used to acquire state parameters of the operating machine in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, operating machine parameters, and target point parameters that the operating device needs to reach in the operating scenario;

[0049] A division module, used for cutting the activity range area corresponding to the working machine into a plurality of fan-shaped radial grid areas according to the working machine parameters, wherein the activity range area is obtained by the range covered by the circular polar coordinate system established based on the support body of the working machine as the origin and the boom of the working machine as the positive semi-axis;

[0050] A processing module is used to determine a first space-time distance vector between the obstacle and the working device in the grid area based on the working device parameters and the obstacle parameters, wherein the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; based on the working device parameters and the target point parameters, determine a second space-time distance vector between the target point and the working device in the grid area, wherein the second space-time distance vector is used to represent the shortest time required for the working device to reach the target point; based on the first space-time distance vector and the second space-time distance vector, determine a moving trajectory from the working device to the target point in the working scene.

[0051] In some optional implementation manners, the division module is specifically used to:

[0052] According to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters, the number of divisions of the working machine based on the amplitude change direction in the activity range area is determined; according to the amplitude change speed of the working machine, the number of divisions of the working machine based on the rotation direction in the activity range area is determined; according to the number of divisions in the amplitude change direction and the number of divisions in the rotation direction, the activity range area is cut into a plurality of sector-shaped radial grid areas.

[0053] In some optional implementation manners, the division module is specifically used to:

[0054] The maximum amplitude variation difference of the operating machine is determined according to the maximum amplitude variation of the operating machine and the minimum amplitude variation of the operating machine; the number of divisions of the amplitude variation direction of the operating machine is determined based on the maximum amplitude variation difference of the operating machine and the maximum rotation speed of the operating machine.

[0055] In some optional implementation manners, the division module is specifically used to:

[0056] The number of divisions of the rotation direction of the working machine is determined based on the ratio of 2π to the maximum rotation speed of the working machine.

[0057] In some optional implementation manners, the processing module is specifically used for:

[0058] The first position of the working device in the grid area is determined according to the working device parameters; the second position of the obstacle in the grid area is determined according to the obstacle parameters; the first amplitude variation space-time distance between the obstacle and the working device in the amplitude variation dimension is determined according to the number of sections in the amplitude variation direction between the first position and the second position in the grid area; the first rotation space-time distance between the obstacle and the working device in the rotation dimension is determined according to the number of sections in the rotation direction between the first position and the second position in the grid area; the first space-time distance vector is obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

[0059] In some optional implementation manners, the processing module is specifically used for:

[0060] The first position of the working device within the grid area is determined according to the working device parameters; the third position of the target point within the grid area is determined according to the target point parameters; the second amplitude variation space-time distance between the target point and the working device in the amplitude variation dimension is determined according to the number of portions in the amplitude variation direction between the first position and the third position within the grid area; the second rotation space-time distance between the target point and the working device in the rotation dimension is determined according to the number of portions in the rotation direction between the first position and the third position within the grid area; the second space-time distance vector is obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

[0061] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes the method for determining the moving trajectory of an operating machine as described in any one of the first aspect; or the processor executes the method for determining the moving trajectory of an operating machine as described in any one of the second aspect.

[0062] In a sixth aspect, an embodiment of the present application provides a chip system, comprising: a processor and an interface, the processor being used to call and execute a computer program from the interface, when the processor executes the computer program, it can implement the method described in the above-mentioned first aspect or any possible design of the first aspect; or it can implement the method described in the above-mentioned second aspect or any possible design of the second aspect.

[0063] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which has a computer program for executing the method described in the above-mentioned first aspect or any possible design of the first aspect; or a computer program for executing the method described in the above-mentioned second aspect or any possible design of the second aspect.

[0064] In an eighth aspect, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed, it can implement the method described in the above-mentioned first aspect or any possible design of the first aspect; or it can implement the method described in the above-mentioned second aspect or any possible design of the second aspect.

[0065] The beneficial effects produced by any one of the third to eighth aspects are the same as those of the first aspect. For details, please refer to the detailed introduction of the first and second aspects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of determining the Euclidean distance provided for related technologies;

[0067] Figure 2 A schematic diagram of a method for determining a moving trajectory of a working machine provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of a method for determining an obstacle state provided in an embodiment of the present application;

[0069] Figure 4 A schematic diagram of a method for determining a target point state provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of an operating scene of an operating machine in a top view provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of another method for determining a moving trajectory of a working machine provided in an embodiment of the present application;

[0072] Figure 7A schematic diagram of a method for dividing a grid area provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of an intelligent operation machine provided in an embodiment of the present application;

[0074] Fig. 9 A schematic diagram of a static obstacle bounding box provided in an embodiment of the present application;

[0075] Fig.10 A schematic diagram of a static obstacle in a top view provided in an embodiment of the present application;

[0076] Fig.11 A schematic diagram of a top view of a dynamic obstacle provided in an embodiment of the present application;

[0077] Fig.12 A schematic diagram of the operation of a dynamic obstacle provided in an embodiment of the present application;

[0078] Fig.13 A schematic diagram of a radiation grid divided into time and space distances provided in an embodiment of the present application;

[0079] Fig.14 A schematic diagram of an action network provided in an embodiment of the present application;

[0080] Fig.15 A value network diagram provided for an embodiment of the present application;

[0081] Fig.16 A schematic diagram of a device for determining a moving trajectory of a working machine provided in an embodiment of the present application;

[0082] Fig.17 A schematic diagram of another device for determining the moving trajectory of a working machine provided in an embodiment of the present application;

[0083] Fig.18 A schematic diagram of another device for determining the moving trajectory of a working machine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0085] As a key technology, reinforcement learning enables computer systems to automatically learn how to make optimal decisions through interaction with the environment, showing great potential in the field of intelligent systems and automation. In the face of safety risks and efficiency issues in traditional tower crane operations, applying reinforcement learning technology to automatic obstacle avoidance for tower cranes has important practical significance and broad market prospects.

[0086] One of the difficulties of reinforcement learning at present lies in the distance measurement standard. Since tower cranes have different motion rules and speeds in rotation and amplitude change, the relevant distance determination method is often not suitable for the distance measurement standard in the current scenario, resulting in low tower crane recognition accuracy.

[0087] For example, see Figure 1 As shown, a polar coordinate system is established with the tower as the origin and the boom as the positive half axis of the x-axis. The polar coordinates of the target point and the four vertices of the obstacle in this polar coordinate system are obtained at a time interval of 1 second. Since the obstacle and target point information obtained every second are the positions in the polar coordinate system established with the boom as the positive half axis of the x-axis, this position is the position coordinate relative to the boom. Figure 1 Point A is located at the outer circle of the x-axis tower crane's range of motion, and point B is a different point of the same amplitude change distance of A. AB is translated in the opposite direction of the x-axis to position A′B′. Under relevant technologies, such as the measurement scale of Euclidean distance, the movement from A to B and from A′ to B′ are in the same direction and distance. However, for the tower crane, A′ to B′ will require a larger rotation arc and at the same time increase the outward amplitude change operation.

[0088] In summary, the scheme of applying reinforcement learning technology to determine the moving trajectory of tower crane has problems such as low accuracy, which is not conducive to ensuring the safety of operation and affects the overall operation efficiency.

[0089] Based on this, the present application provides a method, device and storage medium for determining the mobile trajectory of an operating machine, which introduces a method of determining the state by time and space distance, so as to better locate the state and more accurately determine the mobile trajectory, solve the problem of inaccurate determination of the mobile trajectory of the current operating machine, and effectively improve the accuracy of the identification and positioning of the operating machine and the determination of the mobile trajectory. In addition, the embodiment of the present application also provides a method of dividing the activity range of the operating machine into a plurality of fan-shaped radial grid areas, and performing state positioning based on the grid area, which effectively improves the accuracy of state recognition and is more conducive to determining the mobile trajectory.

[0090] The following will be combined with the accompanying drawings and specific embodiments to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0091] Among them, the method for determining the moving trajectory of the working machinery described in the embodiment of the present application can be applied to working machinery such as tower cranes, truck cranes, crawler cranes, aerial work machinery, pump trucks, etc. The working machinery may include a working device, and the working device can be used to move the working object in the working scene where the working machinery is located, for example, moving the working object to the target point through the working device in the working machinery.

[0092] For example, when the operating machine is a tower crane or a crane, the operating device included in the operating machine may be a hook, and the operating object operated by the operating device may be a hanging object. The operating device in the embodiment of the present application may also be provided in combination with the operating object, that is, the operating device and the operating object are collectively referred to as the operating device after being combined. For example, the operating device may be a combination of a hook and a hanging object; or, the operating device in the embodiment of the present application may also be the same as the operating object, for example, the operating device is the hanging object, etc., which is not limited here.

[0093] For the convenience of description, the embodiment of the present application selects the case where the working machine is a tower crane, the working device is a hook, and the working object is a hanging object for the following introduction.

[0094] The present application embodiment provides a method for determining a moving trajectory of a working machine, such as Figure 2 As shown, the method may include:

[0095] Step S201: Obtaining state parameters of the operating machine in an operating scenario.

[0096] As an example, the state parameters in the embodiment of the present application include operating device parameters, obstacle parameters, and target point parameters that the operating device needs to reach in the operating scenario.

[0097] Step S202: Based on the working device parameters and the obstacle parameters, determine the obstacle state, where the obstacle state is used to indicate the temporal and spatial distance between the obstacle and the working device.

[0098] As an example, the space-time distance between the obstacle and the working device in the embodiment of the present application is used to represent the shortest time required for the obstacle to reach the working device. The space-time distance between the obstacle and the working device may include the variable space-time distance between the obstacle and the working device in the variable dimension, and the rotation space-time distance in the rotation dimension.

[0099] Exemplarily, the rotation space-time distance in the rotation dimension can be determined based on the rotation dimension distance and the rotation dimension speed, and the rotation space-time distance is used to indicate the time to reach the target rotation position; the amplitude variation space-time distance in the amplitude variation dimension can be determined based on the amplitude variation dimension distance and the amplitude variation dimension speed, and the amplitude variation space-time distance is used to indicate the time to reach the target amplitude variation position.

[0100] Furthermore, the embodiments of the present application can be implemented as follows: Figure 3 The process shown in FIG. 1 determines the obstacle state based on the working device parameters and the obstacle parameters:

[0101] Step S202-1: Determine the rotational time-space distance between the obstacle and the operating device:

[0102] The embodiment of the present application can determine the rotational space-time distance between each key point in the at least one key point and the operating device based on the polar coordinates of at least one key point of the bounding box corresponding to the obstacle in the obstacle parameters, and obtain at least one rotational space-time distance between the obstacle and the operating device.

[0103] Specifically, when determining the rotational space-time distance between each key point and the working device, the embodiment of the present application can first determine the arc difference between each key point and the working device in the rotation dimension based on the polar coordinates of each key point, and then determine the rotational space-time distance between each key point and the working device based on the maximum rotation speed of the working device in the working device parameters and the arc difference between each key point and the working device.

[0104] Illustratively, when determining the radian difference between a key point and the working device in the rotational dimension, the embodiment of the present application can confirm the radian difference more carefully based on the movement direction of the working device. For example, when the working device moves in a clockwise direction, the radian difference in the rotational dimension can be directly determined by the difference between the polar coordinates of the key point and the polar coordinates of the working device. For another example, when the working device moves in a counterclockwise direction, the radian difference in the rotational dimension can be determined by subtracting the difference between the polar coordinates from 2π.

[0105] Step S202-2: Determine the luffing temporal and spatial distance between the obstacle and the operating device:

[0106] The embodiment of the present application can determine the variable amplitude time-space distance between each key point and the working device, and obtain at least one variable amplitude time-space distance between the obstacle and the working device.

[0107] Specifically, when determining the amplitude variation space-time distance between each key point and the working device, the embodiment of the present application can first determine the amplitude variation position of each key point according to the polar coordinates of each key point, and then determine the first amplitude variation difference between each key point and the working device according to the amplitude variation position of each key point and the amplitude variation position of the working device in the working device parameters. Finally, determine the amplitude variation space-time distance between each key point and the working device according to the maximum amplitude variation speed of the working device in the working device parameters and the first amplitude variation difference between each key point and the working device.

[0108] Step S202-3: Determine the obstacle state based on the at least one rotation space-time distance and the at least one amplitude variation space-time distance.

[0109] Exemplarily, in the description of the reinforcement learning task, when describing the information of an obstacle, for example, for the convenience of description and calculation, the two-dimensional view of the obstacle in the polar coordinate system is used as an example for introduction. Assuming that the two-dimensional view of the obstacle is a rectangle, the absolute position of the obstacle can be represented by the polar coordinates of the four vertices of the rectangle corresponding to the obstacle (i.e., the key points of the bounding box selected for the obstacle). In order to describe the positional relationship between the obstacle and the working device, it is necessary to calculate the space-time distance between the four vertices of the obstacle and the working device respectively, wherein the space-time distance calculation between each vertex and the working device will obtain two values ​​of the rotation space-time distance and the amplitude-varying space-time distance. Therefore, when the space-time distance calculation is performed based on the four vertices and the working device, eight values ​​will be obtained, including four rotation space-time distances and four amplitude-varying space-time distances, which are used to describe the information of this obstacle.

[0110] It should be noted that the process of calculating the space-time distance based on the polar coordinates of the four vertices of the rectangle corresponding to the obstacle in the embodiment of the present application is only an example and does not constitute a limitation on the embodiment of the present application. For example, in order to calculate the obstacle state more finely, the embodiment of the present application can select multiple key points (for example, greater than four vertices or less than four vertices) for calculation in combination with the actual shape of the obstacle.

[0111] Step S203: Based on the operating device parameters and the target point parameters, determine the target point status, where the target point status is used to indicate the temporal and spatial distance between the target point and the operating device.

[0112] As an example, the space-time distance between the target point and the operating device in the embodiment of the present application is used to represent the shortest time required for the operating device to reach the target point. The space-time distance between the target point and the operating device may include the variable amplitude space-time distance between the target point and the operating device in the variable amplitude dimension, and the rotation space-time distance in the rotation dimension.

[0113] Furthermore, the embodiments of the present application can be implemented as follows: Figure 4 The process shown in FIG. 1 determines the target point state based on the working device parameters and the target point parameters:

[0114] Step S203-1: Determine the rotational time-space distance between the target point and the operating device:

[0115] Specifically, when determining the rotational space-time distance between the target point and the working device, the embodiment of the present application can first determine the arc difference between the target point and the working device in the rotation dimension based on the rotation position of the target point in the target point parameters, and then determine the rotational space-time distance between the target point and the working device based on the maximum rotation speed of the working device in the working device parameters and the arc difference between the target point and the working device.

[0116] Step S203-2: Determine the luffing time-space distance between the target point and the operating device:

[0117] Specifically, when determining the amplitude variation space-time distance between the target point and the working device, the embodiment of the present application can first determine the second amplitude variation difference between the working device and the target point based on the amplitude variation position of the target point in the target point parameters and the amplitude variation position of the working device in the working device parameters, and then determine the amplitude variation space-time distance between the target point and the working device based on the maximum amplitude variation speed of the working device in the working device parameters and the second amplitude variation difference between the target point and the working device.

[0118] Step S203-3: Determine the state of the target point based on the rotation space-time distance between the target point and the working device and the amplitude change space-time distance between the target point and the working device.

[0119] Step S204: Based on the obstacle state and the target point state, determine the movement trajectory of the working device to the target point in the working scene.

[0120] It should be noted that the embodiments of the present application do not limit the order of determining the rotation time-space distance and the amplitude variation time-space distance. For example, the amplitude variation time distance can be determined first, and then the rotation time distance. The amplitude variation time distance and the rotation time distance can also be determined at the same time, or the rotation time distance can be determined first, and then the amplitude variation time distance, etc., which is not limited here.

[0121] For example, in order to better introduce the content of the time and space distance in the embodiment of the present application, the present application takes a tower crane as an example, and selects Figure 5 The tower crane operation scene diagram in the top view is briefly introduced as an example. Figure 5 A represents the location of the hook. Assume that the polar coordinates of A are (θ A ,l A ), the maximum rotation speed and amplitude change speed are vmax slew 、vmax luff ; B is the mission target point or the vertex position of the obstacle, and the polar coordinates are (θ B ,l B ). Among them, the time-space distance is mainly measured by calculating the shortest time required for a moving point to reach another point to measure the position information of two points. Since the circular motion in the polar coordinate system is periodic, the time-space distance between A and B in the rotation dimension of the embodiment of the present application needs to consider the movement direction and distance of the hook A at the same time. When A moves in a clockwise direction, the radian difference from A to B in the rotation dimension is θ B -θ A , and its space-time distance in the rotation dimension is (θ B -θ A ) / vmax slew , when A moves counterclockwise, the space-time distance from A to B in the rotation dimension is (2π-(θ B -θ A )) / vmax slew , the space-time distance from A to B in the variable amplitude dimension is (OB-OA) / vmax luff .

[0122] The present application also provides a method for determining a moving trajectory of a working machine. Figure 6 As shown, the method may include:

[0123] Step S601: Obtain the state parameters of the operating machine in the operating scenario.

[0124] As an example, the state parameters described in the embodiment of the present application include operating device parameters in the operating scenario, obstacle parameters, operating machinery parameters, and target point parameters that the operating device needs to reach.

[0125] Step S602: cutting the activity range area corresponding to the working machine into a plurality of sector-shaped radial grid areas according to the working machine parameters.

[0126] As an example, the activity range area described in the embodiment of the present application is obtained by the range covered by the circular polar coordinate system established based on the support body of the working machine as the origin and the arm of the working machine as the positive semi-axis.

[0127] For example, when the working machine is a tower crane, the supporting body may be a tower body.

[0128] As an example, the embodiments of the present application can be implemented as follows: Figure 7 The process shown in FIG. 1 is to cut the activity range area corresponding to the working machine into a plurality of sector-shaped radial grid areas according to the working machine parameters:

[0129] Step S602-1: Determine the number of divisions of the working machine based on the amplitude change direction in the activity range area according to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters.

[0130] As an example, an embodiment of the present application can determine the difference in the maximum amplitude variation of the operating machinery based on the maximum amplitude variation of the operating machinery and the minimum amplitude variation of the operating machinery, and then determine the number of divisions of the amplitude variation direction of the operating machinery based on the difference in the maximum amplitude variation of the operating machinery and the maximum rotation speed of the operating machinery.

[0131] Step S602-2: Determine the number of divisions of the working machine based on the rotation direction in the activity range area according to the amplitude change speed of the working machine.

[0132] As an example, the embodiment of the present application may determine the number of divisions of the rotation direction of the working machine based on the ratio of 2π to the maximum rotation speed of the working machine.

[0133] Step S602-3: according to the number of divisions in the amplitude variation direction and the number of divisions in the rotation direction, the activity range area is divided into a plurality of sector-shaped radial grid areas.

[0134] Step S603: Based on the working device parameters and the obstacle parameters, determine a first spatiotemporal distance vector between the obstacle and the working device in the grid area.

[0135] The first space-time distance is used to represent the shortest time required for the obstacle to reach the operating device.

[0136] As an example, the first variable-amplitude space-time distance included in the first space-time distance vector is determined in the following manner:

[0137] In the embodiment of the present application, the first position of the working device within the grid area can be determined according to the working device parameters, and the second position of the obstacle within the grid area can be determined according to the obstacle parameters, and then the first amplitude variation space-time distance between the obstacle and the working device in the amplitude variation dimension can be determined according to the number of divisions of the amplitude variation direction between the first position and the second position within the grid area.

[0138] As an example, the first rotation space-time distance included in the first space-time distance vector is determined in the following manner:

[0139] In this embodiment of the application, the first rotational space-time distance between the obstacle and the working device in the rotational dimension can be determined according to the number of divisions of the rotational direction between the first position and the second position in the grid area, and then the first space-time distance vector can be obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

[0140] Step S604: Based on the working device parameters and the target point parameters, determine a second space-time distance vector between the target point and the working device in the grid area.

[0141] The second space-time distance vector is used to represent the shortest time required for the operating device to reach the target point.

[0142] As an example, the second variable-amplitude space-time distance included in the second space-time distance vector is determined in the following manner:

[0143] In the embodiment of the present application, the first position of the working device within the grid area can be determined according to the working device parameters, and the third position of the target point within the grid area can be determined according to the target point parameters, and then the second amplitude variation space-time distance between the target point and the working device in the amplitude variation dimension can be determined according to the number of divisions of the amplitude variation direction between the first position and the third position within the grid area.

[0144] As an example, the second rotation space-time distance included in the second space-time distance vector is determined in the following manner:

[0145] In this embodiment of the application, the second rotational space-time distance between the target point and the working device in the rotation dimension can be determined according to the number of divisions of the rotational direction between the first position and the third position in the grid area, and then the second space-time distance vector can be obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

[0146] Step S605: Based on the first space-time distance vector and the second space-time distance vector, determine the movement trajectory of the working device from the working scene to the target point.

[0147] It should be noted that the embodiments of the present application do not limit the order of determining the number of cuts in the rotation direction and the number of cuts in the amplitude changing direction. For example, the number of cuts in the rotation direction may be determined first, and then the number of cuts in the amplitude changing direction may be determined. The number of cuts in the amplitude changing direction and the number of cuts in the rotation direction may also be determined at the same time. Alternatively, the number of cuts in the amplitude changing direction may be determined first, and then the number of cuts in the rotation direction may be determined, etc. This is not limited here.

[0148] It should be noted that the above-mentioned process steps are only an example of an embodiment of the present application and do not constitute a limitation on the embodiment of the present application. The embodiment of the present application can adjust the sequence of the entire process steps according to actual conditions, add or delete process content, or modify certain process content.

[0149] In order to better introduce the embodiments of the present application, taking the case where the working machine is a tower crane and the working device is a hook as an example, the entire process of determining the moving trajectory of the working machine executed based on the embodiments of the present application is introduced in detail based on different stages.

[0150] Stage 1: Determine the environment model for reinforcement learning involved in the method for determining the movement trajectory of an intelligent tower crane through reinforcement learning provided in an embodiment of the present application.

[0151] As an example, the embodiment of the present application can determine the environmental model of the tower crane reinforcement learning obstacle avoidance task based on the situation of the task goal. For example, the tower crane reinforcement learning obstacle avoidance task can include two tasks, wherein the main task can be to carry an object from the initial position to the target point by means of a hook during the tower crane operation, and the secondary task can be to avoid obstacles during the movement of the object during the tower crane operation. Therefore, the environmental model provided by the embodiment of the present application can include two parts, namely, the environmental composition and the movement of the operating machinery, based on the two tasks listed above.

[0152] For example, the environment components described in the embodiments of the present application may include but are not limited to the following:

[0153] Environment composition 1: Tower crane as a reinforcement learning agent (hereinafter referred to as smart tower crane).

[0154] As an example, the intelligent tower crane to which the method for determining the moving trajectory of an operating machine according to an embodiment of the present application is applied may include a boom, a tower body, a suspension rope and a hook. The operating device of the intelligent tower crane may be a suspension hook, a suspension object or a combination of the two.

[0155] For better understanding, the embodiment of the present application may consider the hanging object to be a mass point, and ignore the swing of the hanging rope caused by inertia and wind during the operation. Figure 8 As shown, a crane provided in an embodiment of the present application can be regarded as consisting of a tower body represented by a line segment OZ, a boom represented by ZX, and a wire rope represented by NY. Point Y represents the location of the hanging object, point N represents the trolley for controlling the boom position, point M represents the minimum boom position, and X is the end of the tower crane boom, which is also the maximum boom position.

[0156] Environmental component 2: obstacles in the work scene.

[0157] As an example, the implementation of this application may adopt an OBB bounding box, for example, a directional bounding box, to describe obstacles in the scene, thereby better reducing errors. It should be noted that the method of using an OBB bounding box in the embodiment of this application to describe obstacles in the scene is only an example of the embodiment of this application, and does not constitute a limitation on the embodiment of this application. For example, the embodiment of this application may also adopt a cylindrical bounding box or an axis-aligned bounding box.

[0158] Among them, the obstacles in the tower crane operation scene described in the embodiment of the present application may include static obstacles (for example, fixed buildings in the operation scene, such as buildings, etc.) and dynamic obstacles that can change their position or shape (for example, other tower cranes in operation in the operation scene, etc.).

[0159] For example, Fig. 9 As shown, in the embodiment of the present application, a directional OBB bounding box can be used to describe static obstacles in the work scene. For example, the static obstacle is regarded as an OBB cube ABCD-EFGH perpendicular to the ground. The rectangle of the cube ABCD-EFGH in the top view is not necessarily parallel to the coordinate axis. For example, the rectangle of the cube ABCD-EFGH in the top view can be as follows: Fig.10 shown.

[0160] For example, Fig.11 As shown, in the embodiment of the present application, the dynamic obstacles of other tower cranes can be completely enclosed by OBB bounding boxes, such as the boom, tower body, hanging objects and wire ropes of other tower cranes, to form a cube ABCD-EFGH, wherein the movement of the tower crane as a dynamic obstacle can be as follows Fig.12 As shown, it is a cubic winding segment O o Z o Rotate.

[0161] Environmental component 3: Unloading area in the work scene.

[0162] As an example, the embodiment of the present application may understand the unloading area as a target point.

[0163] For example, in the embodiment of the present application, the tower crane operation target point can be regarded as a spherical area with a radius of 1 meter, and the task is considered to be completed when the particle where the suspended object is located reaches the sphere.

[0164] Phase 2: Establishing a simulation environment.

[0165] A simulation environment is established through the tower crane model, obstacle model and target point model in the above-mentioned stage one, and the real scene is simulated based on the simulation environment, so as to provide a large amount of interaction data between the intelligent tower crane and the environment for reinforcement learning of the intelligent tower crane. Among them, all simulation environment parameters set in the embodiment of the present application based on this stage two are in line with the actual operation scenario.

[0166] Exemplarily, the interaction data involved in the embodiments of the present application may include but is not limited to the following:

[0167] Tower crane model:

[0168] The tower crane parameters in the tower crane model may include but are not limited to the following:

[0169] The height of the tower crane (for example, the maximum lifting height of the tower crane), the arm length (for example, the maximum luffing distance of the tower crane), the minimum luffing distance, the maximum speed of the slewing and luffing, the maximum acceleration of the slewing and luffing, etc.

[0170] Obstacle model:

[0171] The static obstacle parameters in the obstacle model may include but are not limited to the following:

[0172] The number of static obstacles, center point coordinates, length, width, height, angle between the long side and the positive semi-axis of the x-axis, etc.

[0173] For example, the static obstacle in the embodiment of the present application is lower than the maximum height of the tower crane in the simulation environment. In order to better simulate, the static obstacle refresh position can be set outside the position of the tower crane hoist.

[0174] The dynamic obstacle parameters in the obstacle model may include but are not limited to the following:

[0175] The number of dynamic obstacles, the coordinates of the tower, the length, width and height of the OBB rectangle surrounding it, the angle between the initial boom position and the positive semi-axis of the x-axis, and the rotation speed.

[0176] In order to better simulate, it can be set here that all dynamic obstacles in the operation scene do not overlap with other obstacles when they are refreshed.

[0177] Target point model:

[0178] The target point in the embodiment of the present application is to simulate the actual working scene, and the refresh position can be the upper surface of the static obstacle or the ground, and is within the tower crane operating environment.

[0179] Phase three: performing reinforcement learning on the intelligent tower crane.

[0180] In this stage three, since the operating environment of the tower crane includes three dimensions: lifting, boom length adjustment, and rotation, and the movement rules of the hoisted objects of the intelligent tower crane are not the same in the three dimensions, directly using the three-dimensional state space will increase the complexity of the problem. Therefore, an optional method of the embodiment of the present application is to make decisions in different ways based on different dimensional scenarios.

[0181] Based on the decision scenario one on the amplitude variation and rotation dimensions in the top view, the embodiment of the present application can use reinforcement learning to complete the decision of this scenario one. Based on the movement decision scenario two of the lifting dimension, the embodiment of the present application can use preset rules to complete the decision of this scenario two. The following is an introduction based on the different scenarios in stage three.

[0182] Scenario 1: Use reinforcement learning to make decisions based on the amplitude and rotation dimensions in a bird's-eye view.

[0183] As an example, in the embodiment of the present application, the reinforcement learning process can use a reinforcement learning algorithm to perform reinforcement learning on the intelligent tower crane based on state space, action space, reward function, etc.

[0184] Among them, the main goals of the embodiment of the present application for reinforcement learning of the above-mentioned environmental model include making the suspended object reach the target point from the initial position and avoiding obstacles during the movement of the suspended object.

[0185] Content 1: State space.

[0186] The state space information of the embodiment of the present application can reflect key information of the environment in which the tower crane is used for operation.

[0187] In order to better describe the movement of the tower crane and better measure the distance in the unified rotation and amplitude dimensions, the embodiment of the present application uses the space-time distance as an indicator to measure the position state of the intelligent tower crane to the obstacle in the state space construction.

[0188] Exemplarily, assuming that the operating task of the tower crane includes the main task of reaching the target point and the secondary task of avoiding obstacles, the state space information in the embodiment of the present application may include the position information of the hoisted object to the obstacle, the position information of the hoisted object to the target point, and the information of the hoisted object itself.

[0189] Specifically, when performing state space construction in the embodiment of the present application, the circular ring-shaped activity range area of ​​the tower crane can be cut into fan-shaped grid areas according to the maximum rotation speed and maximum amplitude change speed of the tower crane.

[0190] For example, assuming that the maximum amplitude of the tower crane is max luff , the minimum amplitude is min luff , the maximum amplitude change speed is vmax luff , the maximum rotation speed is vmax slew , then the amplitude change direction is divided into (max luff -min luff ) / vmax luff The rotation direction is divided into 2π / vmax luff After segmentation, it is represented as a radial grid. Substituting the tower crane parameters studied in this scheme, the area range after segmentation is as follows Fig.13 shown.

[0191] Under this setting, set the obstacle state obs is the time-space distance vector between the obstacle and the intelligent tower crane. The rotation luffing position C of a vertex of the obstacle can be determined by referring to the following formula 1:

[0192] c=(c slew ,c luff ) Formula 1

[0193] Obstacle space-time distance vector vec c It can be determined by referring to the following formula 2:

[0194] vec c =(c slew / vmax slew ,(c luff -h luff ) / vmax luff ) Formula 2

[0195] The four vertices of each obstacle can form a vector of length 8 in the counterclockwise direction, and n obstacles can form an array of size n×8.

[0196] Target point state goal is the time-space distance vector between the target point and the intelligent tower crane. The slewing position g of the target point can be determined by referring to the following formula 3:

[0197] g=(g slew ,g luff ) Formula 3

[0198] Obstacle space-time distance vector vec g It can be determined by referring to the following formula 4:

[0199] vec g =(g slew / vmax slew ,(g luff -h luff ) / vmax luff ) Formula 4

[0200] Among them, the state of the hanging object hook h is the variable amplitude position of the load luff .

[0201] It should be noted that when calculating the time-space distance between a certain point (such as the vertex of an obstacle or a target point) and the hook, the rotation position of the hook can be subtracted based on the rotation position of the point, and then the corresponding rotation time-space distance can be obtained based on the rotation difference divided by the maximum rotation speed of the hook. In order to facilitate calculation and better understanding, the embodiment of the present application lists the calculation of the rotation time-space distance by directly dividing the rotation position of the point by the maximum rotation speed of the hook as an example, in which the boom of the tower crane can be used as a reference and the x-axis in this example, and other objects such as obstacles and target points are obtained based on this premise, so the rotation position of the hook is always equal to 0.

[0202] In addition, in order to facilitate calculation and understanding, the embodiments of the present application can set objects with speed attributes to include hooks or hanging objects, while the attributes of objects such as target points and obstacles are only coordinates that describe their positions, and the space-time distance is calculated using this coordinate and the coordinates of the hook. The information contained in the space-time distance is used as the state required for reinforcement learning.

[0203] Content 2: Action Space

[0204] The tower crane's obstacle avoidance task using reinforcement learning is a problem in a continuous action space, which contains two dimensions: rotation action and amplitude variation action.

[0205] As an example, the embodiment of the present application can set the action interval range of the reinforcement learning output to (-1, 1), a rotation action less than 0 indicates clockwise rotation, and greater than 0 indicates counterclockwise rotation; an amplitude variation action less than 0 indicates inward amplitude variation, and greater than 0 indicates outward amplitude variation.

[0206] In addition, since the tower crane has acceleration restrictions in actual operation scenarios, the expected speed of the tower crane at the current time t is Output speed of reinforcement learning at time t And the reinforcement learning output speed at the previous moment And the maximum acceleration of the tower crane rotation and amplitude change acc = [acc slew ,acc luff ] obtained by calculation.

[0207] Under the acceleration limit, the speed of reinforcement learning output act t The interval is If the output speed is less than the left boundary of the interval or greater than the right boundary, the values ​​of the left and right boundaries are taken.

[0208] As an example, based on the above content, the calculation of the expected speed in the embodiment of the present application can refer to the following formula 5:

[0209]

[0210] Content 3: Reward Function

[0211] The setting of the reward function determines whether the reinforcement learning model can complete the task and converge.

[0212] As an example, since the task includes a main task of reaching the target point and a sub-task of avoiding collision, the reward function reward in the embodiment of the present application can be set to four parts as shown in the following formula 6:

[0213]

[0214] The first part r0 can be the reward for the intelligent tower crane to reach the target point, at which point the simulation ends.

[0215] The second part r1 can be the penalty for the collision between the intelligent tower crane and the obstacle, at which point the simulation ends.

[0216] The third part r2 can be the reward during the path-finding process, which represents the projection size of the action vector on the space-time distance vector between the hanging object and the target point. The calculation of this reward function takes into account people's operating habits. Its theoretical maximum value is also the fastest way to reach the target point from the current point.

[0217] The fourth part r3 can be a time penalty. A small penalty will be added to each action to make the smart tower crane actively complete the task.

[0218] Furthermore, when the reward function described in construction 3 is applied to perform reinforcement learning training, it can be divided into multiple training sessions.

[0219] As an example, the embodiment of the present application can first perform reinforcement learning training based on the reward function to obtain model one, and then perform a second reinforcement learning training based on model one as a pre-training model of model two.

[0220] Exemplarily, the first reinforcement learning training uses reward functions r0, r1, and r2, wherein the use of r2 enables the model to converge quickly and obtain a reinforcement learning model 1 that can complete tasks in most scenarios; during the second training, r0, r1, and r3 are used to train the model 1 again.

[0221] Among them, when the model is trained for the second time, r2 is cancelled, so that the newly trained model is not restricted by human operating experience and avoids the model from falling into the local optimal solution. In addition, r3 is added. Since there is a fixed penalty in each step, the model can complete the task as quickly as possible.

[0222] By training the two models in succession, we can effectively avoid the problem of directly using the reward function of the second model to train from scratch. Due to the random motion characteristics of the intelligent tower crane during simulation, the probability of completing the task is very low, and the model cannot converge due to lack of sufficient samples to reach the target point.

[0223] Furthermore, in this stage three, after the deployment of the above-mentioned state space, action space, reward function and other contents, the embodiment of the present application can further perform reinforcement learning of the intelligent tower crane based on the reinforcement learning algorithm.

[0224] Training the reinforcement learning obstacle avoidance strategy requires a large amount of interaction data between the intelligent tower crane and the environment. Based on this, the embodiment of the present application can use the SAC (Soft Actor-Critic) algorithm for reinforcement learning, wherein the SAC algorithm is a model-free offline learning algorithm suitable for tasks such as tower cranes that have continuous action space characteristics and complex operating environments.

[0225] As an example, the SAC algorithm provided in the embodiment of the present application may be composed of an action network (ActorNetwork) and multiple value networks (Q-Network) and an experience replay pool R storing historical training data, wherein the multiple value networks may be value networks obtained based on action-state. For example, the multiple value networks in the embodiment of the present application may include two different initial value networks and two different target value networks, wherein the target value network may be obtained by copying the initial value network.

[0226] Furthermore, due to the common reinforcement learning tasks currently, the network is composed of several fully connected layers, but the fully connected layer needs to accept state inputs of consistent shapes, and the uncertain number of obstacles in the tower crane operation scene will lead to uncertain input state size, and when the number of obstacles varies greatly, too much redundant information will be generated or key information will be lost, increasing the complexity of the problem. Based on this, in order to better realize reinforcement learning training, the embodiment of the present application also provides content for processing obstacle information based on the attention mechanism, so as to better solve the training inconvenience caused by the unequal number of obstacles, wherein the action network and value network provided by this part of the network structure are different except for the input part and the output part, and the intermediate structure is the same.

[0227] For example, Fig.14 As shown, the input of the action network provided in the embodiment of the present application may include three parts: obstacle state, target point state and intelligent tower crane state, such as Fig.15 As shown, the value network provided by the embodiment of the present application includes not only obstacle status, target point status and intelligent tower crane status, but also action as a status.

[0228] Among them, in the middle part of the action network and the value network, the obstacle state extracts features through the attention mechanism to obtain a feature matrix. For example, the feature matrix can be a matrix of 8×8 size, which is not limited here. Then, the embodiment of the present application obtains a feature of length A through a fully connected layer. For the action network, this feature will be connected with the target point state and the intelligent tower crane state to form a feature of length B. For the value network, the action will also be connected to obtain a feature of length C. The feature is connected to two fully connected layers with an output length of D. Among them, the values ​​of A, B, C and D in the embodiment of the present application can be generated or set according to actual conditions. For example, the value of A can be 16, the value of B can be 19, the value of C can be 21, and the value of D can be 256, etc., which are not limited here. After that, in the action network, the mean and standard deviation of the action will be output, and the action will be obtained by random sampling based on the mean standard deviation; the Q value (the value evaluation value of the action) will be output in the value network.

[0229] The embodiment of the present application adopts the above-mentioned action network and value network to provide a solution for processing obstacle information based on the attention mechanism. Based on the attention feature, it can realize the role of paying attention to the relationship between each obstacle, obtain the corresponding result by dynamic weighting, and change the feature of the result from an unfixed length to a fixed length by full connection. This feature also integrates the features of each obstacle information, so that the feature of the obtained result not only learns the relationship and importance of each obstacle, but also fixes the length, which facilitates subsequent training. For example, since each obstacle has the same number of feature values, the attention mechanism provided by the embodiment of the present application can obtain the comprehensive features of each obstacle through the weight matrix of the size set in the attention mechanism, and then through the full connection operation, the obstacle features of the constant number are integrated into the features of fixed length, so as to better solve the training inconvenience caused by the unequal number of obstacles.

[0230] Scenario 2: Use preset rules to complete motion decisions based on the lifting dimension.

[0231] If the starting position is above the target point, the obstacle information of the current height section is obtained first, the tower crane hoisting position remains unchanged, and the reinforcement learning model is used to find the best obstacle avoidance path to the target point. After reaching above the target point, the crane is lifted down to the target point.

[0232] If the starting position is below the target point, it will first lift up to the height of the target point, and then obtain the obstacle information of the current height section. The tower crane will keep the lifting position unchanged, and use the reinforcement learning model to find the best armband path to reach the target point.

[0233] The intelligent working machinery reinforcement learning obstacle avoidance method provided in the embodiment of the present application can enable the working machinery to more accurately and quickly model the construction scene, effectively improve the calculation accuracy, and enhance adaptability.

[0234] like Fig.16 As shown, based on the same inventive concept, the embodiment of the present application provides a working machine movement trajectory determination device 1600, comprising:

[0235] An acquisition module 1610 is used to acquire state parameters of the working machine in the working scene, wherein the state parameters include working device parameters, obstacle parameters and target point parameters that the working device needs to reach in the working scene;

[0236] Processing module 1620 is used to determine the obstacle state based on the working device parameters and the obstacle parameters, the obstacle state is used to indicate the space-time distance between the obstacle and the working device, and the space-time distance between the obstacle and the working device is used to indicate the shortest time required for the obstacle to reach the working device; determine the target point state based on the working device parameters and the target point parameters, the target point state is used to indicate the space-time distance between the target point and the working device, and the space-time distance between the target point and the working device is used to indicate the shortest time required for the working device to reach the target point; determine the movement trajectory of the working device to the target point in the working scene based on the obstacle state and the target point state.

[0237] In some optional implementation manners, the processing module 1620 is specifically used for:

[0238] According to the polar coordinates of at least one key point of the bounding box corresponding to the obstacle in the obstacle parameters, the rotational space-time distance between each key point in the at least one key point and the working device is determined to obtain at least one rotational space-time distance between the obstacle and the working device; the variable amplitude space-time distance between the polar coordinates of each key point in the at least one key point and the working device is determined to obtain at least one variable amplitude space-time distance between the obstacle and the working device; based on the at least one rotational space-time distance and the at least one variable amplitude space-time distance, the obstacle state is determined.

[0239] In some optional implementation manners, the processing module 1620 is specifically used for:

[0240] Determine the amplitude change position of each key point according to the polar coordinates of each key point in the at least one key point; determine the first amplitude change difference between each key point and the working device according to the amplitude change position of each key point and the amplitude change position of the working device in the working device parameters; determine the amplitude change time-space distance between each key point and the working device according to the maximum amplitude change speed of the working device in the working device parameters and the first amplitude change difference between each key point and the working device, and obtain at least one amplitude change time-space distance.

[0241] In some optional implementation manners, the processing module 1620 is specifically used for:

[0242] According to the polar coordinates of each key point in the at least one key point, the arc difference between each key point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters, and the arc difference between each key point and the working device, the rotational space-time distance between each key point and the working device is determined to obtain at least one rotational space-time distance.

[0243] In some optional implementation manners, the processing module 1620 is specifically used for:

[0244] According to the rotation position of the target point in the target point parameters, the arc difference between the target point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters and the arc difference between the target point and the working device, the rotation time and space distance between the target point and the working device is determined; according to the amplitude change position of the target point in the target point parameters and the amplitude change position of the working device in the working device parameters, the second amplitude change difference between the working device and the target point is determined;

[0245] According to the maximum amplitude change speed of the working device in the working device parameters and the second amplitude change difference between the target point and the working device, the amplitude change time-space distance between the target point and the working device is determined; based on the rotation time-space distance between the target point and the working device and the amplitude change time-space distance between the target point and the working device, the state of the target point is determined.

[0246] like Fig.17 As shown, based on the same inventive concept, the embodiment of the present application provides another device 1700 for determining the moving trajectory of a working machine, comprising:

[0247] An acquisition module 1710 is used to acquire state parameters of the working machine in the working scene, wherein the state parameters include working device parameters, obstacle parameters, working machine parameters, and target point parameters that the working device needs to reach in the working scene;

[0248] A division module 1720 is used to divide the activity range area corresponding to the working machine into a plurality of fan-shaped radial grid areas according to the working machine parameters, wherein the activity range area is obtained by the range covered by the circular polar coordinate system established based on the support body of the working machine as the origin and the boom of the working machine as the positive semi-axis;

[0249] Processing module 1730 is used to determine a first space-time distance vector between the obstacle and the working device in the grid area based on the working device parameters and the obstacle parameters, wherein the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; based on the working device parameters and the target point parameters, determine a second space-time distance vector between the target point and the working device in the grid area, wherein the second space-time distance vector is used to represent the shortest time required for the working device to reach the target point; based on the first space-time distance vector and the second space-time distance vector, determine a moving trajectory from the working device to the target point in the working scene.

[0250] In some optional implementation manners, the division module 1720 is specifically used to:

[0251] According to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters, the number of divisions of the working machine based on the amplitude change direction in the activity range area is determined; according to the amplitude change speed of the working machine, the number of divisions of the working machine based on the rotation direction in the activity range area is determined; according to the number of divisions in the amplitude change direction and the number of divisions in the rotation direction, the activity range area is cut into a plurality of sector-shaped radial grid areas.

[0252] In some optional implementation manners, the division module 1720 is specifically used to:

[0253] The maximum amplitude variation difference of the operating machine is determined according to the maximum amplitude variation of the operating machine and the minimum amplitude variation of the operating machine; the number of divisions of the amplitude variation direction of the operating machine is determined based on the maximum amplitude variation difference of the operating machine and the maximum rotation speed of the operating machine.

[0254] In some optional implementation manners, the division module 1720 is specifically used to:

[0255] The number of divisions of the rotation direction of the working machine is determined based on the ratio of 2π to the maximum rotation speed of the working machine.

[0256] In some optional implementation manners, the processing module 1730 is specifically used for:

[0257] The first position of the working device in the grid area is determined according to the working device parameters; the second position of the obstacle in the grid area is determined according to the obstacle parameters; the first amplitude variation space-time distance between the obstacle and the working device in the amplitude variation dimension is determined according to the number of sections in the amplitude variation direction between the first position and the second position in the grid area; the first rotation space-time distance between the obstacle and the working device in the rotation dimension is determined according to the number of sections in the rotation direction between the first position and the second position in the grid area; the first space-time distance vector is obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

[0258] In some optional implementation manners, the processing module 1730 is specifically used for:

[0259] The first position of the working device within the grid area is determined according to the working device parameters; the third position of the target point within the grid area is determined according to the target point parameters; the second amplitude variation space-time distance between the target point and the working device in the amplitude variation dimension is determined according to the number of portions in the amplitude variation direction between the first position and the third position within the grid area; the second rotation space-time distance between the target point and the working device in the rotation dimension is determined according to the number of portions in the rotation direction between the first position and the third position within the grid area; the second space-time distance vector is obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

[0260] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0261] like Fig.18 As shown, based on the same inventive concept, an embodiment of the present application provides an electronic device 1800, including: a processor 1801 and a memory 1802;

[0262] The memory 1802 may be a volatile memory, such as a random-access memory (RAM); the memory 1802 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1802 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1802 may be a combination of the above memories. The processor 1801 may include one or more central processing units (CPUs), graphics processing units (GPUs) or digital processing units, etc.

[0263] The specific connection medium between the memory 1802 and the processor 1801 is not limited in the embodiment of the present application. Fig.18 In the embodiment, the memory 1802 and the processor 1801 are connected via a bus 1803. The bus 1803 is connected to the processor 1801 via a bus 1803. Fig.18 In the figure, the bus 1803 is shown by a bold line, and can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.18 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0264] Optionally, the memory 1802 stores a program code, and when the program code is executed by the processor 1801, the processor 1801 performs the following process:

[0265] Acquire state parameters of the operating machinery in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, and target point parameters that the operating device needs to reach in the operating scenario; determine the obstacle state based on the operating device parameters and the obstacle parameters, wherein the obstacle state is used to indicate the space-time distance between the obstacle and the operating device, and the space-time distance between the obstacle and the operating device is used to indicate the shortest time required for the obstacle to reach the operating device; determine the target point state based on the operating device parameters and the target point parameters, wherein the target point state is used to indicate the space-time distance between the target point and the operating device, and the space-time distance between the target point and the operating device is used to indicate the shortest time required for the operating device to reach the target point; determine the movement trajectory of the operating device to the target point in the operating scenario based on the obstacle state and the target point state.

[0266] In some optional implementation manners, the processor 1801 is specifically configured to:

[0267] According to the polar coordinates of the key points of the bounding box corresponding to the obstacle in the obstacle parameters, the polar coordinates of each vertex and the rotation space-time distance of the working device are determined to obtain four rotation space-time distances from the obstacle to the working device; the polar coordinates of each vertex and the amplitude-changing space-time distance of the working device are determined to obtain four amplitude-changing space-time distances from the obstacle to the working device; based on the four rotation space-time distances and the four amplitude-changing space-time distances, the obstacle state is determined.

[0268] In some optional implementation manners, the processor 1801 is specifically configured to:

[0269] Determine the amplitude variation position of each vertex according to the polar coordinates of each vertex; determine the first amplitude variation difference between the working device and the corresponding vertex according to the amplitude variation position of each vertex and the amplitude variation position of the working device in the working device parameters; determine the amplitude variation time-space distance from each vertex to the working device according to the maximum amplitude variation speed of the working device in the working device parameters and the first amplitude variation difference between each vertex and the working device.

[0270] In some optional implementation manners, the processor 1801 is specifically configured to:

[0271] The arc difference between each vertex and the working device in the rotation dimension is determined according to the polar coordinates of each vertex; the rotational space-time distance from each vertex to the working device is determined according to the maximum rotation speed of the working device in the working device parameters and the arc difference between each vertex and the working device.

[0272] In some optional implementation manners, the processor 1801 is specifically configured to:

[0273] According to the rotation position of the target point in the target point parameters, the arc difference between the target point and the working device in the rotation dimension is determined; according to the maximum rotation speed of the working device in the working device parameters, and the arc difference between the target point and the working device, the rotation time-space distance from the target point to the working device is determined; according to the amplitude variation position of the target point in the target point parameters, and the amplitude variation position of the working device in the working device parameters, the second amplitude variation difference between the working device and the target point is determined; according to the maximum amplitude variation speed of the working device in the working device parameters, and the second amplitude variation difference between the target point and the working device, the amplitude variation time-space distance from the target point to the working device is determined; based on the rotation time-space distance between the target point and the working device and the amplitude variation time-space distance between the target point and the working device, the state of the target point is determined.

[0274] Optionally, the memory 1802 stores a program code, and when the program code is executed by the processor 1801, the processor 1801 performs the following process:

[0275] The state parameters of the working machine in the working scene are obtained, and the state parameters include working device parameters, obstacle parameters, working machine parameters and target point parameters that the working device needs to reach in the working scene; the activity range area corresponding to the working machine is cut into a plurality of fan-shaped radial grid areas according to the working machine parameters, and the activity range area is obtained based on the range covered by the circular polar coordinate system established with the supporting body of the working machine as the origin and the boom of the working machine as the positive semi-axis; based on the working device parameters and the obstacle parameters, a first space-time distance vector between the obstacle and the working device in the grid area is determined, and the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; based on the working device parameters and the target point parameters, a second space-time distance vector between the target point and the working device in the grid area is determined, and the second space-time distance vector is used to represent the shortest time required for the working device to reach the target point; based on the first space-time distance vector and the second space-time distance vector, a moving trajectory of the working device to the target point in the working scene is determined.

[0276] In some optional implementation manners, the processor 1801 is specifically configured to:

[0277] According to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters, the number of divisions of the working machine based on the amplitude change direction in the activity range area is determined; according to the amplitude change speed of the working machine, the number of divisions of the working machine based on the rotation direction in the activity range area is determined; according to the number of divisions in the amplitude change direction and the number of divisions in the rotation direction, the activity range area is cut into a plurality of sector-shaped radial grid areas.

[0278] In some optional implementation manners, the processor 1801 is specifically configured to:

[0279] The maximum amplitude variation difference of the operating machine is determined according to the maximum amplitude variation of the operating machine and the minimum amplitude variation of the operating machine; the number of divisions of the amplitude variation direction of the operating machine is determined based on the maximum amplitude variation difference of the operating machine and the maximum rotation speed of the operating machine.

[0280] In some optional implementation manners, the processor 1801 is specifically configured to:

[0281] The number of divisions of the rotation direction of the working machine is determined based on the ratio of 2π to the maximum rotation speed of the working machine.

[0282] In some optional implementation manners, the processor 1801 is specifically configured to:

[0283] The first position of the working device in the grid area is determined according to the working device parameters; the second position of the obstacle in the grid area is determined according to the obstacle parameters; the first amplitude variation space-time distance between the obstacle and the working device in the amplitude variation dimension is determined according to the number of sections in the amplitude variation direction between the first position and the second position in the grid area; the first rotation space-time distance between the obstacle and the working device in the rotation dimension is determined according to the number of sections in the rotation direction between the first position and the second position in the grid area; the first space-time distance vector is obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

[0284] In some optional implementation manners, the processor 1801 is specifically configured to:

[0285] The first position of the working device within the grid area is determined according to the working device parameters; the third position of the target point within the grid area is determined according to the target point parameters; the second amplitude variation space-time distance in the amplitude variation dimension between the target point and the working device is determined according to the number of portions in the amplitude variation direction between the first position and the third position within the grid area; the second rotation space-time distance in the rotation dimension between the target point and the working device is determined according to the number of portions in the rotation direction between the first position and the third position within the grid area; the second space-time distance vector is obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

[0286] Since the electronic device is the electronic device that executes the method in the embodiment of the present application, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0287] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned intelligent operation machine reinforcement learning method are implemented. The readable storage medium may be a non-volatile readable storage medium.

[0288] The present application is described above with reference to the block diagrams and / or flow charts showing the methods, devices (systems) and / or computer program products according to the embodiments of the present application. It should be understood that a block of the block diagram and / or flow chart diagram and a combination of blocks of the block diagram and / or flow chart diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor and / or other programmable device of a general-purpose computer, a special-purpose computer, to produce a machine, so that the instructions executed by the computer processor and / or other programmable device create a method for implementing the functions / actions specified in the block diagram and / or flow chart block.

[0289] Accordingly, the present application may also be implemented with hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, the present application may take the form of a computer program product on a computer-usable or computer-readable storage medium, which has a computer-usable or computer-readable program code implemented in the medium, for use by an instruction execution system or in conjunction with an instruction execution system. In the context of the present application, a computer-usable or computer-readable medium may be any medium that may contain, store, communicate, transmit, or convey a program for use by an instruction execution system, device, or apparatus, or in conjunction with an instruction execution system, device, or apparatus.

[0290] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0291] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a moving trajectory of a working machine, characterized in that: The method includes: Acquire state parameters of the working machine in the working scene, wherein the state parameters include working device parameters, obstacle parameters and target point parameters that the working device needs to reach in the working scene; Determine the obstacle state based on the working device parameter and the obstacle parameter, wherein the obstacle state is used to indicate the time-space distance between the obstacle and the working device, and the time-space distance between the obstacle and the working device is used to indicate the shortest time required for the obstacle to reach the working device; Based on the working device parameters and the target point parameters, determining the target point state, the target point state is used to indicate the time-space distance between the target point and the working device, and the time-space distance between the target point and the working device is used to indicate the shortest time required for the working device to reach the target point; Based on the obstacle state and the target point state, a moving trajectory of the working device to the target point in the working scene is determined.

2. The method according to claim 1, characterized in that The determining the obstacle state based on the working device parameter and the obstacle parameter includes: According to the polar coordinates of at least one key point of the bounding box corresponding to the obstacle in the obstacle parameters, determine the rotational space-time distance between each key point in the at least one key point and the working device, and obtain at least one rotational space-time distance between the obstacle and the working device; Determine the luffing time-space distance between the polar coordinates of each key point in the at least one key point and the operating device, and obtain at least one luffing time-space distance between the obstacle and the operating device; The obstacle state is determined based on the at least one rotation time-space distance and the at least one amplitude change time-space distance.

3. The method according to claim 2, characterized in that The determining of the obstacle state based on the at least one rotation time-space distance and the at least one amplitude change time-space distance comprises: Determine the amplitude variation position of each key point according to the polar coordinates of each key point in the at least one key point; Determine a first amplitude difference between each key point and the working device according to the amplitude position of each key point and the amplitude position of the working device in the working device parameters; According to the maximum amplitude change speed of the working device in the working device parameters and the first amplitude change difference between each key point and the working device, determine the amplitude change time-space distance between each key point and the working device to obtain at least one amplitude change time-space distance; Determine the arc difference between each key point and the working device in the rotation dimension according to the polar coordinates of each key point in the at least one key point; According to the maximum rotation speed of the working device in the working device parameters and the arc difference between each key point and the working device, determine the rotation time-space distance between each key point and the working device to obtain at least one rotation time-space distance; The obstacle state is determined based on the at least one rotation time-space distance and the at least one amplitude change time-space distance.

4. The method according to claim 1, characterized in that The determining the target point state based on the operating device parameters and the target point parameters includes: Determine the arc difference between the target point and the working device in the rotation dimension according to the rotation position of the target point in the target point parameter; Determine the rotational time-space distance between the target point and the working device according to the maximum rotational speed of the working device in the working device parameters and the arc difference between the target point and the working device; Determine a second amplitude difference between the working device and the target point according to the amplitude position of the target point in the target point parameters and the amplitude position of the working device in the working device parameters; Determine the luffing time-space distance between the target point and the working device according to the maximum luffing speed of the working device in the working device parameters and the second luffing difference between the target point and the working device; The state of the target point is determined based on the rotation time-space distance between the target point and the working device and the amplitude change time-space distance between the target point and the working device.

5. A method for determining a moving trajectory of a working machine, characterized in that: The method includes: Acquire state parameters of the working machine in the working scene, wherein the state parameters include working device parameters, obstacle parameters, working machine parameters, and target point parameters that the working device needs to reach in the working scene; According to the working machine parameters, the activity range area corresponding to the working machine is cut into a plurality of fan-shaped radial grid areas, wherein the activity range area is obtained by the range covered by the circular polar coordinate system established based on the support body of the working machine as the origin and the boom of the working machine as the positive semi-axis; Based on the working device parameters and the obstacle parameters, determining a first space-time distance vector between the obstacle and the working device in the grid area, wherein the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; Based on the working device parameters and the target point parameters, determining a second space-time distance vector between the target point and the working device in the grid area, wherein the second space-time distance is used to represent the shortest time required for the working device to reach the target point; Based on the first space-time distance vector and the second space-time distance vector, a moving trajectory of the working device to the target point in the working scene is determined.

6. The method according to claim 5, characterized in that The step of cutting the activity range area corresponding to the working machine into a plurality of sector-shaped radial grid areas according to the working machine parameters includes: According to the rotation speed of the working machine and the amplitude change speed of the working machine in the working machine parameters, determining the number of divisions of the working machine in the activity range area based on the amplitude change direction; Determining the number of divisions of the working machine in the activity range area based on the rotation direction according to the amplitude change speed of the working machine; According to the number of divisions in the amplitude variation direction and the number of divisions in the rotation direction, the activity range area is cut into a plurality of sector-shaped radial grid areas.

7. The method according to claim 5, characterized in that The step of determining the number of divisions of the amplitude change direction of the operating machine according to the rotation speed and the amplitude change speed of the operating machine comprises: Determining a difference between the maximum amplitude of the working machine and the minimum amplitude of the working machine according to the maximum amplitude of the working machine; The number of divisions in the luffing direction of the working machine is determined based on the difference in the maximum luffing of the working machine and the maximum rotation speed of the working machine.

8. The method according to claim 5, characterized in that The step of determining the number of cuts for the rotation direction of the working machine comprises: The number of divisions of the rotation direction of the working machine is determined based on the ratio of 2π to the maximum rotation speed of the working machine.

9. The method according to any one of claims 6 to 8, characterized in that The determining, based on the working device parameter and the obstacle parameter, a first spatiotemporal distance vector between the obstacle and the working device in the grid area comprises: Determine a first position of the operating device within the grid area according to the operating device parameters; Determine a second position of the obstacle within the grid area according to the obstacle parameter; Determine a first amplitude variation time-space distance between the obstacle and the operating device in the amplitude variation dimension according to the number of divisions of the amplitude variation direction between the first position and the second position in the grid area; Determine a first rotational space-time distance between the obstacle and the working device in the rotational dimension according to the number of divisions of the distance between the first position and the second position in the grid area in the rotational direction; The first space-time distance vector is obtained according to the first amplitude variation space-time distance and the first rotation space-time distance.

10. The method according to any one of claims 6 to 8, characterized in that The determining, based on the working device parameters and the target point parameters, of a second spatiotemporal distance vector between the target point and the working device in the grid area comprises: Determine a first position of the operating device within the grid area according to the operating device parameters; Determine a third position of the target point within the grid area according to the target point parameters; Determine a second amplitude variation time-space distance between the target point and the operating device in the amplitude variation dimension according to the number of divisions of the amplitude variation direction between the first position and the third position in the grid area; Determine a second rotational space-time distance between the target point and the working device in the rotational dimension according to the number of divisions of the distance between the first position and the third position in the grid area in the rotational direction; The second space-time distance vector is obtained according to the second amplitude variation space-time distance and the second rotation space-time distance.

11. A device for determining a moving trajectory of a working machine, characterized in that: The device includes: An acquisition module, used to acquire state parameters of the operating machine in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, and target point parameters that the operating device needs to reach in the operating scenario; A processing module is used to determine the obstacle state based on the working device parameters and the obstacle parameters, the obstacle state is used to indicate the space-time distance between the obstacle and the working device, and the space-time distance between the obstacle and the working device is used to indicate the shortest time required for the obstacle to reach the working device; determine the target point state based on the working device parameters and the target point parameters, the target point state is used to indicate the space-time distance between the target point and the working device, and the space-time distance between the target point and the working device is used to indicate the shortest time required for the working device to reach the target point; determine the movement trajectory of the working device to the target point in the working scene based on the obstacle state and the target point state.

12. A device for determining a moving trajectory of a working machine, characterized in that: The device includes: An acquisition module is used to acquire state parameters of the operating machine in an operating scenario, wherein the state parameters include operating device parameters, obstacle parameters, operating machine parameters, and target point parameters that the operating device needs to reach in the operating scenario; A division module, used for cutting the activity range area corresponding to the working machine into a plurality of fan-shaped radial grid areas according to the working machine parameters, wherein the activity range area is obtained by the range covered by the circular polar coordinate system established based on the support body of the working machine as the origin and the boom of the working machine as the positive semi-axis; A processing module is used to determine a first space-time distance vector between the obstacle and the working device in the grid area based on the working device parameters and the obstacle parameters, wherein the first space-time distance is used to represent the shortest time required for the obstacle to reach the working device; based on the working device parameters and the target point parameters, determine a second space-time distance vector between the target point and the working device in the grid area, wherein the second space-time distance vector is used to represent the shortest time required for the working device to reach the target point; based on the first space-time distance vector and the second space-time distance vector, determine a moving trajectory from the working device to the target point in the working scene.

13. An electronic device, characterized in that: The electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes the method as claimed in any one of claims 1 to 4; or the processor executes the method as claimed in any one of claims 5 to 10.

14. A computer-readable storage medium, characterized in that: It stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the method as claimed in any one of claims 1 to 4; or the electronic device executes the method as claimed in any one of claims 5 to 10.

Images