A method and device for determining a transfer path, and a control system

Abstract

The present application discloses a transfer path determination method, apparatus, and control system. The transfer path determination method includes: obtaining a digital twin model; when a real photovoltaic sweeper performs a sweeping operation, obtaining the first pose information of the real photovoltaic sweeper and the second pose information of the real photovoltaic robot; based on the first pose information and the second pose information, adjusting the virtual relative pose between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model; based on the digital twin model, determining a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper, so that the real photovoltaic robot can control the transfer of the real photovoltaic sweeper according to the first transfer path. The present application simulates the photovoltaic sweeping scenario based on the digital twin model, and obtains the transfer path of the photovoltaic sweeper through the digital twin model, so as to place the photovoltaic sweeper in a suitable position.

Description

Technical Field

[0001] The present application relates to the field of photovoltaic technology, and in particular, to a method and device for determining a transfer path, and a control system, which are applied to the field of photovoltaic cleaning. Background Art

[0002] The cleaning of a photovoltaic array is a heavy and repetitive labor task. Currently, in order to improve the cleaning efficiency and the utilization rate of a photovoltaic cleaning machine, a photovoltaic robot is mostly used to grab the photovoltaic cleaning machine and place the photovoltaic cleaning machine on different photovoltaic arrays to perform various cleaning operations on the photovoltaic panels in the photovoltaic array. If the photovoltaic cleaning machine is not placed in a suitable position, it may cause damage to the photovoltaic cleaning machine or the photovoltaic panel, which not only hinders the cleaning operation, but also affects the power generation efficiency of the photovoltaic panel, and is not conducive to the normal operation of a photovoltaic site (such as a photovoltaic power station).

[0003] Therefore, it is very important to place the photovoltaic cleaning machine in a suitable position. Summary of the Invention

[0004] The present application provides a method and device for determining a transfer path, and a control system, which simulate a photovoltaic cleaning scenario based on a digital twin model, and obtain a transfer path of a photovoltaic cleaning machine through the digital twin model, so as to place the photovoltaic cleaning machine in a suitable position.

[0005] Specifically, the technical solution of the present application is as follows:

[0006] In a first aspect, a method for determining a transfer path is provided, including:

[0007] Obtain a digital twin model; the digital twin model is used to simulate a photovoltaic cleaning scenario of a real photovoltaic cleaning machine;

[0008] When the real photovoltaic cleaning machine performs a cleaning operation, obtain the first pose information of the real photovoltaic cleaning machine and the second pose information of the real photovoltaic robot;

[0009] Based on the first pose information and the second pose information, adjust the virtual relative pose between the virtual photovoltaic cleaning machine and the virtual photovoltaic robot in the digital twin model;

[0010] Based on the digital twin model, determine a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic cleaning machine, so that the real photovoltaic robot controls the transfer of the real photovoltaic cleaning machine according to the first transfer path.

[0011] In one implementation, obtaining the first pose information of the real photovoltaic cleaning machine and the second pose information of the real photovoltaic robot includes:

[0012] Obtain the first travel information of the real photovoltaic sweeper; the first travel information includes the first travel distance of the real photovoltaic sweeper at a first preset time;

[0013] Obtain the second travel information of the real photovoltaic robot; the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot at the first preset time;

[0014] Obtain the relative travel information of the real photovoltaic sweeper and the real photovoltaic robot within the first preset time; the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot;

[0015] Determine the first pose information and the second pose information according to the first travel information, the second travel information and the relative travel information.

[0016] In one implementation,

[0017] The real photovoltaic sweeper includes an inertial measurement unit, and the inertial measurement unit is used to obtain the first motion state information of the real photovoltaic sweeper, and the first travel information is determined based on the first motion state information;

[0018] And / or, the real photovoltaic robot includes a lidar and a camera, the lidar is used to obtain the environmental point cloud data of the photovoltaic cleaning environment, and the second travel information is determined based on the environmental point cloud data;

[0019] And / or, the real photovoltaic sweeper includes a marker, and the relative travel information is determined by taking an image of the marker through the camera.

[0020] In one implementation, the above transfer path determination method further includes:

[0021] Adopt the Slam algorithm to update the pose information of the real photovoltaic robot in real time to obtain the third pose information of the real photovoltaic robot;

[0022] Synchronously update the pose information of the virtual photovoltaic robot in the digital twin model according to the third pose information;

[0023] Obtain the fourth pose information of the real photovoltaic sweeper;

[0024] Synchronously update the pose information of the virtual photovoltaic sweeper in the digital twin model according to the fourth pose information;

[0025] Combine the pose information of the virtual photovoltaic robot and the pose information of the virtual photovoltaic sweeper to dynamically update the digital twin model.

[0026] In one implementation, obtaining the fourth pose information of the real photovoltaic sweeper includes:

[0027] Obtain the first position information of the marker relative to the real photovoltaic sweeper;

[0028] Obtain the first motion state information of the real photovoltaic sweeper;

[0029] According to the first position information and the first motion state information, use the extended Kalman filter algorithm to estimate the fourth pose information in real time, so as to update the pose of the virtual photovoltaic sweeper in the digital twin model.

[0030] In one implementation, the camera is used to collect the image information of the marker. The above transfer path determination method further includes:

[0031] According to the image information, determine the second position information of the marker relative to the camera;

[0032] According to the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera, determine the position information Pm of the real photovoltaic sweeper in the world coordinate system:

[0033]

[0034] Among them, Pm represents the position information of the real photovoltaic sweeper in the world coordinate system, represents the inverse rotation matrix of the camera, Pa represents the second position information, D represents the third position information, Pc represents the fourth position information.

[0035] In one implementation, obtaining the digital twin model includes:

[0036] Obtain the layout information of the photovoltaic cleaning scenario;

[0037] Obtain the environmental information of the photovoltaic cleaning scenario;

[0038] Obtain the first motion state information and the first pose information of the real photovoltaic sweeper;

[0039] Obtain the second motion state information and the second pose information of the real photovoltaic robot;

[0040] According to the environmental information, determine the static model of the photovoltaic cleaning scenario in the digital twin model;

[0041] According to the static model, the layout information, the first motion state information, the first pose information, the second motion state information and the second pose information, construct the digital twin model; obtain the virtual photovoltaic sweeper and the virtual photovoltaic robot from the digital twin model, as well as their virtual relative poses and virtual relative travel information.

[0042] In one implementation, the above transfer path determination method further includes:

[0043] When the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data and the remaining data corresponding to the failure data, in the digital twin model, the virtual photovoltaic sweeper and the virtual photovoltaic robot fit the second transfer path, and the real photovoltaic robot controls the real photovoltaic sweeper to transfer along the second transfer path.

[0044] In a second aspect, a transfer path determination device is provided, including a processor configured to call instructions stored in a memory. When the instructions are called by the processor, the processor executes the transfer path determination method in the first aspect or any implementation of the first aspect above.

[0045] In a third aspect, a control system for a photovoltaic sweeper is provided, including:

[0046] The transfer path determination device provided in the second aspect above, configured to determine a transfer path;

[0047] A photovoltaic robot, coupled to the transfer path determination device, configured to control the transfer of the photovoltaic sweeper according to the transfer path.

[0048] Compared with the prior art, the present application has at least the following beneficial effects:

[0049] 1. The present application restores the photovoltaic cleaning scenario through the digital twin model, and can simulate the dynamic changes of the photovoltaic cleaning scenario; and the operation of the digital twin model is not affected by the actual environment. In this way, an accurate transfer path can be obtained, enabling the real photovoltaic robot to place the real photovoltaic sweeper in a suitable position.

[0050] 2. The present application does not need to rely on an external positioning system. Only by relying on the markers on the real photovoltaic sweeper, the inertial measurement unit and camera on the real photovoltaic robot, etc., the position and attitude of the real photovoltaic sweeper can be estimated, and the digital twin model can be updated in real time according to the estimation result, improving the handling efficiency of the real photovoltaic robot for the real photovoltaic sweeper and promoting the transformation of equipment management towards intelligence and digitization.

[0051] 3. The present application uses the digital twin model to simulate the photovoltaic cleaning scenario, and through the mapping between the real photovoltaic cleaning scenario and the virtual photovoltaic cleaning scenario, makes an overall decision on the behavior of the real photovoltaic robot, improving the robustness of the control and being unaffected by extreme conditions, such as the markers on the real photovoltaic sweeper cannot be effectively recognized under real strong light conditions, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The following briefly introduces the drawings used in the description of the embodiments of the present application:

[0053] Figure 1 It is a schematic structural diagram of a control system of a photovoltaic cleaning machine provided in an embodiment of the present application;

[0054] Figure 2 It is a flowchart of a method for determining a transfer path applied to a photovoltaic cleaning machine provided in an embodiment of the present application;

[0055] Figure 3 It is a schematic diagram of a photovoltaic cleaning scenario provided in an embodiment of the present application;

[0056] Figure 4 It is a schematic structural diagram of a device for determining a transfer path applied to a photovoltaic cleaning machine provided in an embodiment of the present application. Detailed implementation manners

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the specific implementation manners of the present application will be described below with reference to the accompanying drawings. The accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings, and other implementation manners can be obtained. Adjustments and improvements made without departing from the concept of the present application all fall within the protection scope of the present application.

[0058] For the sake of simplicity of the drawings, each accompanying drawing in the embodiments of the present application only schematically shows the parts related to the present application, and they do not represent the actual structure of the product. In addition, for the sake of simplicity and easy understanding of the drawings, in some accompanying drawings, parts with the same structure or function are only schematically shown, and there may actually be more or fewer parts with the same structure or function.

[0059] In the present application, unless otherwise clearly specified and defined, ordinal numbers, such as "first", "second", etc., are only used to distinguish and describe related objects, and cannot be understood as indicating or implying the relative importance or order between related objects; in addition, they do not represent the quantity of related objects. "And / or" is used to describe the relationship between related objects, and it includes any relationship of related objects. For example, "a and / or b" includes: "a alone", "b alone", or "a and b".

[0060] The term "coupled" includes a signal connection between objects, which can be achieved directly through a medium (e.g., wires, traces, etc.) or can be achieved through other components. Similarly, "connected" includes direct connection or indirect connection, or includes electrical connection or signal connection. The objects being connected can be directly connected through a medium (e.g., wires, traces, etc.), or can be indirectly connected through other components, or can be internally connected. "Mounted" can be direct mounting or mounting through other components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0061] In a photovoltaic site (e.g., a photovoltaic power station), to ensure the power generation efficiency of photovoltaic panels, the surfaces of the photovoltaic panels (or photovoltaic modules) need to be cleaned regularly. Currently, a photovoltaic cleaning machine is mostly used to perform the cleaning operation. The photovoltaic cleaning machine can travel on the surface of the photovoltaic panels and perform corresponding cleaning tasks according to the set cleaning path.

[0062] Before the photovoltaic cleaning machine performs the cleaning operation, it needs to be placed in a suitable position on the photovoltaic panel, usually by a photovoltaic robot (e.g., a photovoltaic cleaning vehicle). The photovoltaic panels are installed on photovoltaic brackets. Different photovoltaic brackets may have different heights and installation positions, and the photovoltaic brackets themselves have a certain inclination angle. If the photovoltaic cleaning machine is placed improperly, such as having a deviation in the placement angle, being too high / low in the placement height, etc., it may cause the cleaning machine to collide with and damage the photovoltaic panel, or cause abnormal situations such as extrusion between the photovoltaic cleaning machine and the photovoltaic panel. These abnormal situations may damage the photovoltaic cleaning machine itself or the photovoltaic panel. This not only hinders the cleaning operation but also affects the power generation efficiency of the photovoltaic panel and the service life of the photovoltaic cleaning machine or the photovoltaic panel, which is not conducive to the normal operation of the photovoltaic site (e.g., a photovoltaic power station).

[0063] Based on this, an embodiment of the present application proposes a method for determining a transfer path. This method can be used to determine the transfer path for a photovoltaic robot to control the transfer of a photovoltaic cleaning machine so that the photovoltaic cleaning machine can be placed in a suitable position.

[0064] The following is a description with reference to the drawings.

[0065] Please refer to Figure 1 , which is a schematic structural diagram of a control system for a photovoltaic cleaning machine provided in an embodiment of the present application. As Figure 1As shown, the control system 100 includes a transfer path determination device 110 and a photovoltaic robot 120. Among them, the transfer path determination device 110 is configured to determine the transfer path. The photovoltaic robot 120 is coupled to the transfer path determination device 110 and is configured to control the transfer of the photovoltaic sweeper according to the transfer path. The transfer path determination device 110 can be installed on the photovoltaic robot 120; or it can be set independently of the photovoltaic robot 120, for example, located in a server or a terminal device. Please refer to Figure 2 , which is a flowchart example of a method for determining a transfer path applied to a photovoltaic sweeper provided by an embodiment of the present application. This transfer path determination method can be executed by the above transfer path determination device 110. As Figure 2 shown, this transfer path determination method at least includes the following steps:

[0066] S210: Obtain a digital twin model; the digital twin model is used to simulate the photovoltaic cleaning scenario of the real photovoltaic sweeper;

[0067] S220: When the real photovoltaic sweeper performs a cleaning operation, obtain the first pose information of the real photovoltaic sweeper and the second pose information of the real photovoltaic robot;

[0068] S230: Based on the first pose information and the second pose information, adjust the virtual relative pose between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model;

[0069] S240: Based on the digital twin model, determine the first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper, so that the real photovoltaic robot can control the transfer of the real photovoltaic sweeper according to the first transfer path.

[0070] Digital Twin creates a dynamic virtual model of a physical entity in a digital way. The digital twin model is a virtual digital model, and its core lies in creating a digital copy of a physical entity, which can reflect the state of the physical entity in real time. Users can perform various simulation tests, analyses, and optimizations in the virtual environment based on the digital twin model. The above transfer path determination method simulates the photovoltaic cleaning scenario based on the digital twin model. Please refer to Figure 3 , which is a schematic diagram of a photovoltaic cleaning scenario provided in an embodiment of the present application. As Figure 3As shown, the photovoltaic cleaning scenario includes a photovoltaic support 330, a plurality of photovoltaic panels 340 installed on the photovoltaic support, a photovoltaic cleaning machine 320, and a photovoltaic robot 310. The boxes in the figure are for illustration only and do not represent the actual shape, size, etc. of the photovoltaic support, photovoltaic panel, photovoltaic cleaning machine, or photovoltaic robot. The photovoltaic support installed with photovoltaic panels can also be referred to as a photovoltaic array. For example, the photovoltaic array can include a photovoltaic support 330 and a plurality of photovoltaic panels 340. The photovoltaic cleaning machine 320 can perform a cleaning operation on the photovoltaic array according to a cleaning path.

[0071] In one implementation, the photovoltaic robot 310 can place the photovoltaic cleaning machine 320 at the starting point of the cleaning path. The starting point of the cleaning path can include any point on the photovoltaic array. For example, the starting point of the cleaning path can include the starting point of the photovoltaic array. Exemplarily, the starting point of the photovoltaic array can include any point on the photovoltaic panels at both ends of the bottom of the photovoltaic array, such as Figure 3 point A or point B shown in. In one implementation, the photovoltaic cleaning scenario can include a plurality of photovoltaic arrays. There is an interval between different photovoltaic arrays (such as the interval between different photovoltaic supports or between photovoltaic panels), and this interval can also be referred to as a breakpoint. The photovoltaic robot 310 can grab the photovoltaic cleaning machine 320 and transfer the photovoltaic cleaning machine 320 from the current photovoltaic array to other photovoltaic arrays to be cleaned to prevent the breakpoint from affecting the operation of the photovoltaic cleaning machine 320. Or, the photovoltaic robot 340 can also transfer the photovoltaic cleaning machine 320 to different positions on the same photovoltaic array. In one implementation, the photovoltaic robot 310 can move synchronously when the photovoltaic cleaning machine 320 is moving to facilitate the timely transfer of the photovoltaic cleaning machine 320. Exemplarily, the moving direction of the photovoltaic robot 310 is consistent with the forward direction of the photovoltaic cleaning machine 320. For example, the running direction of the photovoltaic array (such as Figure 3 the arrow shown in).

[0072] The digital twin model can simulate the photovoltaic cleaning scenario. For the convenience of distinction, in the embodiments of this application, the photovoltaic cleaning machine and photovoltaic robot in the real environment are referred to as the real photovoltaic cleaning machine and real photovoltaic robot; the photovoltaic cleaning machine and photovoltaic robot in the digital twin model are referred to as the virtual photovoltaic cleaning machine and virtual photovoltaic robot. The digital twin model can be synchronized with the actual photovoltaic cleaning scenario. For example, the real photovoltaic cleaning machine and the virtual photovoltaic cleaning machine start working synchronously, or the real photovoltaic robot and the virtual photovoltaic robot start working synchronously.

[0073] In one implementation, the real photovoltaic sweeper includes markers. The markers can be set on the real photovoltaic sweeper to determine the pose information of the real photovoltaic sweeper. Exemplarily, the marker can be an ArUco marker code. The ArUco marker code is a square marker composed of a wide black border and an internal binary matrix that determines its identifier (ID). In some embodiments, other marker codes can also be used as markers, such as QR codes, etc. In one implementation, the image information of the marker can be collected, and the pose information of the real photovoltaic sweeper can be determined by combining the image information and the position information of the target photovoltaic panel. The real photovoltaic robot can grasp or place the real photovoltaic sweeper according to the pose information of the real photovoltaic sweeper. However, the image information of the marker is often collected by a camera (such as a camera installed on the photovoltaic robot itself or a camera installed on the photovoltaic bracket / photovoltaic panel). The quality of the image information may be affected by the environment. For example, when the ambient light intensity is strong or weak, the image quality will change, resulting in an inability to accurately obtain the pose information of the real photovoltaic sweeper. The transfer path determination method of the present application can restore the photovoltaic sweeping scenario through a digital twin model and simulate the dynamic changes of the photovoltaic sweeping scenario; moreover, the operation of the digital twin model is not affected by the actual environment. In this way, an accurate transfer path can be obtained, enabling the real photovoltaic robot to place the real photovoltaic sweeper in a suitable position.

[0074] For the convenience of distinction, the pose information of the real photovoltaic sweeper can be referred to as the first pose information, and the pose information of the real photovoltaic robot can be referred to as the second pose information. In one implementation, when the real photovoltaic sweeper performs a sweeping operation, the first pose information of the real photovoltaic sweeper and the second pose information of the real photovoltaic robot can be obtained. Based on the first pose information and the second pose information, the virtual relative pose between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model can be adjusted. Exemplarily, when the real photovoltaic sweeper performs a sweeping operation, its position may change at any time. The real photovoltaic robot can travel synchronously with the real photovoltaic sweeper (refer to the description of the foregoing embodiments). In this way, the relative pose between the real photovoltaic sweeper and the real photovoltaic robot may change at any time. By adjusting the virtual relative pose between the virtual photovoltaic sweeper and the virtual photovoltaic robot and mapping the first pose information and the second pose information into the digital twin model, the digital twin model can simulate or restore the real photovoltaic sweeping scenario in real time, improving the similarity between the digital twin model and the real photovoltaic sweeping scenario, which is beneficial to improving the accuracy of the first transfer path.

[0075] Furthermore, the digital twin model has prior characteristics. Based on the digital twin model, the grasping or placement of the photovoltaic sweeper can be simulated (or predicted), and the first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper can be determined, so that the real photovoltaic robot can control the transfer of the real photovoltaic sweeper according to the first transfer path. In this way, the first transfer path (i.e., the virtual transfer path) generated by simulating with the digital twin model is used as the real transfer path for controlling the transfer of the real photovoltaic sweeper. Each part of the digital twin model is interconnected, and various data generated by the digital twin model have the characteristics of strong correlation and accessibility (for example, some places in a building in reality may be difficult to measure, but the digital twin model dynamically restores the entire scene, enabling the directly acquisition of data that is difficult to measure). Currently, the method for positioning the real photovoltaic sweeper uses multiple independent modules. Each module performs a specific task separately (such as separately collecting the image information of the marker and separately determining), and the real photovoltaic sweeper is positioned through the interaction between the modules. This method lacks integrity. When a certain module fails to perform the task normally, it may cause all tasks to jam or interrupt, affecting the normal operation of the real photovoltaic sweeper. In this application, the digital twin model is used to simulate the photovoltaic cleaning scenario, and through the mapping between the real photovoltaic cleaning scenario and the virtual photovoltaic cleaning scenario, overall decision-making is made on the behavior of the real photovoltaic robot, which can improve robustness. In one implementation, the first transfer path can be the transfer path with the least time consumption and / or the smallest error among multiple transfer paths obtained through multiple simulations. In one implementation, the photovoltaic robot may include a robotic arm. The robotic arm can be used to grasp, transfer, or place the photovoltaic sweeper. Exemplarily, the transfer path may include the movement path of the robotic arm. For example, the movement paths of one or more joints of the robotic arm.

[0076] In one implementation, the pose information of the real photovoltaic sweeper may include the position coordinates and attitude angles of the real photovoltaic sweeper. The pose information of the real photovoltaic robot may include the position coordinates and attitude angles of the real photovoltaic robot. Exemplarily, obtaining the pose information of the real photovoltaic sweeper and the pose information of the real photovoltaic robot includes: obtaining the first travel information of the real photovoltaic sweeper, where the first travel information includes the first travel distance of the real photovoltaic sweeper at a first preset time; obtaining the second travel information of the real photovoltaic robot, where the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot at the first preset time; obtaining the relative travel information between the real photovoltaic sweeper and the real photovoltaic robot within the first preset time, where the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot; and determining the pose information of the real photovoltaic sweeper (for the convenience of distinction, it can be called the first pose information) and the pose information of the real photovoltaic robot (for the convenience of distinction, it can be called the second pose information) according to the first travel information, the second travel information, and the relative travel information.

[0077] Among them, the first travel information includes the first travel distance of the real photovoltaic sweeper at the first preset time. Exemplarily, the first preset time may include a unit time or a time period. In one implementation, the real photovoltaic sweeper includes an accelerometer or an inertial measurement unit (IMU). The accelerometer or the inertial measurement unit can obtain the first motion state information of the real photovoltaic sweeper, such as measuring the acceleration of the real photovoltaic sweeper within a period of time or at any moment. In one implementation, the real photovoltaic sweeper further includes a drive motor. The first motion state information may further include the wheel rotation speed of the walking wheels of the drive motor within a period of time or at any moment. The first travel information is determined based on the above first motion state information. Exemplarily, according to the above acceleration and the wheel rotation speed of the walking wheels, the first travel information can be determined.

[0078] In one implementation, the data obtained by the IMU or the accelerometer can be processed by an advanced filtering algorithm to obtain the first travel information. Exemplarily, when the real photovoltaic sweeper performs a cleaning operation on the photovoltaic panel, integrating the acceleration provided by the accelerometer or the IMU on the real photovoltaic sweeper and the wheel rotation speed provided by the drive motor can obtain the distance traveled by the real photovoltaic sweeper per unit time. In one implementation, the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot at the first preset time. Exemplarily, the real photovoltaic robot includes a lidar. The lidar can obtain the environmental point cloud data of the photovoltaic cleaning scene. A point cloud is a set of multiple spatial points. The point cloud includes multiple discrete points, and each point can correspond to a spatial position, which can be represented by three-dimensional position coordinates (x, y, z). The above environmental point cloud data may include the three-dimensional position coordinates of multiple spatial points in the photovoltaic cleaning scene. The second travel information of the real photovoltaic robot can be determined based on the environmental point cloud data within a certain time interval. For example, registering the spatial points in the environmental point cloud data within a certain time interval can directly obtain the travel speed and the second travel distance of the real photovoltaic robot. Exemplarily, the lidar can be installed on the robotic arm of the real photovoltaic robot, such as the end of the robotic arm. The embodiments of the present application do not limit the number of lidars installed on the real photovoltaic sweeper, and the number of lidars may include one or more. In one implementation, the relative travel information includes the relative distance and the relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot. Exemplarily, the real photovoltaic sweeper includes a marker. The description of the marker can refer to the content of the foregoing embodiments. In one implementation, the real photovoltaic robot further includes a camera. The relative travel information can be determined by the camera capturing the image information of the marker.

[0079] According to the above first movement information, second movement information, and relative movement information, the first pose information and the second pose information can be determined. In one implementation, data fusion is performed on the first movement information, second movement information, and relative movement information to determine the first pose information and the second pose information. Different types of data have different precisions and thus different confidence levels. For example, data fusion can be performed according to the confidence levels of the first movement information, second movement information, and relative movement information. The embodiments of the present application do not limit the high or low confidence levels among the first movement information, second movement information, and relative movement information. For example, point cloud data has high-precision characteristics and a high confidence level; image information has a high confidence level due to its intuitiveness; data obtained from an accelerometer or a drive motor is prone to noise or errors and has a medium confidence level, and so on. By combining the different confidence levels of different types of data, the data obtained above can be fused to obtain the first pose information and the second pose information. In this way, data fusion based on the confidence level can improve the accuracy of the first pose information and the second pose information. In this case, mapping the first pose information and the second pose information to the digital twin model can improve the accuracy and timeliness of the digital twin model.

[0080] In one implementation, the pose information of the real photovoltaic robot and the real sweeper can be estimated in real time, and based on this, the digital twin model can be updated. On the basis of any of the foregoing embodiments, updating the digital twin model may include the following steps: using the Slam algorithm to update the pose information of the real photovoltaic robot in real time to obtain the third pose information of the real photovoltaic robot; according to the third pose information, synchronously updating the pose information of the virtual photovoltaic robot in the digital twin model; obtaining the fourth pose information of the real photovoltaic sweeper; according to the fourth pose information, synchronously updating the pose information of the virtual photovoltaic sweeper in the digital twin model; combining the pose information of the virtual photovoltaic robot and the pose information of the virtual photovoltaic sweeper to dynamically update the digital twin model. In this way, by updating the digital twin model, the timeliness of the model can be improved.

[0081] This application does not need to rely on an external positioning system. Only by relying on the markers on the real photovoltaic sweeper, the inertial measurement unit, camera, etc. on the real photovoltaic robot, the position and attitude of the real photovoltaic sweeper can be estimated, and based on the estimation result, the digital twin model can be updated in real time, improving the handling efficiency of the real photovoltaic robot for the real photovoltaic sweeper and promoting the transformation of equipment management to intelligence and digitization.

[0082] For example, the above obtaining the fourth pose information of the real photovoltaic sweeper includes:

[0083] Obtaining the first position information of the marker relative to the real photovoltaic sweeper;

[0084] Obtaining the first motion state information of the real photovoltaic sweeper;

[0085] According to the first position information and the first motion state information, the extended Kalman filter algorithm is used to estimate the fourth pose information in real time, so as to update the pose of the virtual photovoltaic sweeper in the digital twin model.

[0086] Exemplarily, by applying the extended Kalman filter (EKF) algorithm, the fourth pose information of the real photovoltaic sweeper can be estimated in real time:

[0087] [P global , Θ] = EKF(P local , S motion );

[0088] where P global represents the global position information of the real photovoltaic sweeper, Θ represents the attitude angle, P local represents the first position information of the marker relative to the real photovoltaic sweeper, and S motion represents the first motion state information of the real photovoltaic sweeper. The description of the first motion state information can refer to the content of the foregoing embodiments.

[0089] In one implementation, based on the image information of the marker, the position information of the real photovoltaic sweeper in the world coordinate system can be determined. Specifically, it includes: determining the second position information of the marker relative to the camera according to the image information; determining the position information of the real photovoltaic sweeper in the world coordinate system according to the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera.

[0090] For example, according to the image information of the marker, the second position information of the marker relative to the camera is determined; according to the image information of the marker, the internal parameters of the camera, and the position information of the marker in the world coordinate system, the PnP (Perspective-n-Points) problem is solved to determine the second position information. Let the second position information be P a , and the global position information of the marker can be determined through the inverse transformation of the camera, and the second position information P a is converted into the position information in the global coordinate system. Assume that the displacement of the marker relative to the real photovoltaic sweeper is vector D. Exemplarily, if the marker is exactly located at the geometric center of the real photovoltaic sweeper, the displacement D can be a zero vector.

[0091] The position information of the real photovoltaic sweeper in the world coordinate system Pm can be determined by the following formula:

[0092]

[0093] where represents the inverse rotation matrix of the camera, Pc Represents the fourth position information of the camera in the world coordinate system.

[0094] In one implementation, when the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data corresponding to the failed data and the remaining data (non-failed data) in the first travel information or the second travel information, in the digital twin model, the relative pose between the current real photovoltaic sweeper and the real photovoltaic robot is predicted, and the second transfer path is fitted by the virtual photovoltaic sweeper and the virtual photovoltaic robot. The real photovoltaic robot controls the transfer of the real photovoltaic sweeper along the second transfer path. The failed data includes missing or distorted data. In this way, the historical data corresponding to the failed data is used to replace the failed data to prevent the data failure from affecting the accuracy of the second transfer path. Further, the above historical data can also be used to predict the relative pose between the real photovoltaic sweeper and the real photovoltaic robot within a future time period. In this way, it can prevent the data failure from affecting the accuracy of the relative pose.

[0095] In one implementation, obtaining the digital twin model as described above may include the following steps: obtaining the layout information of the photovoltaic cleaning scenario; obtaining the environmental information of the photovoltaic cleaning scenario; obtaining the first motion state information and the first pose information of the real photovoltaic sweeper; obtaining the second motion state information and the second pose information of the real photovoltaic robot; determining the static model of the photovoltaic cleaning scenario in the digital twin model according to the environmental information; determining the digital twin model according to the first static model, the layout information, the first motion state information, the first pose information, the second motion state information, and the second pose information. The virtual photovoltaic sweeper, the virtual photovoltaic robot, and their virtual relative pose and virtual relative travel information can be obtained from the digital twin model.

[0096] Among them, the description of the first motion state information can refer to the content of the foregoing embodiments, and the description of the second motion state information can refer to the description of the first motion state information. The description of the first pose information and the second pose information can refer to the content of the foregoing embodiments. Exemplarily, the layout information of the photovoltaic cleaning scenario may include one or more of the following: a plane map of the photovoltaic site, the height where the photovoltaic panels are located (such as the average height), the tilt angle of the photovoltaic panels, the coordinates of the starting point of the photovoltaic array, the coordinates of the ending point of the photovoltaic array (similar to the starting point, refer to the description of the starting point of the photovoltaic array in the foregoing embodiments), the coordinates of the breakpoint of the photovoltaic array, and so on. In one implementation, the above layout information can be pre-stored in a database and obtained from the database. Exemplarily, the environmental information may include environmental point cloud data. The description of the environmental point cloud data can refer to the content of the foregoing embodiments. Based on the environmental point cloud data, a static model of the photovoltaic cleaning scenario can be determined. The static model is used to simulate the static environment of the photovoltaic site (which does not change with the movement or motion of a real photovoltaic cleaner or a real photovoltaic robot). The static environment may include photovoltaic panels, the ground, the sky, or other objects; the first motion state information and the second motion state information are dynamic data (which change with the movement or motion of a real photovoltaic cleaner or a real photovoltaic robot). By fusing the static model with the first motion state information and the second motion state information, a digital twin model can be determined.

[0097] In one implementation, the above static model can be determined through the following steps: based on the environmental point cloud data, determine a plurality of feature points; based on the spatial relationship between the plurality of feature points, determine a feature point map; perform feature fusion on the feature point map and the plurality of feature points to determine the static model. For example, preprocess the environmental point cloud data through a deep learning model to identify and classify to obtain a set of feature points:

[0098] F k = DL_Model(P i ), i = 1, 2,..., n;

[0099] Among them, F k represents the set of feature points, P i represents the spatial points in the environmental point cloud, and n represents the total number of spatial points in the environmental point cloud.

[0100] Exemplarily, a graph optimization algorithm can be used to reconstruct the spatial relationship of the feature points:

[0101] G = Optimize(F k , C ij );

[0102] Among them, G represents the feature point map; C ij represents the constraint conditions between the feature points, which are used to indicate the spatial relationship between the plurality of feature points.

[0103] Exemplarily, a multi-scale fusion strategy can be applied to fuse the feature point map with multiple feature points to obtain a static model:

[0104] T = Multiscale_Fusion(G, P i );

[0105] where T represents the static model.

[0106] Based on the same technical concept, an embodiment of the present application further provides a transfer path determination device applied to a photovoltaic robot. Figure 4 FIG. shows a schematic structural diagram of a transfer path determination device applied to a photovoltaic robot provided by an embodiment of the present application. As Figure 4 shown, the transfer path determination device 400 includes a processor 410 for calling instructions stored in a memory 420, and when the instructions are called by the processor 410, the processor 410 executes any one of the transfer path determination methods in the above embodiments.

[0107] It should be noted that the above embodiments can be freely combined according to needs. The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for determining a transfer path, characterized in that, Including: Obtain a digital twin model; the digital twin model is used to simulate the photovoltaic cleaning scenario of a real photovoltaic cleaning machine; When the real photovoltaic cleaning machine performs a cleaning operation, obtain the first pose information of the real photovoltaic cleaning machine and the second pose information of the real photovoltaic robot; Based on the first pose information and the second pose information, adjust the virtual relative pose between the virtual photovoltaic cleaning machine and the virtual photovoltaic robot in the digital twin model; Based on the digital twin model, determine a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic cleaning machine, so that the real photovoltaic robot controls the transfer of the real photovoltaic cleaning machine according to the first transfer path; Wherein, obtaining the first pose information of the real photovoltaic cleaning machine and the second pose information of the real photovoltaic robot includes: Obtain the first travel information of the real photovoltaic cleaning machine; the first travel information includes the first travel distance of the real photovoltaic cleaning machine at a first preset time; Obtain the second travel information of the real photovoltaic robot; the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot at the first preset time; Obtain the relative travel information between the real photovoltaic cleaning machine and the real photovoltaic robot within the first preset time; the relative travel information includes the relative distance and relative travel speed between the real photovoltaic cleaning machine and the real photovoltaic robot; Determine the first pose information and the second pose information according to the first travel information, the second travel information, and the relative travel information.

2. The transfer path determination method according to claim 1, characterized in that The real photovoltaic cleaning machine includes an inertial measurement unit, and the inertial measurement unit is used to obtain the first motion state information of the real photovoltaic cleaning machine, and the first travel information is determined based on the first motion state information; And / or, the real photovoltaic robot includes a lidar and a camera, the lidar is used to obtain environmental point cloud data of the photovoltaic cleaning environment, and the second travel information is determined based on the environmental point cloud data; And / or, the real photovoltaic cleaning machine includes a marker, and the relative travel information is determined by the camera capturing image information of the marker.

3. The transfer path determination method according to claim 2, wherein Further including: Use the Slam algorithm to update the pose information of the real photovoltaic robot in real time to obtain the third pose information of the real photovoltaic robot; According to the third pose information, synchronously update the pose information of the virtual photovoltaic robot in the digital twin model; Obtain the fourth pose information of the real photovoltaic cleaning machine; According to the fourth pose information, synchronously update the pose information of the virtual photovoltaic cleaning machine in the digital twin model; Combine the pose information of the virtual photovoltaic robot and the pose information of the virtual photovoltaic cleaning machine to dynamically update the digital twin model.

4. The transfer path determination method according to claim 3, characterized in that The obtaining the fourth pose information of the real photovoltaic cleaning machine includes: Obtain the first position information of the marker relative to the real photovoltaic cleaning machine; Obtain the first motion state information of the real photovoltaic sweeper; According to the first position information and the first motion state information, use the extended Kalman filter algorithm to estimate the fourth pose information in real time, so as to update the pose of the virtual photovoltaic sweeper in the digital twin model.

5. The transfer path determination method according to claim 4, wherein The camera is used to collect the image information of the marker, and the transfer path determination method further includes: Determine the second position information of the marker relative to the camera according to the image information; According to the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera, determine the position information Pm of the real photovoltaic sweeper in the world coordinate system: wherein, Pm represents the position information of the actual photovoltaic cleaning machine in the world coordinate system, represents the inverse rotation matrix of the camera, Pa represents the second position information, D represents the third position information, and Pc represents the fourth position information.

6. The transfer path determination method according to claim 1, wherein The obtaining of the digital twin model includes: Obtain the layout information of the photovoltaic cleaning scene; Obtain the environmental information of the photovoltaic cleaning scene; Obtain the first motion state information and the first pose information of the real photovoltaic sweeper; Obtain the second motion state information and the second pose information of the real photovoltaic robot; Determine the static model of the photovoltaic cleaning scene in the digital twin model according to the environmental information; Construct the digital twin model according to the static model, the layout information, the first motion state information, the first pose information, the second motion state information and the second pose information; Obtain the virtual photovoltaic sweeper and the virtual photovoltaic robot and their virtual relative poses and virtual relative travel information from the digital twin model.

7. The transfer path determination method according to claim 6, wherein It further includes: When the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data and the remaining data corresponding to the failure data, in the digital twin model, the virtual photovoltaic sweeper and the virtual photovoltaic robot fit the second transfer path, and the real photovoltaic robot controls the real photovoltaic sweeper to transfer along the second transfer path.

8. A transfer path determination device, characterized in that It includes a processor for calling instructions stored in a memory, and when the instructions are called by the processor, the processor executes the transfer path determination method according to any one of claims 1-7.

9. A control system for a photovoltaic cleaning machine, characterized in that, It includes: The transfer path determination device according to claim 8, configured to determine a transfer path; A photovoltaic robot, coupled to the transfer path determination device, configured to control the transfer of the photovoltaic sweeper according to the transfer path.

Images