Transfer path determination method and device, and control system

Abstract

The invention discloses a transfer path determination method and device and a control system. The transfer path determination method comprises the following steps: acquiring a digital twin model; when the real photovoltaic cleaning machine executes cleaning operation, first pose information of the real photovoltaic cleaning machine and second pose information of the real photovoltaic robot are obtained; based on the first pose information and the second pose information, a virtual relative pose between a virtual photovoltaic sweeper and a virtual photovoltaic robot in the digital twin model is adjusted; and based on the digital twin model, determining a first transfer path for the virtual photovoltaic robot to control and transfer the virtual photovoltaic cleaning machine, so that the real photovoltaic robot can control and transfer the real photovoltaic cleaning machine according to the first transfer path. The photovoltaic cleaning scene is simulated based on the digital twinborn model, and the transfer path of the photovoltaic cleaning machine is obtained through the digital twinborn model, so that the photovoltaic cleaning machine can be conveniently placed at a proper position.

Description

Technical Field

[0001] The present application relates to the field of photovoltaic technology, and in particular to a transfer path determination method and device, and a control system, which are applied in the field of photovoltaic cleaning. Background Art

[0002] Photovoltaic array cleaning is a heavy and repetitive labor. At present, in order to improve the cleaning efficiency and the utilization rate of photovoltaic cleaning machines, photovoltaic cleaning machines are often grabbed by photovoltaic robots and placed on different photovoltaic arrays to perform various cleaning operations on the photovoltaic panels in the photovoltaic arrays. If the photovoltaic cleaning machine is not placed in a suitable position, it may cause damage to the photovoltaic cleaning machine or photovoltaic panels, which not only hinders the cleaning operation, but also affects the power generation efficiency of the photovoltaic panels, which is not conducive to the normal operation of the photovoltaic site (such as a photovoltaic power station).

[0003] Therefore, it is very important to place the photovoltaic sweeper in a suitable position. Summary of the invention

[0004] The present application provides a transfer path determination method and device, and a control system, which simulates a photovoltaic cleaning scene based on a digital twin model, and obtains the transfer path of a photovoltaic cleaning machine through the digital twin model to facilitate placing the photovoltaic cleaning machine in a suitable position.

[0005] Specifically, the technical solution of this application is as follows:

[0006] In a first aspect, a transfer path determination method is provided, comprising:

[0007] Obtain a digital twin model; the digital twin model is used to simulate the photovoltaic cleaning scene of a real photovoltaic cleaning machine;

[0008] When the real photovoltaic cleaning machine performs cleaning operations, the first posture information of the real photovoltaic cleaning machine and the second posture information of the real photovoltaic robot are obtained;

[0009] Based on the first pose information and the second pose information, the virtual relative pose between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model is adjusted;

[0010] Based on the digital twin model, a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper is determined, so that the real photovoltaic robot can control the transfer of the real photovoltaic sweeper according to the first transfer path.

[0011] In one implementation, obtaining the first pose information of the real photovoltaic sweeper and the second pose information of the real photovoltaic robot includes:

[0012] Acquire first travel information of a real photovoltaic sweeper; the first travel information includes a first travel distance of the real photovoltaic sweeper at a first preset time;

[0013] Acquire second travel information of the real photovoltaic robot; the second travel information includes a travel speed of the real photovoltaic robot and a second travel distance of the real photovoltaic robot in a first preset time;

[0014] Acquire the relative travel information of the real photovoltaic sweeper and the real photovoltaic robot within a first preset time; the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot;

[0015] The first pose information and the second pose information are determined according to the first travel information, the second travel information and the relative travel information.

[0016] In one implementation,

[0017] The real photovoltaic sweeper includes an inertial measurement unit, the inertial measurement unit is used to obtain first motion state information of the real photovoltaic sweeper, and the first travel information is determined based on the first motion state information;

[0018] And / or, the real photovoltaic robot includes a laser radar and a camera, the laser radar is used to obtain environmental point cloud data of the photovoltaic cleaning environment, and the second travel information is determined based on the environmental point cloud data;

[0019] And / or, the real photovoltaic sweeper includes a marker, and the relative travel information is determined by image information of the marker captured by a camera.

[0020] In one implementation, the above transfer path determination method further includes:

[0021] The Slam algorithm is used to update the pose information of the real photovoltaic robot in real time to obtain the third pose information of the real photovoltaic robot;

[0022] According to the third posture information, the posture information of the virtual photovoltaic robot is synchronously updated in the digital twin model;

[0023] Obtain the fourth posture information of the real photovoltaic sweeper;

[0024] According to the fourth posture information, the posture information of the virtual photovoltaic sweeper is synchronously updated in the digital twin model;

[0025] The digital twin model is dynamically updated by combining the posture information of the virtual photovoltaic robot and the posture information of the virtual photovoltaic sweeper.

[0026] In one implementation, obtaining the fourth posture information of the real photovoltaic sweeper includes:

[0027] Acquire first position information of the marker relative to the real photovoltaic sweeper;

[0028] Obtaining first motion state information of a real photovoltaic sweeper;

[0029] According to the first position information and the first motion state information, an extended Kalman filter algorithm is used to estimate the fourth posture information in real time to update the posture of the virtual photovoltaic sweeper in the digital twin model.

[0030] In one implementation, the camera is used to collect image information of the marker. The above transfer path determination method also includes:

[0031] Determine second position information of the marker relative to the camera based on the image information;

[0032] According to the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera, the position information Pm of the real photovoltaic sweeper in the world coordinate system is determined:

[0033]

[0034] in, Pm Represents the position information of the real photovoltaic sweeper in the world coordinate system, represents the inverse rotation matrix of the camera, Pa represents the second position information, D represents the third position information, Pc Indicates fourth position information.

[0035] In one implementation, obtaining a digital twin model includes:

[0036] Get the layout information of the photovoltaic cleaning scene;

[0037] Obtain environmental information of photovoltaic cleaning scenes;

[0038] Obtain the first motion state information and the first posture information of the real photovoltaic sweeper;

[0039] Acquire the second motion state information and the second posture information of the real photovoltaic robot;

[0040] Determine the static model of the photovoltaic cleaning scene in the digital twin model based on environmental information;

[0041] A digital twin model is constructed according to the static model, layout information, first motion state information, first posture information, second motion state information and second posture information; and the virtual photovoltaic sweeper and the virtual photovoltaic robot and their virtual relative posture and virtual relative travel information are obtained from the digital twin model.

[0042] In one implementation, the above transfer path determination method further includes:

[0043] When the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data and remaining data corresponding to the failure data, in the digital twin model, the virtual photovoltaic sweeper and the virtual photovoltaic robot fit the second transfer path, and the real photovoltaic robot transfers and controls the real photovoltaic sweeper along the second transfer path.

[0044] In a second aspect, a transfer path determination device is provided, comprising a processor, configured to call instructions stored in a memory, wherein when the instructions are called by the processor, the processor executes the transfer path determination method of the first aspect or any one of the implementations of the first aspect.

[0045] In a third aspect, a control system for a photovoltaic sweeper is provided, comprising:

[0046] The transfer path determination device provided in the second aspect above is configured to determine a transfer path;

[0047] The photovoltaic robot is coupled to the transfer path determination device and is configured to control the transfer of the photovoltaic cleaning machine according to the transfer path.

[0048] Compared with the prior art, this application has at least the following beneficial effects:

[0049] 1. This application restores the photovoltaic cleaning scene through the digital twin model, and can simulate the dynamic changes of the photovoltaic cleaning scene; and the operation of the digital twin model will not be affected by the actual environment. In this way, an accurate transfer path can be obtained, so that the real photovoltaic robot can place the real photovoltaic cleaning machine in the appropriate position.

[0050] 2. This application does not require the help of an external positioning system. It can estimate the position and posture of a real photovoltaic sweeper by only using markers on the real photovoltaic sweeper, inertial measurement units on the real photovoltaic robot, cameras, etc., and update the digital twin model in real time based on the estimation results, thereby improving the efficiency of the real photovoltaic robot in carrying the real photovoltaic sweeper and promoting the transformation of equipment management towards intelligence and digitalization.

[0051] 3. This application uses a digital twin model to simulate photovoltaic cleaning scenarios. By mapping the real photovoltaic cleaning scenes and the virtual photovoltaic cleaning scenes, an overall decision is made on the behavior of the real photovoltaic robot, thereby improving the robustness of the control and not being affected by extreme conditions. For example, under real strong light conditions, the markers on the real photovoltaic cleaning machine cannot be effectively identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The following is a brief introduction to the drawings used in describing the embodiments of the present application:

[0053] Figure 1 It is a structural schematic diagram of a control system of a photovoltaic sweeper provided in an embodiment of the present application;

[0054] Figure 2 is a flow chart of a transfer path determination method applied to a photovoltaic sweeper provided in an embodiment of the present application;

[0055] Figure 3 is a schematic diagram of a photovoltaic cleaning scenario provided in an embodiment of the present application;

[0056] Figure 4 It is a structural schematic diagram of a transfer path determination device applied to a photovoltaic sweeper provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other accompanying drawings and other implementation methods can be obtained based on these drawings without creative work. Adjustments and improvements made without departing from the concept of the present application are all within the scope of protection of the present application.

[0058] In order to simplify the drawings, the drawings in the embodiments of the present application only schematically show the parts related to the present application, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some drawings, only parts with the same structure or function are schematically shown, and there may be more or fewer parts with the same structure or function in reality.

[0059] In this application, unless otherwise expressly specified and limited, ordinal numbers, such as "first", "second", etc., are only used to distinguish and describe related objects, and cannot be understood as indicating or implying the relative importance or order between related objects; in addition, they do not represent the number of related objects. "And / or" is used to describe the relationship between related objects, which includes any relationship between related objects, for example, "a and / or b" includes: "a alone", "b alone", or "a and b".

[0060] The term "coupling" includes signal connection between objects, which can be achieved directly through a medium (e.g., a wire, a trace, etc.), or can be achieved through other elements, etc. Similarly, "connection" includes direct connection or indirect connection, or includes electrical connection or signal connection, and the connected objects can be directly connected through a medium (e.g., a wire, a trace, etc.), or can be indirectly connected through other elements, or can be internally connected. "Installation" can be direct installation or installation through other components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0061] In a photovoltaic site (e.g., a photovoltaic power station), in order to ensure the power generation efficiency of the photovoltaic panels, the surface of the photovoltaic panels (or photovoltaic modules) needs to be cleaned regularly. Currently, photovoltaic sweepers are mostly used to perform cleaning operations. The photovoltaic sweepers can travel on the surface of the photovoltaic panels according to a set cleaning path and perform corresponding cleaning tasks.

[0062] Before the photovoltaic sweeper performs the cleaning operation, it needs to be placed in the appropriate position of the photovoltaic panel, which is generally placed by a photovoltaic robot (for example, a photovoltaic sweeper). Photovoltaic panels are installed on photovoltaic brackets. Different photovoltaic brackets may have different heights and installation positions, and the photovoltaic brackets themselves have a certain inclination angle. If the photovoltaic sweeper is placed improperly, such as the placement angle is deviated, the placement height is too high / too low, etc., it may cause the sweeper to collide with the photovoltaic panel, or cause abnormal conditions such as the photovoltaic sweeper and the photovoltaic panel to be squeezed. These abnormal conditions may cause damage to the photovoltaic sweeper itself or the photovoltaic panel. This not only hinders the cleaning operation, but also affects the power generation efficiency of the photovoltaic panel and the service life of the photovoltaic sweeper or photovoltaic panel, which is not conducive to the normal operation and operation of the photovoltaic site (such as a photovoltaic power station).

[0063] Based on this, an embodiment of the present application proposes a transfer path determination method. The method can be used to determine the transfer path of a photovoltaic robot controlling the transfer of a photovoltaic sweeper, so that the photovoltaic sweeper can be placed in a suitable position.

[0064] The following is a description with reference to the accompanying drawings.

[0065] Please refer to Figure 1 , which is a schematic diagram of the structure of a control system of a photovoltaic sweeper provided in an embodiment of the present application. Figure 1As shown, the control system 100 includes a transfer path determination device 110 and a photovoltaic robot 120. The transfer path determination device 110 is configured to determine a transfer path. The photovoltaic robot 120, coupled to the transfer path determination device 110, is configured to control the transfer of the photovoltaic sweeper according to the transfer path. The transfer path determination device 110 can be installed on the photovoltaic robot 120; or it can be set independently of the photovoltaic robot 120, for example, located in a server or terminal device. Please refer to Figure 2 , which is a flowchart of a transfer path determination method for a photovoltaic sweeper provided in an embodiment of the present application. The transfer path determination method can be executed by the above transfer path determination device 110. Figure 2 As shown, the transfer path determination method at least includes the following steps:

[0066] S210: obtaining a digital twin model; the digital twin model is used to simulate a photovoltaic cleaning scene of a real photovoltaic cleaning machine;

[0067] S220: when the real photovoltaic cleaning machine performs a cleaning operation, obtaining first posture information of the real photovoltaic cleaning machine and second posture information of the real photovoltaic robot;

[0068] S230: adjusting the virtual relative posture between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model based on the first posture information and the second posture information;

[0069] S240: Based on the digital twin model, determine a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper, so that the real photovoltaic robot controls the transfer of the real photovoltaic sweeper according to the first transfer path.

[0070] Digital Twin is a dynamic virtual model of a physical entity in a digital way. The core of the digital twin model is to create a digital copy of a physical entity, which can reflect the status of the physical entity in real time. Users can perform various simulation tests, analysis and optimization in a virtual environment based on the digital twin model. The above transfer path determination method simulates the photovoltaic cleaning scene based on the digital twin model. Please refer to Figure 3 , which is a schematic diagram of a photovoltaic cleaning scene provided in an embodiment of the present application. Figure 3As shown, the photovoltaic cleaning scene includes a photovoltaic support 330, a plurality of photovoltaic panels 340 mounted on the photovoltaic support, a photovoltaic cleaning machine 320, and a photovoltaic robot 310. The boxes in the figure are only used for examples and do not represent the actual shape, size, etc. of the photovoltaic support, photovoltaic panel, photovoltaic cleaning machine, or photovoltaic robot. A photovoltaic support with photovoltaic panels installed may also be referred to as a photovoltaic array. For example, a photovoltaic array may include a photovoltaic support 330 and a plurality of photovoltaic panels 340. The photovoltaic cleaning machine 320 may perform cleaning operations on the photovoltaic array according to the cleaning path.

[0071] In one implementation, the photovoltaic robot 310 can place the photovoltaic sweeper 320 at the starting point of the sweeping path. The starting point of the sweeping path can include any point on the photovoltaic array. For example, the starting point of the sweeping path can include the starting point of the photovoltaic array. For example, the starting point of the photovoltaic array can include any point on the photovoltaic panels at both ends of the bottom of the photovoltaic array, such as Figure 3 As shown at point A or point B in the figure. In one implementation, the photovoltaic cleaning scene may include multiple photovoltaic arrays. There are intervals between different photovoltaic arrays (for example, intervals between different photovoltaic brackets or photovoltaic panels), which can also be called breakpoints. The photovoltaic robot 310 can grab the photovoltaic cleaner 320 and transfer the photovoltaic cleaner 320 from the current photovoltaic array to other photovoltaic arrays to be cleaned to prevent the breakpoint from affecting the operation of the photovoltaic cleaner 320. Alternatively, the photovoltaic robot 340 can also transfer the photovoltaic cleaner 320 to different locations on the same photovoltaic array. In one implementation, the photovoltaic robot 310 can move synchronously with the photovoltaic cleaner 320 when it is moving, so as to facilitate the timely transfer of the photovoltaic cleaner 320. For example, the moving direction of the photovoltaic robot 310 is consistent with the forward direction of the photovoltaic cleaner 320. For example, the direction of movement of the photovoltaic array (such as Figure 3 (indicated by the arrow in the middle).

[0072] The digital twin model can simulate the photovoltaic cleaning scene. For the sake of distinction, in the embodiments of the present application, the photovoltaic cleaning machine and photovoltaic robot in the real environment are referred to as real photovoltaic cleaning machines and real photovoltaic robots; the photovoltaic cleaning machine and photovoltaic robot in the digital twin model are referred to as virtual photovoltaic cleaning machines and virtual photovoltaic robots. The digital twin model can be synchronized with the actual photovoltaic cleaning scene. For example, the real photovoltaic cleaning machine and the virtual photovoltaic cleaning machine start working synchronously, or the real photovoltaic robot and the virtual photovoltaic machine start working synchronously.

[0073] In one implementation, the real photovoltaic sweeper includes a marker. The marker can be set on the real photovoltaic sweeper to determine the posture information of the real photovoltaic sweeper. For example, the marker can be an ArUco marker code. The ArUco marker code is a square mark composed of a wide black border and an internal binary matrix that determines its identifier (ID). In some embodiments, other marker codes can also be used as markers, such as QR codes, etc. In one implementation, the image information of the marker can be collected, and the posture information of the real photovoltaic sweeper can be determined by combining the image information and the position information of the target photovoltaic panel. The real photovoltaic robot can grab or place the real photovoltaic sweeper according to the posture information of the real photovoltaic sweeper. However, the image information of the marker is often collected by a camera (such as a camera installed on the photovoltaic robot itself or a camera installed on a photovoltaic bracket / photovoltaic panel). The quality of the image information may be affected by the environment. For example, when the ambient light intensity is strong or weak, the image quality will change, so that the posture information of the real photovoltaic sweeper cannot be accurately obtained. The transfer path determination method of the present application can restore the photovoltaic cleaning scene through the digital twin model, and can simulate the dynamic changes of the photovoltaic cleaning scene; and the operation of the digital twin model will not be affected by the actual environment. In this way, an accurate transfer path can be obtained, so that the real photovoltaic robot can place the real photovoltaic cleaning machine in the appropriate position.

[0074] For the convenience of distinction, the posture information of the real photovoltaic sweeper can be called the first posture information, and the posture information of the real photovoltaic robot can be called the second posture information. In one implementation, when the real photovoltaic sweeper performs the cleaning operation, the first posture information of the real photovoltaic sweeper and the second posture information of the real photovoltaic robot can be obtained. Based on the first posture information and the second posture information, the virtual relative posture between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model can be adjusted. For example, when the real photovoltaic sweeper performs the cleaning operation, its position may change at any time. The real photovoltaic robot can move synchronously with the real photovoltaic sweeper (refer to the description of the aforementioned embodiment). In this way, the relative posture between the real photovoltaic sweeper and the real photovoltaic robot may change at any time. By adjusting the virtual relative posture between the virtual photovoltaic sweeper and the virtual photovoltaic robot, the first posture information and the second posture information are mapped to the digital twin model, so that the digital twin model can simulate or restore the real photovoltaic cleaning scene in real time, improve the similarity between the digital twin model and the real photovoltaic cleaning scene, and help improve the accuracy of the first transfer path.

[0075] Furthermore, the digital twin model has a priori characteristics. Based on the digital twin model, the grabbing or placement of the photovoltaic sweeper can be simulated (or predicted), and the first transfer path of the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper can be determined, so that the real photovoltaic robot can control the transfer of the real photovoltaic sweeper according to the first transfer path. In this way, the first transfer path (i.e., the virtual transfer path) generated by the digital twin model simulation is used as the real transfer path for controlling the transfer of the real photovoltaic sweeper. The various parts of the digital twin model are related to each other, and the various data generated by the digital twin model have strong correlation and obtainable characteristics (for example, some places of a building in reality may be difficult to measure, but the digital twin model dynamically restores the entire scene, so that the difficult-to-measure data can be directly obtained). The current method for locating the real photovoltaic sweeper uses multiple independent modules. Each module performs a specific task separately (for example, separately collects the image information of the marker, and determines it separately), and the real photovoltaic sweeper is located through the interaction between the modules. This method lacks integrity. When one of the modules fails to perform the task normally, it may cause all tasks to get stuck or interrupted, affecting the normal operation of the real photovoltaic sweeper. In this application, a digital twin model is used to simulate the photovoltaic cleaning scene. By mapping the real photovoltaic cleaning scene and the virtual photovoltaic cleaning scene, an overall decision is made on the behavior of the real photovoltaic robot, which can improve the robustness. In one implementation, the first transfer path can be the transfer path with the shortest time and / or the smallest error among multiple transfer paths obtained by multiple simulations. In one implementation, the photovoltaic robot may include a robotic arm. The robotic arm can be used to grasp, transfer or place the photovoltaic cleaning machine. For example, the transfer path may include the motion path of the robotic arm. For example, the motion path of one or more joints of the robotic arm.

[0076] In one implementation, the posture information of the real photovoltaic sweeper may include the position coordinates and attitude angle of the real photovoltaic sweeper. The posture information of the real photovoltaic robot may include the position coordinates and attitude angle of the real photovoltaic robot. For example, obtaining the posture information of the real photovoltaic sweeper and the posture information of the real photovoltaic robot includes: obtaining the first travel information of the real photovoltaic sweeper, the first travel information includes the first travel distance of the real photovoltaic sweeper at the first preset time; obtaining the second travel information of the real photovoltaic robot, the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot at the first preset time; obtaining the relative travel information of the real photovoltaic sweeper and the real photovoltaic robot within the first preset time, the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot; according to the first travel information, the second travel information and the relative travel information, determine the posture information of the real photovoltaic sweeper (for the convenience of distinction, it can be called the first posture information) and the posture information of the real photovoltaic robot (for the convenience of distinction, it can be called the second posture information).

[0077] Among them, the first travel information includes the first travel distance of the real photovoltaic sweeper in the first preset time. For example, the first preset time may include a unit time or a time period. In one implementation, the real photovoltaic sweeper includes an accelerometer or an inertial measurement unit (IMU). The accelerometer or the inertial measurement unit can obtain the first motion state information of the real photovoltaic sweeper, such as measuring the acceleration of the real photovoltaic sweeper within a period of time or at any time. In one implementation, the real photovoltaic sweeper also includes a drive motor. The first motion state information may also include the speed of the walking wheel of the drive motor within a period of time or at any time. The first travel information is determined based on the above-mentioned first motion state information. For example, the first travel information can be determined based on the above acceleration and walking wheel speed.

[0078] In one implementation, the data obtained by the IMU or accelerometer can be processed by an advanced filtering algorithm to obtain the first travel information. For example, when the real photovoltaic sweeper performs a cleaning operation on the photovoltaic panel, the acceleration provided by the accelerometer or IMU on the real photovoltaic sweeper and the walking wheel speed provided by the drive motor are integrated to obtain the distance traveled by the real photovoltaic sweeper in a unit time. In one implementation, the second travel information includes the travel speed of the real photovoltaic robot and the second travel distance of the real photovoltaic robot in the first preset time. For example, the real photovoltaic robot includes a laser radar. The laser radar can obtain the environmental point cloud data of the photovoltaic cleaning scene. The point cloud is a collection of multiple spatial points. The point cloud includes multiple discrete points, each of which can correspond to a spatial position, and this spatial position can be represented by a three-dimensional position coordinate (x, y, z). The above environmental point cloud data may include the three-dimensional position coordinates of multiple spatial points in the photovoltaic cleaning scene. The second travel information of the real photovoltaic robot can be determined based on the environmental point cloud data at a certain time interval. For example, each spatial point in the environmental point cloud data at a certain time interval is aligned to directly obtain the travel speed and the second travel distance of the real photovoltaic robot. For example, the laser radar can be installed on the mechanical arm of the real photovoltaic robot, such as the end of the mechanical arm. The embodiment of the present application does not limit the number of laser radars installed on the real photovoltaic sweeper, and the number of laser radars may include one or more. In one implementation, the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot. For example, the real photovoltaic sweeper includes a marker. For the description of the marker, refer to the content of the aforementioned embodiment. In one implementation, the real photovoltaic robot also includes a camera. The relative travel information can be determined by taking the image information of the marker by the camera.

[0079] According to the above first travel information, second travel information and relative travel information, the first pose information and the second pose information can be determined. In one implementation, the first travel information, the second travel information and the relative travel information are data fused to determine the first pose information and the second pose information. Different types of data have different precisions, so the confidence levels are different. For example, data fusion can be performed based on the confidence levels of the first travel information, the second travel information and the relative travel information. The embodiment of the present application does not limit the confidence level between the first travel information, the second travel information and the relative travel information. For example, point cloud data has high precision characteristics and high confidence; image information has high confidence due to its intuitiveness; data obtained from an accelerometer or a drive motor is prone to noise or errors, and the confidence level is medium, etc. Combined with different confidence levels of different types of data, the above-obtained data is fused to obtain the first pose information and the second pose information. In this way, data fusion based on confidence can improve the accuracy of the first pose information and the second pose information. In this case, mapping the first pose information and the second pose information to the digital twin model can improve the accuracy and timeliness of the digital twin model.

[0080] In one implementation, the posture information of the real photovoltaic robot and the real sweeper can be estimated in real time, and the digital twin model can be updated based on this. Based on any of the aforementioned embodiments, updating the digital twin model may include the following steps: using the Slam algorithm to update the posture information of the real photovoltaic robot in real time, and obtaining the third posture information of the real photovoltaic robot; according to the third posture information, synchronously updating the posture information of the virtual photovoltaic robot in the digital twin model; obtaining the fourth posture information of the real photovoltaic sweeper; according to the fourth posture information, synchronously updating the posture information of the virtual photovoltaic sweeper in the digital twin model; combining the posture information of the virtual photovoltaic robot and the posture information of the virtual photovoltaic sweeper, dynamically updating the digital twin model. In this way, by updating the digital twin model, the timeliness of the model is improved.

[0081] This application does not require the help of an external positioning system, but only relies on markers on the real photovoltaic sweeper, inertial measurement units on the real photovoltaic robot, cameras, etc. to estimate the position and posture of the real photovoltaic sweeper, and updates the digital twin model in real time based on the estimation results, thereby improving the efficiency of the real photovoltaic robot in carrying the real photovoltaic sweeper and promoting the transformation of equipment management towards intelligence and digitalization.

[0082] For example, the fourth posture information of the real photovoltaic sweeper is obtained as follows:

[0083] Acquire first position information of the marker relative to the real photovoltaic sweeper;

[0084] Obtaining first motion state information of a real photovoltaic sweeper;

[0085] According to the first position information and the first motion state information, the extended Kalman filter algorithm is used to estimate the fourth posture information in real time to update the posture of the virtual photovoltaic sweeper in the digital twin model.

[0086] For example, the fourth pose information of the real photovoltaic sweeper can be estimated in real time by applying the extended Kalman filter (EKF) algorithm:

[0087] [P global ,Θ]=EKF(P local ,S motion );

[0088] Among them, P global represents the global position information of the real photovoltaic sweeper, Θ represents the attitude angle, P local Represents the first position information of the marker relative to the real photovoltaic sweeper, S motion Represents the first motion state information of the real photovoltaic sweeper. For the description of the first motion state information, reference can be made to the content of the aforementioned embodiment.

[0089] In one implementation, the position information of the real photovoltaic sweeper in the world coordinate system can be determined based on the image information of the marker. Specifically, the second position information of the marker relative to the camera is determined based on the image information; the position information of the real photovoltaic sweeper in the world coordinate system is determined based on the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera.

[0090] For example, according to the image information of the marker, the second position information of the marker relative to the camera is determined; according to the image information of the marker, the camera internal parameters and the position information of the marker in the world coordinate system, the PnP (Perspective-n-Points) problem is solved to determine the second position information. Let the second position information be P a , the global position information of the marker can be determined by the inverse transformation of the camera, and the second position information P a Transformed into position information in the global coordinate system. Assume that the displacement of the marker relative to the real photovoltaic sweeper is vector D. For example, if the marker is located exactly at the geometric center of the real photovoltaic sweeper, the displacement D can be a zero vector.

[0091] The position information of the real photovoltaic sweeper in the world coordinate system Pm It can be determined by the following formula:

[0092]

[0093] in, Represents the inverse rotation matrix of the camera, Pc Represents the fourth position information of the camera in the world coordinate system.

[0094] In one implementation, when the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data corresponding to the failed data and the remaining data (non-failed data) in the first travel information or the second travel information, the relative posture between the current real photovoltaic sweeper and the real photovoltaic robot is predicted in the digital twin model, and the second transfer path is fitted by the virtual photovoltaic sweeper and the virtual photovoltaic robot. The real photovoltaic robot controls the transfer of the real photovoltaic sweeper with the second transfer path. The failed data includes missing or distorted data. In this way, the failed data is replaced by the historical data corresponding to the failed data to prevent the data failure from affecting the accuracy of the second transfer path. Furthermore, the above historical data can also be used to predict the relative posture between the real photovoltaic sweeper and the real photovoltaic robot in a future time period. In this way, data failure can be prevented from affecting the accuracy of the relative posture.

[0095] In one implementation, the above-mentioned digital twin model acquisition may include the following steps: acquiring the layout information of the photovoltaic cleaning scene; acquiring the environmental information of the photovoltaic cleaning scene; acquiring the first motion state information and the first pose information of the real photovoltaic cleaning machine; acquiring the second motion state information and the second pose information of the real photovoltaic robot; determining the static model of the photovoltaic cleaning scene in the digital twin model according to the environmental information; determining the digital twin model according to the first static model, layout information, first motion state information, first pose information, second motion state information, and second pose information. From the digital twin model, the virtual photovoltaic cleaning machine and the virtual photovoltaic robot and their virtual relative pose and virtual relative travel information can be acquired.

[0096] Among them, the description of the first motion state information can refer to the content of the aforementioned embodiment, and the description of the second motion state information can refer to the description of the first motion state information. The description of the first posture information and the second posture information can refer to the content of the aforementioned embodiment. For example, the layout information of the photovoltaic cleaning scene may include one or more of the following: a photovoltaic site plane map, the height of the photovoltaic panel (for example, the average height), the inclination angle of the photovoltaic panel, the coordinates of the starting point of the photovoltaic array, the coordinates of the end point of the photovoltaic array (similar to the starting point, refer to the description of the starting point of the photovoltaic array in the aforementioned embodiment), the coordinates of the photovoltaic array breakpoint, etc. In one implementation, the above layout information can be pre-stored in a database and obtained from the database. For example, the environmental information may include environmental point cloud data. The description of the environmental point cloud data can refer to the content of the aforementioned embodiment. According to the environmental point cloud data, the static model of the photovoltaic cleaning scene can be determined. The static model is used to simulate the static environment of the photovoltaic site (not changing with the movement or motion of the real photovoltaic sweeper or the real photovoltaic robot), and the static environment may include photovoltaic panels, the ground, the sky or other objects; the first motion state information and the second motion state information are dynamic data (changing with the movement or motion of the real photovoltaic sweeper or the real photovoltaic robot). By fusing the static model with the first motion state information and the second motion state information, a digital twin model can be determined.

[0097] In one implementation, the above static model can be determined by the following steps: determining multiple feature points based on the environmental point cloud data; determining a feature point map based on the spatial relationship between the multiple feature points; and fusing the feature point map with the multiple feature points to determine the static model. For example, the environmental point cloud data is preprocessed by a deep learning model to identify and classify the feature point set:

[0098] F k =DL_Model(P i ),i=1,2,…,n;

[0099] Among them, F k represents the feature point set, P i represents the spatial points in the environment point cloud, and n represents the total number of spatial points in the environment point cloud.

[0100] For example, the graph optimization algorithm can be used to reconstruct the spatial relationship of feature points:

[0101] G=Optimize(F k ,C ij );

[0102] Among them, G represents the feature point map; C ij Represents the constraints between feature points and is used to indicate the spatial relationship between multiple feature points.

[0103] For example, a multi-scale fusion strategy can be applied to fuse the feature point map with multiple feature points to obtain a static model:

[0104] T=Multiscale_Fusion(G,P i );

[0105] Among them, T represents the static model.

[0106] Based on the same technical concept, an embodiment of the present application also provides a transfer path determination device applied to a photovoltaic robot. Figure 4 FIG. 1 is a schematic diagram showing a transfer path determination device for a photovoltaic robot provided in an embodiment of the present application. Figure 4 As shown, the transition path determination device 400 includes a processor 410, which is used to call instructions stored in a memory 420. When the instructions are called by the processor 410, the processor 410 executes any one of the transition path determination methods in the above embodiments.

[0107] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.

Claims

1. A method for determining a transfer path, characterized in that: include: Acquire a digital twin model; the digital twin model is used to simulate a photovoltaic cleaning scene of a real photovoltaic cleaning machine; When the real photovoltaic cleaning machine performs a cleaning operation, obtaining first posture information of the real photovoltaic cleaning machine and second posture information of the real photovoltaic robot; Based on the first posture information and the second posture information, adjusting the virtual relative posture between the virtual photovoltaic sweeper and the virtual photovoltaic robot in the digital twin model; Based on the digital twin model, a first transfer path for the virtual photovoltaic robot to control the transfer of the virtual photovoltaic sweeper is determined, so that the real photovoltaic robot controls the transfer of the real photovoltaic sweeper according to the first transfer path.

2. The method for determining a transition path according to claim 1, characterized in that: The obtaining of the first posture information of the real photovoltaic sweeper and the second posture information of the real photovoltaic robot includes: Acquire first travel information of the real photovoltaic sweeper; the first travel information includes a first travel distance of the real photovoltaic sweeper in a first preset time; Acquire second travel information of the real photovoltaic robot; the second travel information includes a travel speed of the real photovoltaic robot and a second travel distance of the real photovoltaic robot in the first preset time; Acquire the relative travel information of the real photovoltaic sweeper and the real photovoltaic robot within the first preset time; the relative travel information includes the relative distance and relative travel speed between the real photovoltaic sweeper and the real photovoltaic robot; The first position information and the second position information are determined according to the first travel information, the second travel information and the relative travel information.

3. The method for determining a transition path according to claim 2, wherein: The real photovoltaic sweeper includes an inertial measurement unit, and the inertial measurement unit is used to obtain first motion state information of the real photovoltaic sweeper, and the first travel information is determined based on the first motion state information; And / or, the real photovoltaic robot includes a laser radar and a camera, the laser radar is used to obtain environmental point cloud data of the photovoltaic cleaning environment, and the second travel information is determined based on the environmental point cloud data; And / or, the real photovoltaic sweeper includes a marker, and the relative travel information is determined by image information of the marker captured by the camera.

4. The method for determining a transition path according to claim 3, wherein: Also includes: The Slam algorithm is used to update the posture information of the real photovoltaic robot in real time to obtain the third posture information of the real photovoltaic robot; According to the third posture information, synchronously updating the posture information of the virtual photovoltaic robot in the digital twin model; Acquiring fourth posture information of the real photovoltaic sweeper; According to the fourth posture information, synchronously updating the posture information of the virtual photovoltaic sweeper in the digital twin model; The digital twin model is dynamically updated by combining the posture information of the virtual photovoltaic robot and the posture information of the virtual photovoltaic sweeper.

5. The method for determining a transition path according to claim 4, characterized in that: The obtaining of the fourth posture information of the real photovoltaic sweeper includes: Acquire first position information of the marker relative to the real photovoltaic sweeper; Acquire first motion state information of the real photovoltaic sweeper; According to the first position information and the first motion state information, an extended Kalman filter algorithm is used to estimate the fourth posture information in real time to update the posture of the virtual photovoltaic sweeper in the digital twin model.

6. The method for determining a transition path according to claim 5, characterized in that: The camera is used to collect image information of the marker, and the transfer path determination method further includes: Determining second position information of the marker relative to the camera based on the image information; According to the second position information, the third position information of the marker relative to the real photovoltaic sweeper, the fourth position information of the camera in the world coordinate system, and the inverse rotation matrix of the camera, the position information Pm of the real photovoltaic sweeper in the world coordinate system is determined: in, Pm represents the position information of the real photovoltaic sweeper in the world coordinate system, represents the inverse rotation matrix of the camera, Pa represents the second position information, D represents the third position information, Pc Indicates the fourth position information.

7. The method for determining a transition path according to claim 1 or 2, characterized in that: The obtaining of the digital twin model comprises: Obtaining layout information of the photovoltaic cleaning scene; Acquiring environmental information of the photovoltaic cleaning scene; Acquire the first motion state information and the first posture information of the real photovoltaic sweeper; Acquire the second motion state information and the second posture information of the real photovoltaic robot; Determine, according to the environmental information, a static model of the photovoltaic cleaning scene in the digital twin model; The digital twin model is constructed according to the static model, the layout information, the first motion state information, the first posture information, the second motion state information and the second posture information; and the virtual photovoltaic sweeper and the virtual photovoltaic robot and their virtual relative posture and virtual relative travel information are obtained from the digital twin model.

8. The method for determining a transition path according to claim 7, characterized in that: Also includes: When the first travel information of the real photovoltaic sweeper or the second travel information of the real photovoltaic robot fails, based on the historical data and remaining data corresponding to the failure data, in the digital twin model, the virtual photovoltaic sweeper and the virtual photovoltaic robot fit a second transfer path, and the real photovoltaic robot transfers and controls the real photovoltaic sweeper along the second transfer path.

9. A transfer path determination device, characterized in that: The method comprises a processor, which is used to call instructions stored in a memory, and when the instructions are called by the processor, the processor executes the transfer path determination method according to any one of claims 1 to 8.

10. A control system for a photovoltaic sweeper, characterized in that: include: The transition path determination device according to claim 9, configured to determine the transition path; The photovoltaic robot is coupled to the transfer path determination device and is configured to control the transfer of the photovoltaic sweeper according to the transfer path.

Images