End-to-end remote control unmanned agricultural machine operation system

Abstract

The invention discloses an end-to-end remote control unmanned agricultural machine operation system. A control terminal in the system receives a first control instruction which is triggered by a user and used for creating a map for an operation area, positioning data of the operation area is collected through positioning equipment to create map data, and the map data is stored in a cloud server; the cloud server generates an initial navigation control strategy of the operation area according to the map data; the control terminal receives a second control instruction triggered by the user and used for controlling the unmanned agricultural machine to work in the working area, obtains an initial navigation control strategy of the working area from the cloud server, generates a target navigation control instruction according to the initial navigation control strategy, and issues the target navigation control instruction to the unmanned agricultural machine; and the unmanned agricultural machine performs normal operation in the operation area according to the target navigation control instruction. The technical problem that a related remote control unmanned agricultural machine operation system is poor in control flexibility and control reliability is solved.

Description

Technical Field

[0001] The present application relates to the field of remote control technology, and in particular to an end-to-end remote control unmanned agricultural machinery operation system. Background Art

[0002] In the context of modern smart agriculture, remote control technology for unmanned agricultural machinery is becoming an important means of improving agricultural production efficiency and reducing costs. However, existing technologies for remote control of unmanned agricultural machinery still face many challenges, particularly in path planning. Currently, path planning generally involves analyzing pre-built map data to generate a dynamic path that avoids obstacles within the work area. However, this path often fails to avoid dynamic obstacles within the work area, making it impossible for unmanned agricultural machinery to flexibly modify its path, impacting its normal operation.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide an end-to-end remote-controlled unmanned agricultural machinery operation system to at least solve the technical problems of poor control flexibility and control reliability in related remote-controlled unmanned agricultural machinery operation systems.

[0005] According to one aspect of an embodiment of the present application, an end-to-end remote control unmanned agricultural machinery operation system is provided, which includes at least: a control terminal, a target positioning device, a cloud server, and a target unmanned agricultural machinery, wherein the control terminal is used to receive a first control instruction triggered by a target user to create a map of a target operation area, use the target positioning device to collect positioning data of the target operation area to create target map data of the target operation area, and store the target map data on the cloud server, wherein the target positioning device is communicatively connected to the control terminal; the cloud server is used to generate an initial navigation control strategy for the target operation area based on the target map data; the control terminal is also used to receive a second control instruction triggered by the target user to control the target unmanned agricultural machinery to operate in the target operation area, obtain the initial navigation control strategy of the target operation area from the cloud server, generate a target navigation control instruction based on the initial navigation control strategy, and send the target navigation control instruction to the target unmanned agricultural machinery; the target unmanned agricultural machinery is used to perform normal operations in the target operation area according to the target navigation control instruction.

[0006] Optionally, the control terminal includes at least: a map management module and a navigation control module, wherein the map management module is used to receive a first registration instruction triggered by a target user, wherein the first registration instruction carries at least an identity identifier of the target user, and the identity identifier includes at least one of the following: an email address, a mobile phone number, and a social account; the target user is authenticated based on the identity identifier in the first registration instruction, and if the identity authentication is passed, a target Token of the target user is generated based on the identity identifier, a user ID of the target user is created, and the target Token is stored in a preset local database; a navigation control module is used to receive a second registration instruction triggered by the target user, wherein the second registration instruction carries at least an identity identifier of the target user, and the identity identifier includes at least one of the following: an email address, a mobile phone number, and a social account; the target user is authenticated based on the identity identifier in the second registration instruction, and if the identity authentication is passed, a target Token of the target user is generated based on the identity identifier, a user ID of the target user is created, and the target Token is stored in a local database.

[0007] Optionally, the map management module is further used to receive a first control instruction for creating a map of the target operation area triggered by the target user, wherein the first control instruction carries at least the target token and user ID of the target user; the target token in the first control instruction is matched with the local database, and when the matching result is a successful match, a blank map is created in the map generation software, and the target map name of the blank map is determined; a target positioning device based on real-time dynamic carrier phase difference technology is selected from multiple positioning devices for communication connection; the target positioning device is used to collect the positioning data of each of the multiple positioning points in the target operation area, and send the positioning data of each positioning point to the map management module in turn, wherein the positioning data includes at least: latitude and longitude coordinates, altitude Height, positioning status, positioning points include at least: area boundary points, obstacle points, and supply points; the map management module is also used to perform dotting operations on each positioning point in a blank map according to the positioning data, and store the positioning data of each positioning point in a preset MapData array in turn until the dotting operations of all positioning points in the target operation area are completed, and the corresponding target map and target MapData array are obtained. The target map data is composed of the target map and the target MapData array, wherein the MapData array includes at least: a Bound array for storing the positioning data of the area boundary points, an Obstacles array for storing the positioning data of the obstacle points and the corresponding custom obstacle radius, and a Supply Points array for storing the positioning data of the supply points; the association between the user ID and the target map data and the target map name corresponding to the target operation area is packaged into a JSON data packet, and the JSON data packet is uploaded to the cloud server.

[0008] Optionally, the cloud server is further configured to analyze the target map data using a preset path planning algorithm, generate an initial navigation control strategy for the target operation area, and store the user ID, the target map data corresponding to the target operation area, the target map name, and the initial navigation control strategy in a preset cloud database, wherein the initial navigation control strategy includes at least: the position coordinates and recommended driving speeds of a plurality of path points that the target unmanned agricultural machine passes through in sequence according to the global planning path within the target operation area, and the expression for the recommended driving speed of each path point is:

[0009]

[0010] Where, v target Indicates the recommended driving speed for the waypoint, v max Indicates the preset maximum permissible speed, R current Indicates the turning radius of the curve where the path point is located, d obstacle Indicates the custom obstacle radius corresponding to the obstacle point adjacent to the path point, R min Indicates the minimum turning radius, and R min =L / sin(δ max ), L represents the wheelbase of the target unmanned agricultural machine, δ max It represents the maximum value of the steering angle of the target unmanned agricultural machine, and the steering angle expression is: δ=arctan(2eL / L a 2 ), e represents the lateral deviation from the path point to the next path point, L a Indicates the look-ahead distance from a waypoint to the next waypoint.

[0011] Optionally, the navigation control module is used to receive the unmanned agricultural machinery binding instruction triggered by the target user, and select the target unmanned agricultural machinery to be bound from multiple unmanned agricultural machinery within a preset range, wherein the unmanned agricultural machinery binding instruction carries at least the user ID of the target user; obtain the media access control MAC address of the target unmanned agricultural machinery, and send the user ID of the target user and the MAC address of the target unmanned agricultural machinery to the cloud server; the cloud server is also used to store the association record of the user ID of the target user and the MAC address of the target unmanned agricultural machinery in a preset device usage record table, and feedback the corresponding binding feedback instruction to the navigation control module, wherein the binding feedback instruction is used to instruct The navigation control module is further configured to receive, when the target user and the target unmanned agricultural machinery are bound, a second control instruction triggered by the target user to control the target unmanned agricultural machinery to operate within a target operating area, wherein the second control instruction carries at least the user ID of the target user and the target map name corresponding to the target operating area; obtain, from the cloud server, an initial navigation control strategy corresponding to the target operating area based on the user ID and the target map name corresponding to the target operating area in the second control instruction; generate an initial target navigation control instruction based on the initial navigation control strategy, and send the initial target navigation control instruction to the target unmanned agricultural machinery.

[0012] Optionally, the target unmanned agricultural machinery includes at least: an operating status acquisition unit, an environmental perception data acquisition unit, a control unit, and an implementation unit, wherein the operating status acquisition unit is used to collect the first operating status information of the target unmanned agricultural machinery when it travels to the current path point according to the initial navigation control strategy, wherein the first operating status information includes at least: real-time position coordinates and real-time driving speed; the environmental perception data acquisition unit is used to collect the real-time environmental perception data of the target unmanned agricultural machinery when it travels to the current path point according to the initial navigation control strategy; the control unit is used to generate a target navigation control instruction based on the initial navigation control strategy, the real-time environmental perception data and the first operating status information, and send the target navigation control instruction to the implementation unit; the implementation unit is used to control the target unmanned agricultural machinery to perform normal operations in the target operation area according to the target navigation control instruction.

[0013] Optionally, the control unit is also used to determine whether there is a dynamic obstacle when the target unmanned agricultural machinery is traveling to the next path point of the current path point based on real-time environmental perception data, and if so, determine the second operating state information of the dynamic obstacle; use the path adjustment decision model to analyze the initial navigation control strategy, the first motion state information and the second operating state information to obtain the real-time navigation control strategy when the target unmanned agricultural machinery is traveling to the next path point of the current path point.

[0014] Optionally, the real-time environmental perception data includes at least: an image frame sequence, a laser point cloud sequence, wherein the control unit is further used to perform a first preprocessing on the image frame sequence and a second preprocessing on the laser point cloud sequence, wherein the first preprocessing includes at least one of the following: graying, denoising, and white balancing, and the second preprocessing includes at least one of the following: filtering and downsampling; using a pre-trained target detection model to analyze each image frame in the preprocessed image frame sequence to determine the type of object in each image frame, wherein the type includes at least: crop ridges and obstacles; when the type of object in multiple image frames in the image frame sequence is an obstacle, using the preprocessed laser point cloud sequence to determine the state of the obstacle, wherein the state includes: a stationary state or a moving state; when the state of the obstacle is a moving state, determining whether there is a dynamic obstacle in the process of the target unmanned agricultural machinery traveling to the next path point of the current path point.

[0015] Optionally, the laser point cloud sequence includes at least: multiple frames of laser point cloud data and relative motion between frames, wherein the control unit is further used to analyze the preprocessed laser point cloud sequence using a feature point matching method to determine the position and posture information of the target unmanned agricultural machinery at the current path point; cluster the laser point cloud sequence into multiple point cloud clusters according to the obstacles based on the position and posture information, wherein each point cloud cluster corresponds to one obstacle; determine the degree of difference between the center of mass and the normal of the obstacle in the multiple frames of laser point cloud data based on each point cloud cluster, and determine the difference function based on the relative motion between frames and the degree of difference between the center of mass and the normal of each obstacle:

[0016]

[0017] Where, T mn represents the difference function between the mth obstacle and the nth obstacle, Represents the three-dimensional homogeneous coordinate information of the center of mass of the n-th obstacle in the k-1-th frame, represents the relative motion between the k-1th frame and the kth frame, express The rotation matrix of Represents the normal of the nth obstacle in the kth frame; constructs the corresponding correlation matrix based on the difference function in, represents the m obstacles detected in the k-1th frame, represents the n obstacles detected in the kth frame, where m = [1, M] and n = [1, N]. The correlation matrix is ​​used to reflect the difference between each obstacle in two consecutive frames; the state of the obstacle is determined based on the correlation matrix.

[0018] Optionally, the second motion state information includes at least: a motion direction and a motion distance, wherein the control unit is further configured to calculate the motion distance of the dynamic obstacle according to the following formula:

[0019]

[0020] Where, Represents the three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the k-1th frame, Represents the three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the kth frame; based on the acquisition frequency of the lidar sensor and the movement distance of the dynamic obstacle, the movement speed and movement direction of the dynamic obstacle are determined respectively:

[0021] v=Δd*f

[0022]

[0023] Where v represents the speed of the dynamic obstacle, represents the movement direction of the dynamic obstacle, and f represents the acquisition frequency of the lidar sensor.

[0024] Optionally, the path adjustment decision model includes at least: an embedding layer, a fully connected layer, a deep Q network layer, a neural network layer including a bidirectional long short-term memory network, and a strategy generation layer, wherein the control unit is further used to use the embedding layer to perform vector analysis on the initial navigation control strategy, the first motion state information, and the second operating state information, respectively, to obtain the corresponding path feature vector, the first state vector, and the second state vector, respectively; perform feature fusion on the path feature vector, the second state vector, and the second state vector through the fully connected layer to obtain an input vector; input the input vector into the deep Q network layer, calculate the predicted Q values ​​corresponding to different navigation control strategies, and select the navigation control strategy corresponding to the maximum predicted Q value as the preliminary decision result; input the preliminary decision result into the neural network layer including a bidirectional long short-term memory network for decision optimization to obtain an optimized decision result; input the optimized decision result into the strategy generation layer for strategy generation, and output the corresponding real-time navigation control strategy, wherein the real-time navigation control strategy includes at least: a driving speed adjustment value and a steering angle adjustment value.

[0025] Optionally, the implementation unit includes at least: a servo and a drive motor, wherein the control unit is further used to convert the real-time navigation control strategy into binary data, and parse the binary data into a pulse width modulation (PWM) signal, and send it to the implementation unit; the implementation unit is also used to control the servo and the drive motor according to the PWM signal.

[0026] Optionally, the control terminal is used to receive a return control instruction triggered by the target user and send the return control instruction to the return control module, wherein the return control instruction carries a preset return destination; the control unit is used to return in the target operating area from the current path point according to the return control instruction until the return destination is reached, and send a return completion instruction to the control terminal.

[0027] Optionally, the control terminal is used to receive a stop driving control instruction triggered by a target user and send the stop driving control instruction to the control unit; the control unit is used to trigger mechanical braking and cut off power according to the stop driving control instruction.

[0028] Optionally, the control terminal is used to receive human remote control instructions triggered by the target user and send the human remote control instructions to the control unit; the control unit is used to pause receiving target navigation control instructions and control the target unmanned agricultural machinery to perform normal operations in the target operation area according to the human remote control instructions.

[0029] Optionally, the end-to-end remote-controlled unmanned agricultural machinery operation system also includes: an encryption module, wherein the encryption module is used to encrypt control instructions using a first encryption algorithm, wherein the control instructions include at least: a first control instruction, a second control instruction, a target navigation control instruction, a return control instruction, a stop control instruction, and a manual remote control control instruction; and to encrypt non-real-time data using a second encryption algorithm, wherein the non-real-time data includes at least: target map data and an initial navigation control strategy; wherein the encryption accuracy of the first encryption algorithm is higher than that of the second encryption algorithm.

[0030] In an embodiment of the present application, a control terminal can receive a first control instruction triggered by a target user to create a map of a target operating area, utilize a low-cost and high-precision target positioning device to collect positioning data of the target operating area, create target map data of the target operating area, and store the target map data on a cloud server. The control terminal can also receive a second control instruction triggered by the target user to control a target unmanned agricultural machine to operate within the target operating area, obtain an initial navigation control strategy generated based on the target map data of the target operating area from the cloud server, generate a target navigation control instruction based on the initial navigation control strategy, and issue it to the target unmanned agricultural machine, so that the target unmanned agricultural machine can perform autonomous navigation operations within the target operating area according to the target navigation control instruction. That is, through the technical effects of high-precision path planning, autonomous navigation control, and remote secure transmission of autonomous navigation control instructions, the purpose of optimizing the unmanned agricultural machine operation process, improving operation efficiency, and ensuring operational safety is achieved, thereby resolving the technical problems of poor control flexibility and control reliability in related remote control unmanned agricultural machine operation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0032] Figure 1 1 is a schematic structural diagram of an optional end-to-end remote control unmanned agricultural machinery operation system according to an embodiment of the present application;

[0033] Figure 2 It is a detailed structural diagram of another optional end-to-end remote control unmanned agricultural machinery operation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0035] It should be noted that the terms "first", "second", etc. in the specification, claims, and drawings of the present application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0036] In order to better understand the embodiments of the present application, some nouns or terms that appear in the description of the embodiments of the present application are first translated and explained as follows:

[0037] JWT (JSON Web Token): is a lightweight authentication mechanism that defines a compact and independent way to securely transmit information between different systems using JSON objects. It consists of three parts: a header, a payload, and a signature. The header typically contains the token type and the signature algorithm; the payload contains claims, which are information about the user and permissions; and the signature is encrypted using a specified algorithm by combining the encoded header and payload with a key.

[0038] Real-time kinematic (RTK) technology is a method for measuring the difference of carrier phase observations between two measurement stations in real time. The carrier phase collected by the base station is sent to the user receiver for coordinate difference calculation to provide the three-dimensional positioning results of the measurement station in the specified coordinate system with centimeter-level accuracy.

[0039] DTLS (Datagram Transport Layer Security) protocol: It is an extension of the TLS (Transport Layer Security) protocol based on UDP (User Datagram Protocol), providing secure communication while retaining the low latency characteristics of UDP.

[0040] SRTP (Secure Real-time Transport Protocol): is an extension of the RTP (Real-time Transport Protocol) protocol, designed to provide security mechanisms such as data encryption, message authentication, integrity protection, and replay protection. SRTP uses a symmetric encryption algorithm to encrypt the payload of the RTP data packet and uses a signature algorithm to provide integrity protection and message authentication.

[0041] WebRTC (Web Real-Time Communications) is a real-time communication technology that allows web applications or sites to establish peer-to-peer connections between browsers without the help of an intermediary to achieve the transmission of video streaming and / or beverages or other arbitrary data.

[0042] HTTP (Hypertext Transfer Protocol) protocol: is a client-to-server request-response protocol that usually runs on top of TCP (Transmission Control Protocol), which specifies what kind of messages the client may send to the server and what kind of responses it may get. TLS1.3 (Transport Layer Security Protocol version 1.3): is the latest version of the TLS protocol, used to provide secure communications on computer networks. Compared with previous versions, TLS1.3 has significant improvements in security and performance, which are specifically reflected in a faster handshake process (TLS1.3 reduces the number of round trips in the handshake process, thereby speeding up the establishment of connections), stronger security (TLS1.3 removes insecure encryption algorithms and protocols and adopts stronger encryption methods), forward security (TLS1.3 enables forward security by default, ensuring that even if the key is leaked, past communication content cannot be decrypted), etc.

[0043] AES (Advanced Encryption Standard): A standard symmetric encryption algorithm used to protect electronic data. AES supports key lengths of 128, 192, and 256 bits, with AES-128 being the most commonly used. It uses a 128-bit (16-byte) key for encryption and decryption. AES is a block cipher, operating on 128-bit (16-byte) blocks of data.

[0044] Example 1

[0045] According to an embodiment of the present application, an end-to-end remote control method for an unmanned agricultural machinery operation system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Figure 1 FIG. 1 is a structural diagram of an end-to-end remote control unmanned agricultural machinery operation system 10 provided according to an embodiment of the present application. Figure 1 As shown, the system 10 includes at least: a control terminal 11, a target positioning device 12, a cloud server 13, and a target unmanned agricultural machine 14, and the interaction process between these four parts is as follows:

[0047] Step 1: The control terminal 11 first receives the first control instruction triggered by the target user to create a map of the target operation area, uses the target positioning device 12 to collect the positioning data of the target operation area to create the target map data of the target operation area, and stores the target map data on the cloud server 13, wherein the target positioning device is in communication with the control terminal.

[0048] Step 2: The cloud server 13 generates an initial navigation control strategy for the target operation area based on the target map data.

[0049] Step 3: The control terminal 11 then receives the second control instruction triggered by the target user to control the target unmanned agricultural machinery 14 to operate in the target operation area, obtains the initial navigation control strategy of the target operation area from the cloud server 13, generates the target navigation control instruction based on the initial navigation control strategy, and sends the target navigation control instruction to the target unmanned agricultural machinery 14.

[0050] Step 4: The target unmanned agricultural machine 14 performs normal operations in the target operation area according to the target navigation control instructions.

[0051] Through the above interaction process, it can be seen that the embodiment of the present application integrates the image transmission function in the control terminal, avoiding the deployment of an additional expensive image transmission module in the system and reducing system costs. Secondly, the unmanned agricultural machinery can autonomously complete the operation task according to the initial navigation control strategy generated by the cloud server, and at the same time has the ability to avoid obstacles and adjust the vehicle speed in real time, improving operation efficiency and safety. This effectively solves the problems of high cost and complex operation existing in related technologies, provides a more efficient, stable and secure remote control method for unmanned agricultural machinery operations, and significantly improves the intelligence level and operation efficiency of unmanned agricultural machinery operations.

[0052] The following will be combined Figure 2 The detailed structural diagram shown illustrates the functions of each module of the end-to-end remote control unmanned agricultural machinery operation system 10.

[0053] Before controlling an unmanned agricultural machine, detailed environmental modeling of the operating area is required, including identifying field boundaries, obstacle locations, crop distribution, and other factors. This modeling information forms the basis for the unmanned agricultural machine's autonomous navigation. Therefore, the control terminal 11 includes at least a map management module 111, which can be understood as "map generation software" installed on the control terminal 11.

[0054] In order to prevent the system from being affected by malicious operations due to unauthorized access, the target user needs to register on this module in advance before using the map management module 111, so as to ensure that only authorized users can access and use it.

[0055] Specifically, the target user's registration and identity verification process on the map management module 111 includes:

[0056] First, a first registration instruction triggered by a target user is received, wherein the first registration instruction carries at least the identity identifier of the target user, and the identity identifier includes but is not limited to: email address, mobile phone number, social account, etc.

[0057] Then, the target user is authenticated based on the identity identifier in the first registration instruction. Among them, if the identity identifier is a mobile phone number, the system can call the SMS gateway API to send a text message containing a verification code (such as a combination of four or six digits / letters) to the target user's mobile phone number. The target user can enter the received verification code on the control terminal within the preset validity period, and then the system verifies the correctness of the verification code to authenticate the target user; if the identity identifier is an email address, the system can send an email containing a confirmation link to the target user's email address, where the link usually includes a randomly generated token, and then the target user can click the link and be redirected to the confirmation page or directly authenticate the identity; if the identity identifier is a social account, the open API of the social platform can be called to request user authorization, and the social platform will ask the target user in a pop-up window whether he agrees to share necessary personal information (such as nickname, avatar, social media ID, etc.) to the map management module 111. If he agrees, an authorization code can be fed back to the map management module 111. The map management module 111 can use the authorization code to request an access token or user information from the social platform. After verifying the authorization code, the social platform provides an access token or directly returns user information.

[0058] If authentication is successful, a target token is generated for the target user based on the identity. The identity can be processed using JWT (JSON Web Token), so the generated target token encapsulates the target user's identity. At the same time, a (unique) user ID for the target user is created.

[0059] Finally, the target Token is stored in the preset local database.

[0060] Among them, the target Token obtained after the target user is registered and authenticated is stored in the local database so that when the user makes subsequent operation requests (such as map editing), he can directly carry the corresponding Token in the request without having to authenticate the identity through the network request again, thereby improving the operation reception speed and reducing network delay.

[0061] Furthermore, after completing the above registration and identity verification, the map management module 111 may generate map data of the operation area according to the following steps, including:

[0062] Step 1: The map management module 111 first receives a first control instruction triggered by a target user to create a map of a target operation area, wherein the first control instruction carries at least a target token and a user ID of the target user.

[0063] Step 2: The map management module 111 matches the target Token in the first control instruction with the local database. If the matching result is successful, a blank map is created in the map generation software and a target map name of the blank map is determined.

[0064] Specifically, if the match result is successful, it indicates that the user's identity has been verified and the next step is allowed, including: the target user clicking the "New Map" button in the map generation software. The map generation software then uses the Universally Unique Identifier (UUID) to generate a temporary map ID and target map name mapName, and then jumps to the "dot interface" in the map generation software with this temporary map ID and target map name mapName. It should be noted that the custom target map name mapName is used to verify the uniqueness of the map, so duplicate target map names are not allowed.

[0065] Step 3: Select a target positioning device based on real-time dynamic carrier phase difference technology from multiple positioning devices for communication connection.

[0066] Specifically, the third step is to use the wireless communication module to perform device scanning in the "dot interface" of the map management module 111 to find available positioning devices in the surrounding area. In the list of scanned devices, the target user can select the target positioning device 12 according to the device name, service UUID or other identifiers. For example, a positioning device with "RTK" in the device name can be selected for high-precision positioning. After the target user selects the target positioning device 12 to be connected, the map management module 111 can communicate with the target positioning device 12. If the distance between the target positioning device 12 and the map management module 111 is short, the two can be connected via Bluetooth. If the distance between the target positioning device 12 and the map management module 111 is far, the two can be connected via a 4G / 5G network.

[0067] Step 4: The target positioning device 12 collects positioning data of each of the multiple positioning points in the target operation area, and sends the positioning data of each positioning point to the map management module 111 in sequence.

[0068] Among them, positioning points include at least: area boundary points, obstacle points (i.e., trees, stones, telephone poles or any other physical obstacles in the operating area that hinder the normal operation of unmanned agricultural machinery), and supply points (i.e., points where unmanned agricultural machinery needs to be resupplied or maintained during operation, such as refueling, charging or parts replacement locations). In addition, the positioning data includes at least: longitude and latitude coordinates (based on the WGS84 coordinate system), altitude, and positioning status. If the positioning status of a point is displayed as "FIX", it means that the point has been successfully positioned; if the positioning status of a point is displayed as "NO FIX", it means that the positioning of the point has failed.

[0069] Step 5: The map management module 111 monitors the serial port data sent by the target positioning device 12, and parses it through the NAME-0183 protocol to extract the positioning data of each positioning point; then, based on the positioning data, each positioning point is marked in turn in the blank map, and the positioning data of each positioning point is stored in the preset MapData array in turn, until the marking operation of all positioning points in the target operation area is completed, and the corresponding target map and target MapData array are obtained. The target map data is composed of the target map and the target MapData array.

[0070] The MapData array includes at least: a Bound array for storing the location data of area boundary points; an Obstacles array for storing the location data of obstacle points and their corresponding customized obstacle radius; and a SupplyPoints array for storing the location data of supply points. Specifically, when the map management module 111 performs a dotting operation on a blank map based on the location data of area boundary points, it can store the location data of each completed area boundary point in the Bound array in order of dotting. For example, the first dot is displayed as "Area Boundary Point 1," the second as "Area Boundary Point 2," and so on. Furthermore, when the map management module 111 performs a dotting operation on a blank map based on the location data of obstacle points, the target user can customize the obstacle radius, thereby setting an obstacle object on the blank map. The relevant data of the set obstacle object (obstacle point and obstacle radius) is stored in the Obstacles array. Similarly, when the map management module 111 performs a dotting operation on a blank map based on the location data of supply points, it can store the location data of each completed supply point in the SupplyPoints array in order of dotting.

[0071] Step 6: The map management module 111 packages the association relationship between the user ID and the target map data corresponding to the target operation area and the target map name into a JSON data packet, and uploads the JSON data packet to the cloud server 13.

[0072] The JSON data packet may also include a generation timestamp of the target map data and other identification information, which is not specifically limited in this application. Furthermore, the map management module 111 may upload the JSON data packet to the cloud server 13 via the HTTP protocol.

[0073] Furthermore, the cloud server 13 receives the JSON data packet and parses the data packet to obtain target map data of the target operation area, and analyzes the target map data using a preset path planning algorithm to generate an initial navigation control strategy for the target operation area.

[0074] The path planning algorithm can take into account many factors, such as the target operating area's boundary points, obstacles, and supply points, to generate the optimal initial navigation control strategy. The initial navigation control strategy includes at least the position coordinates (i.e., ordered coordinate points) of the multiple path points that the target unmanned agricultural machine passes through in the target operating area along the global planning path, as well as the recommended driving speed for each path point generated by combining the path characteristics of the global planning path and the characteristics of the unmanned agricultural machine. Therefore, the expression for the recommended driving speed for each path point in the global planning path is:

[0075]

[0076] Where, v target Indicates the recommended driving speed for the waypoint; v max Indicates the preset maximum permissible speed, which can be set according to the actual application scenario and the parameters of the unmanned agricultural machinery; R current Indicates the turning radius of the curve where the waypoint is located, which can be obtained from the target map data; d obstacle Indicates the custom obstacle radius corresponding to the obstacle point adjacent to the path point, which can also be obtained from the target map data; R min Indicates the minimum turning radius, and R min =L / sin(δ max ), L represents the wheelbase of the target unmanned agricultural machine, δ max It represents the maximum value of the steering angle of the target unmanned agricultural machine, and the steering angle expression is: δ=arctan(2eL / L a 2 ), e represents the lateral deviation from the path point to the next path point, L a Indicates the look-ahead distance from a waypoint to the next waypoint.

[0077] Furthermore, the cloud server 13 feeds back the initial navigation control strategy corresponding to the target operating area to the map management module 111. The map management module 111 can update the target map data (including the target MapData array and the target map) based on the initial navigation control strategy, including: calling a map development tool to render a path on the target map to update the target map; the target MapData array also includes a Path array for storing path data. Therefore, the location coordinates of each path point on the global planning path and the recommended driving speed can be stored in the Path array to update the target MapData array. The map management module 111 re-uploads the updated target map data to the cloud database of the cloud server 13 via the HTTP protocol.

[0078] Therefore, the cloud database stores relevant information such as user ID, target map data corresponding to the target operation area, target map name, and initial navigation control strategy.

[0079] It should be noted that if the target user needs to modify the target map data, the map management module 111 can receive a map modification instruction triggered by the target user, wherein the map modification instruction carries at least the target user's user ID and the target map name, and then obtain the target map data associated with the user ID and the target map name from the cloud database of the cloud server 13; perform the corresponding modification operation on the target map data, and re-upload the modified target map data to the cloud server 13. If the target user needs to delete the target map data, the cloud server 13 can directly receive a map deletion instruction triggered by the target user through the control terminal, wherein the map modification instruction carries at least the target user's user ID and the target map name, and then query the cloud database of the cloud server 13 for the target map data associated with the user ID and the target map name for deletion.

[0080] When controlling the autonomous navigation operation of the unmanned agricultural machinery, it is also necessary to issue accurate operation instructions to the unmanned agricultural machinery, such as starting operation, pausing, turning, speed adjustment, etc., to guide it to perform operations according to the established path and strategy. Therefore, the control terminal 11 also includes a navigation control module 112, and the navigation control module 112 can be understood as the "automatic navigation software" installed on the control terminal 11.

[0081] Likewise, in order to prevent the system from being affected by malicious operations due to unauthorized access, the target user needs to register on the navigation control module 112 in advance before using it, thereby ensuring that only authorized users can access and use it.

[0082] Specifically, the target user's registration and identity verification process on the navigation control module 112 includes:

[0083] First, a second registration instruction triggered by the target user is received, wherein the first registration instruction carries at least the target user's identity identifier and user ID, and the identity identifier includes but is not limited to: email address, mobile phone number, social account, etc.

[0084] Then, the target user is authenticated based on the identity identifier in the second registration instruction, where the identity identifier also includes but is not limited to: email address, mobile phone number, social account. Therefore, the navigation control module 112 can use the same or similar mechanism as the map management module 111 to authenticate the target user.

[0085] If the authentication is successful, a target token for the target user is generated based on the identity identifier. The target token also encapsulates the user ID and other necessary permission information. At the same time, a (unique) user ID for the target user is created.

[0086] Finally, the target Token and user ID are stored in the local database.

[0087] It should be noted that the map management module 111 and the navigation control module 112 share the same database to achieve data sharing, that is, the target user can use his or her own user ID to query the corresponding user information, such as passwords, map data, etc., in the map management module 111 and the navigation control module 112 respectively, to ensure that the user can seamlessly switch between different functional modules. In addition, the target Token generated by the registration and identity authentication of the target object in the map management module 111 and the navigation control module 112 is consistent. This allows the target Token to use the user Token as an identity credential when the user switches between different functional modules (for example, from the map management module 111 to the navigation control module 112), without the need to re-login or verify the identity, thereby greatly improving the user experience and ensuring smoother operation.

[0088] Furthermore, after completing the above registration and identity verification, the navigation control module 111 may issue control instructions to the target unmanned agricultural machine according to the following steps, including:

[0089] Step 1: The navigation control module 112 first receives the unmanned agricultural machinery binding instruction triggered by the target user, and selects a target unmanned agricultural machinery to be bound from multiple unmanned agricultural machinery within a preset range, wherein the unmanned agricultural machinery binding instruction carries at least the user ID of the target user.

[0090] Step 2: The navigation control module 112 obtains the MAC address of the target unmanned agricultural machine and sends the user ID of the target user and the MAC address of the target unmanned agricultural machine to the cloud server 13 .

[0091] Step 3: The cloud server 13 stores the association record between the user ID of the target user and the MAC address of the target unmanned agricultural machinery in a preset device usage record table, wherein the device usage record table includes usage records of multiple users using the unmanned agricultural machinery, and each usage record includes but is not limited to: user ID, vehicle name, vehicle identification code, vehicle MAC address, vehicle width, vehicle length, safe expansion distance, minimum turning radius and other information.

[0092] Step 4: The cloud server 13 feeds back a corresponding binding feedback instruction to the navigation control module, wherein the binding feedback instruction is used to indicate whether the target user and the target unmanned agricultural machine are successfully bound.

[0093] Step 5: When the target user and the target unmanned agricultural machine are bound, the navigation control module 112 receives a second control instruction triggered by the target user to control the target unmanned agricultural machine to operate within the target operation area. The second control instruction carries at least the user ID of the target user and the target map name corresponding to the target operation area.

[0094] Step 6: The navigation control module 112 obtains the initial navigation control strategy corresponding to the target operation area from the cloud server 13 based on the user ID in the second control instruction and the target map name corresponding to the target operation area; generates an initial target navigation control instruction based on the initial navigation control strategy, and sends the initial target navigation control instruction to the target unmanned agricultural machinery 14.

[0095] Furthermore, the target unmanned agricultural machine 14 relies on its own components and the cloud server to work together to achieve real-time autonomous navigation of the target unmanned agricultural machine. The target unmanned agricultural machine 14 includes at least: an operating status acquisition unit 141, an environmental perception data acquisition unit 142, a control unit (i.e., an "industrial computer") 143, and an implementation unit (i.e., a "bottom-level main control board") 144. The target unmanned agricultural machine can generate a real-time navigation control strategy at preset time intervals through the following steps to achieve autonomous navigation:

[0096] First, the operating status collection unit 141 can collect first operating status information of the target unmanned agricultural machine as it travels to the current path point according to the initial navigation control strategy. The first operating status information includes at least real-time location coordinates and real-time travel speed. Simultaneously, the environmental perception data collection unit 142 collects real-time environmental perception data of the target unmanned agricultural machine as it travels to the current path point according to the initial navigation control strategy.

[0097] Then, the control unit 143 generates a target navigation control instruction according to the initial navigation control strategy, the real-time environment perception data and the first operating state information, and sends the target navigation control instruction to the implementation unit 144;

[0098] Finally, the implementation unit 144 controls the target unmanned agricultural machine to perform normal operations in the target operation area according to the target navigation control instruction.

[0099] The operating status acquisition unit 141 may be an RTK positioning device for acquiring real-time operating status information of the target unmanned agricultural machinery.

[0100] The environmental perception data acquisition unit 142 can be a camera and a lidar sensor. The camera, as the core optical sensor for environmental perception, can transmit the image stream captured by the camera via the WebRTC data channel. The image stream can be encoded using H.264 encoding, and the acquisition frame rate can be set based on the actual application scenario to balance image quality and bandwidth. The lidar sensor is one of the core sensors for obstacle detection and surrounding environment perception. It can also transmit the laser point cloud sequence captured by the lidar sensor via the WebRTC data channel. The laser point cloud sequence is binary data.

[0101] Specifically, the control unit 143 is the "core brain" of the target unmanned agricultural machine 14, and it can undertake key tasks such as data processing, decision-making and algorithm operation. Therefore, the control unit 143 can generate a real-time navigation control strategy according to the following steps, including:

[0102] Step S1: Determine whether there are dynamic obstacles when the target unmanned agricultural machine 14 is traveling to the next path point of the current path point based on real-time environmental perception data. The real-time environmental perception data includes at least: an image frame sequence and a laser point cloud sequence.

[0103] Specifically, the implementation of step S1 includes:

[0104] Step S11: Perform a first preprocessing on the image frame sequence and a second preprocessing on the laser point cloud sequence. The first preprocessing includes, but is not limited to, grayscaling (converting color images to grayscale images), denoising (eliminating randomly distributed noise points in the image to improve image quality), and white balancing (adjusting the true color reproduction of white points in the image). The second preprocessing includes, but is not limited to, filtering (eliminating invalid points and outliers to improve the purity of the point cloud data) and downsampling (reducing the density of the point cloud data).

[0105] Step S12: Analyze each image frame in the pre-processed image frame sequence using a pre-trained object detection model to determine the type of object in each image frame, wherein the type includes at least: crop ridges and obstacles.

[0106] Step S13: If the object in multiple frames of the image frame sequence is an obstacle, the state of the obstacle is determined using the pre-processed laser point cloud sequence, wherein the laser point cloud sequence includes at least multiple frames of laser point cloud data and relative motion between frames.

[0107] Therefore, the control unit 143 may determine the state of the obstacle according to the following steps, including:

[0108] Step 1: Use the feature point matching method to analyze the preprocessed laser point cloud sequence to determine the position information of the target unmanned agricultural machinery at the current path point.

[0109] Step 2: Based on the pose information, the laser point cloud sequence is clustered into multiple point cloud clusters according to the obstacles, where each point cloud cluster corresponds to one obstacle.

[0110] Step 3: Determine the degree of difference between the center of mass and normal of the obstacle in the multi-frame laser point cloud data based on each point cloud cluster, and determine the difference function based on the relative motion between frames and the degree of difference between the center of mass and normal of each obstacle:

[0111]

[0112] Where, T mn represents the difference function between the mth obstacle and the nth obstacle, Represents the three-dimensional homogeneous coordinate information of the center of mass of the n-th obstacle in the k-1-th frame, represents the relative motion between the k-1th frame and the kth frame, express The rotation matrix of Represents the normal of the nth obstacle in the kth frame;

[0113] Step 4: Construct the corresponding correlation matrix based on the difference function in, represents the m obstacles detected in the k-1th frame, represents the n obstacles detected in the kth frame, where m = [1, M] and n = [1, N]. The correlation matrix is ​​used to reflect the difference between each obstacle in two consecutive frames.

[0114] Step 5: Determine the state of the obstacle based on the correlation matrix, where the state includes: static state or moving state.

[0115] Specifically, if the difference between any obstacle in two consecutive frames corresponding to the correlation matrix is ​​less than a preset threshold value, it indicates that the state of the obstacle is in motion; conversely, if the difference between any obstacle in two consecutive frames corresponding to the correlation matrix is ​​greater than the preset threshold value, it indicates that the state of the obstacle is stationary.

[0116] In step S14, when the state of the obstacle is a moving state, it is determined that there is a dynamic obstacle in the process of the target unmanned agricultural machine traveling to the next path point of the current path point.

[0117] Step S2: If the obstacle exists, determine the second movement state information of the dynamic obstacle, wherein the second movement state information at least includes: movement direction and movement distance.

[0118] Therefore, in the technical solution provided in step S2 above, the control unit 143 may determine the second operating state information of the dynamic obstacle according to the following steps, including:

[0119] First, calculate the movement distance of the dynamic obstacle according to the following formula:

[0120]

[0121] Where, Represents the three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the k-1th frame, The three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the k-th frame;

[0122] Then, based on the acquisition frequency of the lidar sensor and the movement distance of the dynamic obstacle, the movement speed and direction of the dynamic obstacle are determined respectively:

[0123] v=Δd*f

[0124]

[0125] Where v represents the speed of the dynamic obstacle, represents the movement direction of the dynamic obstacle, and f represents the acquisition frequency of the lidar sensor.

[0126] In step S3, the path adjustment decision model is used to analyze the initial navigation control strategy, the first motion state information, and the second operation state information to obtain a real-time navigation control strategy when the target unmanned agricultural machine travels to the next path point of the current path point.

[0127] Specifically, the path adjustment decision model includes at least: an embedding layer, a fully connected layer, a deep Q network layer, a neural network layer including a bidirectional long short-term memory network, and a strategy generation layer. Therefore, the control unit 143 can use the embedding layer to perform vector analysis on the initial navigation control strategy, the first motion state information, and the second operating state information, respectively, to obtain the corresponding path feature vector, first state vector, and second state vector. The fully connected layer then performs feature fusion on the path feature vector, the second state vector, and the second state vector to obtain an input vector. The input vector is then input into the deep Q network layer to calculate the predicted Q values ​​corresponding to different navigation control strategies, and the navigation control strategy corresponding to the maximum predicted Q value is selected as the preliminary decision result. The preliminary decision result is then input into the neural network layer including a bidirectional long short-term memory network for decision optimization, obtaining an optimized decision result. The optimized decision result is then input into the strategy generation layer for strategy generation, which outputs the corresponding real-time navigation control strategy, wherein the real-time navigation control strategy includes at least a driving speed adjustment value and a steering angle adjustment value.

[0128] It should be noted that in order to ensure the normal operation of the control unit 143, the priority order of the tasks executed by the control unit 143 is set in the embodiment of the present application, among which real-time tasks (such as the generation of real-time navigation control strategies) are the first priority tasks, computing tasks (such as obstacle identification, obstacle status analysis) are the second priority tasks, and network tasks (such as clustering of laser point cloud sequences) are the third priority tasks.

[0129] Furthermore, the control unit 143 can convert the generated real-time navigation control strategy into binary data, and use the WebRTC data channel to send the binary data to the GPIO (General Purpose Input / Output) interface at a preset frequency to convert the binary data into a PWM signal and send it to the implementation unit 144.

[0130] The implementation unit 144 includes at least a steering gear 1441 and a drive motor 1442. Therefore, the implementation unit 144 can use PWM signals to control the steering gear 1441 and the drive motor 1442. The PWM pulse width determines the rotation angle of the steering gear 1441, while the PWM duty cycle determines the speed and torque of the drive motor 1442. This converts the target navigation control command output by the control unit 143 into the core mission of actual movement, thereby driving the target unmanned agricultural machine to operate normally.

[0131] In addition to the autonomous navigation function, the target unmanned agricultural machine 14 can also realize the return function, mission cancellation function, and switching to manual remote control function.

[0132] As for the return function of the target unmanned agricultural machine 14, its implementation process includes:

[0133] The control terminal 11 first receives the return control instruction triggered by the target user, and sends the return control instruction to the control unit 143, wherein the return control instruction carries a preset return destination, which can generally be set as the first path point in the global planning path.

[0134] The control unit 143 then returns to the target operating area from the current path point according to the return control command, until it reaches the return destination, and sends a return completion command to the control terminal. In other words, upon receiving the return control command, the control module 143 will first stop the current operation, activate the return mode, and generate a return control strategy at preset intervals. The specific generation process can be referred to above steps S1-S3, and will not be further described here. Once the target unmanned agricultural machine 14 reaches the return destination, the corresponding return completion command is fed back to the control terminal 11.

[0135] As for the task cancellation function of the target unmanned agricultural machine 14, its implementation process includes:

[0136] The control terminal 11 first receives a stop driving control instruction (such as an “emergency stop instruction”) triggered by a target user, and sends the stop driving control instruction to the control unit 143 .

[0137] Control unit 143 then triggers the mechanical brakes and cuts off power to the drive motor in response to the stop control command. Simultaneously, it clears temporary data stored within control unit 143, including the target map data and real-time navigation control strategy for the current task. It also stores the progress of unfinished tasks in the unmanned agricultural machine's local database, indicating the task execution status as "interrupted." The target can then choose to resume or cancel the task.

[0138] As for the switching of the target unmanned agricultural machine 14 to the manual remote control function, the implementation process includes:

[0139] The control terminal 11 first receives the manual remote control command triggered by the target user, and sends the manual remote control command to the control unit 143;

[0140] The control unit 143 then suspends receiving the target navigation control instruction to prevent conflicts between the automatic target navigation control instruction and the manual remote control instruction, and controls the target unmanned agricultural machine to perform normal operations in the target operation area according to the manual remote control instruction.

[0141] Specifically, when the target unmanned agricultural machinery 14 enters the manual remote control mode, the control terminal 11 can receive the remote control selection instruction triggered by the target user, and select the target remote control from multiple remote controls within a preset range, wherein the target remote control must have a battery charge higher than a preset threshold value; furthermore, the target remote control can send analog control signals to the control unit via different channels, such as channel 1 for steering control and channel 2 for throttle control, so as to control the target unmanned agricultural machinery to perform normal operations within the target operating area.

[0142] At the same time, the target unmanned agricultural machinery 14 can send the first operating status information and real-time environmental perception data it collects to the control terminal 11 through the cloud server 13, so that the location information of the unmanned agricultural machinery can be displayed in real time even in remote control mode.

[0143] Furthermore, to prevent the unmanned agricultural machinery from executing incorrect commands due to network or signal interference, an encryption module is provided in the end-to-end remote control unmanned agricultural machinery operation system 10. This encryption module is configured to encrypt control commands using a first encryption algorithm, wherein the control commands include at least: a first control command, a second control command, a target navigation control command, a return control command, a stop control command, and a manual remote control command; and to encrypt non-real-time data using a second encryption algorithm, wherein the non-real-time data includes at least: target map data and an initial navigation control strategy. The first encryption algorithm has a higher encryption accuracy than the second encryption algorithm.

[0144] For example, non-real-time data such as target map data and initial navigation control strategies transmitted via HTTP can be encrypted using TLS 1.3. Control commands transmitted via WebRTC can be encrypted using the SRTP algorithm of the DTLS protocol to ensure the secure transmission of real-time control commands. Information transmitted via Bluetooth can be encrypted using the AES-128 encryption algorithm to prevent near-field signal interception. Furthermore, control commands triggered by the target user can be accompanied by an incremental sequence number and millisecond-level timestamp to ensure the legitimacy of the command source.

[0145] In summary, the end-to-end remote control unmanned agricultural machinery operation system 10 provided in the embodiment of the present application can perform end-to-end reliable control of the target unmanned agricultural machinery through the control terminal by integrating functions such as high-precision map construction, intelligent operation navigation, and flexible remote operation, thereby solving the technical problems of high control cost and complex operation of traditional control systems.

[0146] In the embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0147] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0148] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0150] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An end-to-end remote control unmanned agricultural machinery operation system, characterized in that: The end-to-end remote control unmanned agricultural machinery operation system at least includes: a control terminal, a target positioning device, a cloud server, and a target unmanned agricultural machinery, wherein: The control terminal is configured to receive a first control instruction triggered by a target user to create a map of a target operation area, collect positioning data of the target operation area using the target positioning device to create target map data of the target operation area, and store the target map data on the cloud server, wherein the target positioning device is in communication with the control terminal; The cloud server is configured to generate an initial navigation control strategy for the target operating area based on the target map data; The control terminal is further configured to receive a second control instruction triggered by the target user to control the target unmanned agricultural machine to operate in the target operation area, obtain an initial navigation control strategy for the target operation area from the cloud server, generate a target navigation control instruction based on the initial navigation control strategy, and send the target navigation control instruction to the target unmanned agricultural machine; The target unmanned agricultural machine is used to perform normal operations in the target operating area according to the target navigation control instructions.

2. The system according to claim 1, wherein: The control terminal at least includes: a map management module and a navigation control module, wherein: The map management module is configured to receive a first registration instruction triggered by the target user, wherein the first registration instruction carries at least an identity identifier of the target user, and the identity identifier includes at least one of the following: an email address, a mobile phone number, or a social account; authenticate the target user based on the identity identifier in the first registration instruction, and if the identity identifier is successfully authenticated, generate a target token for the target user based on the identity identifier, create a user ID for the target user, and store the target token in a preset local database; The navigation control module is used to receive a second registration instruction triggered by the target user, wherein the second registration instruction carries at least the identity identifier of the target user, and the identity identifier includes at least one of the following: an email address, a mobile phone number, and a social account; the target user is authenticated based on the identity identifier in the second registration instruction, and if the identity authentication is passed, a target token of the target user is generated based on the identity identifier, a user ID of the target user is created, and the target token is stored in the local database.

3. The system according to claim 2, characterized in that The map management module is further configured to receive a first control instruction triggered by the target user to create a map of the target operation area, wherein the first control instruction carries at least a target token and a user ID of the target user; match the target token in the first control instruction with the local database, and if the matching result is successful, create a blank map in the map generation software and determine a target map name for the blank map; and select the target positioning device based on real-time dynamic carrier phase difference technology from a plurality of positioning devices for communication connection; The target positioning device is used to collect positioning data of each of the plurality of positioning points in the target operation area, and sequentially send the positioning data of each positioning point to the map management module, wherein the positioning data includes at least: latitude and longitude coordinates, altitude, and positioning status, and the positioning points include at least: area boundary points, obstacle points, and supply points; The map management module is further used to perform a marking operation on each of the positioning points in the blank map in sequence based on the positioning data, and store the positioning data of each positioning point in a preset MapData array in sequence until the marking operation of all positioning points in the target operation area is completed, thereby obtaining a corresponding target map and a target MapData array. The target map data is composed of the target map and the target MapData array, wherein the MapData array at least includes: a Bound array for storing the positioning data of the area boundary points, an Obstacles array for storing the positioning data of the obstacle points and the corresponding custom obstacle radius, and a Supply Points array for storing the positioning data of the supply points; the association relationship between the user ID and the target map data and target map name corresponding to the target operation area is packaged into a JSON data packet, and the JSON data packet is uploaded to the cloud server.

4. The system according to claim 1, wherein: The cloud server is further configured to analyze the target map data using a preset path planning algorithm, generate an initial navigation control strategy for the target operation area, and store the user ID, the target map data corresponding to the target operation area, the target map name, and the initial navigation control strategy in a preset cloud database, wherein the initial navigation control strategy includes at least: the position coordinates and recommended driving speeds of a plurality of path points that the target unmanned agricultural machine sequentially passes through in the target operation area according to the global planning path, and the expression for the recommended driving speed of each path point is: Where, v target Indicates the recommended driving speed for the path point, v max Indicates the preset maximum permissible speed, R current Indicates the turning radius of the curve where the path point is located, d obstacle Indicates the custom obstacle radius corresponding to the obstacle point adjacent to the path point, R min Indicates the minimum turning radius, and R min =L / sin(δ max ), L represents the wheelbase of the target unmanned agricultural machine, δ max represents the maximum value of the steering angle of the target unmanned agricultural machine, and the expression of the steering angle is: δ = arctan (2eL / L a 2 ), e represents the lateral deviation from the path point to the next path point, L a Indicates the forward distance from the path point to the next path point.

5. The system according to claim 1, wherein: The navigation control module is configured to receive an unmanned agricultural machine binding instruction triggered by the target user, and select a target unmanned agricultural machine to be bound from a plurality of unmanned agricultural machines within a preset range, wherein the unmanned agricultural machine binding instruction carries at least the user ID of the target user; obtain a media access control MAC address of the target unmanned agricultural machine, and send the user ID of the target user and the MAC address of the target unmanned agricultural machine to the cloud server; The cloud server is further configured to store the association record between the user ID of the target user and the MAC address of the target unmanned agricultural machine in a preset device usage record table, and to feed back a corresponding binding feedback instruction to the navigation control module, wherein the binding feedback instruction is used to indicate whether the target user and the target unmanned agricultural machine are successfully bound; The navigation control module is also used to receive a second control instruction triggered by the target user to control the target unmanned agricultural machinery to operate within the target operation area when the target user and the target unmanned agricultural machinery are bound, wherein the second control instruction carries at least the user ID of the target user and the target map name corresponding to the target operation area; based on the user ID in the second control instruction and the target map name corresponding to the target operation area, obtain the initial navigation control strategy corresponding to the target operation area from the cloud server; generate an initial target navigation control instruction based on the initial navigation control strategy, and send the initial target navigation control instruction to the target unmanned agricultural machinery.

6. The system according to claim 5, characterized in that The target unmanned agricultural machine at least includes: an operating status acquisition unit, an environmental perception data acquisition unit, a control unit, and an implementation unit, wherein: The operating status collecting unit is used to collect first operating status information of the target unmanned agricultural machine when it travels to the current path point according to the initial navigation control strategy, wherein the first operating status information at least includes: real-time position coordinates and real-time driving speed; The environmental perception data collection unit is used to collect real-time environmental perception data of the target unmanned agricultural machine when it travels to the current path point according to the initial navigation control strategy; The control unit is configured to generate a target navigation control instruction based on the initial navigation control strategy, the real-time environment perception data, and the first operating state information, and send the target navigation control instruction to the implementation unit; The implementation unit is used to control the target unmanned agricultural machine to perform normal operations in the target operation area according to the target navigation control instruction.

7. The system according to claim 6, characterized in that The control unit is further configured to determine, based on the real-time environmental perception data, whether there is a dynamic obstacle when the target unmanned agricultural machine is traveling to a next path point after the current path point, and if so, determine second operating state information of the dynamic obstacle; The path adjustment decision model is used to analyze the initial navigation control strategy, the first motion state information, and the second operating state information to obtain a real-time navigation control strategy when the target unmanned agricultural machine travels to a next path point of the current path point.

8. The system according to claim 7, characterized in that The real-time environment perception data includes at least: image frame sequence, laser point cloud sequence, wherein, The control unit is further used to perform a first preprocessing on the image frame sequence and a second preprocessing on the laser point cloud sequence, wherein the first preprocessing includes at least one of the following: grayscale, denoising, and white balance, and the second preprocessing includes at least one of the following: filtering and downsampling; using a pre-trained target detection model to analyze each image frame in the preprocessed image frame sequence to determine the type of object in each image frame, wherein the type includes at least: crop ridges and obstacles; when the type of object in multiple image frames in the image frame sequence is an obstacle, using the preprocessed laser point cloud sequence to determine the state of the obstacle, wherein the state includes: a stationary state or a moving state; when the state of the obstacle is the moving state, it is determined that there is a dynamic obstacle in the process of the target unmanned agricultural machinery traveling to the next path point of the current path point.

9. The system according to claim 8, characterized in that The laser point cloud sequence at least includes: multiple frames of laser point cloud data and relative motion between frames, wherein: The control unit is further configured to analyze the pre-processed laser point cloud sequence using a feature point matching method to determine the pose information of the target unmanned agricultural machinery at the current path point; cluster the laser point cloud sequence into a plurality of point cloud clusters according to the obstacles based on the pose information, wherein each point cloud cluster corresponds to one obstacle; determine the degree of difference between the center of mass and the normal of the obstacle in the multi-frame laser point cloud data based on each point cloud cluster, and determine a difference function based on the inter-frame relative motion and the degree of difference between the center of mass and the normal of each obstacle: Where, T mn represents the difference function between the mth obstacle and the nth obstacle, Represents the three-dimensional homogeneous coordinate information of the center of mass of the n-th obstacle in the k-1-th frame, represents the relative motion between the k-1th frame and the kth frame, express The rotation matrix of Represents the normal of the nth obstacle in the kth frame; constructs the corresponding correlation matrix based on the difference function in, represents the m obstacles detected in the k-1th frame, represents n obstacles detected in the kth frame, where m = [1, M] and n = [1, N]. The association matrix is ​​used to reflect the difference between each obstacle in two consecutive frames; the state of the obstacle is determined based on the association matrix.

10. The system according to claim 7, wherein: The second motion state information at least includes: motion direction and motion distance, wherein: The control unit is further configured to calculate the movement distance of the dynamic obstacle according to the following formula: Where, Represents the three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the k-1th frame, The three-dimensional homogeneous coordinate information of the center of mass of the obstacle in the kth frame is represented; the movement speed and movement direction of the dynamic obstacle are determined according to the acquisition frequency of the lidar sensor and the movement distance of the dynamic obstacle: v=Δd*f Where v represents the speed of the dynamic obstacle, represents the moving direction of the dynamic obstacle, and f represents the acquisition frequency of the lidar sensor.

11. The system according to claim 7, wherein: The path adjustment decision model at least includes: an embedding layer, a fully connected layer, a deep Q network layer, a neural network layer including a bidirectional long short-term memory network, and a strategy generation layer, wherein: The control unit is further used to use the embedding layer to perform vector analysis on the initial navigation control strategy, the first motion state information and the second operating state information, respectively, to obtain the corresponding path feature vector, first state vector and second state vector, respectively; perform feature fusion on the path feature vector, the second state vector and the second state vector through the fully connected layer to obtain an input vector; input the input vector into the deep Q network layer, calculate the predicted Q values ​​corresponding to different navigation control strategies, and select the navigation control strategy corresponding to the maximum predicted Q value as the preliminary decision result; input the preliminary decision result into the neural network layer including the bidirectional long short-term memory network for decision optimization to obtain an optimized decision result; input the optimized decision result into the strategy generation layer for strategy generation, and output the corresponding real-time navigation control strategy, wherein the real-time navigation control strategy includes at least: a driving speed adjustment value and a steering angle adjustment value.

12. The system according to claim 10, wherein: The implementation unit at least includes: a steering gear and a drive motor, wherein: The control unit is further configured to convert the real-time navigation control strategy into binary data, and parse the binary data into a pulse width modulation (PWM) signal, and send the signal to the implementation unit; The implementation unit is further configured to control the steering gear and the drive motor according to the PWM signal.

13. The system according to claim 6, wherein: The control terminal is configured to receive a return control instruction triggered by the target user and send the return control instruction to the return control module, wherein the return control instruction carries a preset return destination; The control unit is used to return from the current path point in the target operation area according to the return control instruction until reaching the return destination, and send a return completion instruction to the control terminal.

14. The system according to claim 6, wherein: The control terminal is configured to receive a stop driving control instruction triggered by the target user and send the stop driving control instruction to the control unit; The control unit is used to trigger mechanical braking and cut off power according to the stop driving control instruction.

15. The system according to claim 6, wherein: The control terminal is configured to receive a manual remote control instruction triggered by the target user and send the manual remote control instruction to the control unit; The control unit is used to suspend receiving the target navigation control instruction and control the target unmanned agricultural machine to perform normal operations in the target operation area according to the manual remote control instruction.

16. The system according to any one of claims 1, 13-15, characterized in that: The end-to-end remote control unmanned agricultural machinery operation system further includes: an encryption module, wherein The encryption module is used to encrypt control instructions using a first encryption algorithm, wherein the control instructions at least include: the first control instruction, the second control instruction, the target navigation control instruction, the return control instruction, the stop driving control instruction, and the manual remote control control instruction; and to encrypt non-real-time data using a second encryption algorithm, wherein the non-real-time data at least includes: the target map data and the initial navigation control strategy; wherein the encryption accuracy of the first encryption algorithm is higher than that of the second encryption algorithm.

Images