Picking vehicle rail butt joint method, device, equipment and program product
Through the trajectory detection and displacement calculation model, combined with the autoencoder and time series network, the high accuracy and high adaptability of track positioning of picking vehicles in greenhouse plantations is solved, and efficient and reliable track docking is achieved, reducing equipment failures and costs.
Patent Information
- Application Number
- CN202510788570.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-07-18
- Estimated Expiration
- 2045-06-13
AI Technical Summary
In greenhouse plantations, the track positioning of picking vehicles faces the problems of high accuracy, high adaptability and high reliability. The existing technologies such as UWB and GNSS have problems such as low positioning accuracy, limited coverage, susceptibility to environmental interference, high equipment costs, and large computing resource requirements when positioning indoors.
The trajectory detection model and displacement calculation model are used to obtain reference track images and reference track images, process track points and calculate relative displacement data, adjust the picking vehicle position to achieve track docking, and use the autoencoder and time series network for feature extraction and displacement calculation.
It realizes high-precision track docking, reduces equipment damage and operating failures, improves picking efficiency and operation stability, reduces costs, and adapts to complex greenhouse environments.
Smart Images

Figure CN120335456A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of intelligent control and robotics, and particularly to a method, device, equipment and program product for docking a picking vehicle with a track. Background Art
[0002] In the greenhouse plantation environment, the track positioning of the picking vehicle faces many complex challenges. Firstly, the internal space of the greenhouse is limited and the structure is complex, and the track layout is restricted by many factors. The picking vehicle needs to accurately position and operate on a narrow and possibly curved track. Secondly, the irregular distribution of the crop planting area and the dynamic changes of the picking tasks require the picking vehicle to be able to quickly adapt to the track switching and adjustment to achieve efficient operation scheduling. At the same time, the picking vehicle also needs to avoid collisions with other facilities and equipment in the greenhouse during operation to ensure operation safety. These factors together increase the difficulty of the track positioning of the picking vehicle, and there is an urgent need for a high-precision, highly adaptable and highly reliable positioning technology to solve this problem. Summary of the Invention
[0003] In view of the above problems, the present invention provides a method, device, equipment and program product for docking a picking vehicle with a track.
[0004] According to a first aspect of the present invention, there is provided a method for docking a picking vehicle with a track, including: acquiring a reference track image and a benchmark track image, where the reference track image is an image obtained by photographing the track before the picking vehicle enters the track, the benchmark track image is an image obtained by photographing the track after the picking vehicle enters the track, and the reference track image contains the movement trajectory information of the controlled picking vehicle; respectively processing the reference track image and the benchmark track image based on a trajectory detection model to obtain a target reference trajectory point and a target benchmark trajectory point; processing the target reference trajectory point and the target benchmark trajectory point based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; and in the case where the relative displacement data does not meet a preset condition, adjusting the driving position of the picking vehicle based on the relative displacement data to control the picking vehicle to perform track docking.
[0005] According to an embodiment of the present invention, the displacement calculation model includes an autoencoder and a time series network. Processing the target reference trajectory point and the target benchmark trajectory point based on the displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track includes: respectively processing the target reference trajectory point and the target benchmark trajectory point based on the autoencoder to obtain a reference compressed feature and a benchmark compressed feature; fusing the reference compressed feature and the benchmark compressed feature to obtain a fused track feature corresponding to the reference trajectory point; and processing the fused track features corresponding to multiple reference trajectory points based on the time series network to obtain relative displacement data of the picking vehicle relative to the track.
[0006] According to an embodiment of the present invention, based on a time series network to process the fusion track features corresponding to multiple reference track points, relative displacement data of the picking vehicle relative to the track is obtained, including: based on the time series network to process the fusion track features corresponding to multiple reference track points, hidden features are obtained; the hidden features are processed using an activation function to obtain predicted track points; the predicted track points are compared with target reference track points to obtain relative displacement data.
[0007] According to an embodiment of the present invention, based on a track detection model to process a reference track image and a reference benchmark image respectively, target reference track points and target reference benchmark points are obtained, including: based on the track detection model to process the reference track image, classification probability values of multiple reference track points are obtained, and based on the track detection model to process the reference benchmark image, classification probability values of multiple reference benchmark points are obtained; non-maximum suppression is used to process the classification probability values of multiple reference track points to obtain target reference track points; non-maximum suppression is used to process the classification probability values of multiple reference benchmark points to obtain target reference benchmark points.
[0008] According to an embodiment of the present invention, the relative displacement data includes a relative angle and a relative moving distance; in the case where the relative displacement data does not meet a preset condition, based on the relative displacement data, the driving position of the picking vehicle is adjusted to control the picking vehicle to perform track docking, including: in the case where the relative angle does not meet a preset angle and the relative moving distance meets a preset moving distance, based on the relative angle, the picking vehicle is controlled to move to move the picking vehicle onto the track; in the case where the relative angle meets the preset angle and the relative moving distance does not meet the preset moving distance, based on the relative moving distance, the picking vehicle is controlled to move to move the picking vehicle onto the track; in the case where the relative angle does not meet the preset angle and the relative moving distance does not meet the preset moving distance, based on the relative angle and the relative moving distance, the picking vehicle is controlled to move to move the picking vehicle onto the track.
[0009] According to an embodiment of the present invention, the picking vehicle track docking method further includes: in the case where the picking vehicle has completed the picking task between the first ridges, controlling the picking vehicle to disengage from the track between the first ridges and move to a preset position; and in response to performing the picking task between the second ridges, controlling the picking vehicle to dock with the track between the second ridges, where the first ridges and the second ridges are adjacent.
[0010] According to an embodiment of the present invention, the track detection model is trained based on the following steps: obtaining a sample track image and labeled track points; iteratively performing the following operations: processing the sample track image based on an initial track detection model to obtain sample track points; determining the number of track points where the labeled track points and the sample track points overlap; subtracting the coordinate value of the (i + 1)-th track point from the coordinate value of the i-th track point to obtain an angular difference, where the coordinate value of the i-th track point represents subtracting the coordinate value of the i-th labeled track point from the coordinate value of the i-th sample track point, and the coordinate value of the (i + 1)-th track point represents subtracting the coordinate value of the (i + 1)-th labeled track point from the coordinate value of the (i + 1)-th sample track point, and i > 0; dividing the angular difference by the number of track points to obtain an average angular change; processing the average angular change using an angular loss function to obtain an angular loss value; and training the initial track detection model using the angular loss value to obtain a trained track detection model.
[0011] The second aspect of the present invention provides a track docking device for a picking vehicle, including: an acquisition module for acquiring a reference track image and a benchmark track image, where the reference track image is an image obtained by photographing the track before the picking vehicle enters the track, and the benchmark track image is an image obtained by photographing the track after the picking vehicle enters the track, and the reference track image contains the movement track information of the controlled picking vehicle; a track point obtaining module for respectively processing the reference track image and the benchmark track image based on a track detection model to obtain a target reference track point and a target benchmark track point; a relative displacement data obtaining module for processing the target reference track point and the target benchmark track point based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; and a track docking module for adjusting the driving position of the picking vehicle based on the relative displacement data when the relative displacement data does not meet a preset condition, so as to control the picking vehicle to perform track docking.
[0012] The third aspect of the present invention provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, where the above one or more processors execute the above one or more computer programs to implement the steps of the above method.
[0013] The fourth aspect of the present invention further provides a computer program product, including a computer program or instruction, where the above computer program or instruction implements the steps of the above method when executed by a processor.
[0014] According to an embodiment of the present invention, a reference track image and a benchmark track image are processed based on a trajectory detection model to obtain a target reference trajectory point and a target benchmark trajectory point; a relative displacement data of the picking vehicle relative to the track is obtained by processing the target reference trajectory point and the target benchmark trajectory point based on a displacement calculation model; in the case where the relative displacement data does not meet a preset condition, the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking. By acquiring images in real time to accurately extract trajectory points and calculate relative displacements, position deviations of the picking vehicle can be discovered and corrected in a timely manner, thereby achieving high-precision track docking, and equipment damage or operation failures caused by inaccurate track docking can be significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above content and other objects, features, and advantages of the present invention will become clearer through the following description of the embodiments of the present invention with reference to the accompanying drawings.
[0016] Figure 1 FIG. shows an application scenario diagram of a track docking method for a picking vehicle according to an embodiment of the present invention.
[0017] Figure 2 FIG. shows a flowchart of a track docking method for a picking vehicle according to an embodiment of the present invention.
[0018] Figure 3 FIG. shows a network structure diagram of an autoencoder according to an embodiment of the present disclosure.
[0019] Figure 4 FIG. shows a network structure diagram of a trajectory detection model according to an embodiment of the present invention.
[0020] Figure 5 FIG. shows a flowchart of a track docking method for a picking vehicle according to an embodiment of the present invention.
[0021] Figure 6 FIG. shows a flowchart of a track docking method for a picking vehicle according to another embodiment of the present invention.
[0022] Figure 7 FIG. shows a structural block diagram of a track docking device for a picking vehicle according to an embodiment of the present invention.
[0023] Figure 8 FIG. shows a block diagram of an electronic device suitable for implementing a track docking method for a picking vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.
[0025] The terms used herein are merely for the purpose of describing specific embodiments and are not intended to limit the present invention. The terms "comprising", "including" and the like as used herein indicate the presence of the described features, steps, operations and / or components, but do not preclude the presence or addition of one or more other features, steps, operations or components.
[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0027] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C).
[0028] In the greenhouse plantation environment, the track positioning of the picking vehicle faces many complex challenges. In the existing technology, the positioning cost of Ultra-Wideband (UWB) is relatively high. The equipment is expensive, and a large number of base stations and possible wiring are required during deployment, increasing the amount of engineering construction and cost; the coverage range is limited, the signal propagation distance is short, and it is easily affected by obstacles, resulting in a reduced coverage range; the signal is easily interfered by the environment; the positioning accuracy is limited. The non-line-of-sight problem and hardware factors affect the accuracy, and it is difficult to generally achieve the ideal high accuracy; the functions are limited. Generally, angle information cannot be directly obtained, and the data type is single; the battery life is short. Due to the high transmission frequency, the equipment power consumption is large, making the battery life of battery-powered equipment short. In addition, Simultaneous Localization and Mapping (SLAM) has high requirements for computing resources, and processing a large amount of data requires strict hardware requirements. Real-time problems are likely to occur in large-scale scenarios; the accuracy is greatly affected by the environment. Lack of features, dynamic objects, different lighting and weather conditions, etc. will all interfere with positioning; there is a problem of cumulative error. The error accumulates gradually, and loop detection is also difficult to completely eliminate; the initialization is relatively difficult. Incorrect initial information acquisition will affect the subsequent process; the algorithm robustness is insufficient. Different algorithms are applicable to limited scenarios, and problems are likely to occur when the sensor fails and interferes. The Global Navigation Satellite System (GNSS) has many disadvantages in indoor positioning. Due to the limited signal penetration ability, it is blocked by buildings, and the signal is greatly attenuated or even interrupted; the indoor multipath effect is serious, and the reflected objects cause the signal to be chaotic and it is difficult to accurately model and compensate; the positioning accuracy is limited. The signal quality is poor and the satellite visibility is low, and the indoor accuracy often drops to dozens of meters or even hundreds of meters; there are many indoor electromagnetic interference sources, and the interference is difficult to eliminate, affecting the positioning accuracy and stability. Therefore, there is an urgent need for a positioning technology with high accuracy, high adaptability and high reliability to solve this problem.
[0029] In view of this, the present invention provides a method, a device and equipment for docking the track of a picking vehicle. The method includes: obtaining a reference track image and a reference track image, the reference track image characterizing the movement trajectory of the picking vehicle before entering the track, the reference track image characterizing the picking vehicle entering the track, and the reference track image characterizing the movement trajectory of the controlled picking vehicle; respectively processing the reference track image and the reference track image based on a trajectory detection model to obtain a target reference trajectory point and a target reference trajectory point; processing the target reference trajectory point and the target reference trajectory point based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; in the case where the relative displacement data does not meet the preset conditions, adjusting the driving position of the picking vehicle based on the relative displacement data to control the picking vehicle to perform track docking.
[0030] It should be noted that the picking vehicle track docking method and the picking vehicle track docking device provided by the present invention can be used in the field of agricultural machinery, and can also be used in any field other than agricultural machinery, such as the field of intelligentization and automation. Therefore, the application fields of the picking vehicle track docking method and the picking vehicle track docking device provided by the present invention are not limited.
[0031] Figure 1 Fig. shows an application scenario diagram of the picking vehicle track docking method according to an embodiment of the present invention.
[0032] As Figure 1 shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.
[0033] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).
[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.
[0035] The server 105 may be a server providing various services, such as a background management server (only as an example) that supports the websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0036] It should be noted that the picking vehicle track docking method provided by the embodiments of the present invention can generally be executed by the server 105. Correspondingly, the picking vehicle track docking device provided by the embodiments of the present invention can generally be set in the server 105. The picking vehicle track docking method provided by the embodiments of the present invention can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the picking vehicle track docking device provided by the embodiments of the present invention can also be set in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.
[0037] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in
[0038] Figure 2 FIG. shows a flowchart of the picking vehicle track docking method according to an embodiment of the present invention.
[0039] As Figure 2 shown, the picking vehicle track docking method of this embodiment includes operations S210 to S240, and this picking vehicle track docking method can be executed by an electronic device.
[0040] In operation S210, a reference track image and a reference track image are obtained.
[0041] In operation S220, the reference track image and the reference track image are respectively processed based on the trajectory detection model to obtain a target reference trajectory point and a target reference trajectory point.
[0042] In operation S230, the target reference trajectory point and the target reference trajectory point are processed based on the displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track.
[0043] In operation S240, in the case where the relative displacement data does not meet the preset conditions, the driving position of the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking.
[0044] According to an embodiment of the present invention, the reference track image is an image obtained by photographing the track before the picking vehicle enters the track, the reference track image is an image obtained by photographing the track after the picking vehicle enters the track, and the reference track image contains the movement trajectory information of the controlled picking vehicle.
[0045] According to an embodiment of the present invention, a single - eye camera or other sensors can be equipped on the picking vehicle to obtain a reference track image and a benchmark track image. The single - eye camera or other sensors can be used to collect environmental data in real time. By being real - time, the single - eye camera ensures that the acquired data can fully reflect the position information of the picking vehicle relative to the greenhouse plantation track. The collected reference track image and benchmark track image have three channels of red, green, and blue, and can contain the track information of the greenhouse plantation.
[0046] According to an embodiment of the present invention, a trajectory detection model can be used to obtain trajectory points in an image. Based on processing the reference track image by the trajectory detection model, target reference trajectory points can be obtained. Based on processing the benchmark track image by the trajectory detection model, target benchmark trajectory points can be obtained.
[0047] According to an embodiment of the present invention, by processing the target reference trajectory points and the target benchmark trajectory points based on a displacement calculation model, relative displacement data of the picking vehicle relative to the track can be obtained. The relative displacement data can include the relative angle, relative moving distance, relative height difference, etc. of the position where the picking vehicle is located from the track.
[0048] According to an embodiment of the present invention, when the relative displacement data does not meet the preset conditions, the driving position of the picking vehicle can be adjusted based on the relative displacement data to control the picking vehicle to perform track docking.
[0049] According to an embodiment of the present invention, the reference track image and the benchmark track image are respectively processed based on the trajectory detection model to obtain target reference trajectory points and target benchmark trajectory points; the target reference trajectory points and the target benchmark trajectory points are processed based on the displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; when the relative displacement data does not meet the preset conditions, the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking. By acquiring images in real time to accurately extract trajectory points and calculate relative displacements, the position deviation of the picking vehicle can be detected and corrected in time, so as to achieve high - precision track docking. This method has a low cost and high adaptability to the greenhouse track environment, and can significantly reduce equipment damage or operation failures caused by inaccurate track docking.
[0050] According to an embodiment of the present invention, the displacement calculation model includes an auto - encoder and a time - series network. Processing the target reference trajectory points and the target benchmark trajectory points based on the displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track includes: respectively processing the target reference trajectory points and the target benchmark trajectory points based on the auto - encoder to obtain a reference compressed feature and a benchmark compressed feature; fusing the reference compressed feature and the benchmark compressed feature to obtain a fused track feature corresponding to the reference trajectory point; processing the fused track features corresponding to multiple reference trajectory points based on the time - series network to obtain relative displacement data of the picking vehicle relative to the track.
[0051] According to an embodiment of the present invention, by processing target reference trajectory points based on an autoencoder, reference compression features can be obtained, and by processing target reference trajectory points based on an autoencoder, reference compression features can be obtained. The autoencoder can adopt an autoencoder architecture based on a deep siamese neural network for further processing the features output by the trajectory detection model.
[0052] According to an embodiment of the present invention, by using an autoencoder, unsupervised learning can be performed and features can be automatically learned from trajectory points. Through the non-linear modeling ability, complex non-linear relationships in the data can be captured, and compression features can be extracted from the overall image data. The extracted low-dimensional compression features can be used to achieve dimensionality reduction and effectively remove noise and redundancy, and a hierarchical feature representation of the track can be obtained.
[0053] Figure 3 The network structure diagram of the autoencoder according to an embodiment of the present disclosure is shown.
[0054] As Figure 3 shown, taking the input of target reference trajectory points as an example, the target reference trajectory points 301 are input into the encoder 302, the encoded features of the target reference trajectory points are output, the encoded features are processed by the max pooling layer 303 to obtain max pooling features. The max pooling features are input into the fully connected layer 304, and the compression features 305 (i.e., reference compression features) are output. The compression features 305 are input into the fully connected layer 306, and the decoded features are output; the decoded features are processed by the max pooling layer 307 to obtain pooled features; the pooled features are processed by the decoder 308 to output the first target reference trajectory points 309.
[0055] According to an embodiment of the present invention, the autoencoder consists of an encoder and a decoder. The encoder maps high-dimensional input data to a low-dimensional latent space. Through a series of convolutional layers, linear transformations, non-linear activation layers, etc., the encoder can learn the internal structure and feature patterns of the input data, extract the most representative feature information, and compress it into one-dimensional data, but not limited to this. The embodiments of the present invention do not limit the dimension. The decoder can decode the one-dimensional data generated by the encoder into an output with the same dimension and form as the original input data. The decoder restores the one-dimensional feature information into the original data by reversing the process of the encoder.
[0056] According to an embodiment of the present invention, during the encoding and decoding process, the loss function of the autoencoder can be constituted by calculating the Euclidean distance between the data output by the decoder and the original data, as shown in formula (1).
[0057] ,
[0058]
[0059] (1);
[0060] Wherein, X represents the position of the trajectory point input to the encoder, represents the position of the nth input trajectory point, Y represents the position of the trajectory point output by the decoder, represents the position of the nth trajectory point output by the decoder, and d(X, Y) represents the Euclidean distance between the position of the trajectory point input to the encoder and the position of the trajectory point output by the decoder.
[0061] According to an embodiment of the present invention, the reference compression feature and the reference compression feature can be fused to obtain a fused track feature corresponding to the reference trajectory point; based on the time series network processing the fused track features corresponding to multiple reference trajectory points respectively, the relative displacement data of the picking vehicle relative to the track can be obtained.
[0062] According to an embodiment of the present invention, by using an autoencoder to process the reference trajectory point and the reference trajectory point respectively, the reference compression feature and the reference compression feature are obtained, which can effectively reduce the feature dimension, retain key information and improve the efficiency of subsequent processing; fusing the reference compression feature and the reference compression feature to obtain a fused track feature can provide more learning information for the subsequent processing of the time series network; based on the time series network processing the fused track feature, the relative displacement value of the position of the picking vehicle relative to the track is obtained, and combined with the autoencoder compression feature, high-precision real-time positioning can be realized. The time series network can process dynamically changing features to ensure that the system maintains high precision during real-time operation. By using the feature compression of the autoencoder and the efficient processing of the time series network, the calculation cost is reduced and the operation efficiency is improved.
[0063] According to an embodiment of the present invention, based on the time series network processing the fused track features corresponding to multiple reference trajectory points respectively, the relative displacement data of the picking vehicle relative to the track is obtained, including: based on the time series network processing the fused track features corresponding to multiple reference trajectory points respectively, hidden features are obtained; the hidden features are processed by an activation function to obtain predicted trajectory points; the predicted trajectory points are compared with the target reference trajectory points to obtain relative displacement data.
[0064] According to an embodiment of the present invention, the time series network may include a long short-term memory network, etc., but is not limited thereto. The embodiment of the present disclosure does not limit the network type. Based on the time series network processing the fused track features corresponding to multiple reference trajectory points respectively, hidden features can be obtained; the hidden features are processed by an activation function to obtain predicted trajectory points; the predicted trajectory points are compared with the target reference trajectory points to obtain relative displacement data.
[0065] According to an embodiment of the present invention, the selected and encoded one-dimensional data can be input into a time series network to output relative displacement data. For example, the time series network is a long short-term memory network. The long short-term memory network (LSTM) has multiple gating units, including a reset gate and an update gate, where (1 ≤ ≤ ) represents the th layer of the LSTM. The activation value of the hidden feature is a linear interpolation between the previous activation value and the candidate hidden activation value, and the specific process is shown in formulas (2), (3), (4), and (5):
[0066] = ( , ) (2);
[0067] = ( , ) (3);
[0068] = ( , ) (4);
[0069] = ( ) + (5);
[0070] Among them, represents the value of the reset gate in the th layer, represents the activation function, represents the weight matrix of the reset gate, represents the hidden feature of the th layer, represents the value of the update gate in the th layer, represents the weight matrix of the update gate, represents the candidate hidden feature in the th layer, Denote the final hidden feature of the l-th layer. Denote the complement of the update gate. Denote the value of the update gate.
[0071] According to an embodiment of the present invention, by processing the final hidden feature of the l-th layer of the long short-term memory network using an activation function, relative displacement data can be obtained.
[0072] According to an embodiment of the present invention, by using a time series network to process the fused track features, noise can be filtered more effectively, key features can be adaptively extracted, and at the same time, the processing ability for complex dynamic changes can be enhanced. With the help of multi-layer supervised learning, the time series network can better combine earlier information with current information, thereby inferring relative displacement data more accurately and stably, improving the overall performance and the accuracy of inference.
[0073] According to an embodiment of the present invention, processing the reference track image and the reference benchmark image respectively based on a track detection model to obtain a target reference track point and a target reference benchmark point, including: processing the reference track image based on the track detection model to obtain classification probability values of multiple reference track points, and processing the reference benchmark image based on the track detection model to obtain classification probability values of multiple reference benchmark points; using non-maximum suppression to process the classification probability values of multiple reference track points to obtain the target reference track point; using non-maximum suppression to process the classification probability values of multiple reference benchmark points to obtain the target reference benchmark point.
[0074] According to an embodiment of the present invention, by using a track detection model to identify the tracks from the reference track image and the reference benchmark image, the tracks in the greenhouse plantation have characteristic points such as being slender, so the tracks are abstracted as straight lines and further transformed into a series of characteristic points (i.e., track points). Processing the reference track image based on the track detection model to obtain multiple reference track points, each reference track point corresponding to a classification probability value, and processing the reference benchmark image based on the track detection model to obtain multiple reference benchmark points, each reference benchmark point corresponding to a classification probability value.
[0075] According to an embodiment of the present invention, using non-maximum suppression to process the classification probability values of multiple reference track points can obtain the target reference track point; using non-maximum suppression to process the classification probability values of multiple reference benchmark points can obtain the target reference benchmark point. Among them, the two largest reference track points can be selected as the target reference track point according to the positive classification probability values among the classification probability values of multiple reference track points. The two largest reference benchmark points can be selected as the target reference benchmark point according to the positive classification probability values among the classification probability values of multiple reference benchmark points.
[0076] According to an embodiment of the present invention, the relative displacement data includes a relative angle and a relative moving distance; in the case where the relative displacement data does not meet the preset conditions, the traveling position of the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking, including: in the case where the relative angle does not meet the preset angle and the relative moving distance meets the preset moving distance, controlling the picking vehicle to move based on the relative angle to move the picking vehicle onto the track; in the case where the relative angle meets the preset angle and the relative moving distance does not meet the preset moving distance, controlling the picking vehicle to move based on the relative moving distance to move the picking vehicle onto the track; in the case where the relative angle does not meet the preset angle and the relative moving distance does not meet the preset moving distance, controlling the picking vehicle to move based on the relative angle and the relative moving distance to move the picking vehicle onto the track.
[0077] According to an embodiment of the present invention, when the relative displacement data does not meet the preset conditions, it indicates that there is a large deviation between the picking vehicle and the track, and it is still necessary to control the picking vehicle to move. When the relative displacement data meets the preset conditions, the distance deviation between the picking vehicle and the track can be ignored, and there is no need to adjust the picking vehicle anymore, and track docking can be performed.
[0078] According to an embodiment of the present invention, in the case where the relative angle does not meet the preset angle and the relative moving distance meets the preset moving distance, the picking vehicle can be controlled to move based on the relative angle to move the picking vehicle onto the track; in the case where the relative angle meets the preset angle and the relative moving distance does not meet the preset moving distance, the picking vehicle can be controlled to move based on the relative moving distance to move the picking vehicle onto the track; in the case where the relative angle does not meet the preset angle and the relative moving distance does not meet the preset moving distance, the picking vehicle can be controlled to move based on the relative angle and the relative moving distance to move the picking vehicle onto the track.
[0079] According to an embodiment of the present invention, the method for docking the picking vehicle to the track further includes: in the case where the picking vehicle has completed the picking task between the first ridges, controlling the picking vehicle to disengage from the track between the first ridges and move to a preset position; and in response to performing the picking task between the second ridges, controlling the picking vehicle to dock with the track between the second ridges, where the first ridges and the second ridges are adjacent.
[0080] According to an embodiment of the present invention, in the picking operation environment of a greenhouse plantation, fixed tracks can be provided between the ridges, and the spacing between each ridge remains constant. In the case where the picking vehicle has completed the picking task between the first ridges, the picking vehicle can be controlled to disengage from the track between the first ridges and move to a preset position; and in response to performing the picking task between the second ridges, the picking vehicle can be controlled to dock with the track between the second ridges, where the first ridges and the second ridges are adjacent.
[0081] According to an embodiment of the present invention, the picking vehicle can quickly switch to the next ridge after completing the task of one ridge, reducing the waiting time caused by track switching, thereby significantly improving the overall picking efficiency. Through the automated track detachment and docking mechanism, manual intervention is reduced, labor costs are lowered, and at the same time, the stability and reliability of the operation are improved.
[0082] According to an embodiment of the present invention, the track detection model is trained based on the following steps: obtaining a sample track image and labeled track points; iteratively performing the following operations: processing the sample track image based on an initial track detection model to obtain sample track points; determining the number of overlapping track points between the labeled track points and the sample track points; subtracting the coordinate value of the (i + 1)-th track point from the coordinate value of the i-th track point to obtain an angular difference, where the coordinate value of the i-th track point represents subtracting the coordinate value of the i-th labeled track point from the coordinate value of the i-th sample track point, and the coordinate value of the (i + 1)-th track point represents subtracting the coordinate value of the (i + 1)-th labeled track point from the coordinate value of the (i + 1)-th sample track point, i > 0; dividing the angular difference by the number of track points to obtain an average angular change; processing the average angular change using an angular loss function to obtain an angular loss value; training the initial track detection model using the angular loss value to obtain a trained track detection model.
[0083] According to an embodiment of the present invention, processing the sample track image based on the initial track detection model can obtain sample track points.
[0084] According to an embodiment of the present invention, the number of overlapping track points between the labeled track points and the sample track points can be determined by determining the maximum value between the start index of the labeled track points on the true track line and the start index of the sample track points, and determining the minimum value between the end index of the labeled track points on the true track line and the end index of the sample track points.
[0085] According to an embodiment of the present invention, the coordinate value of the i-th track point can be subtracted from the coordinate value of the (i + 1)-th track point to obtain an angular difference, where the coordinate value of the i-th track point represents subtracting the coordinate value of the i-th labeled track point from the coordinate value of the i-th sample track point, and the coordinate value of the (i + 1)-th track point represents subtracting the coordinate value of the (i + 1)-th labeled track point from the coordinate value of the (i + 1)-th sample track point, i > 0.
[0086] According to an embodiment of the present invention, dividing the angular difference by the number of track points can obtain an average angular change, as shown in formula (6).
[0087] (6);
[0088] where , , 。where , They are respectively the end and start indices of the label trajectory points, , which are respectively the end and start indices of the sample trajectory points. and They are respectively the abscissas corresponding to the sample trajectory points and the label trajectory points at the same index.
[0089] According to an embodiment of the present invention, by using an angular loss function to process the average value of angular changes, an angular loss value can be obtained; by using the angular loss value to train an initial track detection model, a trained track detection model can be obtained. The angular loss function is shown in formula (7).
[0090] (7);
[0091] where g represents the average value of angular changes, and represents the angular loss value.
[0092] Figure 4 FIG. shows the network structure diagram of the track detection model according to an embodiment of the present invention.
[0093] As Figure 4 shown, first, the track image 401 is input into the bottom-up backbone network 402. Its convolutional layer is shown in formula (8).
[0094] (8);
[0095] where, is the input feature map , the height of the input feature map is, the number of channels is , the width is , the number of channels is , represents the weight parameter, represents the input data with coordinates ([[]]END]] ), the channel dimension is l, and the dimension is , is the length of the convolutional kernel, is the width of the convolutional kernel, the number of input channels is , the number of output channels is , represents the bias value, is the output feature map, the dimension , the height of the output feature map , the width of the output feature map , is the padding size, is the stride.
[0096] According to an embodiment of the present invention, after the convolutional layer, the ReLU (Rectified Linear Unit) non-linear activation function is used, as shown in formula (9):
[0097] (9);
[0098] max represents the operation of finding the maximum value. For the output of the convolutional layer , after passing through the activation function ReLU, it is as shown in formula (10):
[0099] (10);
[0100] Then, downsampling is performed through the max pooling layer, as shown in formula (11):
[0101] (11);
[0102] In max pooling, represents the value at the position (i, j) in the k-th channel of the output feature map. x is the input feature map, i and j are the coordinate indices of the output feature map y, m and n are the local coordinate indices within the pooling window, S is the pooling stride, and max represents the operation of finding the maximum value. By taking the maximum value of the elements of the input feature map x within the pooling window that moves with a stride of S, the value at the corresponding position (i, j) in the output feature map y is obtained, thereby achieving downsampling of the feature map and retaining the main features.
[0103] By repeating the above operations, the feature layers of the last three layers are obtained to construct the top-down and lateral connection channels. Through the top-down and lateral connection channels, the features in the backbone network 402 can be input into the pyramid network 403. The top-down channels are constructed through the upsampling operation of bilinear interpolation, and the upsampling is performed by the method of bilinear interpolation , and the side is dimension-reduced through a 1×1 convolutional kernel , and then feature fusion is performed, as shown in formula (12):
[0104] (12);
[0105] Among them, represents the feature map, represents the feature map after fusing the i-th layer, represents the feature map after fusing the (i + 1)-th layer.
[0106] According to an embodiment of the present invention, the feature map of the topmost layer of the pyramid network 403 can be output to the prediction head 404. The prediction head 404 uniformly samples in the picture for region alignment. The predefined prior feature points have a structure including the starting coordinates of the track line and the angle of the track line, as well as the abscissas of several track line feature points. The region of interest is aligned through the prior feature points and the feature map, and after operations such as convolution and fully connected layers, the track prior feature is obtained. , and the global feature that is adjusted to 1D after scaling and flattening , to construct an attention matrix , as shown in formulas (13) and (14).
[0107] (13);
[0108] (14);
[0109] where is the product of matrix dimensions, is the result after enhancement by adding the attention matrix and the transpose of the global feature matrix and the track prior feature .
[0110] According to an embodiment of the present invention, After being adjusted to the output format through a series of linear layers, it is appended after the initialized prior feature point list. Repeat the above operations until the bottom layer of the pyramid network 403. Then, through non-maximum suppression, that is, by sorting the positive sample probabilities and intercepting, the trajectory point detection result 405 of the track image is output.
[0111] According to an embodiment of the present invention, the angle loss function (i.e., angle loss) can be used to process the sample trajectory points and the label trajectory points to generate an angle loss value; the intersection over union loss function of the lane line (i.e., lane ious loss) can be used to process the sample trajectory points and the label trajectory points to generate an intersection over union loss value; the focal loss function (i.e., Focal loss) can be used to process the sample trajectory points and the label trajectory points to generate a focal loss value; the total loss value of the angle loss value, the intersection over union loss value, and the focal loss value can be calculated, and the initial trajectory detection model can be trained using the total loss value to obtain the trained trajectory detection model.
[0112] According to an embodiment of the present invention, the trajectory detection model can comprehensively and accurately capture the global and local features in the environment. It can screen out valuable information from complex environmental data, and then generate a preliminary path prediction, providing a reliable basis for subsequent path planning.
[0113] Figure 5The flowchart of the track docking method of the picking vehicle according to an embodiment of the present invention is shown.
[0114] As Figure 5 shown, a monocular camera 501 is used to collect a reference track image 502 and a reference track image 503 respectively. The reference track image 502 represents an image obtained by photographing the track before the picking vehicle enters the track, and the reference track image 503 represents an image obtained by photographing the track after the picking vehicle enters the track. The reference track image 502 represents the movement track of the controlled picking vehicle. The track detection model 504 can be used to process the reference track image 502 and the reference track image 503 respectively to obtain a target reference track point 505 and a target reference track point 506. Based on the autoencoder 507, the target reference track point 505 and the target reference track point 506 are processed respectively to obtain a reference compressed feature 508 and a reference compressed feature 509; the reference compressed feature 508 and the reference compressed feature 509 are fused to obtain a fused track feature 510 corresponding to the reference track point; based on the time series network 511 (i.e., LSTM), the fused track feature 510 corresponding to each of the multiple reference track points is processed, and then the fused track feature 510 is processed by an activation function (i.e., softmax), and the relative displacement data 512 of the picking vehicle relative to the track can be obtained. It is judged whether the relative displacement data meets the condition 513. When the relative displacement data does not meet the preset condition, the driving position of the picking vehicle is adjusted based on the relative displacement data, and the monocular camera 501 is used to re-collect the reference track image 502 to control the picking vehicle to perform track docking. When the relative displacement data meets the preset condition, the picking vehicle can be controlled to perform track docking.
[0115] According to an embodiment of the present invention, Figure 5 A positioning and track alignment system applicable to the greenhouse environment based on monocular vision and a corresponding greenhouse picking vehicle are disclosed. The system consists of four main modules. Module A is used to collect reference track images, Module B is used to obtain a target reference track point 506 and a target reference track point 507, Module C is used to obtain a reference compressed feature 508 and a reference compressed feature 509, and Module D is used for path prediction and planning to finally realize the precise track alignment function of the picking vehicle under the standard greenhouse plantation. The green lines in Module B and Module C represent the standard reference path of the track, while the red lines represent the real-time track path. Through the closed-loop feedback mechanism, the system can optimize the path planning in real time to ensure that the robot can accurately perform track positioning.
[0116] Figure 6 The flowchart of the track docking method of the picking vehicle according to another embodiment of the present invention is shown.
[0117] As Figure 6As shown, a reference track image 601 and a reference track image 602 can be obtained. The reference track image 601 and the reference track image 602 are respectively processed based on the track detection model 603 to obtain a target reference track point 604 and a target reference track point 605. The target reference track point 604 and the target reference track point 605 are respectively processed based on the encoder 606 in the autoencoder to obtain a reference compressed feature and a reference compressed feature; the reference compressed feature and the reference compressed feature are fused to obtain a fused track feature corresponding to the reference track point. The fused track features corresponding to multiple reference track points are processed based on the long short-term memory network 607 to obtain relative displacement data 608 of the picking vehicle relative to the track. When the relative displacement data does not meet the preset conditions (that is, the relative displacement data is greater than the threshold), it indicates that the picking vehicle has not reached the predetermined track position 609, and the movement of the picking vehicle can be adjusted based on the relative displacement data 611. When the relative displacement data meets the preset conditions (that is, the relative displacement data is less than or equal to the threshold), it indicates that the picking vehicle has reached the predetermined track position 610.
[0118] Figure 7 Fig. shows a structural block diagram of a picking vehicle track docking device according to an embodiment of the present invention.
[0119] As Figure 7 shown, the picking vehicle track docking device of this embodiment includes an acquisition module 710, a track point obtaining module 720, a relative displacement data obtaining module 730, and a track docking module 740.
[0120] The acquisition module 710 is configured to acquire a reference track image and a reference track image. The reference track image is an image obtained by photographing the track before the picking vehicle enters the track, and the reference track image is an image obtained by photographing the track after the picking vehicle enters the track. The reference track image contains the movement track information of the controlled picking vehicle.
[0121] The track point obtaining module 720 is configured to respectively process the reference track image and the reference track image based on the track detection model to obtain a target reference track point and a target reference track point.
[0122] The relative displacement data obtaining module 730 is configured to process the target reference track point and the target reference track point based on the displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track.
[0123] The track docking module 740 is configured to, when the relative displacement data does not meet the preset conditions, adjust the driving position of the picking vehicle based on the relative displacement data to control the picking vehicle to perform track docking.
[0124] According to an embodiment of the present invention, the reference track image and the reference track image are respectively processed based on the track detection model to obtain the target reference track point and the target reference track point; the relative displacement data of the picking vehicle relative to the track is obtained by processing the target reference track point and the target reference track point based on the displacement calculation model; in the case where the relative displacement data does not meet the preset conditions, the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking. By obtaining images in real time to accurately extract track points and calculate relative displacements, the position deviation of the picking vehicle can be discovered and corrected in time, so as to achieve high-precision track docking, which can significantly reduce equipment damage or operation failures caused by inaccurate track docking.
[0125] According to an embodiment of the present invention, the displacement calculation model includes an autoencoder and a time series network.
[0126] According to an embodiment of the present invention, the relative displacement data obtaining module includes: a compressed feature obtaining unit, a fused track feature obtaining unit, and a relative displacement data obtaining unit.
[0127] The compressed feature obtaining unit is configured to process the target reference track point and the target reference track point respectively based on the autoencoder to obtain a reference compressed feature and a reference compressed feature.
[0128] The fused track feature obtaining unit is configured to fuse the reference compressed feature and the reference compressed feature to obtain a fused track feature corresponding to the reference track point.
[0129] The relative displacement data obtaining unit is configured to process the fused track features corresponding to multiple reference track points based on the time series network to obtain the relative displacement data of the picking vehicle relative to the track.
[0130] According to an embodiment of the present invention, the relative displacement data obtaining unit includes: a hidden feature subunit, a predicted track point subunit, and a relative displacement data subunit.
[0131] The hidden feature subunit is configured to process the fused track features corresponding to multiple reference track points based on the time series network to obtain hidden features.
[0132] The predicted track point subunit is configured to process the hidden features using an activation function to obtain predicted track points.
[0133] The relative displacement data subunit is configured to compare the predicted track points with the target reference track points to obtain relative displacement data.
[0134] According to an embodiment of the present invention, the track point obtaining module includes: a classification probability value unit, a target reference track point unit, and a target reference track point unit.
[0135] A classification probability value unit for processing a reference track image based on a track detection model to obtain classification probability values of multiple reference track points, and processing a reference track image based on the track detection model to obtain classification probability values of multiple reference track points.
[0136] A target reference track point unit for using non-maximum suppression to process the classification probability values of multiple reference track points to obtain target reference track points.
[0137] A target reference track point unit for using non-maximum suppression to process the classification probability values of multiple reference track points to obtain target reference track points.
[0138] According to an embodiment of the present invention, the relative displacement data includes a relative angle and a relative moving distance.
[0139] According to an embodiment of the present invention, the track docking module includes: a first unit, a second unit, and a third unit.
[0140] The first unit is configured to control the picking vehicle to move based on the relative angle when the relative angle does not satisfy a preset angle and the relative moving distance satisfies a preset moving distance, so as to move the picking vehicle onto the track.
[0141] The second unit is configured to control the picking vehicle to move based on the relative moving distance when the relative angle satisfies the preset angle and the relative moving distance does not satisfy the preset moving distance, so as to move the picking vehicle onto the track.
[0142] The third unit is configured to control the picking vehicle to move based on the relative angle and the relative moving distance when the relative angle does not satisfy the preset angle and the relative moving distance does not satisfy the preset moving distance, so as to move the picking vehicle onto the track.
[0143] According to an embodiment of the present invention, the picking vehicle track docking device of this embodiment further includes: a detachment module and a docking module.
[0144] The detachment module is configured to control the picking vehicle to detach from the track between the first ridges and move to a preset position when the picking vehicle has completed the picking task between the first ridges.
[0145] The docking module is configured to control the picking vehicle to dock with the track between the second ridges in response to performing the picking task between the second ridges, and the first ridges and the second ridges are adjacent.
[0146] According to an embodiment of the present invention, a sample track image and labeled track points are obtained; the following operations are iteratively performed: processing the sample track image based on an initial track detection model to obtain sample track points; determining the number of overlapping track points between the labeled track points and the sample track points; subtracting the coordinate value of the (i + 1)-th track point from the coordinate value of the i-th track point to obtain an angular difference, where the coordinate value of the i-th track point represents subtracting the coordinate value of the i-th labeled track point from the coordinate value of the i-th sample track point, and the coordinate value of the (i + 1)-th track point represents subtracting the coordinate value of the (i + 1)-th labeled track point from the coordinate value of the (i + 1)-th sample track point, and i > 0; dividing the angular difference by the number of track points to obtain an average angular change; processing the average angular change using an angular loss function to obtain an angular loss value; and training the initial track detection model using the angular loss value to obtain a trained track detection model.
[0147] According to an embodiment of the present invention, any of the acquisition module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be combined and implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable means such as integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the acquisition module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be at least partially implemented as a computer program module, and when the computer program module is run, it can execute the corresponding functions.
[0148] Figure 8 A block diagram of an electronic device suitable for implementing the gyroscope drift error compensation method according to an embodiment of the present invention is shown.
[0149] As Figure 8As shown, the electronic device according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a ROM 802 (i.e., read-only memory) or a program loaded from a storage section 808 into a RAM 803 (i.e., random access memory). The processor 801 may include, for example, a general microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.
[0150] In the RAM 803, various programs and data required for the operation of the electronic device are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to an embodiment of the present invention by executing a program in the ROM 802 and / or the RAM 803. It should be noted that the program may also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flow according to an embodiment of the present invention by executing a program stored in one or more memories.
[0151] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 805, and the input / output (I / O) interface 805 is also connected to the bus 804. The electronic device may further include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A driver 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the driver 810 as needed, so that a computer program read from it can be installed into the storage section 808 as needed.
[0152] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to an embodiment of the present invention is implemented.
[0153] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM 803), read-only memory (ROM 802), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or apparatus. For example, according to an embodiment of the present invention, the computer-readable storage medium may include one or more memories other than the above-described ROM 802 and / or RAM 803 and / or ROM 802 and RAM 803.
[0154] An embodiment of the present invention further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the gyroscope drift error compensation method provided by the embodiment of the present invention.
[0155] When the computer program is executed by the processor 801, it executes the above functions defined in the system / apparatus of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.
[0156] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 809, and / or be installed from the removable medium 811. The program code contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0157] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or be installed from the removable medium 811. When the computer program is executed by the processor 801, it executes the above functions defined in the system of the embodiment of the present invention. According to an embodiment of the present invention, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.
[0158] According to embodiments of the present invention, program code for executing the computer programs provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming languages include, but are not limited to, programming languages such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).
[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0160] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.
[0161] The above describes the embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.
Claims
1. A method for docking the track of a picking vehicle, characterized in that, Including: Obtain a reference track image and a reference track image. The reference track image is an image obtained by photographing the track before the picking vehicle enters the track. The reference track image is an image obtained by photographing the track after the picking vehicle enters the track. The reference track image contains the movement trajectory information of the controlled picking vehicle. Process the reference track image and the reference track image respectively based on the trajectory detection model to obtain a target reference trajectory point and a target reference trajectory point. Process the target reference trajectory point and the target reference trajectory point based on the displacement calculation model to obtain the relative displacement data of the picking vehicle relative to the track. In the case where the relative displacement data does not meet the preset conditions, adjust the driving position of the picking vehicle based on the relative displacement data to control the picking vehicle to perform track docking.
2. The method according to claim 1, wherein The displacement calculation model includes an autoencoder and a time series network. The process of processing the target reference trajectory point and the target reference trajectory point based on the displacement calculation model to obtain the relative displacement data of the picking vehicle relative to the track includes: Process the target reference trajectory point and the target reference trajectory point respectively based on the autoencoder to obtain a reference compressed feature and a reference compressed feature. Fuse the reference compressed feature and the reference compressed feature to obtain a fused track feature corresponding to the reference trajectory point. Process the fused track features corresponding to multiple reference trajectory points based on the time series network to obtain the relative displacement data of the picking vehicle relative to the track.
3. The method according to claim 2, wherein The process of processing the fused track features corresponding to multiple reference trajectory points based on the time series network to obtain the relative displacement data of the picking vehicle relative to the track includes: Process the fused track features corresponding to multiple reference trajectory points based on the time series network to obtain a hidden feature. Process the hidden feature using an activation function to obtain a predicted trajectory point. Compare the predicted trajectory point with the target reference trajectory point to obtain the relative displacement data.
4. The method according to claim 1, wherein The process of processing the reference track image and the reference track image respectively based on the trajectory detection model to obtain a target reference trajectory point and a target reference trajectory point includes: Process the reference track image based on the trajectory detection model to obtain the classification probability values of multiple reference trajectory points, and process the reference track image based on the trajectory detection model to obtain the classification probability values of multiple reference trajectory points. Use non-maximum suppression to process the classification probability values of multiple reference trajectory points to obtain the target reference trajectory point. Use non-maximum suppression to process the classification probability values of multiple reference trajectory points to obtain the target reference trajectory point.
5. The method according to claim 1, wherein The relative displacement data includes a relative angle and a relative movement distance. In the case where the relative displacement data does not meet the preset conditions, adjust the driving position of the picking vehicle based on the relative displacement data to control the picking vehicle to perform track docking, including: When the relative angle does not meet the preset angle and the relative movement distance meets the preset movement distance, control the picking vehicle to move based on the relative angle so as to move the picking vehicle onto the track; When the relative angle meets the preset angle and the relative movement distance does not meet the preset movement distance, control the picking vehicle to move based on the relative movement distance so as to move the picking vehicle onto the track; When the relative angle does not meet the preset angle and the relative movement distance does not meet the preset movement distance, control the picking vehicle to move based on the relative angle and the relative movement distance so as to move the picking vehicle onto the track.
6. The method according to claim 1, the method further comprising: When the picking vehicle has completed the picking task in the first row interval, control the picking vehicle to disengage from the track in the first row interval and move to a preset position; and In response to executing the picking task in the second row interval, control the picking vehicle to dock with the track in the second row interval, and the first row interval is adjacent to the second row interval.
7. The method according to claim 1, the track detection model is trained based on the following steps Obtain a sample track image and labeled track points; Iteratively perform the following operations: Process the sample track image based on the initial track detection model to obtain sample track points; Determine the number of overlapping track points between the labeled track points and the sample track points; Subtract the coordinate value of the (i + 1)-th track point from the coordinate value of the i-th track point to obtain an angle difference, where the coordinate value of the i-th track point represents subtracting the coordinate value of the i-th labeled track point from the coordinate value of the i-th sample track point, and the coordinate value of the (i + 1)-th track point represents subtracting the coordinate value of the (i + 1)-th labeled track point from the coordinate value of the (i + 1)-th sample track point, i > 0; Divide the angle difference by the number of track points to obtain an average angle change; Process the average angle change using an angle loss function to obtain an angle loss value; Train the initial track detection model using the angle loss value to obtain a trained track detection model.
8. A track docking device for a picking vehicle, characterized in that, including: An acquisition module for acquiring a reference track image and a reference benchmark image, where the reference track image is an image obtained by photographing the track before the picking vehicle enters the track, the reference benchmark image is an image obtained by photographing the track after the picking vehicle enters the track, and the reference track image contains the movement trajectory information of the controlled picking vehicle; A track point obtaining module for processing the reference track image and the reference benchmark image respectively based on a track detection model to obtain target reference track points and target reference benchmark points; A relative displacement data obtaining module for processing the target reference track points and the target reference benchmark points based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; The orbital docking module is used to adjust the driving position of the picking vehicle based on the relative displacement data when the relative displacement data does not meet the preset conditions, so as to control the picking vehicle to perform orbital docking.
9. An electronic device, comprising: One or more processors; A memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Vehicle control method, device and equipment
CN111038477A
Position prediction method, system, equipment and medium
CN114862946A
Vehicle drift trajectory control method, device and equipment and storage medium
CN116215537A
Vehicle trajectory prediction method and device and trajectory prediction model training method and device
CN118864529A
Track planning method, vehicle and computer program product
CN119027917A
