Picking vehicle track docking method, device, equipment and program product

Abstract

The application provides a picking vehicle track docking method, device, equipment and program product, which can be applied to the fields of intelligent control and robot technology. The method comprises the following steps: acquiring a reference track image and a reference track image, the reference track image representing the movement track of the controlled picking vehicle before the picking vehicle enters the track, and the reference track image representing the picking vehicle entering the track; processing the reference track image and the reference track image based on a track detection model to obtain target reference track points and target reference track points; processing the target reference track points and the target reference track points based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; and adjusting the driving position of the picking vehicle based on the relative displacement data in the case that the relative displacement data does not satisfy a preset condition, so as to control the picking vehicle to perform track docking.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent control and robotics, in particular to a picking vehicle track docking method, device, equipment and program product. BACKGROUND

[0002] In the greenhouse plantation environment, the track positioning of the picking vehicle faces many complex challenges. First, the interior space of the greenhouse is limited and the structure is complex, and the track layout is subject to many restrictions, and the picking vehicle needs to be accurately positioned and operated on a narrow and possibly curved track. Second, the irregular distribution of crop planting areas and the dynamic changes of picking tasks require the picking vehicle to quickly adapt to the switching and adjustment of the track to achieve efficient job scheduling. At the same time, the picking vehicle also needs to avoid collision with other facilities and equipment in the greenhouse during operation to ensure job safety. These factors together increase the difficulty of track positioning of the picking vehicle, and a high-precision, high-adaptability and high-reliability positioning technology is needed to solve this problem. SUMMARY

[0003] In view of the above problems, the present application provides a picking vehicle track docking method, device, equipment and program product.

[0004] According to a first aspect of the present application, a picking vehicle track docking method is provided, comprising: acquiring a reference track image and a reference track image, the reference track image being an image obtained by photographing the track before the picking vehicle enters the track, the reference track image being an image obtained by photographing the track after the picking vehicle enters the track, the reference track image containing the motion trajectory information of the controlled picking vehicle; processing the reference track image and the reference track image based on the trajectory detection model to obtain target reference trajectory points and target reference trajectory points; processing the target reference trajectory points and the target reference trajectory points based on the displacement calculation model to obtain the relative displacement data of the picking vehicle relative to the track; in the case that the relative displacement data does not satisfy the preset condition, adjusting the driving position of the picking vehicle based on the relative displacement data to realize the control of the picking vehicle to perform track docking.

[0005] According to an embodiment of the present application, the displacement calculation model includes an autoencoder and a time series network, and the processing of the target reference trajectory points and the target reference trajectory points based on the displacement calculation model to obtain the relative displacement data of the picking vehicle relative to the track includes: processing the target reference trajectory points and the target reference trajectory points based on the autoencoder to obtain reference compression features and reference compression features; fusion of reference compression features and reference compression features to obtain fusion track features corresponding to reference trajectory points; processing the fusion track features corresponding to each of the plurality of reference trajectory points based on the time series network to obtain the relative displacement data of the picking vehicle relative to the track.

[0006] According to an embodiment of the present application, the relative displacement data of the picking vehicle relative to the track is obtained by processing the fused track features corresponding to the plurality of reference track points based on the time series network, including: obtaining hidden features by processing the fused track features corresponding to the plurality of reference track points based on the time series network; obtaining predicted track points by processing the hidden features using an activation function; and obtaining the relative displacement data by comparing the predicted track points with the target reference track points.

[0007] According to an embodiment of the present application, the target reference track points and the target reference track points are obtained by processing the reference track image and the reference track image based on the track detection model, including: obtaining classification probability values of the plurality of reference track points by processing the reference track image based on the track detection model, and obtaining classification probability values of the plurality of reference track points by processing the reference track image based on the track detection model; obtaining the target reference track points by processing the classification probability values of the plurality of reference track points using non-maximum suppression; and obtaining the target reference track points by processing the classification probability values of the plurality of reference track points using non-maximum suppression.

[0008] According to an embodiment of the present application, the relative displacement data includes a relative angle and a relative movement distance; and in a case where the relative displacement data does not satisfy a preset condition, the driving position of the picking vehicle is adjusted based on the relative displacement data to control the picking vehicle to perform track docking, including: in a case where the relative angle does not satisfy a preset angle and the relative movement distance satisfies a preset movement distance, the picking vehicle is controlled to move based on the relative angle to move the picking vehicle onto the track; in a case where the relative angle satisfies the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle is controlled to move based on the relative movement distance to move the picking vehicle onto the track; and in a case where the relative angle does not satisfy the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle is controlled to move based on the relative angle and the relative movement distance to move the picking vehicle onto the track.

[0009] According to an embodiment of the present application, the picking vehicle track docking method further includes: in a case where the picking vehicle completes the picking task in the first row, the picking vehicle is controlled to detach from the track in the first row and move to a preset position; and in response to performing the picking task in the second row, the picking vehicle is controlled to dock with the track in the second row, and the first row and the second row are adjacent.

[0010] According to an embodiment of the present application, the track detection model is trained based on the following steps: obtaining sample track images and label trajectory points; iteratively performing the following operations: processing the sample track images based on an initial track detection model to obtain sample trajectory points; determining the number of trajectory points overlapping the label trajectory points and the sample trajectory points; subtracting the coordinate value of the i trajectory point from the coordinate value of the i+1 trajectory point to obtain an angle difference value, the coordinate value of the i trajectory point representing the coordinate value of the i sample trajectory point minus the coordinate value of the i label trajectory point, the coordinate value of the i+1 trajectory point representing the coordinate value of the i+1 sample trajectory point minus the coordinate value of the i+1 label trajectory point, i>0; dividing the angle difference value by the number of trajectory points to obtain an average angle change value; processing the average angle change value using an angle loss function to obtain an angle loss value; training the initial track detection model using the angle loss value to obtain a trained track detection model.

[0011] The second aspect of the present application provides a picking vehicle track docking device, comprising: an acquisition module, configured to acquire a reference track image and a reference track image, the reference track image being an image obtained by photographing the track before the picking vehicle enters the track, the reference track image being an image obtained by photographing the track after the picking vehicle enters the track, the reference track image containing the motion trajectory information of the controlled picking vehicle; a trajectory point obtaining module, configured to process the reference track image and the reference track image based on a trajectory detection model to obtain target reference trajectory points and target reference trajectory points; a relative displacement data obtaining module, configured to process the target reference trajectory points and the target reference trajectory points based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; a track docking module, configured to adjust the driving position of the picking vehicle based on the relative displacement data when the relative displacement data does not satisfy a preset condition, so as to control the picking vehicle to perform track docking.

[0012] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0013] The fourth aspect of the present application further provides a computer program product, comprising computer programs or instructions, which are executed by a processor to implement the steps of the above method.

[0014] According to an embodiment of the present application, the reference track image and the reference track image are respectively processed based on a track detection model to obtain target reference track points and target reference track points; the target reference track points and the target reference track points are processed based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; in the case that the relative displacement data does not satisfy a preset condition, the picking vehicle is adjusted based on the relative displacement data to realize control of the picking vehicle to perform track docking. By real-time acquisition of images to accurately extract track points and calculate relative displacement, the position deviation of the picking vehicle is discovered and corrected in time, so as to realize high-precision track docking, which can significantly reduce equipment damage or operation failure caused by inaccurate track docking. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 A scene diagram of application of a picking vehicle track docking method according to an embodiment of the present application is shown.

[0017] Figure 2 A flowchart of a picking vehicle track docking method according to an embodiment of the present application is shown.

[0018] Figure 3 A network structure diagram of a self-encoder according to an embodiment of the present disclosure is shown.

[0019] Figure 4 A network structure diagram of a track detection model according to an embodiment of the present application is shown.

[0020] Figure 5 A flowchart of a picking vehicle track docking method according to an embodiment of the present application is shown.

[0021] Figure 6 A flowchart of a picking vehicle track docking method according to another embodiment of the present application is shown.

[0022] Figure 7 A structure block diagram of a picking vehicle track docking device according to an embodiment of the present application is shown.

[0023] Figure 8 A block diagram of an electronic device suitable for implementing a picking vehicle track docking method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] Embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood, however, that the description which follows is illustrative only and is not intended to limit the scope of the present application. In the detailed description, procedures, apparatuses, and methods that are well known and commonly used in the art will not be described in detail in order to avoid obscuring the concept of the present application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological equivalents thereof, means the inclusion of the stated features, steps, operations, and / or components but not to the exclusion of one or more other features, steps, operations, or components.

[0026] All terms used herein including technical and scientific terms have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification, and not be interpreted in an idealized or overly formal way.

[0027] In the case where expressions such as "at least one of A, B, and C, etc." are used, it generally should be interpreted to include any of one, two, three, four, five, six, seven, eight, nine, ten, or more than ten of A, B, and C, etc. (e.g., "a system having at least one of A, B, and C" should include, but not be limited to, a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0028] In a greenhouse plantation environment, the track positioning of the picking vehicle faces many complex challenges. In the prior art, the Ultra-Wideband (UWB) positioning technology has high cost, requires a large number of base stations and possible wiring during deployment, and increases the amount of engineering construction and cost; the coverage range is limited, the signal propagation distance is short and is easily affected by obstacles to reduce the coverage range; the signal is easily affected by environmental interference; the positioning accuracy is limited, the non-line-of-sight problem and hardware factors affect the accuracy, and it is difficult to achieve ideal high accuracy; the function is limited, and the angle information cannot be directly obtained, the data type is single; the battery life is short, the device power consumption is large due to the high transmission frequency, and the battery-powered device has short endurance. In addition, the Simultaneous Localization and Mapping (SLAM) requires high computing resources, and the hardware requirements are high for processing a large amount of data, and real-time problems are prone to occur in large-scale scenes; the accuracy is greatly affected by the environment, and features, dynamic objects, and different light and weather conditions can interfere with positioning; there is a cumulative error problem, the error is gradually accumulated, and loop detection is also difficult to completely eliminate; the initialization is difficult, and inaccurate initial information acquisition will affect the follow-up; the algorithm robustness is insufficient, and different algorithms are limited in applicable scenarios, and sensor failure interference is prone to problems. The Global Navigation Satellite System (GNSS) has many shortcomings in indoor positioning. Due to the limited signal penetration ability, the signal is greatly attenuated or even interrupted due to building shielding; the indoor multipath effect is serious, and the signal is chaotic and difficult to accurately model and compensate due to reflection objects; the positioning accuracy is limited, the signal quality is poor, the satellite visibility is low, and the indoor accuracy is often reduced to tens of meters or even hundreds of meters; there are many indoor electromagnetic interference sources, and the interference is difficult to eliminate, affecting the positioning accuracy and stability. Therefore, there is an urgent need for a high-precision, high-adaptability, and high-reliability positioning technology to solve this problem.

[0029] Therefore, the present application provides a picking vehicle track docking method, a picking vehicle track docking device and equipment. The method comprises: obtaining a reference track image and a reference track image, the reference track image representing the picking vehicle entering the track before the picking vehicle enters the track, and the reference track image representing the motion trajectory of the controlled picking vehicle; based on the trajectory detection model, the reference track image and the reference track image are processed respectively to obtain the target reference trajectory point and the target reference trajectory point; based on the displacement calculation model, the target reference trajectory point and the target reference trajectory point are processed to obtain the relative displacement data of the picking vehicle relative to the track; in the case that the relative displacement data does not satisfy the preset condition, the driving position of the picking vehicle is adjusted based on the relative displacement data to realize the control of the picking vehicle to perform track docking.

[0030] It should be noted that the picking vehicle track docking method and the picking vehicle track docking device provided by the present application can be used in the field of agricultural machinery, and can also be used in any field other than agricultural machinery, such as the field of intelligence and automation, and therefore the application field of the picking vehicle track docking method and the picking vehicle track docking device provided by the present application is not limited.

[0031] Figure 1 The application scenario diagram of the picking vehicle track docking method according to the embodiment of the present application is shown.

[0032] As Figure 1 shown, the application scenario 100 according to the embodiment can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0033] A user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0035] The server 105 can be a server providing various services, such as a background management server supporting a website browsed by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0036] It should be noted that the picking vehicle track docking method provided in the embodiments of the present application can be generally executed by the server 105. Accordingly, the picking vehicle track docking device provided in the embodiments of the present application can be generally arranged in the server 105. The picking vehicle track docking method provided in the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the picking vehicle track docking device provided in the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the above-mentioned system is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0038] Figure 2 A flowchart of the picking vehicle track docking method according to an embodiment of the present application is shown.

[0039] As Figure 2 shown, the picking vehicle track docking method of this embodiment includes operations S210-S240, which can be executed by an electronic device.

[0040] In operation S210, a reference track image and a reference track image are acquired.

[0041] In operation S220, the reference track image and the reference track image are respectively processed based on a track detection model to obtain target reference track points and target reference track points.

[0042] In operation S230, the target reference track points and the target reference track points are processed based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track.

[0043] In operation S240, in the case that the relative displacement data does not satisfy a preset condition, the driving position of the picking vehicle is adjusted based on the relative displacement data to realize control of the picking vehicle to perform track docking.

[0044] According to the embodiments of the present application, the reference track image is an image obtained by photographing the track before the picking vehicle enters the track, and the reference track image is an image obtained by photographing the track after the picking vehicle enters the track. The reference track image contains the motion track information of the controlled picking vehicle.

[0045] According to the embodiment of the present application, the picking vehicle can be equipped with a monocular camera or other sensors to obtain reference track images and reference track images, which can be used to collect environmental data in real time. The monocular camera ensures that the obtained data can fully reflect the position information of the picking vehicle relative to the track of the greenhouse planting yard in real time. The collected reference track images and reference track images have three channels of red, green and blue, and can contain track information of the greenhouse planting yard.

[0046] According to the embodiment of the present application, the trajectory detection model can be used to obtain the trajectory points in the image. Based on the trajectory detection model processing the reference track image, the target reference trajectory points can be obtained, and based on the trajectory detection model processing the reference track image, the target reference trajectory points can be obtained.

[0047] According to the embodiment of the present application, based on the displacement calculation model processing the target reference trajectory points and the target reference trajectory points, the relative displacement data of the picking vehicle relative to the track can be obtained. The relative displacement data can include the relative angle, the relative moving distance, the relative height difference and the like between the position of the picking vehicle and the track.

[0048] According to the embodiment of the present application, in the case that the relative displacement data does not satisfy the preset condition, the driving position of the picking vehicle can be adjusted based on the relative displacement data, so as to realize the control of the picking vehicle to perform track docking.

[0049] According to the embodiment of the present application, based on the trajectory detection model, the reference track image and the reference track image are processed to obtain the target reference trajectory points and the target reference trajectory points; based on the displacement calculation model, the target reference trajectory points and the target reference trajectory points are processed to obtain the relative displacement data of the picking vehicle relative to the track; in the case that the relative displacement data does not satisfy the preset condition, the picking vehicle is adjusted based on the relative displacement data, so as to realize the control of the picking vehicle to perform track docking. By obtaining the image in real time to accurately extract the trajectory points and calculate the relative displacement, the position deviation of the picking vehicle can be discovered and corrected in time, so as to realize the high-precision track docking. The method has low cost and high adaptability to the greenhouse track environment, and can significantly reduce the damage or operation failure of the equipment caused by inaccurate track docking.

[0050] According to the embodiment of the present application, the displacement calculation model includes an autoencoder and a time series network, and based on the displacement calculation model processing the target reference trajectory points and the target reference trajectory points, the relative displacement data of the picking vehicle relative to the track is obtained, including: based on the autoencoder, the target reference trajectory points and the target reference trajectory points are processed to obtain reference compression features and reference compression features; the reference compression features and the reference compression features are fused to obtain fusion track features corresponding to the reference trajectory points; based on the time series network, the fusion track features corresponding to the plurality of reference trajectory points are processed to obtain the relative displacement data of the picking vehicle relative to the track.

[0051] According to an embodiment of the present application, the reference compressed features can be obtained based on the autoencoder processing the target reference trajectory points, and the benchmark compressed features can be obtained based on the autoencoder processing the target benchmark trajectory points. The autoencoder can adopt an autoencoder architecture based on a deep twin neural network, for further processing the features output by the trajectory detection model.

[0052] According to an embodiment of the present application, by using the autoencoder, unsupervised learning can be performed and features can be automatically learned from the trajectory points, the complex nonlinear relationship of the data can be captured through the nonlinear modeling capability, the compressed features of low dimension can be extracted from the overall image data, the dimensionality reduction can be achieved and the noise and redundancy can be effectively removed, and the hierarchical feature representation of the trajectory can be obtained.

[0053] Figure 3 A network structure diagram of an autoencoder according to an embodiment of the present application is shown.

[0054] As shown in Figure 3 , taking the input target reference trajectory points as an example, the target reference trajectory points 301 are input into the encoder 302, and the encoded features of the target reference trajectory points are output. The encoded features are processed by the max-pooling layer 303 to obtain the max-pooling features. The max-pooling features are input into the fully connected layer 304, and the compressed features 305 (i.e., the reference compressed features) are output. The compressed features 305 are input into the fully connected layer 306, and the decoded features are output. The decoded features are processed by the max-pooling layer 307 to obtain the pooling features. The pooling features are processed by the decoder 308, and the first target reference trajectory points 309 are output.

[0055] According to an embodiment of the present application, the autoencoder is composed of an encoder and a decoder. The encoder is used to map the high-dimensional input data to the low-dimensional latent space. Through a series of convolutional layers, linear transformations, and nonlinear activation layers, etc., the encoder can learn the internal structure and feature patterns of the input data, and extract the most representative feature information, which can be compressed into one-dimensional data, but is not limited thereto. The embodiment of the present application does not limit the dimension. The decoder can decode the one-dimensional data generated by the encoder into output data with the same dimension and form as the original input data. The decoder restores the one-dimensional feature information to the original data by reversing the process of the encoder.

[0056] According to an embodiment of the present application, in the process of encoding and decoding, the loss function of the autoencoder can be constituted by calculating the Euclidean distance between the data output by the decoder and the original data, as shown in formula (1).

[0057] ,

[0058]

[0059] (1) ;

[0060] wherein X represents the position of the track point input to the encoder, represents the position of the n th track point input, Y represents the position of the track point output by the decoder, represents the position of the n th track point output by the decoder, and d (X, Y) represents the Euclidean distance between the position of the track point input to the encoder and the position of the track point output by the decoder.

[0061] According to the embodiment of the present application, the reference compression feature and the reference compression feature can be fused to obtain the fusion track feature corresponding to the reference track point; based on the time sequence network processing the fusion track feature corresponding to each of the plurality of reference track points, the relative displacement data of the picking vehicle relative to the track can be obtained.

[0062] According to the embodiment of the present application, by using the autoencoder to process the reference track point and the reference track point respectively, the reference compression feature and the reference compression feature are obtained, which can effectively reduce the feature dimension, retain the key information and improve the efficiency of subsequent processing; the reference compression feature and the reference compression feature are fused to obtain the fusion track feature, which can provide more learning information for the subsequent processing of the time sequence network; based on the time sequence network processing the fusion track feature, the relative displacement value of the position of the picking vehicle relative to the position of the track is obtained, which combines the feature compression of the autoencoder and the efficient processing of the time sequence network, reduces the calculation cost and improves the operation efficiency.

[0063] According to the embodiment of the present application, based on the time sequence network processing the fusion track feature corresponding to each of the plurality of reference track points, the relative displacement data of the picking vehicle relative to the track is obtained, comprising: based on the time sequence network processing the fusion track feature corresponding to each of the plurality of reference track points, the hidden feature is obtained; the hidden feature is processed by using the activation function to obtain the predicted track point; the predicted track point is compared with the target reference track point to obtain the relative displacement data.

[0064] According to the embodiment of the present application, the time sequence network can include long short-term memory network and the like, but is not limited thereto, and the embodiment of the present application does not limit the type of network. Based on the time sequence network processing the fusion track feature corresponding to each of the plurality of reference track points, the hidden feature can be obtained; the hidden feature is processed by using the activation function to obtain the predicted track point; the predicted track point is compared with the target reference track point to obtain the relative displacement data.

[0065] According to an embodiment of the present application, the one-dimensional data after encoding can be input to a time series network, and relative displacement data is output. For example, the time series network is a long short-term memory network (LSTM), which has multiple gating units, including a reset gate and an update gate, wherein (1≤ ≤ ) represents the value of the reset gate of the LSTM at the lth layer. The activation value of the hidden feature is a linear interpolation between the previous activation value and the candidate hidden activation value, and the specific process is shown in equations (2), (3), (4) and (5):

[0066] = ( [ , ]) (2);

[0067] = ( [ , ]) (3);

[0068] = ( [ , ]) (4);

[0069] =( ) + (5);

[0070] wherein, represents the value of the reset gate at the lth layer, represents an activation function, represents a weight matrix of the reset gate, represents the hidden feature of the (l-1)th layer, represents an input at time step t, represents the value of the update gate at the lth layer, represents a weight matrix of the update gate, represents the candidate hidden feature at the lth layer, represents a hyperbolic tangent activation function, represents a weight matrix of the candidate hidden feature, represents a product, represents the final hidden feature of the lth layer, represents a complement of the update gate, represents a value of the update gate.

[0071] According to an embodiment of the present application, the final hidden feature of the lth layer of the long short-term memory network is processed by using an activation function, and relative displacement data can be obtained.

[0072] According to an embodiment of the present application, by processing the fused track features by using the time series network, noise can be filtered more effectively, key features can be adaptively extracted, and the processing capability for complex dynamic changes can be enhanced. With the multi-layer supervised guidance learning, the time series network can better combine earlier information and current information, so that the relative displacement data can be more accurately and stably inferred, and the overall performance and inference accuracy are improved.

[0073] According to an embodiment of the present application, the reference track image and the reference track image are processed based on the track detection model respectively to obtain the target reference track point and the target reference track point, including: processing the reference track image based on the track detection model to obtain classification probability values of a plurality of reference track points, and processing the reference track image based on the track detection model to obtain classification probability values of a plurality of reference track points; the classification probability values of the plurality of reference track points are processed by using non-maximum suppression to obtain the target reference track point; and the classification probability values of the plurality of reference track points are processed by using non-maximum suppression to obtain the target reference track point.

[0074] According to an embodiment of the present application, by using the track detection model to identify the track from the reference track image and the reference track image, the track in the greenhouse planting yard has characteristics such as elongation, and therefore the track is abstracted as a straight line and further converted into a series of feature points (i.e. track points). The reference track image is processed based on the track detection model to obtain a plurality of reference track points, each of which corresponds to a classification probability value, and the reference track image is processed based on the track detection model to obtain a plurality of reference track points, each of which corresponds to a classification probability value.

[0075] According to an embodiment of the present application, the classification probability values of the plurality of reference track points are processed by using non-maximum suppression to obtain the target reference track point, and the classification probability values of the plurality of reference track points are processed by using non-maximum suppression to obtain the target reference track point. Among them, according to the positive classification probability values in the classification probability values of the plurality of reference track points, the two largest reference track points can be selected as the target reference track points. According to the positive classification probability values in the classification probability values of the plurality of reference track points, the two largest reference track points can be selected as the target reference track points.

[0076] According to an embodiment of the present application, the relative displacement data includes a relative angle and a relative movement distance; in a case where the relative displacement data does not satisfy a preset condition, the driving position of the picking vehicle is adjusted based on the relative displacement data to realize control of the picking vehicle to perform track docking, including: in a case where the relative angle does not satisfy a preset angle and the relative movement distance satisfies a preset movement distance, the picking vehicle is controlled to move based on the relative angle to realize movement of the picking vehicle onto the track; in a case where the relative angle satisfies the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle is controlled to move based on the relative movement distance to realize movement of the picking vehicle onto the track; and in a case where the relative angle does not satisfy the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle is controlled to move based on the relative angle and the relative movement distance to realize movement of the picking vehicle onto the track.

[0077] According to an embodiment of the present application, in a case where the relative displacement data does not satisfy a preset condition, it indicates that the picking vehicle has a large deviation from the track, and the picking vehicle still needs to be controlled to move. In a case where the relative displacement data satisfies the preset condition, the distance deviation between the picking vehicle and the track can be ignored, and the picking vehicle does not need to be adjusted and can be docked with the track.

[0078] According to an embodiment of the present application, in a case where the relative angle does not satisfy a preset angle and the relative movement distance satisfies a preset movement distance, the picking vehicle can be controlled to move based on the relative angle to realize movement of the picking vehicle onto the track; in a case where the relative angle satisfies the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle can be controlled to move based on the relative movement distance to realize movement of the picking vehicle onto the track; and in a case where the relative angle does not satisfy the preset angle and the relative movement distance does not satisfy the preset movement distance, the picking vehicle can be controlled to move based on the relative angle and the relative movement distance to realize movement of the picking vehicle onto the track.

[0079] According to an embodiment of the present application, the picking vehicle track docking method further includes: in a case where the picking vehicle has completed a picking task in a first row, controlling the picking vehicle to disengage from the track of the first row and move to a preset position; and in response to performing a picking task in a second row adjacent to the first row, controlling the picking vehicle to dock with the track of the second row.

[0080] According to an embodiment of the present application, in a picking operation environment of a greenhouse planting farm, fixed tracks can be arranged between rows, and the distance between the rows is kept constant. In a case where the picking vehicle has completed a picking task in a first row, the picking vehicle can be controlled to disengage from the track of the first row and move to a preset position; and in response to performing a picking task in a second row adjacent to the first row, the picking vehicle can be controlled to dock with the track of the second row.

[0081] According to the embodiment of the present application, the picking vehicle can quickly switch to the next row after completing a row task, reducing the waiting time caused by track switching, thereby significantly improving the overall picking efficiency. Through the automatic track disengagement and docking mechanism, manual intervention is reduced, labor cost is reduced, and the stability and reliability of the operation are improved.

[0082] According to the embodiment of the present application, the track detection model is trained based on the following steps: obtaining sample track images and label trajectory points; iteratively performing the following operations: processing the sample track images based on the initial track detection model to obtain sample trajectory points; determining the number of trajectory points overlapping the label trajectory points and the sample trajectory points; subtracting the coordinate value of the i-th trajectory point from the coordinate value of the i+1-th trajectory point to obtain an angle difference value, the coordinate value of the i-th trajectory point representing subtracting the coordinate value of the i-th label trajectory point from the coordinate value of the i-th sample trajectory point, the coordinate value of the i+1-th trajectory point representing subtracting the coordinate value of the i+1-th label trajectory point from the coordinate value of the i+1-th sample trajectory point, i>0; dividing the angle difference value by the number of trajectory points to obtain an average angle change value; processing the average angle change value using an angle loss function to obtain an angle loss value; training the initial track detection model using the angle loss value to obtain a trained track detection model.

[0083] According to the embodiment of the present application, the sample trajectory points can be obtained by processing the sample track images based on the initial track detection model.

[0084] According to the embodiment of the present application, the maximum value between the start index of the label trajectory points on the real track line and the start index of the sample trajectory points can be determined, the minimum value between the end index of the label trajectory points on the real track line and the end index of the sample trajectory points can be determined, and the number of trajectory points overlapping the label trajectory points and the sample trajectory points can be determined.

[0085] According to the embodiment of the present application, the coordinate value of the i-th trajectory point can be subtracted from the coordinate value of the i+1-th trajectory point to obtain an angle difference value, the coordinate value of the i-th trajectory point representing subtracting the coordinate value of the i-th label trajectory point from the coordinate value of the i-th sample trajectory point, the coordinate value of the i+1-th trajectory point representing subtracting the coordinate value of the i+1-th label trajectory point from the coordinate value of the i+1-th sample trajectory point, i>0.

[0086] According to the embodiment of the present application, the angle difference value can be divided by the number of trajectory points to obtain an average angle change value, as shown in formula (6).

[0087] (6);

[0088] wherein , , . wherein , respectively are the end and start indexes of the label track point, , respectively are the end and start indexes of the sample track point. and respectively are the horizontal coordinates of the sample track point and the label track point at the same index.

[0089] According to the embodiment of the present application, the average value of the angle change is processed by using the angle loss function, so as to obtain an angle loss value; the initial track detection model is trained by using the angle loss value, so as to obtain a trained track detection model. The angle loss function is shown in formula (7).

[0090] (7);

[0091] wherein g represents the average value of the angle change, and represents the angle loss value.

[0092] Figure 4 Fig. 1 shows a network structure diagram of a track detection model according to an embodiment of the present application.

[0093] As shown in Fig. 1, first, the track image 401 is input into the bottom-up backbone network 402. The convolution layer thereof is shown in formula (8). Figure 4

[0094] (8);

[0095] wherein, is the input feature map , the height of the input feature map is , the width is , and the channel number is , represents a weight parameter, represents the input data with the coordinate being , the channel dimension being l, the dimension being , is the length of the convolution kernel, is the width of the convolution kernel, the input channel number is , and the output channel number is , represents a bias value, is the output feature map, the dimension , the output feature map height , and the output feature map width , is the padding size, is the step length.

[0096] ​According to an embodiment of the present application, after the convolution layer, a ReLU (Rectified Linear Unit) nonlinear activation function is used, as shown in equation (9):

[0097] (9);

[0098] max represents a maximum value operation, and for the output of the convolution layer , after the activation function ReLU, as shown in equation (10):

[0099] (10);

[0100] Then down-sampling is performed through a max-pooling layer, as shown in equation (11):

[0101] (11);

[0102] In the max-pooling, represents the value of the kth channel at position (i, j) in the output feature map, x is the input feature map, i and j are coordinate indexes of the output feature map y, m and n are local coordinate indexes in the pooling window, S is the pooling step, max represents a maximum value operation, and the value of the corresponding position (i, j) in the output feature map y is obtained by taking the maximum value of the elements of the input feature map x in the pooling window moving with a step S, so as to realize down-sampling of the feature map and retain the main features.

[0103] By repeating the above operation, the feature map layers of the last 3 layers are obtained to construct a top-down and lateral connection channel, and through the top-down and lateral connection channel, the features in the backbone network 402 can be input into the pyramid network 403. The top-down channel is constructed by a bilinear interpolation up-sampling operation, and the up-sampling is performed by a bilinear interpolation method , the side is reduced in dimension by a 1x1 convolution kernel , and then feature fusion is performed, as shown in equation (12):

[0104] (12);

[0105] wherein, represents a feature map, represents a feature map after fusion of the ith layer, represents a feature map after fusion of the i+1th layer.

[0106] According to an embodiment of the present application, the feature map at the top layer of the pyramid network 403 can be output into the prediction head 404, which uniformly samples in the picture for region alignment, predefines prior feature points, and the structure thereof includes the coordinates of the starting point of the track line and the angle of the track line, as well as the horizontal coordinates of a plurality of track line feature points. Through the region alignment of the prior feature points and the feature map, the track prior feature is obtained through convolution and full connection layer operations and the like , and the global feature is converted into one dimension after scaling and flattening , and the attention matrix is constructed , as shown in formulas (13) and (14).

[0107] (13);

[0108] (14);

[0109] wherein is the product of the matrix dimensions, is the result after the attention matrix is multiplied by the global feature matrix , and is added to the enhanced result of the track prior feature .

[0110] According to an embodiment of the present application, the After being adjusted into the output format through a series of linear layers, the initialized prior feature point list is appended. The above operation is repeated until the bottom layer of the pyramid network 403. Then, through non-maximum suppression, that is, through sorting and cutting of positive sample probabilities, the track point detection result 405 of the track image is output.

[0111] According to an embodiment of the present application, the sample track point and the label track point can be processed by using an angle loss function (i.e., angle loss) to generate an angle loss value; the sample track point and the label track point can be processed by using a lane ious loss function (i.e., lane ious loss) to generate an intersection-over-union loss value; the sample track point and the label track point can be processed by using a focal loss function (i.e., Focal loss) to generate a focal loss value; and the total loss value of the angle loss value, the intersection-over-union loss value and the focal loss value can be calculated, and the total loss value is used to train the initial track detection model to obtain the trained track detection model.

[0112] According to an embodiment of the present application, the track detection model can comprehensively and accurately capture global and local features in the environment. It can filter out valuable information in complex environmental data, and then generate a preliminary path prediction to provide a reliable basis for subsequent path planning.

[0113] Figure 5A flowchart of a harvesting vehicle track docking method according to an embodiment of the present invention is shown.

[0114] like Figure 5 As shown, a monocular camera 501 is used to acquire a reference track image 502 and a reference track image 503. The reference track image 502 represents the image obtained by taking pictures of the track before the picking vehicle enters the track, and the reference track image 503 represents the image obtained by taking pictures of the track after the picking vehicle enters the track. The reference track image 502 represents the movement trajectory of the controlled picking vehicle. The trajectory detection model 504 can be used to process the reference track image 502 and the reference track image 503 respectively to obtain the target reference trajectory point 505 and the target reference trajectory point 506. Based on the autoencoder 507, the target reference trajectory point 505 and the target baseline trajectory point 506 are processed respectively to obtain reference compressed features 508 and baseline compressed features 509. The reference compressed features 508 and baseline compressed features 509 are fused to obtain fused track features 510 corresponding to the reference trajectory points. The fused track features 510 corresponding to each of the multiple reference trajectory points are processed using a time series network 511 (i.e., LSTM), and then the fused track features 510 are processed using an activation function (i.e., softmax) to obtain the relative displacement data 512 of the picking vehicle relative to the track. It is determined whether the relative displacement data meets the preset conditions 513. If the relative displacement data does not meet the preset conditions, the driving position of the picking vehicle is adjusted based on the relative displacement data, and the reference track image 502 is re-acquired using a monocular camera 501 to control the picking vehicle to perform track docking. If the relative displacement data meets the preset conditions, the picking vehicle can be controlled to perform track docking.

[0115] According to an embodiment of the present invention, Figure 5 A monocular vision-based positioning and alignment system suitable for greenhouse environments, along with a corresponding greenhouse harvesting vehicle, is disclosed. The system consists of four main modules: Module A acquires reference track images; Module B obtains target reference trajectory points 506 and target baseline trajectory points 507; Module C obtains reference compression features 508 and baseline compression features 509; and Module D performs path prediction and planning, ultimately enabling the harvesting vehicle to accurately align with the track in a standard greenhouse plantation. In modules B and C, green lines represent standard reference tracks, while red lines represent real-time track paths. Through a closed-loop feedback mechanism, the system can optimize path planning in real time, ensuring accurate track positioning for the robot.

[0116] Figure 6 A flowchart of a harvester track docking method according to another embodiment of the present invention is shown.

[0117] like Figure 6As shown, the reference track image 601 and the reference track image 602 can be obtained, the reference track image 601 and the reference track image 602 are respectively processed based on the track detection model 603 to obtain the target reference track point 604 and the target reference track point 605, the target reference track point 604 and the target reference track point 605 are respectively processed based on the encoder 606 in the auto-encoder to obtain the reference compression feature and the reference compression feature, and the reference compression feature and the reference compression feature are fused to obtain the fusion track feature corresponding to the reference track point. The fusion track feature corresponding to each of the plurality of reference track points is processed based on the long short-term memory network 607 to obtain the relative displacement data 608 of the picking vehicle relative to the track. In the case where the relative displacement data does not satisfy the preset condition (i.e., the relative displacement data is greater than the threshold value), it is indicated that the picking vehicle has not reached the predetermined track position 609, and the movement 611 of the picking vehicle can be adjusted based on the relative displacement data. In the case where the relative displacement data satisfies the preset condition (i.e., the relative displacement data is less than or equal to the threshold value), it is indicated that the picking vehicle reaches the predetermined track position 610.

[0118] Figure 7 A structural block diagram of a picking vehicle track docking device according to an embodiment of the application is shown.

[0119] As Figure 7 shown, the picking vehicle track docking device of this embodiment includes an acquisition module 710, a track point obtaining module 720, a relative displacement data obtaining module 730, and a track docking module 740.

[0120] The acquisition module 710 is configured to acquire a reference track image and a reference track image, the reference track image being an image obtained by photographing the track before the picking vehicle enters the track, the reference track image being an image obtained by photographing the track after the picking vehicle enters the track, and the reference track image containing the motion track information of the controlled picking vehicle.

[0121] The track point obtaining module 720 is configured to process the reference track image and the reference track image based on the track detection model to obtain the target reference track point and the target reference track point.

[0122] The relative displacement data obtaining module 730 is configured to process the target reference track point and the target reference track point based on the displacement calculation model to obtain the relative displacement data of the picking vehicle relative to the track.

[0123] The track docking module 740 is configured to, in the case where the relative displacement data does not satisfy the preset condition, adjust the driving position of the picking vehicle based on the relative displacement data to realize the control of the picking vehicle to perform track docking.

[0124] According to an embodiment of the present application, the reference track image and the reference track image are respectively processed based on a track detection model to obtain target reference track points and target reference track points; the target reference track points and the target reference track points are processed based on a displacement calculation model to obtain relative displacement data of the picking vehicle relative to the track; and in the case that the relative displacement data does not satisfy a preset condition, the picking vehicle is adjusted based on the relative displacement data to realize control of the picking vehicle to perform track docking. By real-time acquisition of images to accurately extract track points and calculate relative displacement, position deviation of the picking vehicle is discovered and corrected in time, so that high-precision track docking is realized, and equipment damage or operation failure caused by inaccurate track docking can be significantly reduced.

[0125] According to an embodiment of the present application, the displacement calculation model comprises a self-encoder and a time series network.

[0126] According to an embodiment of the present application, the relative displacement data obtaining module comprises a compressed feature obtaining unit, a fused track feature obtaining unit and a relative displacement data obtaining unit.

[0127] The compressed feature obtaining unit is configured to process the target reference track points and the target reference track points based on the self-encoder respectively to obtain reference compressed features and reference compressed features.

[0128] The fused track feature obtaining unit is configured to fuse the reference compressed features and the reference compressed features to obtain fused track features corresponding to the reference track points.

[0129] The relative displacement data obtaining unit is configured to process the fused track features corresponding to the plurality of reference track points based on the time series network to obtain the relative displacement data of the picking vehicle relative to the track.

[0130] According to an embodiment of the present application, the relative displacement data obtaining unit comprises a hidden feature subunit, a predicted track point subunit and a relative displacement data subunit.

[0131] The hidden feature subunit is configured to process the fused track features corresponding to the plurality of reference track points based on the time series network to obtain hidden features.

[0132] The predicted track point subunit is configured to process the hidden features by using an activation function to obtain predicted track points.

[0133] The relative displacement data subunit is configured to compare the predicted track points with the target reference track points to obtain the relative displacement data.

[0134] According to an embodiment of the present application, the track point obtaining module comprises a classification probability value unit, a target reference track point unit and a target reference track point unit.

[0135] The classification probability value unit is configured to process the reference track image based on the track detection model to obtain classification probability values of the reference track points, and process the reference track image based on the track detection model to obtain classification probability values of the reference track points.

[0136] The target reference track point unit is configured to process the classification probability values of the reference track points by using the non-maximum suppression to obtain the target reference track point.

[0137] The target reference track point unit is configured to process the classification probability values of the reference track points by using the non-maximum suppression to obtain the target reference track point.

[0138] According to an embodiment of the present application, the relative displacement data comprises a relative angle and a relative movement distance.

[0139] According to an embodiment of the present application, the track docking module comprises a first unit, a second unit and a third unit.

[0140] The first unit is configured to control the picking vehicle to move based on the relative angle to realize movement of the picking vehicle to the track when the relative angle does not satisfy a preset angle and the relative movement distance satisfies a preset movement distance.

[0141] The second unit is configured to control the picking vehicle to move based on the relative movement distance to realize movement of the picking vehicle to the track when the relative angle satisfies the preset angle and the relative movement distance does not satisfy the preset movement distance.

[0142] The third unit is configured to control the picking vehicle to move based on the relative angle and the relative movement distance to realize movement of the picking vehicle to the track when the relative angle does not satisfy the preset angle and the relative movement distance does not satisfy the preset movement distance.

[0143] According to an embodiment of the present application, the picking vehicle track docking device of the embodiment further comprises a disengaging module and a docking module.

[0144] The disengaging module is configured to control the picking vehicle to disengage from the track of the first row and move to a preset position when the picking vehicle completes the picking task of the first row.

[0145] The docking module is configured to control the picking vehicle to dock with the track of the second row in response to performing the picking task of the second row, the first row and the second row being adjacent.

[0146] According to an embodiment of the present application, sample track images and label track points are acquired; the following operations are iteratively performed: processing the sample track images based on an initial track detection model to obtain sample track points; determining the number of track points overlapped by the label track points and the sample track points; subtracting the coordinate value of the (i+1)th track point from the coordinate value of the ith track point to obtain an angle difference value, the coordinate value of the ith track point representing the coordinate value of the ith sample track point minus the coordinate value of the ith label track point, the coordinate value of the (i+1)th track point representing the coordinate value of the (i+1)th sample track point minus the coordinate value of the (i+1)th label track point, i>0; dividing the angle difference value by the number of track points to obtain an average angle change value; processing the average angle change value using an angle loss function to obtain an angle loss value; training the initial track detection model using the angle loss value to obtain a trained track detection model.

[0147] According to an embodiment of the present application, any one or more of the obtaining module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the obtaining module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. hardware or firmware, or any one of software, hardware and firmware three implementation ways or in any appropriate combination of any of them. Alternatively, at least one of the obtaining module 710, the track point obtaining module 720, the relative displacement data obtaining module 730, and the track docking module 740 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0148] Figure 8 A block diagram of an electronic device suitable for implementing the gyroscope drift error compensation method according to an embodiment of the present application is shown.

[0149] As Figure 8As shown, the electronic device according to embodiments of the present application includes a processor 801 which can perform various appropriate actions and processes according to a program stored in a ROM 802 (i.e., read only memory) or a program loaded into a RAM 803 (i.e., random access memory) from a storage section 808. The processor 801 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or multiple processing units to perform the various actions of the method processes according to embodiments of the present application.

[0150] In the RAM 803, various programs and data required for operation of the electronic device are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 802 and / or the RAM 803. Note that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0151] According to embodiments of the present application, the electronic device can also include an input / output (I / O) interface 805 which is also connected to the bus 804. The electronic device can also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read therefrom is installed into the storage section 808 as necessary.

[0152] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which when executed, implement the method according to embodiments of the present application.

[0153] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but not limited to: a portable computer diskette, a hard disk, a random access memory (RAM 803), a read-only memory (ROM 802), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this application, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer readable storage medium can include the ROM 802 and / or the RAM 803 described above, and / or one or more memory chips other than the ROM 802 and the RAM 803.

[0154] Embodiments of the present application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the gyro drift error compensation method provided by the embodiments of the present application.

[0155] The above functions defined in the system / device of the embodiments of the present application are performed when the computer program is executed by the processor 801. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0156] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on a network medium, and be downloaded and installed through the communication part 809, and / or installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

[0157] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the detachable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0158] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language can include, but is not limited to, Java, C++, python, "C" language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0159] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0160] Those skilled in the art will appreciate that the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly noted in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways without departing from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0161] The embodiments of the present application have been described above. However, these embodiments are merely for the purpose of illustration and are not intended to limit the scope of the present application. Although the respective embodiments are described above separately, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Numerous alternatives and modifications of embodiments of the present application can be made by those skilled in the art without departing from the scope of the present application, and all such alternatives and modifications are to be included within the scope of the present application.

Claims

1. A picking cart track docking method, characterized by, The method comprises the following steps: acquiring a reference track image and a reference track image, the reference track image being an image obtained by photographing the track before the picking vehicle enters the track, and the reference track image being an image obtained by photographing the track after the picking vehicle enters the track, the reference track image containing the motion track information of the controlled picking vehicle; processing the reference track image and the reference track image based on a track detection model to obtain target reference track points and target reference track points; processing the target reference track points and the target reference track points based on a deep twin neural network autoencoder to obtain reference compressed features and reference compressed features, and a displacement calculation model comprising the autoencoder and a time series network; fusing the reference compressed features and the reference compressed features to obtain fused track features corresponding to the target reference track points; processing the fused track features corresponding to each of the target reference track points based on the time series network to obtain hidden features; processing the hidden features using an activation function to obtain predicted track points; comparing the predicted track points with the target reference track points to obtain relative displacement data; in the case that the relative displacement data does not meet the preset condition, adjusting the target reference track points corresponding to the picking vehicle based on the relative displacement data, so that the target reference track points are aligned with the target reference track points, to realize control of the picking vehicle to perform track docking.

2. The method of claim 1, wherein, The method comprises the following steps: processing the reference track image and the reference track image based on a track detection model to obtain target reference track points and target reference track points; processing the reference track image based on the track detection model to obtain classification probability values of a plurality of reference track points, and processing the reference track image based on the track detection model to obtain classification probability values of a plurality of reference track points; processing the classification probability values of the plurality of reference track points using non-maximum suppression to obtain the target reference track points; 3. The method of claim 1, wherein, processing the classification probability values of the plurality of reference track points using non-maximum suppression to obtain the target reference track points. The relative displacement data comprises a relative angle and a relative movement distance. In the case that the relative displacement data does not meet the preset condition, adjusting the driving position of the picking vehicle based on the relative displacement data to realize control of the picking vehicle to perform track docking, comprising: in the case that the relative angle does not meet the preset angle and the relative movement distance meets the preset movement distance, controlling the picking vehicle to move based on the relative angle to realize the movement of the picking vehicle to the track; in the case that the relative angle meets the preset angle and the relative movement distance does not meet the preset movement distance, controlling the picking vehicle to move based on the relative movement distance to realize the movement of the picking vehicle to the track; in the case that the relative angle does not meet the preset angle and the relative movement distance does not meet the preset movement distance, controlling the picking vehicle to move based on the relative angle and the relative movement distance to realize the movement of the picking vehicle to the track.

4. The method of claim 1, further comprising: controlling the picking vehicle to disengage from a track of a first row and move to a preset position when the picking vehicle finishes a picking task of the first row; and controlling the picking vehicle to dock with a track of a second row adjacent to the first row in response to performing a picking task of the second row.

5. The method of claim 1, wherein the track detection model is trained based on the following steps, obtaining sample track images and labeled track points; iteratively performing the following operations: processing the sample track images based on an initial track detection model to obtain sample track points; determining a number of track points that overlap between the labeled track points and the sample track points; subtracting a coordinate value of an i+1th track point from a coordinate value of an ith track point to obtain an angle difference value, wherein the coordinate value of the ith track point represents a coordinate value of an ith sample track point minus a coordinate value of an ith labeled track point, and the coordinate value of the i+1th track point represents a coordinate value of an i+1th sample track point minus a coordinate value of an i+1th labeled track point, i>0; dividing the angle difference value by the number of track points to obtain an average angle change value; processing the average angle change value using an angle loss function to obtain an angle loss value; training the initial track detection model using the angle loss value to obtain a trained track detection model. comprising:

6. A picking cart track docking device, characterized in that, an obtaining module, configured to obtain a reference track image and a benchmark track image, the reference track image being an image obtained by photographing a track before a picking vehicle enters the track, the benchmark track image being an image obtained by photographing the track after the picking vehicle enters the track, the reference track image containing motion track information of the picking vehicle controlled; a track point obtaining module, configured to process the reference track image and the benchmark track image based on a track detection model to obtain target reference track points and target benchmark track points; a relative displacement data obtaining module, configured to process the target reference track points and the target benchmark track points based on a deep twin neural network autoencoder to obtain reference compressed features and benchmark compressed features, a displacement calculation model comprising the autoencoder and a time series network; fusing the reference compressed features and the benchmark compressed features to obtain fused track features corresponding to the target reference track points; processing the fused track features corresponding to the target reference track points based on the time series network to obtain hidden features; processing the hidden features using an activation function to obtain predicted track points; comparing the predicted track points with the target benchmark track points to obtain the relative displacement data; a track docking module, configured to adjust target reference track points corresponding to the picking vehicle based on the relative displacement data when the relative displacement data does not satisfy a preset condition, so that the target reference track points are aligned with the target benchmark track points, to realize control of the picking vehicle to perform track docking.

7. An electronic device, comprising: one or more processors; ​ a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-5.

8. A computer program product comprising computer programs or instructions, characterized in that, the computer program or instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-5.

Images