Method and equipment for detecting getting-on and getting-off of articles

Abstract

The invention provides a method and equipment for detecting getting-on and getting-off of articles, and the method comprises the steps: detecting the weight of articles on a warehouse forklift; reading the article labels on the warehouse forklift and the number of the article labels; identifying the types and the visual quantity of articles on the warehouse forklift; based on the weight of the articles on the warehouse forklift, the article labels, the number of the article labels, the types of the articles and the visual number of the articles, the types and the number of the articles on the warehouse forklift are calculated; and the current warehouse-in and warehouse-out states of the warehouse forklift are confirmed, the current warehouse-in and warehouse-out states, the article types and the article number are synchronized to a warehouse management system, and the current warehouse-in and warehouse-out states comprise the warehouse-out state and the warehouse-in state. According to the invention, getting-on and getting-off article detection of the warehouse forklift is realized, and the management efficiency of reference warehouse-in and warehouse-out is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the warehouse management technical field, and in particular to a method and device for detecting the loading and unloading of goods. BACKGROUND

[0002] In a warehouse management system, the loading and unloading of goods is registered and managed by a warehouse manager. With the development of radio frequency identification technology, the warehouse manager can read the tags of the goods being loaded and unloaded by a reader and directly synchronize them to the system to confirm the loading and unloading of the goods, thereby improving the efficiency of warehouse management. However, the single loading and unloading management is too much work for the warehouse manager, who needs to scan each item being loaded and unloaded one by one. If the warehouse manager completely relies on RFID detection of the loading and unloading of goods, there will be a certain error rate. SUMMARY

[0003] The present application aims to solve the problem of loading and unloading management of goods and proposes a method and device for detecting the loading and unloading of goods.

[0004] The present application proposes a method for detecting the loading and unloading of goods, which is set in a warehouse forklift. The method comprises: detecting the weight of the goods on the warehouse forklift; reading the tags of the goods on the warehouse forklift and the number of tags; identifying the type of goods on the warehouse forklift and the visual quantity of goods; calculating the type of goods and the quantity of goods on the warehouse forklift based on the weight of the goods, the tags of the goods, the number of tags, the type of goods, and the visual quantity of goods; and confirming the current loading and unloading state of the warehouse forklift and synchronizing the current loading and unloading state, the type of goods, and the quantity of goods to the warehouse management system, wherein the current loading and unloading state includes the loading state and the unloading state. In some embodiments, reading the tags of the goods on the warehouse forklift and the number of tags comprises: respectively using two readers set at the diagonal positions of the base of the warehouse forklift to read the tags of the goods and the number of tags.

[0005] In some embodiments, calculating the type of goods and the quantity of goods on the warehouse forklift based on the weight of the goods, the tags of the goods, the number of tags, the type of goods, and the visual quantity of goods comprises: when the reading results of the two readers are consistent, using the tags of the goods and the number of tags as the type of goods and the quantity of goods on the warehouse forklift.

[0006] In some embodiments, calculating the type and number of items on a warehouse forklift based on the weight of the items, the item labels, the number of item labels, the item types, and the visual number of items on the warehouse forklift includes: When the reading results of two readers are inconsistent, the structure data is constructed based on the output results of the readers, the weight of the items on the warehouse forklift, the type of items, and the visual number of items; The structure data is fed into a pre-trained inbound and outbound identification model, which outputs the type and quantity of items on the warehouse forklift; Identifying the type of items and the visual quantity of items on a warehouse forklift, including starting image acquisition when a weight sensor detects that a weight is applied to the warehouse forklift and stopping image acquisition when the weight sensor detects that the weight applied to the warehouse forklift has not changed; Identify the type and visual quantity of items on warehouse forklifts based on multiple captured images.

[0007] In some embodiments, the inbound and outbound identification model includes: A first feature extraction network is used to obtain a feature vector of difference information of a reader reading result; The second feature extraction network is used to obtain the feature vector of the object image; The output network has inputs including: a difference information feature vector, an item image feature vector, and an item weight. The output of the output network is the item type and the item quantity on the warehouse forklift.

[0008] In some embodiments, the first feature extraction network includes: An imaging module, used for encoding the output information of the reader tag into a difference information feature image; The image feature extraction network is used to extract the difference information feature vector of the difference information feature image.

[0009] In some embodiments, the output network comprises: a flattening layer, configured to flatten the difference information feature vector to the same length or width as the object image feature vector; A splicing layer, configured to splice the difference information feature vector and the object image feature vector into a splicing feature vector in a length direction or a width direction; Multi-layer fully connected layers, used to fuse the concatenated feature vectors; The output layer is used to output the type and quantity of items on the warehouse forklift based on the concatenated feature vector and the weight of the items.

[0010] In some embodiments, a feature secondary extraction layer is further arranged between the concatenation layer and the multi-layer full connection layer, and is used to abstract the concatenation feature vector.

[0011] In some embodiments, the input of the first feature extraction network includes the weight of the article, so that the first feature extraction network calculates the difference information feature vector with reference to the weight of the article; and / or, The input of the second feature extraction network includes the weight of the article, so that the second feature extraction network calculates the article image feature vector with reference to the weight of the article.

[0012] The present application also proposes a device for detecting the loading and unloading of articles, which comprises: a weight detection device for detecting the weight of the article on the warehouse forklift; an RFID reader for reading the article label and the number of article labels on the warehouse forklift an image recognition device for collecting the image of the article on the warehouse forklift; one or more processors for calculating the type and quantity of the article on the warehouse forklift based on the weight of the article on the warehouse forklift, the article label, the number of article labels, the type of the article and the visual quantity of the article; and confirming the current warehouse in-out state of the warehouse forklift, and synchronizing the current warehouse in-out state, the type of the article and the quantity of the article to the warehouse management system, wherein the current warehouse in-out state comprises the warehouse-out state and the warehouse-in state.

[0013] The present application improves the reliability of the RFID reading result by using the weight of the article on the warehouse forklift to assist verification when the reader reads the article label and the number of articles on the forklift, and when the RFID reading result is inconsistent, uses the image collected by the image sensor to identify the type and visual quantity of the article on the warehouse forklift to provide a reference information, and then uses the RFID reading result and the identification result of the image collected by the image sensor to comprehensively confirm the type and quantity of the article on the warehouse forklift; finally, the type and quantity of the article on the warehouse forklift and the current warehouse in-out state of the warehouse forklift are sent to the warehouse management system for secondary confirmation, thereby improving the reliability of the warehouse management system. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 a flowchart of an embodiment of the method for detecting the loading and unloading of articles of the present application; Figure 2 a warehouse in-out model block diagram of an embodiment of the method for detecting the loading and unloading of articles of the present application. DETAILED DESCRIPTION

[0015] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described in the description of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0016] In the description of the present application, the terms "first", "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly and specifically limited.

[0017] In the description of the present application, the term "for example" is used to indicate "as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily interpreted as more preferred or more advantageous than other embodiments. The following description is given in order to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present application can be implemented without using these specific details. In other examples, well-known structures and processes will not be described in detail in order to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the shown embodiments, but is consistent with the broadest scope of the principles and characteristics disclosed.

[0018] Example 1 The present application provides a method for detecting the loading and unloading of goods, which is arranged on a warehouse forklift. In the present application, the warehouse forklift is provided with a weighing sensor for detecting the weight of goods, a reader for reading RFID tags, a camera for collecting images of goods on the warehouse forklift, a communication module in communication connection with a warehouse management system, and a processor.

[0019] The method can confirm the number and type of goods on the forklift, and repeatedly confirm the number and type of goods in combination with the warehouse management system. It should be noted that in the warehouse management system of the present application, there is a detection of warehouse entry and exit, and the repeated confirmation is realized in combination with the detection of goods on the warehouse forklift.

[0020] Reference Figure 1 In some embodiments, the method comprises: S100, detecting the weight of goods on the warehouse forklift; Exemplarily, the weighing sensor can be arranged at the bottom of the warehouse forklift to detect the weight of the articles stacked on the warehouse forklift. It should be noted that the warehouse management system has pre-recorded the types and weights of various articles, so that the processor can calculate the quantity of the articles according to the weight of the articles combined with the type of the articles, or infer the type of the articles according to the weight of the articles combined with the quantity of the articles, so that the weight is an important reference information for the article loading and unloading detection of the present application.

[0021] S200, reading the article label and the quantity of the article label on the warehouse forklift; Exemplarily, one or more readers can be arranged on the warehouse forklift to read the RFID label of the article on the forklift, and then obtain the quantity and type of the article. It should be noted that during the reading of the label by the reader, the problems of label overlapping, metal shielding and mutual crosstalk may occur, resulting in inaccurate reading results of the reader.

[0022] In the present application, multiple readers can be arranged to confirm the quantity and type of the article combined with the reading results of the multiple readers. When the multiple detection results are consistent, it is considered that the reading result of the reader is correct, and further, when the multiple detection results are consistent, the weight of each type is multiplied by the quantity to obtain the article weight of the forklift based on the detection result and the type and quantity information, and the correctness of the detection result is further verified combined with the article weight collected by the weighing sensor.

[0023] Exemplarily, if the reading results of the multiple readers are inconsistent, the type and quantity of the article on the warehouse forklift need to be confirmed combined with the article weight and the image information collected in step S300, and the confirmation result is output to the warehouse management system and the warehouse management personnel are reminded to compare and confirm the detection result. In this way, the warehouse management personnel do not need to manually input the detection result, but only need to detect the result on the forklift, and when it is confirmed that there is no problem, it can be directly clicked to confirm, thereby improving the warehouse management efficiency.

[0024] S300, identifying the type and visual quantity of the article on the warehouse forklift; Exemplarily, an image sensor can be arranged at the position of the push handle of the forklift. When the weight sensor detects that the weight is applied to the warehouse forklift, the image collection is started, and the image collection is stopped when the weight sensor detects that the weight applied to the warehouse forklift does not change. In this way, the collected images are all images in the stacking process, reducing the redundancy of the images and greatly reducing the algorithmic expenditure of the processor.

[0025] In the present application, if the collection results of the RFID reader and / or the weighing sensor are consistent, the detection results can be directly output, and the image data collected by the image sensor does not need to be used. However, if the collection results of the RFID reader and / or the weighing sensor are inconsistent, the image data collected by the image sensor needs to be used to assist in confirming the number and type of the goods on the forklift. It should be noted that during the loading and unloading of the goods, the RFID reads the RFID tag in real time, and the error rate is very low, so the frequency of using the image sensor to collect data is low. Therefore, the present application is configured to use the image data collected by the image sensor to assist in confirming the number and type of the goods on the forklift only when the collection results of the RFID reader and / or the weighing sensor are inconsistent. This reduces the amount of image processing and greatly reduces the computational power consumption of the processor.

[0026] Further, a processor with poor computational power can be used to record the time stamp when the collection results of the RFID reader and / or the weighing sensor are inconsistent during the stacking process, then extract the corresponding video data collected by the image sensor to identify the number and type of the goods, and perform auxiliary verification. This process can continue until the stacking is completed, prompting the stacking personnel to compare and confirm the detection results, and confirming that there is no problem, then directly clicking to confirm, thereby improving the warehouse management efficiency.

[0027] The needs of embedding image processing algorithms with low computational power into warehouse forklifts are met.

[0028] S400, based on the weight of the goods on the warehouse forklift, the goods label, the number of goods labels, the type of goods, and the visual quantity of goods, calculating the type and number of goods on the warehouse forklift; and Exemplarily, this step distinguishes between the case where the collection results of the RFID reader and / or the weighing sensor are consistent, and the case where the collection results of the RFID reader and / or the weighing sensor are inconsistent.

[0029] When the collection results are consistent, a large amount of computational power is not needed, and the reading results of the RFID can be directly used as the final results. When the collection results are inconsistent, the time stamp when the collection results of the RFID reader and / or the weighing sensor are inconsistent is recorded, then the corresponding video data collected by the image sensor is extracted to identify the number and type of the goods, and auxiliary verification is performed.

[0030] S500, confirming the current in-out warehouse state of the warehouse forklift, and synchronizing the current in-out warehouse state, the type of goods, and the number of goods to the warehouse management system, wherein the current in-out warehouse state includes an out-of-warehouse state and an in-warehouse state. Exemplarily, the warehouse forklift and the host of the warehouse management system establish a communication connection, and the item type and the item quantity are synchronized to the warehouse management system, the warehouse management system itself already knows the in-out warehouse situation of the batch of items, and then the current in-out warehouse state, the item type and the item quantity are synchronized to the warehouse management system, repeated confirmation is performed to ensure the reliability of the in-out warehouse detection of the items.

[0031] The embodiment of the application improves the reliability of the RFID reading result by using the item weight on the warehouse forklift to assist verification when the reader reads the item label and the item quantity on the forklift, and when the RFID reading result is inconsistent, using the image collected by the image sensor to identify the item type and the item visual quantity on the warehouse forklift to provide a reference information, and then using the RFID reading result and the identification result of the image collected by the image sensor to comprehensively confirm the item type and the item quantity on the warehouse forklift; finally, the item type and the item quantity on the warehouse forklift and the current in-out warehouse state of the warehouse forklift are sent to the warehouse management system for secondary confirmation, thereby improving the reliability of the warehouse management system.

[0032] In some embodiments, reading the item label and the item label quantity on the warehouse forklift comprises: respectively using two readers arranged at the diagonal positions of the base of the warehouse forklift to read the item label and the item label quantity. The two readers arranged at the diagonal positions of the base of the warehouse forklift are far apart in position and can well cover the position of the items on the forklift, reducing the shielding and improving the reading accuracy of the readers.

[0033] In some embodiments, calculating the item type and the item quantity on the warehouse forklift based on the item weight, the item label, the item label quantity, the item type and the item visual quantity on the warehouse forklift comprises: When the reading results of the two readers are consistent, the item label and the item label quantity are used as the item type and the item quantity on the warehouse forklift.

[0034] When the reading results of the two readers are consistent, it can be considered that the reading result is correct.

[0035] In other embodiments, when the reading results of the two readers are consistent, the item weight can also be combined to assist verification of whether the reading result of the two readers is reliable, that is, the product of the total amount and the quantity of the item reading result of the reader is used to obtain the theoretical item weight of the forklift, and when the theoretical item weight is consistent with the weight collected by the weight sensor, it is considered that the reading result of the reader is reliable.

[0036] In some embodiments, calculating the item type and the item quantity on the warehouse forklift based on the item weight, the item label, the item label quantity, the item type and the item visual quantity on the warehouse forklift comprises: When the reading results of the two readers are inconsistent, structural data is constructed based on the output results of the readers, the weight of the article on the warehouse forklift, the type of the article, and the visual quantity of the article; The structural data is input into a pre-trained warehouse identification model, and the pre-trained warehouse identification model outputs the type of the article on the warehouse forklift and the quantity of the article; In the present application, the pre-trained warehouse identification model can use a regression model, obtain a warehouse identification model based on training data and data labeling, and then transplant the warehouse model to the warehouse forklift after lightening the warehouse model.

[0037] In the present application, image recognition can use a single image, that is, an image after the stacking is completed, but the error is large. Image recognition can also use full-process video data, but at this time, the input of the warehouse model has a large amount of redundant images, resulting in large computational overhead and interference with the recognition result. In order to control the cost, the processor of the warehouse forklift has low computing power and is difficult to bear a large amount of calculation.

[0038] Further, identifying the type of the article on the warehouse forklift and the visual quantity of the article includes starting to collect images when the weight sensor detects that a weight is applied to the warehouse forklift, and stopping collecting images when the weight sensor detects that the weight applied to the warehouse forklift has not changed. The type of the article on the warehouse forklift and the visual quantity of the article are identified based on the collected multiple images.

[0039] Reference Figure 2 In some embodiments, the warehouse identification model includes: A first feature extraction network for obtaining a difference information feature vector of the reading result of the reader; A second feature extraction network for obtaining an article image feature vector; An output network inputting the difference information feature vector, the article image feature vector, and the weight of the article, and the output of the output network being the type of the article on the warehouse forklift and the quantity of the article.

[0040] In the embodiments of the present application, the first feature extraction network can be a recurrent neural network, such as an RNN, an LSTM, or a GRU network. In order to improve the recognition efficiency, the first feature extraction network of the present application needs to encode the output information of the reader label into a difference information feature image, then extract the image feature vector, and finally splice the feature vectors corresponding to the image sensor collected image to realize the same dimension or type of the feature vectors, thereby improving the recognition efficiency. Specifically, in some embodiments, the first feature extraction network includes: an imaging module configured to encode the output information of the reader tag into a difference information feature image; an image feature extraction network configured to extract a difference information feature vector from the difference information feature image.

[0041] The imaging module can use Gram angle field transformation or Markov transformation for image encoding, or obtain a time-frequency transformation of the output information of the reader tag, such as wavelet transformation or short-time Fourier transformation, and then convert it into a heat map.

[0042] The image feature extraction network is preferably a convolutional neural network, which can efficiently extract image feature information.

[0043] In some embodiments, the output network comprises: a flattening layer configured to flatten the difference information feature vector to the same length or width as the item image feature vector; a concatenation layer configured to concatenate the difference information feature vector and the item image feature vector into a concatenated feature vector in the length direction or the width direction; a feature secondary extraction layer configured to abstract the concatenated feature vector; a multi-layer fully connected layer configured to fuse the concatenated feature vector; an output layer configured to output the type and quantity of the item on the warehouse forklift according to the concatenated feature vector and the item weight.

[0044] In a convolutional neural network (CNN) architecture, the flattening layer (Flatten Layer) serves as a connection between the convolution / pooling layer and the fully connected layer. In this application, the flattening layer can serve the purpose of adjusting the feature vector, flattening the difference information feature vector to the same length or width as the item image feature vector, so as to perform image splicing. Therefore, the flattening here is a modification of the dimension of the feature vector to meet the splicing requirement.

[0045] In the subsequent process, the concatenated feature vector is flattened and sent to the multi-layer fully connected layer, and finally the output layer is used to output the result.

[0046] In some embodiments, the input of the first feature extraction network comprises the item weight, so that the first feature extraction network calculates the difference information feature vector with reference to the item weight.

[0047] It should be noted that the feature extraction network only needs to be trained according to the labeled information and the input information. In the training process, high-value input information can improve the effect of the feature extraction network. The item weight is used as reference information as one of the inputs of the feature extraction network, thereby improving the effect of feature extraction.

[0048] Similarly, the input of the second feature extraction network includes the weight of the article, so that the second feature extraction network calculates the article image feature vector with reference to the weight of the article.

[0049] Specifically, the feature network can have two input ends to input the reading result of the reader or the image collected by the image sensor and the weight of the article.

[0050] In other embodiments, the weight of the article can be input into one of the intermediate layers of the feature extraction network.

[0051] Example 2 The present application also proposes a device for detecting the loading and unloading of articles, exemplarily a warehouse forklift or an upgrade assembly for upgrading the warehouse forklift. The device comprises: a weight detection device for detecting the weight of the article on the warehouse forklift; an RFID reader for reading the article label and the number of article labels on the warehouse forklift an image recognition device for collecting the image of the article on the warehouse forklift; one or more processors for calculating the type and number of articles on the warehouse forklift based on the weight of the article, the article label, the number of article labels, the type of article and the visual number of articles on the warehouse forklift; and confirming the current warehouse-in and warehouse-out state of the warehouse forklift, and synchronizing the current warehouse-in and warehouse-out state, the type of article and the number of articles to the warehouse management system, wherein the current warehouse-in and warehouse-out state includes the warehouse-out state and the warehouse-in state.

[0052] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0053] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0054] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0055] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0056] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0057] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments described and shown, and it is therefore intended that the application cover any and all variations of the preferred embodiments which fall within the scope of the present application. Accordingly, the appended claims are intended to cover all such modifications and variations as falling within the scope of the application.

[0058] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for detecting items getting on and off a vehicle, provided on a warehouse forklift, characterized in that: include: Detecting the weight of items on warehouse forklifts; Read item labels and item quantity on warehouse forklifts; Identify the type of items and visual quantity of items on warehouse forklifts; Calculate the item type and item quantity on the warehouse forklift based on the item weight, item label, item label quantity, item type, and item visual quantity; as well as Confirm the current in-and-out status of the warehouse forklift, and synchronize the current in-and-out status, item type, and item quantity to the warehouse management system. The current in-and-out status includes: outbound status and inbound status.

2. The method for detecting objects getting on and off a vehicle according to claim 1, wherein: Read the item labels and item quantity on warehouse forklifts, including: Two readers, each located at a diagonal position on a warehouse forklift base, are used to read the item tags and the number of item tags.

3. The method for detecting objects getting on and off a vehicle according to claim 2, wherein: Calculate the item type and quantity on warehouse forklifts based on item weight, item label, number of item labels, item type, and visual quantity of items, including: When the reading results of the two readers are consistent, the item tags and the number of item tags are used as the item type and the number of items on the warehouse forklift.

4. The method for detecting objects getting on and off a vehicle according to claim 2, wherein: Calculate the item type and quantity on warehouse forklifts based on item weight, item label, number of item labels, item type, and visual quantity of items, including: When the reading results of two readers are inconsistent, the structure data is constructed based on the output results of the readers, the weight of the items on the warehouse forklift, the type of items, and the visual number of items; The structure data is fed into a pre-trained inbound and outbound identification model, which outputs the type and quantity of items on the warehouse forklift; Identifying the type of items and the visual quantity of items on a warehouse forklift, including starting image acquisition when a weight sensor detects that a weight is applied to the warehouse forklift and stopping image acquisition when the weight sensor detects that the weight applied to the warehouse forklift has not changed; Identify the type and visual quantity of items on warehouse forklifts based on multiple captured images.

5. The method for detecting objects getting on and off a vehicle according to claim 4, wherein: The inbound and outbound identification model includes: A first feature extraction network is used to obtain a feature vector of difference information of a reader reading result; The second feature extraction network is used to obtain the feature vector of the object image; The output network has inputs including: a difference information feature vector, an item image feature vector, and an item weight. The output of the output network is the item type and the item quantity on the warehouse forklift.

6. The method for detecting objects getting on and off a vehicle according to claim 5, wherein: The first feature extraction network includes: An imaging module, used for encoding the output information of the reader tag into a difference information feature image; The image feature extraction network is used to extract the difference information feature vector of the difference information feature image.

7. The method for detecting objects getting on and off a vehicle according to claim 5, wherein: The output network includes: a flattening layer, configured to flatten the difference information feature vector to the same length or width as the object image feature vector; A splicing layer, configured to splice the difference information feature vector and the object image feature vector into a splicing feature vector in a length direction or a width direction; Multi-layer fully connected layers, used to fuse the concatenated feature vectors; The output layer is used to output the type and quantity of items on the warehouse forklift based on the concatenated feature vector and the weight of the items.

8. The method for detecting objects getting on and off a vehicle according to claim 7, wherein: A feature secondary extraction layer is provided between the splicing layer and the multi-layer fully connected layer to abstract the splicing feature vector.

9. The method for detecting objects getting on and off a vehicle according to claim 7, wherein: The input of the first feature extraction network includes the weight of the item, so that the first feature extraction network calculates the difference information feature vector with reference to the weight of the item; and / or, The input of the second feature extraction network includes the weight of the item, so that the second feature extraction network calculates the item image feature vector with reference to the weight of the item.

10. A device for detecting objects getting on and off a vehicle, characterized in that: include: Weight detection device, used to detect the weight of items on warehouse forklifts; RFID reader, reads the item tags and item tag quantity on the warehouse forklift Image recognition device, used to capture images of items on warehouse forklifts; One or more processors calculate the type and quantity of items on the warehouse forklift based on the weight of the items, item labels, the number of item labels, the type of items, and the visual quantity of the items on the warehouse forklift; and confirm the current inbound and outbound status of the warehouse forklift, and synchronize the current inbound and outbound status, item type, and item quantity to the warehouse management system, where the current inbound and outbound status includes: outbound status and inbound status.

Images