Automatic picking method and system based on visual recognition

Abstract

The present invention provides an automatic picking method and system based on visual recognition, the method comprising: collecting image information of goods to be picked by an image acquisition device; identifying the category of the goods in the image information by a preset visual recognition model; and transporting the goods to a corresponding processing platform according to the category of the goods by a transportation system. The present invention can automatically identify the goods, assist or replace the picking work of employees, thereby reducing employee detection errors and improving picking efficiency.

Description

Technical Field

[0001] The present invention relates to the field of logistics technology, and in particular to an automatic picking method and system based on visual recognition. Background Art

[0002] Currently, many workplaces, especially logistics centers, require a lot of inspection work. The picking work in logistics centers still relies mainly on visual inspection. Employees not only need to pick according to the order, but also need to make sure the type and quantity of goods are correct. Due to long hours and repetitive work, mistakes are often made when picking. In addition, foreign workers also have problems due to language barriers. The wrong type or quantity of goods will result in incomplete inventory and resources will be spent to correct the errors, reducing work efficiency. Summary of the invention

[0003] One object of the present invention is to provide an automatic picking method based on visual recognition, which automatically identifies goods and assists or replaces employees' picking work, thereby reducing employee detection errors and improving picking efficiency. Another object of the present invention is to provide an automatic picking system based on visual recognition. Another object of the present invention is to provide a computer device. Another object of the present invention is to provide a readable medium.

[0004] In order to achieve the above objectives, the present invention discloses an automatic picking method based on visual recognition, comprising:

[0005] Collecting image information of goods to be picked by an image acquisition device;

[0006] Identify the product category of the product in the image information by using a preset visual recognition model;

[0007] The transportation system transports goods to the corresponding processing platform according to the category of goods.

[0008] Preferably, the method further comprises, before identifying the product category of the product in the image information by a preset visual recognition model:

[0009] Collecting distance information of goods to be picked up through distance sensor devices;

[0010] The recognition range of the image information is determined according to the distance information so that a preset visual recognition model can recognize the commodity category of the commodity in the recognition range of the image information.

[0011] Preferably, the method further comprises, before collecting image information of the goods to be picked by the image acquisition device:

[0012] The goods to be picked are heated and defrosted.

[0013] Preferably, the method further comprises, before collecting image information of the goods to be picked by the image acquisition device:

[0014] Collecting image information of goods to be picked by an image acquisition device;

[0015] Annotating the image information to obtain a training data set;

[0016] The visual recognition model is obtained by training the machine learning model using the training data set.

[0017] The present invention also discloses an automatic picking system based on visual recognition, comprising:

[0018] An image acquisition device for acquiring image information of goods to be picked;

[0019] A product recognition module, used to recognize the product category of the product in the image information through a preset visual recognition model;

[0020] The cargo transportation system is used to transport cargo to the corresponding processing platform according to the cargo category.

[0021] Preferably, it further comprises a distance sensor device and an image processing module,

[0022] The distance sensor device is used to collect distance information of the goods to be picked;

[0023] The image processing module is used to determine the recognition range of the image information according to the distance information before identifying the commodity category of the commodity in the image information through the preset visual recognition model so that the preset visual recognition model can identify the commodity category of the commodity in the recognition range of the image information.

[0024] Preferably, it further comprises a defrosting device for heating and defrosting the goods to be picked before the image information of the goods to be picked is collected by the image collection device.

[0025] Preferably, it further includes a model training module, which is used to collect image information of the goods to be picked by the image acquisition device before collecting image information of the goods to be picked by the image acquisition device, annotate the image information to obtain a training data set, and train the machine learning model by the training data set to obtain the visual recognition model.

[0026] The present invention also discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0027] When the processor executes the program, the method described above is implemented.

[0028] The present invention also discloses a computer readable medium having a computer program stored thereon.

[0029] When the program is executed by a processor, the method described above is implemented.

[0030] The present invention provides a set of automatic picking methods and systems based on visual recognition suitable for use in picking. Through the trained visual recognition model, the goods can be quickly and accurately identified in real time, and then the goods of different categories identified can be transported to the corresponding processing platform through the logistics goods transportation system. When in use, the image acquisition device and other equipment can be installed on the picking transportation line or trolley. The automatic recognition function assists or replaces the employee's picking work, thereby reducing employee detection errors, saving manpower inspection, and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 One of the flowcharts showing a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0033] Figure 2 A schematic diagram showing a transportation system in a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0034] Figure 3 A second flowchart showing a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0035] Figure 4 A schematic diagram showing the range of goods on a trolley in a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0036] Figure 5 A schematic diagram showing a defrosting device in a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0037] Figure 6 A third flowchart showing a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0038] Figure 7 A schematic diagram showing an image acquisition device in a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0039] Figure 8A schematic diagram showing another image acquisition device in a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0040] Fig. 9 A fourth flowchart showing a specific embodiment of the automatic picking method based on visual recognition of the present invention;

[0041] Fig.10 One of the structural diagrams of a specific embodiment of the automatic picking system based on visual recognition of the present invention is shown;

[0042] Fig.11 A second structural diagram showing a specific embodiment of the automatic picking system based on visual recognition of the present invention;

[0043] Fig.12 A third structural diagram showing a specific embodiment of the automatic picking system based on visual recognition of the present invention;

[0044] Fig.13 A fourth structural diagram showing a specific embodiment of the automatic picking system based on visual recognition of the present invention;

[0045] Fig.14 A schematic diagram showing the structure of a computer device suitable for implementing an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] According to one aspect of the present invention, this embodiment discloses an automatic picking method based on visual recognition. Figure 1 As shown, in this embodiment, the method includes:

[0048] S100: Capture image information of the goods 3 to be picked up by the image acquisition device 11. It is understandable that the image acquisition device 11 can be a camera, a webcam, or other device with image or video acquisition function. Further, when the image acquisition device 11 acquires a video including the goods 3 to be picked up, it is necessary to extract the video frames in the video to obtain the image information including the goods 3.

[0049] S200: Identify the product category of the product 3 in the image information through a preset visual recognition model. The preset visual model can be obtained by using the image acquisition device 11 of this embodiment to collect image information, manually or machine-label the product category to form a training data set, and then obtain the preset visual recognition model through machine learning technology. The visual recognition model can automatically and accurately identify the product category in the collected image information, ensure recognition accuracy, and can replace and assist manual operation.

[0050] S300: Transport goods 3 to corresponding processing platforms according to the goods category through the transport system 2. It can be understood that in this embodiment, the goods transport system 2 has the function of transferring the classified goods 3 to the corresponding processing platforms, that is, after obtaining the goods category of the goods 3 through the visual recognition model, the transport system 2 is controlled to adjust the transport direction of the goods 3 according to the goods category, so that the goods 3 arrive at the corresponding processing platform, and then the goods are picked up by human hands or robots and placed on designated containers or shelves, such as Figure 2 shown.

[0051] The present invention provides a set of automatic picking methods and systems based on visual recognition suitable for use in picking. Through the trained visual recognition model, the goods 3 can be quickly and accurately identified in real time, and then the goods 3 of different categories identified can be transported to the corresponding processing platform through the logistics goods transportation system 2. When in use, the image acquisition device 11 and other equipment can be installed on the picking transportation line or trolley. The automatic recognition function assists or replaces the employee's picking work, thereby reducing employee detection errors, saving manpower inspection, and improving work efficiency.

[0052] In a preferred embodiment, Figure 3 As shown, the method further comprises before S200:

[0053] S021: Collect distance information of the goods 3 to be picked up through a distance sensor device.

[0054] S022: Determine a recognition range of the image information according to the distance information so that a preset visual recognition model can recognize the product category of the product 3 in the recognition range of the image information.

[0055] Among them, the distance sensing device can be a 3D snapshot camera, for example, a laser snapshot 3D camera. Of course, in other embodiments, other distance sensing devices with distance measurement functions can also be used. It can be understood that by using a 3D snapshot camera to shoot the goods 3, the depth information of the goods 3 and the environment can be obtained, that is, the distance information between the goods 3 and the distance sensing device can be obtained. The goods 3 that are far away from the image acquisition device 11 will have a smaller recognition range; on the contrary, the closer the goods 3 are to the image acquisition device 11, the larger the recognition range. According to the predetermined correspondence between the range of the goods 3 on the image information of the goods 3 collected by the image acquisition device 11 and the goods 3 and the distance information, combined with the real-time distance information, the range of the goods 3 on the real-time collected image information can be obtained, so that the recognition range of the visual recognition model can be limited, and the recognition efficiency of the visual recognition model can be improved.

[0056] In a specific embodiment of the present application, Figure 4 As shown, the goods 3 are arranged in layers in the trolley, and the image acquisition device 11 is arranged on the top of the trolley. When a distance sensor device is further arranged, the distance sensor device is preferably also arranged on the top of the trolley, at the same height as the image acquisition device 11. The visual recognition model can be set on an industrial PC, and the industrial PC is connected to the image acquisition device 11 and the distance sensor device respectively. After the image acquisition device 11 acquires the image information of the goods 3 in the trolley and the distance sensor device acquires the distance information of the goods 3 in the trolley, the image information and the distance information are respectively transmitted to the working PC for data processing and goods category identification.

[0057] The image processing model preset in the working PC determines the distance between the goods 3 and the distance acquisition device according to the acquired distance information, and can obtain the distance between the goods 3 and the image acquisition device 11, and can determine the specific layer number of the trolley where the goods 3 are located. Further, the range of goods on the acquired image information can be determined according to the distance information, that is, the recognition range can be obtained, so that the visual recognition model can perform image recognition within the recognition range, which can reduce the image recognition process for useless areas, thereby improving the recognition efficiency of goods 3.

[0058] In a preferred embodiment, the method further comprises before S100:

[0059] S011: Heating and defrosting the goods 3 to be picked.

[0060] It is understandable that, since the present application uses image information to identify the category of goods 3, when some refrigerated products begin to be transported, a layer of frost usually forms on their surface. Therefore, in this preferred embodiment, the goods 3 to be picked can be first subjected to a heating and defrosting treatment, such as defrosting by hot air treatment, to remove the frost on the surface of the goods 3, so that the image acquisition device 11 can clearly capture the surface features of the goods 3 for visual recognition, such as Figure 5 shown.

[0061] In a preferred embodiment, Figure 6 As shown, the method further includes, before collecting image information of the goods 3 to be picked up by the image acquisition device 11:

[0062] S010: The image acquisition device 11 acquires image information of the goods 3 to be picked.

[0063] S020: Annotate the image information to obtain a training data set.

[0064] S030: Training the machine learning model using the training data set to obtain the visual recognition model.

[0065] In a specific embodiment, before training the model, the training data set must first be uploaded to the server. The training data set is generally a processed picking video, which is annotated video frame. Annotations are performed manually, and circles are drawn on each video frame to set the recognition range of the goods 3. The annotation data will be converted into a text file and uploaded to the server together with the video frame image. These data will be processed by the instance segmentation model convolutional neural network (Mask R-CNN) method, and features will be extracted from the RGB data within the circled range. The extracted features are matched with the manually labeled product categories to obtain a visual recognition model. In actual application, the visual recognition model can read the color data (RGB) in the image information to be recognized, and then use the instance segmentation model convolutional neural network (Mask R-CNN) mode to process the data to obtain the features within the range to identify the goods 3 in the image.

[0066] In this embodiment, Figure 7 As shown, the image acquisition device 11 can collect image information of the goods 3 at one angle, and identify the features of the goods 3 at one angle through the visual recognition model to obtain the category of the goods. Figure 8As shown, the image acquisition device 11 can also collect image information of the goods 3 at multiple angles, and after manual annotation, a visual recognition model that can identify the category of goods at multiple angles is obtained through machine learning technology. By collecting image information of the goods 3 at multiple angles, the goods 3 can be identified at multiple angles, which can improve the accuracy of identifying the category of goods. Among them, the image acquisition device 11 such as a camera can be set at different positions of the goods 3 to collect image information of the goods 3 at multiple angles.

[0067] The present invention provides a low-cost automatic picking method for the logistics industry through visual recognition technology. It is easy to implement and can start working after the software is trained before use. Compared with robotic automation, traditional logistics centers can enjoy improved work efficiency without modifying the operating mode. The automatic picking method can improve work efficiency for many industries. In terms of inspection, the present invention can quickly and accurately identify the type and quantity of items. Compared with manual inspection, the present invention can reduce inventory errors. General recognition systems rely on network connections to transmit data to image recognition servers before processing image data. The present invention can operate independently. As long as the trained visual recognition model is downloaded on the device, it can operate without a network connection. Therefore, the present invention can operate quickly without the need for high-bandwidth Internet infrastructure.

[0068] In a preferred embodiment, Fig. 9 As shown, the method further comprises:

[0069] S400: Obtaining the quantity of different commodity categories according to the commodity categories of each commodity 3 obtained by identification. In this preferred embodiment, the quantity of commodities 3 of different categories can be counted after the commodity categories are identified, providing an automatic counting function for commodities 3.

[0070] In a preferred embodiment, the cargo transportation system includes a controller and a transmission device.

[0071] The transmission device includes a main transport channel provided with a plurality of turntables and a branch channel connected to each turntable and a corresponding processing platform.

[0072] The controller is used to control the rotation of the turntable according to the category of the goods so that the goods are transported to the corresponding branch channels and then to the corresponding processing platforms.

[0073] It is understandable that the controller of the cargo transportation system can receive the cargo category of the cargo 3 transmitted by the cargo identification module, and can determine the processing platform to which the cargo category currently received should be transported based on the correspondence between the cargo category and the processing platform, and then determine the turntable corresponding to the processing platform. The position of the cargo 3 can be determined by the transportation of the transmission device, and when the cargo 3 is transported to the corresponding turntable, the turntable is controlled to rotate to transfer the cargo 3 to the corresponding branch channel, and then transported to the corresponding processing platform for manual or robot processing.

[0074] Based on the same principle, this embodiment also discloses an automatic picking system based on visual recognition. Fig.10 As shown, the system includes an image acquisition device 11, a product identification module 12 and a product transportation system 2.

[0075] The image acquisition device 11 acquires image information of the goods 3 to be picked up. It is understandable that the image acquisition device 11 can be a camera, a webcam, or other device with image or video acquisition function. Further, when the image acquisition device 11 acquires a video including the goods 3 to be picked up, it is necessary to extract the video frames in the video to obtain the image information including the goods 3.

[0076] The product identification module 12 is used to identify the product category of the product 3 in the image information through a preset visual recognition model 121. The preset visual model can be obtained by using the image acquisition device 11 of this embodiment to collect image information, manually or machine-labeling the product category to form a training data set, and then obtaining the preset visual recognition model 121 through machine learning technology. The visual recognition model 121 can automatically and accurately identify the product category in the collected image information, ensure recognition accuracy, and can replace and assist manual operation.

[0077] The goods transportation system 2 is used to transport the goods 3 to the corresponding processing platform according to the goods category. It can be understood that in this embodiment, the goods transportation system 2 has the function of transferring the classified goods 3 to the corresponding processing platform, that is, after obtaining the goods category of the goods 3 through the visual recognition model 121, the transportation system 2 is controlled to adjust the transportation direction of the goods 3 according to the goods category, so that the goods 3 arrive at the corresponding processing platform, and then the goods are picked up by human hands or robots and placed on the designated container or shelf.

[0078] In a preferred embodiment, Fig.11 As shown, the system further includes a distance sensor device 14 and an image processing module 122 .

[0079] The distance sensor device 14 is used to collect distance information of the goods 3 to be picked.

[0080] The image processing module 122 is used to determine the recognition range of the image information according to the distance information before the preset visual recognition model 121 recognizes the product category of the product 3 in the image information so that the preset visual recognition model 121 can recognize the product category of the product 3 in the recognition range of the image information.

[0081] Among them, the distance sensor device 14 can be a 3D snapshot camera, for example, a laser snapshot 3D camera. Of course, in other embodiments, other distance sensor devices 14 with distance measurement functions can also be used. It can be understood that by using a 3D snapshot camera to shoot the goods 3, the depth information of the goods 3 and the environment can be obtained, that is, the distance information between the goods 3 and the distance sensor device 14 can be obtained. The goods 3 that are far away from the image acquisition device 11 will have a smaller recognition range; on the contrary, the closer the goods 3 are to the image acquisition device 11, the larger the recognition range. According to the predetermined correspondence between the range of the goods 3 on the image information of the goods 3 collected by the image acquisition device 11 and the goods 3 and the distance information, combined with the real-time distance information, the range of the goods 3 on the real-time collected image information can be obtained, so that the recognition range of the visual recognition model 121 can be limited, and the recognition efficiency of the visual recognition model 121 can be improved.

[0082] In a preferred embodiment, see Figure 5 , the system further includes a defrosting device 16. The defrosting device 16 is used to heat and defrost the goods 3 to be picked before the image information of the goods 3 to be picked is collected by the image acquisition device 11. It can be understood that since the present application classifies the goods 3 through image information, when some refrigerated products begin to be transported, a layer of frost usually forms on their surfaces. Therefore, in this preferred embodiment, the goods 3 to be picked can be first subjected to a heating and defrosting treatment. For example, a hot air channel can be set on the goods transportation system, so that the goods 3 are transported through the hot air channel, and the frost on the surface of the goods 3 is removed by hot air treatment, so that the image acquisition device 11 can clearly collect the surface features of the goods 3 for visual identification.

[0083] In a preferred embodiment, Fig.12 As shown, the system further includes a model training module 120. The model training module 120 is used to collect image information of the goods 3 to be picked up by the image acquisition device 11 before collecting image information of the goods 3 to be picked up by the image acquisition device 11, annotate the image information to obtain a training data set, and train the machine learning model by the training data set to obtain the visual recognition model 121.

[0084] In a specific embodiment, before training the model, the training data set must first be uploaded to the server. The training data set is generally a processed picking video, which is annotated video frame. Annotations are performed manually, and circles are drawn on each video frame to set the recognition range of the goods 3. The annotation data will be converted into a text file and uploaded to the server together with the video frame image. These data will be processed by the instance segmentation model convolutional neural network (Mask R-CNN) method, and features will be extracted from the RGB data within the circled range. The extracted features are matched with the manually labeled product categories to obtain the visual recognition model 121. In actual application, the visual recognition model 121 can read the color data (RGB) in the image information to be recognized, and then use the instance segmentation model convolutional neural network (Mask R-CNN) mode to process the data to obtain the features within the range to identify the goods 3 in the image.

[0085] In this embodiment, see Figure 7 , the image acquisition device 11 can collect image information of the product 3 at one angle, and obtain the product category by identifying the features of the product 3 at one angle through the visual recognition model 121. In other preferred embodiments, see Figure 8 The image acquisition device 11 can also collect image information of the goods 3 at multiple angles, and obtain a visual recognition model 121 that can identify the category of goods from multiple angles through machine learning technology after manual annotation. By collecting image information of the goods 3 from multiple angles, the goods 3 can be identified from multiple angles, which can improve the accuracy of identifying the category of goods. Among them, the image acquisition device 11 such as a camera can be set at different positions of the goods 3 to collect image information of the goods 3 from multiple angles.

[0086] In a preferred embodiment, Fig.13 As shown, the system further includes a data statistics module 123. The data statistics module 123 is used to obtain the quantity of different commodity categories according to the commodity categories of each commodity 3 obtained by identification. In this preferred embodiment, the quantity of commodities 3 of different categories can be counted after the commodity categories are identified, providing an automatic statistics function for commodities 3.

[0087] Since the principle of solving the problem by this system is similar to that of the above method, the implementation of this system can refer to the implementation of the method and will not be repeated here.

[0088] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer device, and specifically, the computer device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0089] In a typical example, a computer device specifically includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the method described above is implemented when the processor executes the program.

[0090] Reference below Fig.14 , which shows a schematic diagram of the structure of a computer device 600 suitable for implementing an embodiment of the present application.

[0091] like Fig.14 As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate operations and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0092] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal feedback device (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 606 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed as needed as the storage section 608.

[0093] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 609, and / or installed from the removable medium 611.

[0094] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0095] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0096] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0101] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0102] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An automatic picking method based on visual recognition, characterized in that: include: Collecting image information of goods to be picked by an image acquisition device; Identify the product category of the product in the image information by using a preset visual recognition model; Transport goods to corresponding processing platforms according to their categories through the transportation system; The method further comprises, before identifying the product category of the product in the image information by using a preset visual recognition model: Collect distance information of goods to be picked up through distance sensor devices; Determine the recognition range of the image information according to the distance information so that a preset visual recognition model can recognize the product category of the product in the recognition range of the image information; Among them, the goods are arranged in layers on the trolley, and the image acquisition device is arranged on the top of the trolley; when a distance sensing device is provided, the distance sensing device is also arranged on the top of the trolley, and is located at the same height as the image acquisition device; the visual recognition model is set on the industrial PC, and the industrial PC is connected to the image acquisition device and the distance sensing device respectively; the image processing model preset in the industrial PC determines the distance between the goods and the distance sensing device according to the collected distance information, and determines the specific layer number of the goods on the trolley.

2. The automatic picking method based on visual recognition according to claim 1 is characterized in that: The method further comprises, before collecting image information of the goods to be picked by the image acquisition device: The goods to be picked are heated and defrosted.

3. The automatic picking method based on visual recognition according to claim 1 is characterized in that: The method further comprises, before collecting image information of the goods to be picked by the image acquisition device: Collecting image information of goods to be picked by an image acquisition device; Annotating the image information to obtain a training data set; The visual recognition model is obtained by training the machine learning model using the training data set.

4. An automatic picking system based on visual recognition, characterized in that: include: An image acquisition device for acquiring image information of goods to be picked; A product recognition module, used to recognize the product category of the product in the image information through a preset visual recognition model; The cargo transportation system is used to transport the cargo to the corresponding processing platform according to the cargo category; A distance sensor device is used to collect distance information of goods to be picked; An image processing module, used for determining a recognition range of the image information according to the distance information before recognizing the commodity category of the commodity in the image information by using a preset visual recognition model so that the preset visual recognition model recognizes the commodity category of the commodity in the recognition range of the image information; Among them, the goods are arranged in layers on the trolley, and the image acquisition device is arranged on the top of the trolley; when a distance sensing device is provided, the distance sensing device is also arranged on the top of the trolley, and is located at the same height as the image acquisition device; the visual recognition model is set on the industrial PC, and the industrial PC is connected to the image acquisition device and the distance sensing device respectively; the image processing model preset in the industrial PC determines the distance between the goods and the distance sensing device according to the collected distance information, and determines the specific layer number of the goods on the trolley.

5. The automatic picking system based on visual recognition according to claim 4 is characterized in that: It further includes a defrosting device for heating and defrosting the goods to be picked before the image information of the goods to be picked is collected by the image collection device.

6. The automatic picking system based on visual recognition according to claim 4 is characterized in that: It further includes a model training module, which is used to collect image information of the goods to be picked by the image acquisition device before collecting image information of the goods to be picked by the image acquisition device, annotate the image information to obtain a training data set, and train the machine learning model by the training data set to obtain the visual recognition model.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images