An online perception system and method for congestion of conveyor line items based on machine vision

Abstract

The present invention discloses an online perception system and method for congestion of conveyor line items based on machine vision, wherein the system comprises: an image acquisition module, an image intelligent classification module, a congestion assessment module and a fault decision module; the method comprises: S1. collecting and storing images of items transported during the transportation process of the conveyor line; S2. identifying the images of items transported by a trained machine learning model, thereby obtaining the accumulation of items on the conveyor line, and obtaining the total area of ​​item accumulation, and determining whether transportation congestion has occurred on the conveyor line according to the size of the total area of ​​item accumulation; S3. obtaining congestion morphological parameters according to the item transport images, and quantifying the degree of congestion according to the congestion morphological parameters, wherein the congestion morphological parameters include congestion range, congestion intensity and congestion time; S4. analyzing and processing decisions according to the degree of congestion, and analyzing the causes of congestion. The present invention can provide an efficient and simple online perception system and method for congestion of conveyor line items.

Description

Technical Field

[0001] The present invention relates to the field of intelligent industrial technology, and more specifically to an online perception system and method for congestion of conveyor line items based on machine vision. Background Art

[0002] Conveyor lines are a key part of the intelligent logistics field. With the development of science and technology and the popularization of intelligence, automated conveyor lines have been implemented in many factories, greatly improving the efficiency of logistics sorting. However, the improvement of conveyor line efficiency has put forward higher requirements for the monitoring of the operating status of items on the conveyor line. At present, the transportation of items on conveyor lines relies on sensors, such as photoelectric sensors, which judge and evaluate congestion based on the number of items passing through the sensor per unit time. However, during the transportation of items, items of different shapes and sizes are blocked, and there are multiple items passing through the sensor in parallel. In addition, the movement speed of items on the conveyor line is fast and the operating environment is complex, which leads to misjudgment of congestion by photoelectric sensors. In addition, due to the small detection range of the sensor, the longer the length of the same conveyor line, the more sensors need to be installed, which makes the installation complicated and the cost increased.

[0003] At present, the existing line conveyor congestion relies on sensors, but due to the small detection range of sensors, long-distance conveyor lines need to deploy a large number of sensors. At the same time, with the improvement of conveying efficiency, due to factors such as different sizes and shapes, there are errors in the congestion judgment of sensor detection results, which leads to the inability to sort the conveyor line products in time or even the need to shut down.

[0004] Therefore, how to provide an online perception system and method for conveyor line item congestion based on machine vision that can achieve efficient sorting and scheduling is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides an online perception system for conveying congestion of objects on a conveyor line based on machine vision, which is used to provide efficient and simple online perception of conveying congestion of objects on a conveyor line.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] An online perception system for conveyor line congestion based on machine vision, comprising: an image acquisition module, an image intelligent classification module, a congestion assessment module and a fault decision module;

[0008] The image acquisition module is used to acquire and store images of items transported during the transport process of the conveyor line;

[0009] The image intelligent classification module is used to identify the article transport image through the trained machine learning model, so as to obtain the accumulation of articles on the conveyor line and the total accumulation area of ​​the articles, and determine whether transportation congestion occurs on the conveyor line according to the size of the total accumulation area of ​​the articles;

[0010] The congestion assessment module is used to obtain congestion morphology parameters according to the article transportation image, and quantify the degree of congestion according to the congestion morphology parameters, wherein the congestion morphology parameters include congestion range, congestion intensity and congestion time, the congestion range is the total area of ​​the article stacking, the congestion intensity is the area of ​​the overlapping part of the articles, and the congestion time is the total time experienced by the beginning and end congestion image frames, the beginning image frame is the image frame where the congestion is initially recognized, and the end image frame is the image frame where the article currently congested is transported to the end of the conveyor line;

[0011] The fault decision module is used to analyze and process decisions based on the congestion level and analyze the cause of the congestion.

[0012] Preferably, the image acquisition module includes at least one camera, the camera is arranged above the conveyor line, and the frequency of the camera collecting images at least meets the efficiency of conveying items passing through the conveyor line per unit time.

[0013] Preferably, a server is further included, and the image acquisition module stores the acquired item transportation images in the server and provides the item transportation images to the image intelligent classification module.

[0014] Preferably, it also includes an image analysis module;

[0015] The image analysis module includes an image preprocessing unit and an image noise reduction unit. The image preprocessing unit performs image preprocessing on the item transportation image data to filter blank item images. The image noise reduction unit performs image noise reduction on the preprocessed image.

[0016] Preferably, it also includes an image feature extraction module;

[0017] The image feature extraction module is used to extract image features from the denoised item transportation image, including collecting image data within a preset time range, marking the type of congestion, training the machine learning model, and also extracting features from the denoised item transportation image acquired within the actual time period to be detected, and inputting the features into the trained machine learning model.

[0018] A method for online perception of congestion of conveyor line items based on machine vision, comprising the following steps:

[0019] S1. Collect and store images of items transported during the transportation process of the conveyor line;

[0020] S2. Identify the article transport image through the trained machine learning model to obtain the accumulation of articles on the conveyor line and the total accumulation area of ​​the articles, and determine whether congestion occurs on the conveyor line according to the size of the total accumulation area of ​​the articles;

[0021] S3. Obtain congestion morphology parameters according to the item transport image, and quantify the degree of congestion according to the congestion morphology parameters, wherein the congestion morphology parameters include congestion range, congestion intensity and congestion time, wherein the congestion range is the total area of ​​the item stacking, the congestion intensity is the area of ​​the overlapping part of the items, and the congestion time is the total duration of the beginning and end congestion image frames, wherein the beginning image frame is the image frame where the congestion is initially identified, and the end image frame is the image frame where the items that are currently congested are transported to the end of the conveyor line;

[0022] S4. Analyze and make decisions based on the congestion level and analyze the causes of the congestion.

[0023] Preferably, in S1, the frequency of collecting images at least satisfies the efficiency of conveying items passing through the conveyor line per unit time.

[0024] Preferably, before performing S2, image analysis is also included, which specifically includes the following contents:

[0025] Performing image preprocessing on the item transport image data to filter blank item images;

[0026] Perform image denoising on the preprocessed image.

[0027] Preferably, feature extraction is also included before S2, specifically including the following contents:

[0028] Image feature extraction is performed on the denoised item transportation image, which includes collecting image data within a preset time range, labeling the type of congestion, training the machine learning model, and also includes feature extraction on the denoised item transportation image obtained within the actual time period to be detected, and inputting the feature into the trained machine learning model.

[0029] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses an online perception system and method for conveyor line item congestion based on machine vision. The present invention has a simple structure, and the corresponding installation and use processes are not complicated. It can better avoid the problem of needing to install too many sensors in the prior art. The present invention uses machine learning to accurately and quickly identify congestion through images, and can identify a larger conveyor line range, which solves the problem of relying on statistics on the throughput of conveyed items to evaluate congestion in traditional technologies. It plays a good auxiliary role on conveyor lines in high-speed, high-efficiency intelligent factories, and can also evaluate the health of equipment conveyor lines through congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 The accompanying drawing is a schematic diagram of the structure provided by the present invention;

[0032] Figure 2 The accompanying drawing is a schematic diagram of camera installation;

[0033] Figure 3 The attached figure is a schematic diagram of conveying items on the conveyor line;

[0034] 101-image acquisition module, 102-image analysis module, 103-image feature extraction module, 104-image intelligent classification module, 105-congestion assessment module, 106-fault decision module, 201-conveyor line, 202-items, 203-camera, 204-server, 301-total area of ​​item accumulation, 302-area of ​​overlapping parts of items. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The embodiment of the present invention discloses an online perception system for conveyor line congestion based on machine vision, such as Figure 1As shown, it includes: an image acquisition module 101, an image intelligent classification module 104, a congestion assessment module 105 and a fault decision module 106;

[0037] The image acquisition module 101 is used to acquire and store images of the object 202 during the transportation process of the conveyor line 201;

[0038] The image intelligent classification module 104 is used to identify the object transportation image through the trained machine learning model, so as to obtain the accumulation of the objects 202 on the conveyor line 201 and obtain the total accumulation area 301 of the objects, and determine whether the conveyor line 201 is congested according to the size of the total accumulation area 301 of the objects;

[0039] The congestion assessment module 105 is used to obtain congestion morphological parameters according to the item transport image, and quantify the degree of congestion according to the congestion morphological parameters, wherein the congestion morphological parameters include congestion range, congestion intensity and congestion time, the congestion range is the total area 301 of item accumulation, the congestion intensity is the area 302 of the overlapping part of items, and the congestion time is the total time experienced by the beginning and end congestion image frames, the beginning image frame is the image frame where the congestion is initially recognized, and the end image frame is the image frame where the current congested item 202 is recognized and transported to the end of the conveyor line 201;

[0040] The fault decision module 106 is used to analyze and make decisions based on the degree of congestion and analyze the causes of congestion.

[0041] In order to further implement the above technical solution, the image acquisition module 101 includes at least one camera 203, which is arranged above the conveyor line 201, and the frequency of image acquisition by the camera 203 at least meets the conveying efficiency of the conveyor line 201 of the items 202 passing through the conveyor line 201 per unit time.

[0042] In order to further implement the above technical solution, a server 204 is also included. The image acquisition module 101 stores the acquired object transportation images in the server 204 and provides the object transportation images to the image intelligent classification module 104 .

[0043] In order to further implement the above technical solution, it also includes an image analysis module 102;

[0044] The image analysis module 102 includes an image preprocessing unit and an image noise reduction unit. The image preprocessing unit performs image preprocessing on the item transport image data to filter blank item images. The image noise reduction unit performs image noise reduction on the preprocessed image.

[0045] In order to further implement the above technical solution, it also includes an image feature extraction module 103;

[0046] The image feature extraction module 103 is used to extract image features from the denoised item transportation images, including collecting image data within a preset time range, labeling the type of congestion, training the machine learning model, and also extracting features from the denoised item transportation images obtained within the actual time period to be detected, and inputting them into the trained machine learning model.

[0047] A method for online perception of congestion of conveyor line items based on machine vision, comprising the following steps:

[0048] S1. Collect and store images of items transported during the transport process of the conveyor line 201;

[0049] S2. The trained machine learning model is used to identify the transport image of the items, thereby obtaining the accumulation of items 202 on the conveyor line 201 and obtaining the total accumulation area 301 of the items, and determining whether a transport congestion has occurred on the conveyor line 201 according to the size of the total accumulation area 301 of the items;

[0050] S3. Obtain congestion morphology parameters according to the item transport image, and quantify the degree of congestion according to the congestion morphology parameters, wherein the congestion morphology parameters include congestion range, congestion intensity and congestion time, the congestion range is the total area 301 of item accumulation, the congestion intensity is the area 302 of the overlapping part of items, the congestion time is the total time experienced by the beginning and end congestion image frames, the beginning image frame is the image frame where congestion is initially recognized, and the end image frame is the image frame where the current congested item 202 is recognized and transported to the end of the conveyor line 201;

[0051] S4. Analyze and make decisions based on the degree of congestion and analyze the causes of congestion.

[0052] In order to further implement the above technical solution, in S1, the frequency of collecting images at least satisfies the efficiency of conveying items passing through the conveyor line 201 per unit time.

[0053] In order to further implement the above technical solution, image analysis is also included before S2, specifically including the following contents:

[0054] Perform image preprocessing on the item transportation image data and filter out blank item images;

[0055] Perform image denoising on the preprocessed image.

[0056] In order to further implement the above technical solution, feature extraction is also included before S2, which specifically includes the following contents:

[0057] Image feature extraction is performed on the denoised item transportation images, which includes collecting image data within a preset time range, labeling the type of congestion, training the machine learning model, and also includes feature extraction on the denoised item transportation images obtained within the actual time period to be detected, and inputting them into the trained machine learning model.

[0058] It should be noted that:

[0059] The camera 203 is installed as follows Figure 2 As shown, the installation angle θ and the height h need to be reasonably set to ensure that the viewing angle of the camera can cover the area l in the diagram and can distinguish the farthest object 202.

[0060] The total area 301 of the piled items and the area 302 of the overlapping items are as follows: Figure 3 shown.

[0061] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0062] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A machine vision-based conveyor line congestion perception system, characterized in that: include: Image acquisition module, image intelligent classification module, congestion assessment module and fault decision module; The image acquisition module is used to acquire and store images of items transported during the transport process of the conveyor line; The image intelligent classification module is used to identify the article transport image through the trained machine learning model, so as to obtain the accumulation of articles on the conveyor line and the total accumulation area of ​​the articles, and determine whether transportation congestion occurs on the conveyor line according to the size of the total accumulation area of ​​the articles; The congestion assessment module is used to obtain congestion morphology parameters according to the article transportation image, and quantify the degree of congestion according to the congestion morphology parameters, wherein the congestion morphology parameters include congestion range, congestion intensity and congestion time, the congestion range is the total area of ​​the article stacking, the congestion intensity is the area of ​​the overlapping part of the articles, and the congestion time is the total time experienced by the beginning and end congestion image frames, the beginning image frame is the image frame where the congestion is initially recognized, and the end image frame is the image frame where the article currently congested is transported to the end of the conveyor line; The fault decision module is used to analyze and process decisions based on the congestion level and analyze the cause of the congestion.

2. According to the machine vision-based online perception system for conveyor line congestion of items in claim 1, it is characterized in that: The image acquisition module includes at least one camera, which is arranged above the conveyor line, and the frequency of the camera collecting images at least meets the efficiency of conveying items passing through the conveyor line per unit time.

3. According to the machine vision-based online perception system for conveyor line congestion of items in claim 1, it is characterized in that: It also includes a server, and the image acquisition module stores the acquired item transportation image in the server and provides the item transportation image to the image intelligent classification module.

4. According to the machine vision-based online perception system for conveyor line congestion of items in claim 1, it is characterized in that: Also includes an image analysis module; The image analysis module includes an image preprocessing unit and an image noise reduction unit. The image preprocessing unit performs image preprocessing on the item transportation image data to filter blank item images. The image noise reduction unit performs image noise reduction on the preprocessed image.

5. According to the machine vision-based online perception system for conveyor line congestion of items in claim 1, it is characterized in that: It also includes an image feature extraction module; The image feature extraction module is used to extract image features from the denoised item transportation image, including collecting image data within a preset time range, marking the type of congestion, training the machine learning model, and also extracting features from the denoised item transportation image acquired within the actual time period to be detected, and inputting the features into the trained machine learning model.

6. A method for online perception of congestion in conveyor line items based on machine vision, characterized in that: The following steps are involved: S1. Collect and store images of items transported during the transportation process of the conveyor line; S2. Identify the article transport image through the trained machine learning model to obtain the accumulation of articles on the conveyor line and the total accumulation area of ​​the articles, and determine whether congestion occurs on the conveyor line according to the size of the total accumulation area of ​​the articles; S3. Obtain congestion morphology parameters according to the item transport image, and quantify the degree of congestion according to the congestion morphology parameters, wherein the congestion morphology parameters include congestion range, congestion intensity and congestion time, wherein the congestion range is the total area of ​​the item stacking, the congestion intensity is the area of ​​the overlapping part of the items, and the congestion time is the total duration of the beginning and end congestion image frames, wherein the beginning image frame is the image frame where the congestion is initially identified, and the end image frame is the image frame where the items that are currently congested are transported to the end of the conveyor line; S4. Analyze and make decisions based on the congestion level and analyze the causes of the congestion.

7. The method for online perception of congestion of conveyor line objects based on machine vision according to claim 6 is characterized in that: In S1, the frequency of collecting images at least satisfies the efficiency of conveying items passing through the conveyor line per unit time.

8. The method for online perception of congestion of conveyor line objects based on machine vision according to claim 6 is characterized in that: Image analysis is also included before S2, including the following: Performing image preprocessing on the item transport image data to filter blank item images; Perform image denoising on the preprocessed image.

9. The method for online perception of congestion of conveyor line objects based on machine vision according to claim 6 is characterized in that: Before S2, feature extraction is also included, which specifically includes the following: Image feature extraction is performed on the denoised item transportation image, which includes collecting image data within a preset time range, labeling the type of congestion, training the machine learning model, and also includes feature extraction on the denoised item transportation image obtained within the actual time period to be detected, and inputting the feature into the trained machine learning model.

Images