A combined feeding port control system based on machine vision

Abstract

The present invention discloses a combined feed port control system based on machine vision, comprising: a feed port detection unit, a feed port control unit, and a terminal display unit; the feed port detection unit is used to collect information data corresponding to the goods put into the feed port, and the information data includes image information data of the goods in the feed port and weight image information data on the goods packaging; the feed port control unit is used to analyze the collected information data, obtain the corresponding feed port analysis results, and control the feed port according to the feed port analysis results; the terminal display unit is used to view the information data corresponding to the feed port and control the feed port through manual operation. By analyzing the feed port image information data and the goods weight data, the standard control orientation of the goods is obtained, and the feed port is controlled to reduce the impact force during the transmission of the goods. Through control, the tilted goods can also be straightened to ensure that the goods enter the vending machine normally.

Description

Technical Field

[0001] The present invention relates to the technical field of feeding port control, and in particular to a combined feeding port control system based on machine vision. Background Art

[0002] The feed port is a key component of a vending machine, serving as a transitional device between the product delivery channel and the beverage storage chamber. Currently, the feed port is connected to a length adjustment module, allowing it to move at a fixed length. However, beverages vary in length, making it impossible to adjust to the length of the beverage, preventing the device from slowing down and correcting the flow of most beverages. Furthermore, the feed port's status cannot be monitored in real time, preventing it from properly aligning and smoothly controlling the product's delivery, significantly increasing the risk of failure. Summary of the Invention

[0003] The present invention provides a combined feed port control system based on machine vision to address the existing problems of the feed port, a key component of a vending machine, serving as a transitional device between the product delivery channel and the beverage storage chamber. Current feed ports on the market are connected to a length adjustment module for movement, and their length is fixed. However, beverages vary in length, making it impossible to adjust to the length of the beverages, resulting in an inability to slow down and correct the flow of most beverages. Furthermore, the system cannot monitor the status of products being delivered to the feed port in real time, preventing smooth control of product delivery, significantly increasing the failure rate.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A combined feeding port control system based on machine vision, comprising: a feeding port detection unit, a feeding port control unit and a terminal display unit;

[0006] The feeding port detection unit is used to collect information data corresponding to the goods put into the feeding port, and the information data includes image information data of the goods in the feeding port and weight image information data on the goods packaging;

[0007] The feeding port control unit is used to analyze the collected information data, obtain the corresponding feeding port analysis results, and control the feeding port according to the feeding port analysis results;

[0008] The terminal display unit is used to view information data corresponding to the feeding port and to control the feeding port through manual operation.

[0009] Wherein, the feeding port detection unit includes: a feeding port machine vision module and a product packaging machine vision module;

[0010] The feeding port machine vision module is used to collect the current product image information data when the product is put into the specified position of the feeding port, and determine the angle of the corresponding product packaging by collecting the product image;

[0011] The product packaging machine vision module is used to collect the corresponding product packaging image after the product is rotated to a set angle under the control of the feeding port control unit.

[0012] The feeding port control unit includes: a rotation angle control module, an identification control module, an opening control module and a length adjustment control module;

[0013] The rotation angle control module is used to determine the angle of the corresponding product packaging based on the product image collected by the feeding port machine vision module, and to obtain the angle required to rotate when the current product packaging angle is within the shooting angle of the product packaging machine vision module through analysis. The rotation angle control module controls the product to rotate to a position where the product packaging machine vision module can capture the product packaging;

[0014] The recognition control module is used to recognize the corresponding product packaging image collected, and obtain the basic information of the current product through recognition, which includes the weight, category and location of the product;

[0015] The opening control module is used to judge the collected current commodity image information data. If it is judged that the commodity is located at the feeding port, the opening control module controls the feeding port switch device to open the feeding port entrance, and the commodity is fed into the feeding port through the entrance;

[0016] The length adjustment control module is used to confirm the location and weight of the goods through the collected corresponding product packaging images, and by analyzing the location and weight of the goods, obtain the preset length value that currently ensures the stable transportation of the goods. The length adjustment control module adjusts the pallet length to the preset length value.

[0017] Wherein, the terminal display unit includes: an information viewing module and an operation module;

[0018] The information viewing module is used to view the commodity information data collected by the feeding port detection unit;

[0019] The operation module is used for the staff to send control instructions to the feeding port control unit through key operations, and the feeding port control unit controls the feeding port according to the received control instructions.

[0020] The product packaging machine vision module includes: a line array camera submodule and an image data control submodule;

[0021] The line array camera submodule is used to collect key information of the product by taking pictures of the corresponding product packaging. The line array camera submodule adjusts the scanning line frequency according to the dynamic changes of the product in the high-definition image data captured. The key information of the product includes: the corresponding name of the product, basic information of the corresponding name of the product, and the location information of the feeding port where the product is located;

[0022] The image data control submodule is used to preprocess high-definition image data and store image data. During the image acquisition process, the collected high-definition image data of the feeding port is stored through the image data control submodule. The image data control submodule adopts multi-threading technology to increase the response speed of image storage.

[0023] Among them, the feeding port control unit identifies the goods in the image and analyzes the image. After identifying the goods, the basic information of the goods is saved in the EXCEL table of the information viewing module. The staff queries the corresponding product information through the information viewing module and views the image information of the goods when they are located at the feeding port through the information viewing module.

[0024] Wherein, the identification control module includes: a deep learning submodule and a feeding abnormality identification submodule;

[0025] The deep learning module is used to extract features of template image information and search image information based on the product image information data when the product is located at the feeding port, and convolve the feature map of the template image information with the feature map of the search image as a convolution kernel to obtain a response map after convolution. The peak point in the response map is mapped to the original image to obtain an estimated target position, and the current position of the product is judged to see whether it meets the estimated target position;

[0026] The feeding abnormality identification module is used to perform data analysis on the collected image data, identify whether there is an abnormality in the feeding port through data analysis, and automatically alarm if an abnormality is found in the feeding port.

[0027] The deep learning submodule inputs image data based on a deep learning algorithm and outputs learning results through training;

[0028] Establishing a Gaussian background model and obtaining image feature data based on the Gaussian background model;

[0029] Based on the distribution of the Gaussian background model, the matching degree between the pixel information of each pixel in the image feature data and the Gaussian background model is obtained. The foreground and background points are classified according to the matching degree between the pixel information and the Gaussian background model. The foreground is regarded as the detected target. The Gaussian distribution parameters are updated through each pixel in the image to obtain the updated Gaussian model until the training is completed.

[0030] The collected images are preprocessed by performing Gaussian smoothing filtering on the images to remove noise. The background and foreground are determined using the Gaussian modeling method. The foreground is eroded and expanded, i.e., closed operations are performed to eliminate interference. The corresponding area of the image is selected, and the image area and threshold are used for screening to extract the target in the image.

[0031] Among them, when the feeding port machine vision module detects that the product is located at the feeding port, the feeding port control unit triggers the linear array camera submodule to enter the photo-taking state. After the linear array camera submodule obtains the target image, it analyzes the target image through the recognition control module to obtain the product information and position information in the target image, and obtains the corresponding analysis results based on the product information and position information. The length adjustment control module adjusts the length of the pallet according to the corresponding analysis results, and straightens the posture of the product by adjusting the length of the pallet. The product is smoothly transferred from the feeding port to the storage warehouse through the pallet.

[0032] The length of the pallet is regulated based on fuzzy control, wherein the fuzzy control includes: fuzzification of input quantity, fuzzy reasoning, and defuzzification of output quantity;

[0033] Fuzzy input: The actual input variable of pallet length is expressed as a fuzzy variable through corresponding rules. The length adjustment control module analyzes the pallet length and product data to determine the range of the actual change. The range is divided into several levels according to the weight of the corresponding product and the stability of its location.

[0034] Fuzzy reasoning: The corresponding fuzzy control rules are converted into a control rule table. Based on the change in pallet length when different products are in different positions obtained by the deep learning submodule, fuzzy reasoning is performed based on the change value.

[0035] Output defuzzification: Based on the reasoning of fuzzy rules, the location of the product, weight data and the current length of the pallet are output, and the center of gravity method is used to defuzzify the change pattern of the pallet length.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] A combined feed port control system based on machine vision comprises: a feed port detection unit, a feed port control unit, and a terminal display unit; the feed port detection unit is used to collect information data corresponding to the goods put into the feed port, the information data including the image information data of the goods in the feed port and the weight image information data on the goods packaging; the feed port control unit is used to analyze the collected information data, obtain the corresponding feed port analysis results, and control the feed port according to the feed port analysis results; the terminal display unit is used to view the information data corresponding to the feed port and control the feed port through manual operation. By analyzing the feed port image information data and the goods weight data, the standard product control orientation is obtained, and the feed port is controlled to reduce the impact force during the goods transmission. Through control, the tilted goods can also be straightened to ensure the normal entry of the goods into the vending machine.

[0038] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0039] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0041] Figure 1 This is a structural diagram of a combined feeding port control system based on machine vision in an embodiment of the present invention;

[0042] Figure 2 This is a flow chart of a combined feeding port control system based on machine vision in an embodiment of the present invention;

[0043] Figure 3 This is a structural diagram of a feeding port control unit in a combined feeding port control system based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0045] The embodiment of the present invention provides a combined feeding port control system based on machine vision, please refer to Figures 1 to 3, including: a feeding port detection unit, a feeding port control unit and a terminal display unit;

[0046] The feeding port detection unit is used to collect information data corresponding to the goods put into the feeding port, and the information data includes image information data of the goods in the feeding port and weight image information data on the goods packaging;

[0047] The feeding port control unit is used to analyze the collected information data, obtain the corresponding feeding port analysis results, and control the feeding port according to the feeding port analysis results;

[0048] The terminal display unit is used to view information data corresponding to the feeding port and to control the feeding port through manual operation.

[0049] The working principle of the above technical solution is as follows: the feed port detection unit is used to collect information data corresponding to the goods put into the feed port, and the information data includes the image information data of the goods in the feed port and the weight image information data on the goods packaging; the feed port control unit is used to analyze the collected information data, obtain the corresponding feed port analysis results, and control the feed port according to the feed port analysis results; the terminal display unit is used to view the information data corresponding to the feed port and control the feed port through manual operation. By analyzing the feed port image information data and the goods weight data, the standard product control orientation is obtained, and the feed port is controlled to reduce the impact force during the goods transmission. Through control, the tilted goods can also be straightened to ensure the normal entry of the goods into the vending machine.

[0050] The beneficial effects of the above technical solution are as follows: the feed port detection unit is used to collect information data corresponding to the goods put into the feed port, and the information data includes the image information data of the goods in the feed port and the weight image information data on the goods packaging; the feed port control unit is used to analyze the collected information data, obtain the corresponding feed port analysis results, and control the feed port according to the feed port analysis results; the terminal display unit is used to view the information data corresponding to the feed port and control the feed port through manual operation. By analyzing the feed port image information data and the goods weight data, the standard product control orientation is obtained, and the feed port is controlled to reduce the impact force during the goods transmission. Through control, the tilted goods can also be straightened to ensure that the goods enter the vending machine normally.

[0051] In another embodiment, the feeding port detection unit includes: a feeding port machine vision module and a product packaging machine vision module;

[0052] The feeding port machine vision module is used to collect the current product image information data when the product is put into the specified position of the feeding port, and determine the angle of the corresponding product packaging by collecting the product image;

[0053] The product packaging machine vision module is used to collect the corresponding product packaging image after the product is rotated to a set angle under the control of the feeding port control unit.

[0054] The working principle of this technical solution is as follows: the inlet machine vision module is used to capture image data of the product when it is dropped into the designated position of the inlet, and the angle of the corresponding product packaging is determined by the captured product image. The product packaging machine vision module is used to capture the corresponding product packaging image after the product is rotated to a set angle by the inlet control unit. By capturing the product packaging image, all product information can be obtained, facilitating product analysis.

[0055] The beneficial effects of the above technical solution are as follows: the feeding port machine vision module is used to capture image data of the current product when the product is dropped into the designated position of the feeding port, and the angle of the corresponding product packaging is determined by the captured product image; the product packaging machine vision module is used to capture the corresponding product packaging image after the product is rotated to a set angle by the feeding port control unit. By capturing the product packaging image, all product information can be obtained, facilitating product analysis.

[0056] In another embodiment, the feeding port control unit includes: a rotation angle control module, an identification control module, an opening control module and a length adjustment control module;

[0057] The rotation angle control module is used to determine the angle of the corresponding product packaging based on the product image collected by the feeding port machine vision module, and to obtain the angle required to rotate when the current product packaging angle is within the shooting angle of the product packaging machine vision module through analysis. The rotation angle control module controls the product to rotate to a position where the product packaging machine vision module can capture the product packaging;

[0058] The recognition control module is used to recognize the corresponding product packaging image collected, and obtain the basic information of the current product through recognition, which includes the weight, category and location of the product;

[0059] The opening control module is used to judge the collected current commodity image information data. If it is judged that the commodity is located at the feeding port, the opening control module controls the feeding port switch device to open the feeding port entrance, and the commodity is fed into the feeding port through the entrance;

[0060] The length adjustment control module is used to confirm the location and weight of the goods through the collected corresponding product packaging images, and by analyzing the location and weight of the goods, obtain the preset length value that currently ensures the stable transportation of the goods. The length adjustment control module adjusts the pallet length to the preset length value.

[0061] The working principle of the above technical solution is as follows: the rotation angle control module is used to determine the angle of the corresponding product packaging based on the product image collected by the feeding port machine vision module, and obtain the number of angles required to rotate when the current product packaging is at the shooting angle of the product packaging machine vision module through analysis. The rotation angle control module controls the product to rotate to the position where the product packaging machine vision module can shoot the product packaging; the recognition control module is used to identify the corresponding product packaging image collected, and obtain the basic information of the current product through recognition. The basic information includes the weight of the product, the category of the product and the location; the opening control module is used to judge the collected current product image information data. If it is judged that the product is located at the feeding port, the opening control module controls the feeding port switch device to open the feeding port entrance, and the product is fed into the feeding port through the entrance; the length adjustment control module is used to confirm the location and weight of the product through the collected corresponding product packaging image, and obtain the preset length value for ensuring the stable transportation of the product by analyzing the location and weight of the product. The length adjustment control module adjusts the pallet length to the preset length value. By controlling the length of the pallet, the stability of the product transmission is ensured. The control can also be used to straighten the tilted products to ensure that the products enter the vending machine normally.

[0062] The beneficial effects of the above technical solution are as follows: the rotation angle control module is used to determine the angle of the corresponding product packaging based on the product image collected by the feeding port machine vision module, and obtain the number of angles required to rotate when the current product packaging is at the shooting angle of the product packaging machine vision module through analysis. The rotation angle control module controls the product to rotate to the position where the product packaging machine vision module can shoot the product packaging; the recognition control module is used to identify the corresponding product packaging image collected, and obtain the basic information of the current product through recognition, and the basic information includes the weight of the product, the category of the product and the location; the opening control module is used to judge the collected current product image information data. If it is judged that the product is located at the feeding port, the opening control module controls the feeding port switch device to open the feeding port entrance, and the product is fed into the feeding port through the entrance; the length adjustment control module is used to confirm the location and weight of the product through the collected corresponding product packaging image, and obtain the preset length value for ensuring the stable transportation of the product by analyzing the location and weight of the product. The length adjustment control module adjusts the pallet length to the preset length value. By controlling the length of the pallet, the stability of the product transmission is ensured. The control can also be used to straighten the tilted products to ensure that the products enter the vending machine normally.

[0063] In another embodiment, the terminal display unit includes: an information viewing module and an operation module;

[0064] The information viewing module is used to view the commodity information data collected by the feeding port detection unit;

[0065] The operation module is used for the staff to send control instructions to the feeding port control unit through key operations, and the feeding port control unit controls the feeding port according to the received control instructions.

[0066] The working principle of the above technical solution is as follows: the information viewing module is used to view the commodity information data collected by the feeding port detection unit; the operation module is used by the staff to send control instructions to the feeding port control unit through key operation, and the feeding port control unit controls the feeding port according to the received control instructions. This facilitates the staff to operate the feeding port.

[0067] The beneficial effects of the above technical solution are as follows: the information viewing module is used to view the commodity information data collected by the feeding port detection unit; the operation module is used for the staff to send control instructions to the feeding port control unit through key operation, and the feeding port control unit controls the feeding port according to the received control instructions. This facilitates the staff to operate the feeding port.

[0068] In another embodiment, the product packaging machine vision module includes: a line array camera submodule, an image data control submodule;

[0069] The line array camera submodule is used to collect key information of the product by taking pictures of the corresponding product packaging. The line array camera submodule adjusts the scanning line frequency according to the dynamic changes of the product in the high-definition image data captured. The key information of the product includes: the corresponding name of the product, basic information of the corresponding name of the product, and the location information of the feeding port where the product is located;

[0070] The image data control submodule is used to preprocess high-definition image data and store image data. During the image acquisition process, the collected high-definition image data of the feeding port is stored through the image data control submodule. The image data control submodule adopts multi-threading technology to increase the response speed of image storage.

[0071] The working principle of the above technical solution is: the line array camera submodule is used to collect key information of the product by shooting pictures of the corresponding product packaging. The line array camera submodule adjusts the scanning line frequency according to the dynamic changes of the product in the high-definition image data obtained by shooting. The key information of the product includes: the corresponding name of the product, the basic information of the corresponding name of the product, and the position information of the feeding port where the product is located; the image data management submodule is used to pre-process the high-definition image data and store the image data. During the image acquisition process, the collected high-definition image data of the feeding port is stored by the image data management submodule. The image data management submodule adopts multi-threading technology to increase the response speed of image storage.

[0072] The beneficial effects of the above technical solution are as follows: the line array camera submodule is used to collect key information of the product by shooting pictures of the corresponding product packaging, and the line array camera submodule adjusts the scanning line frequency according to the dynamic changes of the product in the high-definition image data obtained by shooting. The key information of the product includes: the corresponding name of the product, the basic information of the corresponding name of the product, and the position information of the feeding port where the product is located; the image data control submodule is used to pre-process the high-definition image data and store the image data, wherein, during the image acquisition process, the collected high-definition image data of the feeding port is stored by the image data control submodule, and the image data control submodule adopts multi-threading technology to increase the response speed of image storage.

[0073] In another embodiment, the feeding port control unit identifies the goods in the image and analyzes the image. After identifying the goods, the basic information of the goods is saved in the EXCEL table of the information viewing module. The staff queries the corresponding product information through the information viewing module and views the image information of the goods when they are located at the feeding port through the information viewing module.

[0074] The working principle of the above technical solution is: the feeding port control unit identifies the goods in the image and analyzes the image. After identifying the goods, the basic information of the goods is saved in the EXCEL table of the information viewing module. The staff queries the corresponding product information through the information viewing module and views the image information of the goods when they are located at the feeding port through the information viewing module.

[0075] The beneficial effects of the above technical solution are: the feeding port control unit identifies the goods in the image and analyzes the image, and after identifying the goods, saves the basic information of the goods into the EXCEL table of the information viewing module. The staff queries the corresponding product information through the information viewing module and views the image information of the goods when they are located at the feeding port through the information viewing module.

[0076] In another embodiment, the identification control module includes: a deep learning submodule and a feeding abnormality identification submodule;

[0077] The deep learning module is used to extract features of template image information and search image information based on the product image information data when the product is located at the feeding port, and convolve the feature map of the template image information with the feature map of the search image as a convolution kernel to obtain a response map after convolution. The peak point in the response map is mapped to the original image to obtain an estimated target position, and the current position of the product is judged to see whether it meets the estimated target position;

[0078] The feeding abnormality identification module is used to perform data analysis on the collected image data, identify whether there is an abnormality in the feeding port through data analysis, and automatically alarm if an abnormality is found in the feeding port.

[0079] The working principle of the above technical solution is: the deep learning module is used to extract the features of the template image information and the search image information based on the product image information data when the product is located at the feeding port, and the feature map of the template image information is used as the convolution kernel to convolve with the feature map of the search image. After the convolution, a response map is obtained, and the peak point in the response map is mapped to the original image to obtain the estimated target position, and the current position of the product is judged to see whether it meets the estimated target position; the feeding abnormality recognition module is used to perform data analysis on the collected image data, and identify whether there is an abnormality in the feeding port through data analysis. If an abnormality is found in the feeding port, an alarm is automatically issued.

[0080] The beneficial effects of the above technical solution are as follows: the deep learning module is used to extract the features of the template image information and the search image information based on the product image information data when the product is located at the feeding port, and the feature map of the template image information is used as the convolution kernel to convolve with the feature map of the search image. After the convolution, a response map is obtained, and the peak points in the response map are mapped to the original image to obtain the estimated target position, and the current position of the product is judged to see whether it meets the estimated target position; the feeding abnormality recognition module is used to perform data analysis on the collected image data, and identify whether there is an abnormality in the feeding port through data analysis. If an abnormality is found in the feeding port, an alarm is automatically issued.

[0081] In another embodiment, the deep learning submodule inputs image data based on a deep learning algorithm and outputs learning results through training;

[0082] Establishing a Gaussian background model and obtaining image feature data based on the Gaussian background model;

[0083] Based on the distribution of the Gaussian background model, the matching degree between the pixel information of each pixel in the image feature data and the Gaussian background model is obtained. The foreground and background points are classified according to the matching degree between the pixel information and the Gaussian background model. The foreground is regarded as the detected target. The Gaussian distribution parameters are updated through each pixel in the image to obtain the updated Gaussian model until the training is completed.

[0084] The collected images are preprocessed by performing Gaussian smoothing filtering on the images to remove noise. The background and foreground are determined using the Gaussian modeling method. The foreground is eroded and expanded, i.e., closed operations are performed to eliminate interference. The corresponding area of the image is selected, and the image area and threshold are used for screening to extract the target in the image.

[0085] The working principle of the above technical solution is as follows: the deep learning submodule inputs image data based on the deep learning algorithm and outputs learning results through training;

[0086] Establishing a Gaussian background model and obtaining image feature data based on the Gaussian background model;

[0087] Based on the distribution of the Gaussian background model, the matching degree between the pixel information of each pixel in the image feature data and the Gaussian background model is obtained. The foreground and background points are classified according to the matching degree between the pixel information and the Gaussian background model. The foreground is regarded as the detected target. The Gaussian distribution parameters are updated through each pixel in the image to obtain the updated Gaussian model until the training is completed.

[0088] The collected images are preprocessed by performing Gaussian smoothing filtering on the images to remove noise. The background and foreground are determined using the Gaussian modeling method. The foreground is eroded and expanded, i.e., closed operations are performed to eliminate interference. The corresponding area of the image is selected, and the image area and threshold are used for screening to extract the target in the image.

[0089] The beneficial effects of the above technical solution are as follows: the deep learning submodule inputs image data based on the deep learning algorithm and outputs learning results through training; establishes a Gaussian background model and obtains image feature data based on the Gaussian background model; obtains the matching degree between the pixel information of each pixel in the image feature data and the Gaussian background model based on the distribution of the Gaussian background model, classifies the foreground and background points according to the matching degree between the pixel information and the Gaussian background model, uses the foreground as the detected target, updates the Gaussian distribution parameters through each pixel in the image, obtains the update of the Gaussian model until the training is completed; pre-processes the collected image, performs Gaussian smoothing filtering on the image to remove noise, uses Gaussian modeling to determine the background and foreground, performs corrosion and expansion processing (i.e., closing operation) on the foreground to eliminate interference, selects the corresponding area of the image, uses the image area and threshold for screening, and extracts the target in the image. In this way, accurate image information is obtained, and the feeding port is controlled by analyzing the image.

[0090] In another embodiment, when the feeding port machine vision module detects that the product is located at the feeding port, the feeding port control unit triggers the linear array camera submodule to enter the photo taking state. After the linear array camera submodule obtains the target image, it analyzes the target image through the recognition control module to obtain the product information and position information in the target image, and obtains the corresponding analysis results based on the product information and position information. The length adjustment control module adjusts the length of the pallet according to the corresponding analysis results, and by adjusting the length of the pallet to straighten the posture of the product, the product is smoothly transferred from the feeding port to the storage warehouse through the pallet.

[0091] The working principle of the above technical solution is as follows: when the feeding port machine vision module detects that a product is located at the feeding port, the feeding port control unit triggers the linear array camera submodule to enter the photo state. After the linear array camera submodule acquires the target image, the recognition control module analyzes the target image to obtain product information and location information in the target image. Based on the product information and location information, a corresponding analysis result is obtained. The length adjustment control module adjusts the length of the pallet according to the corresponding analysis result. By adjusting the pallet length, the product is straightened, and the product is smoothly transferred from the feeding port to the storage bin via the pallet. This control can also be used to straighten the position of tilted products to ensure that the products enter the vending machine normally.

[0092] The beneficial effects of the above technical solution are as follows: when the feeding port machine vision module detects that a product is located at the feeding port, the feeding port control unit triggers the linear array camera submodule to enter the photo capture state. After the linear array camera submodule acquires the target image, the recognition control module analyzes the target image to obtain product information and location information in the target image. Based on the product information and location information, a corresponding analysis result is obtained. The length adjustment control module adjusts the length of the pallet based on the corresponding analysis result. By adjusting the pallet length, the product is aligned, and the product is smoothly transferred from the feeding port to the storage bin via the pallet. This control can also be used to straighten the position of tilted products, ensuring that the products enter the vending machine normally.

[0093] In another embodiment, the length of the pallet is regulated based on fuzzy control, wherein the fuzzy control includes: fuzzification of input quantity, fuzzy reasoning, and defuzzification of output quantity;

[0094] Fuzzy input: The actual input variable of pallet length is expressed as a fuzzy variable through corresponding rules. The length adjustment control module analyzes the pallet length and product data to determine the range of the actual change. The range is divided into several levels according to the weight of the corresponding product and the stability of its location.

[0095] Fuzzy reasoning: The corresponding fuzzy control rules are converted into a control rule table. Based on the change in pallet length when different products are in different positions obtained by the deep learning submodule, fuzzy reasoning is performed based on the change value.

[0096] Output defuzzification: Based on the reasoning of fuzzy rules, the location of the product, weight data and the current length of the pallet are output, and the center of gravity method is used to defuzzify the change pattern of the pallet length.

[0097] The working principle of the above technical solution is: the length of the pallet is regulated based on fuzzy control, wherein the fuzzy control includes: fuzzification of input quantity, fuzzy reasoning, and defuzzification of output quantity;

[0098] Fuzzy input: The actual input variable of pallet length is expressed as a fuzzy variable through corresponding rules. The length adjustment control module analyzes the pallet length and product data to determine the range of the actual change. The range is divided into several levels according to the weight of the corresponding product and the stability of its location.

[0099] Fuzzy reasoning: The corresponding fuzzy control rules are converted into a control rule table. Based on the change in pallet length when different products are in different positions obtained by the deep learning submodule, fuzzy reasoning is performed based on the change value.

[0100] Output defuzzification: Based on the reasoning of fuzzy rules, the location of the product, weight data and the current length of the pallet are output, and the center of gravity method is used to defuzzify the change pattern of the pallet length.

[0101] The beneficial effects of the above technical solution are: regulating the length of the pallet based on fuzzy control, wherein the fuzzy control includes: input fuzzification, fuzzy reasoning, and output defuzzification; input fuzzification: the actual input variable of the pallet length is expressed as a fuzzy quantity through the corresponding rules, and the length adjustment control module analyzes the pallet length and product data, determines the range of change of the actual change through analysis, and divides the change range into several levels according to the weight of the corresponding product and the stability of the position; fuzzy reasoning: converting the corresponding fuzzy control rules into a control rule table, and performing fuzzy reasoning based on the change value of the pallet length when different products are in different positions obtained by the deep learning submodule; output defuzzification: outputting the rules of the product position, weight data and the current length of the pallet based on the reasoning of fuzzy rules, and using the center of gravity method to defuzzify the rule of the pallet length change.

[0102] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A combined feeding port control system based on machine vision, characterized in that: include: Feeding port detection unit, feeding port control unit and terminal display unit; The feeding port detection unit is used to collect information data corresponding to the goods put into the feeding port, and the information data includes image information data of the goods in the feeding port and weight image information data on the goods packaging; The feeding port control unit is used to analyze the collected information data, obtain the corresponding feeding port analysis results, and control the feeding port according to the feeding port analysis results; The terminal display unit is used to view the information data corresponding to the feeding port and to control the feeding port through manual operation; The feeding port detection unit includes: a feeding port machine vision module and a product packaging machine vision module; The feeding port machine vision module is used to collect the current product image information data when the product is put into the specified position of the feeding port, and determine the angle of the corresponding product packaging by collecting the product image; The product packaging machine vision module is used to collect the corresponding product packaging image after the product is rotated to a set angle by the feeding port control unit; When the feeding port machine vision module detects that the product is located at the feeding port, the feeding port control unit triggers the linear array camera submodule to enter the photo taking state. After the linear array camera submodule obtains the target image, it analyzes the target image through the recognition control module to obtain the product information and position information in the target image. According to the product information and position information, the corresponding analysis results are obtained. The length adjustment control module adjusts the length of the pallet according to the corresponding analysis results. By adjusting the length of the pallet to correct the posture of the product, the product is smoothly transferred from the feeding port to the storage bin through the pallet. The length of the pallet is regulated based on fuzzy control, wherein the fuzzy control includes: fuzzification of input quantity, fuzzy reasoning, and defuzzification of output quantity; Fuzzy input: The actual input variable of pallet length is expressed as a fuzzy variable through corresponding rules. The length adjustment control module analyzes the pallet length and product data to determine the range of the actual change. The range is divided into several levels according to the weight of the corresponding product and the stability of its location. Fuzzy reasoning: The corresponding fuzzy control rules are converted into a control rule table. Based on the change in pallet length when different products are in different positions obtained by the deep learning submodule, fuzzy reasoning is performed based on the change value. Output defuzzification: Based on the reasoning of fuzzy rules, the location of the product, weight data and the current length of the pallet are output, and the center of gravity method is used to defuzzify the change pattern of the pallet length.

2. The combined feeding port control system based on machine vision according to claim 1, characterized in that: The feeding port control unit includes: a rotation angle control module, an identification control module, an opening control module and a length adjustment control module; The rotation angle control module is used to determine the angle of the corresponding product packaging based on the product image collected by the feeding port machine vision module, and to obtain the angle required to rotate when the current product packaging angle is within the shooting angle of the product packaging machine vision module through analysis. The rotation angle control module controls the product to rotate to a position where the product packaging machine vision module can capture the product packaging; The recognition control module is used to recognize the corresponding product packaging image collected, and obtain the basic information of the current product through recognition, which includes the weight, category and location of the product; The opening control module is used to judge the collected current commodity image information data. If it is judged that the commodity is located at the feeding port, the opening control module controls the feeding port switch device to open the feeding port entrance, and the commodity is fed into the feeding port through the entrance; The length adjustment control module is used to confirm the location and weight of the goods through the collected corresponding product packaging images, and by analyzing the location and weight of the goods, obtain the preset length value that currently ensures the stable transportation of the goods. The length adjustment control module adjusts the pallet length to the preset length value.

3. The combined feeding port control system based on machine vision according to claim 1, characterized in that: The terminal display unit includes: an information viewing module and an operation module; The information viewing module is used to view the commodity information data collected by the feeding port detection unit; The operation module is used for the staff to send control instructions to the feeding port control unit through key operations, and the feeding port control unit controls the feeding port according to the received control instructions.

4. The combined feeding port control system based on machine vision according to claim 2, characterized in that: The product packaging machine vision module includes: a line array camera submodule and an image data control submodule; The line array camera submodule is used to collect key information of the product by taking pictures of the corresponding product packaging. The line array camera submodule adjusts the scanning line frequency according to the dynamic changes of the product in the high-definition image data captured. The key information of the product includes: the corresponding name of the product, basic information of the corresponding name of the product, and the location information of the feeding port where the product is located; The image data control submodule is used to preprocess high-definition image data and store image data. During the image acquisition process, the collected high-definition image data of the feeding port is stored through the image data control submodule. The image data control submodule adopts multi-threading technology to increase the response speed of image storage.

5. The combined feeding port control system based on machine vision according to claim 3, characterized in that: The feeding port control unit identifies the product in the image and analyzes the image. After identifying the product, the basic information of the product is saved in the EXCEL table of the information viewing module. The staff queries the corresponding product information through the information viewing module and views the image information of the product when it is located at the feeding port through the information viewing module.

6. The combined feeding port control system based on machine vision according to claim 2, characterized in that: The identification control module includes: a deep learning submodule and a feeding abnormality identification submodule; The deep learning submodule is used to extract features of template image information and search image information based on the product image information data when the product is located at the feeding port, and convolve the feature map of the template image information with the feature map of the search image as the convolution kernel. After the convolution, a response map is obtained, and the peak point in the response map is mapped to the original image to obtain an estimated target position, and the current position of the product is judged to see whether it meets the estimated target position; The feeding abnormality identification submodule is used to perform data analysis on the collected image data, identify whether there is an abnormality in the feeding port through data analysis, and automatically alarm if an abnormality is found in the feeding port.

7. The combined feeding port control system based on machine vision according to claim 6, characterized in that: The deep learning submodule inputs image data based on a deep learning algorithm and outputs learning results through training; Establishing a Gaussian background model and obtaining image feature data based on the Gaussian background model; Based on the distribution of the Gaussian background model, the matching degree between the pixel information of each pixel in the image feature data and the Gaussian background model is obtained. The foreground and background points are classified according to the matching degree between the pixel information and the Gaussian background model. The foreground is regarded as the detected target. The Gaussian distribution parameters are updated through each pixel in the image to obtain the updated Gaussian model until the training is completed. The collected images are preprocessed by performing Gaussian smoothing filtering on the images to remove noise. The background and foreground are determined using the Gaussian modeling method. The foreground is eroded and expanded, i.e., closed operations are performed to eliminate interference. The corresponding area of the image is selected, and the image area and threshold are used for screening to extract the target in the image.

Images