An industrial visual inspection system based on computing resource sharing

Abstract

The present invention relates to an industrial visual inspection system for computing resource sharing, comprising: a resource pool, including several industrial computer devices; a computing resource management center, used to manage the busy and idle states of several industrial computer devices in the resource pool; an image acquisition device, used to acquire product images; a gateway device, used to receive product images, and send the product images to one of the industrial computer devices in the resource pool that has redundant computing resources and is available through the computing resource management center; the industrial computer device is used to perform visual analysis on the product image, and send the visual analysis result to the gateway device; the gateway device is also used to receive the visual analysis result, and send the visual analysis result to the sorting device; the sorting device is used to sort the product according to the visual analysis result. By adopting a resource pool composed of consumer-grade industrial computer devices for resource sharing, the effective utilization rate of the equipment is improved and the cost of visual inspection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of industrial control technology, and in particular to an industrial vision inspection system based on computing resource sharing. Background Art

[0002] At present, in the process of promoting the digital upgrade of industrial manufacturing and implementing intelligent manufacturing, one of the key issues is how to use automated technology to inspect the quality of industrial products manufactured on the production line, so as to realize an end-to-end automated production line from raw material processing, production to back-end quality inspection. With the development of computer vision technology, many companies have begun to use image detection methods to automatically detect product defects and conduct sorting. In recent years, due to the stable and accurate recognition effect of deep learning algorithms, more and more factories have begun to adopt deep learning-based methods to replace traditional image analysis-based methods, and have achieved very good application results. Different from visual inspection methods in other fields, industrial visual inspection often has the following characteristics:

[0003] 1. High real-time requirements. In industrial visual inspection, in order to connect with the actual industrial system and meet the production rhythm of actual industrial production, the analysis of each product image must be carried out in real time, and the analysis results must be returned within the range allowed by the production rhythm. There is no distinction between high and low priorities between different image analysis tasks;

[0004] 2. High stability requirements. Any failure in image analysis or a freeze in the analysis process may disrupt or stop the normal production rhythm, affecting the production process. More seriously, it may cause the misalignment or loss of analysis results of different products before and after, resulting in errors in the sorting of good and defective products.

[0005] Therefore, in actual visual analysis, the analysis of an image is often performed on only one hardware device, rather than using a model scheduling architecture with multiple hardware devices. Because if a model scheduling architecture with multiple hardware devices is used to analyze a single image, in order to ensure real-time performance, it is necessary to ensure that the image can be transmitted in real time between different models, so the network connection between hardware devices needs to use a 10G network or even more expensive network connection. The implementation cost of this is more expensive than enhancing the configuration of a single device, and the network connection method further increases the possibility of errors and reduces the stability of the analysis.

[0006] In the computer vision quality inspection method based on deep learning on a single device, in order to improve the speed of reasoning, it is more common to use a GPU (Graphics Processing Unit)-based method to perform reasoning operations on the deep learning model. Therefore, in order to meet the needs of real-time control and stability on the industrial production line, the existing practice is to place an industrial computer with a GPU on a machine, directly connect the camera through the industrial computer to obtain the captured image, and use the industrial computer to call the quality inspection algorithm for analysis. According to the analysis results, the industrial computer is connected to the sorting equipment on the machine, and the sorting signal is directly sent to the sorting equipment to realize the control of the sorting equipment. As for the method of placing an industrial computer on each machine, the cost is relatively high due to the high cost of GPU. It is relatively difficult to operate and maintain such a GPU. Once a GPU fails, it is easy to cause a large area of ​​machines to be unusable, which is difficult to promote on a large scale in actual production. To address this problem, the general solution is to transplant the deep learning model to a cheaper hardware device such as NUC or to use the reasoning method based on ARM architecture hardware devices to reduce the cost of hardware. However, the computing power of cheap hardware is generally relatively weak. For visual quality inspection algorithms that require higher computing resources, there is no way to replace them through such transplantation. On the other hand, since the visual quality inspection algorithm is computationally intensive, the power consumption is relatively high, and the demand for resources is relatively high, it is easy to have abnormalities after running for a long time. For the solution of one computing hardware for one machine, single point failure is easy to occur. Once the computing resources are abnormal, the machine cannot perform normal analysis. This phenomenon is more likely to occur in hot summer weather. Summary of the invention

[0007] The present invention aims to overcome at least one defect in the above-mentioned prior art and provide an industrial visual inspection system based on computing resource sharing, which adopts a resource pool composed of consumer-grade industrial computer devices for resource sharing, improves the effective utilization rate of equipment and reduces the cost of visual inspection.

[0008] The scheme adopted by the present invention is:

[0009] An industrial visual inspection system based on computing resource sharing is provided, comprising:

[0010] Several image acquisition devices, several gateway devices, a computing resource management center, a resource pool and several sorting devices, one of the gateway devices is respectively connected to one or more of the image acquisition devices and one of the sorting devices, several of the gateway devices are respectively connected to the computing resource management center, and the resource pool includes several industrial computer devices;

[0011] The computing resource management center is used to manage the computing resources of several industrial computer devices in the resource pool;

[0012] The image acquisition device is used to acquire product images and send the acquired product images to the gateway device;

[0013] The gateway device is used to receive the product image, send the product image to an industrial computer device in the resource pool that has redundant computing resources and is available through the computing resource management center, and maintain a connection with the industrial computer device to directly send the subsequent product image to the industrial computer device;

[0014] The industrial computer device is used to perform visual analysis on the product image, and send the visual analysis result and its current load condition and health condition to the gateway device;

[0015] The gateway device is also used to receive the visual analysis result and the current load condition and health condition sent by the connected industrial computer device, send the visual analysis result to the sorting device, and judge whether the industrial computer device is abnormal according to the current load condition and health condition of the industrial computer device. If so, disconnect the connection with the industrial computer device, and send the next product image to another industrial computer device with redundant computing resources and available in the resource pool through the computing resource management center when sending the product image next time, and keep connected with the other industrial computer device to send the subsequent product image directly to the other industrial computer device;

[0016] The sorting equipment is used to sort the products according to the visual analysis results.

[0017] A gateway device is placed at the location of each machine to replace the industrial computer device in the prior art. Compared with the method of connecting the cloud to all sorting devices in the prior art, the method of connecting a gateway device to a sorting device has higher stability. The gateway device sends the product image collected by the image acquisition device to an industrial computer device with excess computing resources and available in the resource pool through the computing resource management center. After the industrial computer device performs visual analysis on the product image, the gateway device sends the visual analysis result to the sorting device, so that the sorting device performs sorting according to the visual analysis result, thereby performing real-time visual analysis on each product image to meet the production rhythm and real-time requirements of actual industrial production; since the gateway device does not have a GPU, it only needs to be compared A relatively low-configuration CPU, memory, and hard disk configuration can meet its communication and storage functions. While feeding back the visual analysis results to the gateway device, the industrial computer device also sends its own load and health status to the gateway device. When the gateway device determines that the connected industrial computer device has an abnormality, it actively switches to another industrial computer device with excess computing resources and available, thereby ensuring that the production rhythm is not interrupted by abnormal industrial computer equipment without additional communication costs. A resource pool is formed by using several consumer-grade industrial computer devices, and the resource pool shares computing resources between different machines to provide external visual analysis services, thereby improving the effective utilization rate of industrial computer equipment, reducing equipment procurement costs, and reducing the hardware cost of visual quality inspection.

[0018] Furthermore, the gateway device is used to receive the product image, and send the product image to an industrial computer device in the resource pool that has redundant computing resources and is available through the computing resource management center, including:

[0019] The gateway device is used to receive the product image, generate a visual analysis request according to the product image, and send the product image and the visual analysis request to the computing resource management center, so that the computing resource management center receives the visual analysis request, obtains an industrial computer device with excess computing resources and available from the resource pool according to the visual analysis request, and generates feedback information to the gateway device;

[0020] The gateway device is also used to send the product image to the industrial computer device with redundant computing resources and available according to the feedback information;

[0021] The gateway device is also used to determine whether the industrial computer device is abnormal according to the current load and health status of the industrial computer device. If so, disconnect the connection with the industrial computer device, and send the next product image to another industrial computer device with redundant computing resources and available in the resource pool through the computing resource management center when sending the product image next time, including:

[0022] The gateway device is also used to determine whether the industrial computer device is abnormal according to the current load condition and health condition of the industrial computer device;

[0023] If so, disconnect the connection with the industrial computer device, generate a switching request when sending the product image next time, and send the switching request to the computing resource management center, so that the computing resource management center receives the switching request, re-acquires an industrial computer device with redundant computing resources and available from the resource pool according to the switching request, and generates new feedback information to the gateway device;

[0024] The gateway device is further configured to send the product image to be sent next time to the industrial computer device with redundant computing resources and available according to the new feedback information.

[0025] The gateway device obtains an industrial computer device with excess computing resources and available through the computing resource management center, and maintains a connection with the industrial computer device during subsequent transmissions, and sends the subsequently received product images directly to the industrial computer device, ensuring a certain degree of loose coupling between the gateway device and the computing resource management center, so that during normal production, even if a brief abnormality occurs in the computing resource management center, it will not affect the normal production.

[0026] Furthermore, the gateway device is used to receive the product image, and send the product image to an industrial computer device in the resource pool that has redundant computing resources and is available through the computing resource management center, including:

[0027] The gateway device is used to receive the product image, generate a visual analysis request according to the product image, and send the product image and the visual analysis request to the computing resource management center, so that the computing resource management center receives the visual analysis request, and obtains an industrial computer device with excess computing resources and available from the resource pool according to the visual analysis request, generates feedback information, and sends the product image to the industrial computer device with excess computing resources and available, and sends the feedback information to the gateway device, so that the gateway device is connected to the corresponding industrial computer device according to the feedback information;

[0028] The gateway device is also used to determine whether the industrial computer device is abnormal according to the current load and health status of the industrial computer device. If so, disconnect the connection with the industrial computer device, and send the next product image to another industrial computer device with redundant computing resources and available in the resource pool through the computing resource management center when sending the product image next time, including:

[0029] The gateway device is also used to determine whether the industrial computer device is abnormal according to the current load condition and health condition of the industrial computer device;

[0030] If so, disconnect the connection with the industrial computer device, generate a switching request when sending the product image next time, and send the switching request to the computing resource management center, so that the computing resource management center receives the switching request, and re-acquires an industrial computer device with excess computing resources and available from the resource pool according to the switching request, generates new feedback information to the gateway device, and sends the product image to the industrial computer device with excess computing resources and available, and sends the new feedback information to the gateway device, so that the gateway device connects to the corresponding industrial computer device according to the new feedback information.

[0031] The gateway device sends the product image to the computing resource management center, which then sends the product image to an industrial computer device with excess computing resources and available in the resource pool. In this way, each product image can be sent to an industrial computer device with excess computing resources and available in the resource pool, and the industrial computer device performs real-time visual analysis to meet the production rhythm and real-time requirements of actual industrial production.

[0032] Furthermore, the industrial computer device is also used to generate a visual analysis abnormality result to the gateway device when there is an abnormality in the visual analysis of the product image;

[0033] The gateway device is also used to send a switching request to the computing resource management center when the product image fails to be sent to the industrial computer device or the visual analysis abnormal result is received, so that the next product image to be sent can be sent to another industrial computer device in the resource pool with excess computing resources and available through the computing resource management center.

[0034] When sending the product image to an industrial computer device with excess computing resources and available fails, the gateway device sends a switching request to the computing resource management center to switch to another industrial computer device with excess computing resources and available. Through this automatic switching mechanism, it can be ensured that when some industrial computer devices have abnormalities, the front-end gateway device can still find available industrial computer devices for connection to achieve visual analysis of product images, thereby ensuring that the production process is continuous from the perspective of the machine, and will not be interrupted due to the abnormality of a certain industrial computer device, ensuring the normal production rhythm and meeting the high stability requirements of industrial visual inspection.

[0035] Furthermore, the computing resource management center is used to manage the computing resources of several industrial computer devices in the resource pool, including:

[0036] The computing resource management center is used to obtain the heartbeat and load conditions of several industrial computer devices in the resource pool, and determine the computing resources of the industrial computer devices according to the heartbeat and load conditions of the industrial computer devices.

[0037] The computing resource management center obtains the heartbeat and load conditions of several industrial computer devices in the resource pool, and can determine the resources of the industrial computer devices in real time based on the heartbeat and load conditions of the industrial computer devices, so as to manage and allocate the computing resources of the resource pool.

[0038] Furthermore, the computing resource management center is also used to determine whether there is a new industrial computer device connected to it, and if so, add the new industrial computer device to the resource pool.

[0039] When it is determined that a new industrial computer device is connected to the computing resource management center, the new industrial computer device is added to the resource pool so that the computing resources of the resource pool can meet production requirements.

[0040] Furthermore, the industrial computer device is also used to detect its own operating status according to a preset period, generate a health status according to the operating status, and report the health status to the computing resource management center;

[0041] The industrial computer device is also used to automatically repair itself by taking corresponding repair actions according to the operating conditions when detecting that its own operating conditions are abnormal;

[0042] After the industrial computer device is connected to the gateway device, when it detects that its own operation is abnormal, it sends the detection result of the abnormality to the gateway device, so that the gateway device disconnects from the industrial computer device, and sends the next product image to another industrial computer device with excess computing resources and available in the resource pool through the computing resource management center when the product image is sent next time.

[0043] Industrial computer equipment regularly checks its own operating status, and when it detects that its own operating status is abnormal, it takes corresponding repair actions according to the operating status for automatic repair, so that it can return to normal in a timely and rapid manner, ensuring that the resource pool has enough industrial computer equipment in an available state to meet the computing requirements of production.

[0044] Furthermore, the computing resource management center is also used to determine whether the industrial computer device is abnormal according to the health status of the industrial computer device. If so, the abnormal industrial computer device is removed from the resource pool.

[0045] When it is determined that the industrial computer device is abnormal based on the health status of the industrial computer device, the abnormal industrial computer device is removed from the resource pool to ensure that the industrial computer devices in the resource pool are normally available as much as possible.

[0046] Furthermore, the industrial computer device is also used to detect its own operating status according to a preset period, including: the industrial computer device is also used to detect the current operating status of its own GPU and / or the current operating status of its own visual detection model according to a preset period, and the visual detection model is used to perform visual analysis on the product image.

[0047] The industrial computer equipment regularly checks its own operation status from both the hardware and software levels, and takes corresponding repair actions to repair it when an abnormality is detected, so that both the hardware and software levels of the industrial computer equipment are in normal status.

[0048] Furthermore, the industrial computer device is used to perform visual analysis on the product image, including:

[0049] The industrial computer device is used to call the visual inspection model to perform visual analysis on the product image.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention replaces the industrial computer equipment in the prior art by placing a gateway device at the location of each machine. Compared with the method of connecting the cloud to all sorting devices in the prior art, the method of connecting one gateway device to one sorting device is more stable. The gateway device sends the product image captured by the image acquisition device to an industrial computer device with excess computing resources and available in the resource pool through the computing resource management center. After the industrial computer device performs visual analysis on the product image, the gateway device sends the visual analysis result to the sorting device, so that the sorting device performs sorting according to the visual analysis result, thereby performing real-time visual analysis on each product image, meeting the production rhythm and real-time requirements of actual industrial production.

[0052] While feeding back the visual analysis results to the gateway device, the IPC device also sends its own load and health status to the gateway device. When the gateway device determines that the connected IPC device has an abnormality, it actively switches to another IPC device with excess computing resources and available, thereby ensuring that the production rhythm is not interrupted by the abnormality of the IPC device without increasing the communication cost.

[0053] Since the gateway device does not have a GPU, it only needs a relatively low-configuration CPU, memory, and hard disk configuration to meet its communication and storage functions. A resource pool is formed by using several consumer-grade industrial computers, and the resource pool shares computing resources between different machines to provide external visual analysis services, thereby improving the effective utilization rate of industrial computer equipment, reducing equipment procurement costs, and reducing the hardware cost of visual quality inspection.

[0054] When sending the product image to an industrial computer device with excess computing resources and available fails, the gateway device sends a switching request to the computing resource management center to switch to another industrial computer device with excess computing resources and available. Through this automatic switching mechanism, it can be ensured that when some industrial computer devices have abnormalities, they can automatically switch to normal and available industrial computer devices to perform visual analysis on product images. From the perspective of the machine, it is ensured that the production process is continuous and will not be interrupted due to the abnormality of a certain industrial computer device, thus meeting the high stability requirements of industrial visual inspection.

[0055] The industrial computer equipment regularly checks its own operating status, and when it detects that its own operating status is abnormal, it takes corresponding repair actions according to the operating status to restore it to normal in a timely and rapid manner, ensuring that the resource pool has enough industrial computer equipment in an available state to meet the computing requirements of production. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the modules of the industrial visual inspection system based on computing resource sharing described in the embodiment.

[0057] Figure numerals: image acquisition device 100, gateway device 200, computing resource management center 300, resource pool 400, industrial computer device 410, sorting device 500. DETAILED DESCRIPTION

[0058] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0059] In order to improve the speed of reasoning and meet the requirements of real-time control and stability on industrial production lines, the existing technology adopts a method of placing an industrial computer with a GPU on each machine, connecting the camera to the industrial computer to obtain the captured images, and using the industrial computer to call the quality inspection algorithm for analysis. According to the analysis results, the industrial computer is connected to the sorting device on the machine, and the sorting signal is directly sent to the sorting device to realize the control of the sorting device. Although this method can meet the requirements of real-time and stability, due to the high cost of GPU, equipping each machine with an industrial computer with a GPU will result in very high production costs, which is difficult to promote on a large scale in actual production.

[0060] In order to solve the above problem, in this embodiment, Figure 1 As shown, an industrial visual inspection system based on computing resource sharing is provided, comprising:

[0061] Several image acquisition devices 100, several gateway devices 200, a computing resource management center 300, a resource pool 400 and several sorting devices 500, one gateway device 200 is respectively connected to one or more image acquisition devices 100 and one sorting device 500, several gateway devices 200 are respectively connected to the computing resource management center 300, and the resource pool 400 includes several industrial computer devices 410;

[0062] The computing resource management center 300 is used to manage the computing resources of several industrial computer devices 410 in the resource pool 400;

[0063] The image acquisition device 100 is used to acquire product images and send the acquired product images to the gateway device 200;

[0064] The gateway device 200 is used to receive the product image, send the product image to an industrial computer device 410 with redundant computing resources and available in the resource pool 400 through the computing resource management center 300, and maintain a connection with the industrial computer device 410 to send subsequent product images directly to the industrial computer device 410;

[0065] The industrial computer device 410 is used to perform visual analysis on the product image, and send the visual analysis results and its current load and health status to the connected gateway device 200;

[0066] The gateway device 200 is also used to receive the visual analysis results and the current load and health status sent by the connected industrial computer device 410, send the visual analysis results to the sorting device 500, and judge whether the industrial computer device 410 is abnormal according to the current load and health status of the industrial computer device 410. If so, disconnect the connection with the industrial computer device 410, and send the next product image to another industrial computer device 410 with redundant computing resources and available in the resource pool 400 through the computing resource management center 300, and maintain connection with the other industrial computer device 410 to directly send subsequent product images to the other industrial computer device 410;

[0067] The sorting device 500 is used to sort the products according to the visual analysis results.

[0068] Specifically, a gateway device 200 is placed at the location of each machine. The gateway device 200 is only used to connect to external devices on the machine, such as the image acquisition device 100, the sorting device 500, the computing resource management center 300, the industrial computer device 410, etc. It can be understood that the gateway device 200 can be connected to the external devices on the machine by wireless or wired means. Since the gateway device 200 does not have a GPU, only a relatively low-configuration CPU, memory and hard disk configuration are required to meet its communication and storage functions, thereby reducing costs; all gateway devices 200 and resource pools 400 are connected to a computing resource management center 3 00, the resource pool 400 is composed of several industrial computer devices 410, that is, all gateway devices 200 and all industrial computer devices 410 are connected to a computing resource management center 300, and the computing resource center uniformly manages all industrial computer devices 410 in the resource pool 400, wherein the industrial computer devices 410 can be consumer-grade, and several consumer-grade industrial computer devices 410 are used to form the resource pool 400, and the resource pool 400 shares computing resources among different machines and provides visual analysis services to the outside world, which can improve the effective utilization rate of the industrial computer devices 410, reduce equipment procurement costs, and reduce the hardware cost of visual quality inspection.

[0069] Specifically, the image acquisition device 100 may be a camera. After acquiring the product image, the image acquisition device 100 sends the product image to the gateway device 200. The gateway device 200 sends the product image to an industrial computer device 410 with redundant computing resources in the resource pool 400 through the computing resource management center 300. After the industrial computer device 410 performs visual analysis on the product image, the industrial computer device 410 sends the visual analysis result and its own current load and health status to the gateway device 200. The gateway device 200 sends the visual analysis result to the sorting device 500, so that the sorting device 500 performs sorting according to the visual analysis result, so that each product can be sorted. Real-time visual analysis of product images is performed to meet the production rhythm and real-time requirements of actual industrial production; when the gateway device 200 determines that the connected industrial computer device 410 has an abnormality based on the load condition and health status of the industrial computer device 410, it actively switches to another industrial computer device 410 with excess computing resources and available, thereby ensuring that the production rhythm is not interrupted by the abnormality of the industrial computer device 410 without additional communication costs; it can be understood that for products that require multiple product images to cover, the gateway device 200 will merge the visual analysis results of multiple product images to form a visual analysis result of the entire product and then send it to the inspection device 500.

[0070] In one embodiment, the gateway device 200 is used to receive the product image, and send the product image to an industrial computer device 410 with redundant computing resources and available in the resource pool 400 through the computing resource management center 300, including:

[0071] The computing resource management center 300 receives the visual analysis request, obtains an industrial computer device 410 with redundant computing resources and available from the resource pool 400 according to the visual analysis request, and generates feedback information to the gateway device 200;

[0072] The gateway device 200 is also used to send the product image to the industrial computer device 410 with redundant computing resources and available according to the feedback information;

[0073] The gateway device 200 is also used to determine whether the industrial computer device 410 is abnormal according to the current load and health status of the industrial computer device 410. If so, the connection with the industrial computer device 410 is disconnected, and the next time the product image is sent, the product image to be sent is sent to another industrial computer device in the resource pool with redundant computing resources and available through the computing resource management center 300, including:

[0074] The gateway device 200 is also used to determine whether the industrial computer device 410 is abnormal according to the current load and health status of the industrial computer device 410;

[0075] If yes, disconnect the connection with the industrial computer device 410, generate a switching request when sending the product image next time, and send the switching request to the computing resource management center 300, so that the computing resource management center 300 receives the switching request, re-acquires an industrial computer device 410 with redundant computing resources and available from the resource pool 400 according to the switching request, and generates new feedback information to the gateway device 200;

[0076] The gateway device 200 is also used to send the product image to be sent next time to the industrial computer device 410 that has redundant computing resources and is available according to the new feedback information.

[0077] The gateway device 200 obtains an industrial computer device 410 with excess computing resources and available through the computing resource management center 300, and still maintains a connection with the industrial computer device 410 in subsequent transmissions, and directly sends the subsequently received product images to the industrial computer device 410, ensuring a certain degree of loose coupling between the gateway device 200 and the computing resource management center 300, so that during the normal production process, even if the computing resource management center 300 encounters a brief abnormality, it will not affect the normal progress of production.

[0078] In one embodiment, the gateway device 200 is used to receive the product image, and send the product image to an industrial computer device 410 with redundant computing resources and available in the resource pool 400 through the computing resource management center 300, including:

[0079] The gateway device 200 is used to receive a product image, generate a visual analysis request according to the product image, and send the product image and the visual analysis request to the computing resource management center 300, so that the computing resource management center 300 receives the visual analysis request, and obtains an industrial computer device 410 with excess computing resources and available from the resource pool 400 according to the visual analysis request, generates feedback information, and sends the product image to the industrial computer device 410 with excess computing resources and available, and sends the feedback information to the gateway device 200, so that the gateway device 200 is connected to the corresponding industrial computer device 410 according to the feedback information;

[0080] The gateway device 200 is also used to determine whether the industrial computer device 410 is abnormal according to the current load and health status of the industrial computer device 410. If so, the connection with the industrial computer device 410 is disconnected, and the next time the product image is sent, the product image to be sent next time is sent to another industrial computer device 410 with redundant computing resources and available in the resource pool 400 through the computing resource management center 300, including:

[0081] The gateway device 200 is also used to determine whether the industrial computer device 410 is abnormal according to the current load and health status of the industrial computer device 410;

[0082] If so, disconnect the connection with the industrial computer device 410, generate a switching request when sending the product image next time, and send the switching request to the computing resource management center 300, so that the computing resource management center 300 receives the switching request and re-acquires an industrial computer device 410 with excess computing resources and available from the resource pool 400 according to the switching request, generates new feedback information to the gateway device 200, and sends the product image to the industrial computer device 410 with excess computing resources and available, and sends the new feedback information to the gateway device 200, so that the gateway device 200 connects to the corresponding industrial computer device 410 according to the new feedback information.

[0083] The gateway device 200 sends the product image to the computing resource management center 300, and the computing resource management center 300 sends the product image to an industrial computer device 410 in the resource pool 400 that has excess computing resources and is available. In this way, each product image can be sent to the industrial computer device 410 in the resource pool 400 that has excess computing resources and is available, and the industrial computer device 410 performs real-time visual analysis to meet the production rhythm and real-time requirements of actual industrial production.

[0084] Specifically, when the image acquisition device 100 sends the product image to the gateway device 200, the gateway device 200 will obtain the industrial computer device 410 for visual analysis in one of the following two ways. The specific way to be adopted can be set by the configuration of the gateway device 200 and the computing resource management center 300 according to the actual situation. The first way is that after the gateway device 200 receives the product image, it generates a visual analysis request and directly sends the product image and the visual analysis request to the computing resource management center 300. The computing resource management center 300 sends the product image to one of the industrial computer devices 410 according to the computing resources of each industrial computer device 410. The gateway device 200 performs visual analysis on an industrial computer device 410 that has excess computing resources and is available. The second method is that after the gateway device 200 receives the product image, it generates a visual analysis request, sends the visual analysis request to the computing resource management center 300, and then obtains an industrial computer device 410 that has excess computing resources and is available through the computing resource management center 300, and then directly sends the product image to the industrial computer device 410 that has excess computing resources and is available. Regardless of which method is used, the gateway device 200 will automatically obtain an available industrial computer device 410 from the resource pool 400 through the computing resource management center 300 to perform visual analysis on the current product image.

[0085] In one embodiment, the industrial computer device 410 is further used to generate a visual analysis abnormality result to the gateway device 200 when there is an abnormality in the visual analysis of the product image;

[0086] The gateway device 200 is also used to send a switching request to the computing resource management center 300 when the product image fails to be sent to the industrial computer device 410 or an abnormal visual analysis result is received, so that the next product image to be sent can be sent through the computing resource management center 300 to another industrial computer device 410 in the resource pool 400 that has excess computing resources and is available.

[0087] When sending the product image to the industrial computer device 410 that has excess computing resources and is available fails, the gateway device 200 sends a switching request to the computing resource management center 300 to switch to another industrial computer device 410 that has excess computing resources and is available. Through this automatic switching mechanism, it can be ensured that when some industrial computer devices 410 have abnormalities, they can automatically switch to normal and available industrial computer devices 410 to perform visual analysis on the product image, thereby ensuring that the production process is continuous from the perspective of the machine, and will not cause generation interruptions due to the abnormality of a certain industrial computer device 410, thereby ensuring the normal production rhythm and meeting the high stability requirements of industrial visual inspection.

[0088] In one embodiment, the computing resource management center 300 is used to manage the computing resources of several industrial computer devices 410 in the resource pool 400, including:

[0089] The computing resource management center 300 is used to obtain the heartbeat and load conditions of several industrial computer devices 410 in the resource pool 400, and determine the computing resources of the industrial computer devices 410 according to the heartbeat and load conditions of the industrial computer devices 410.

[0090] In one embodiment, the computing resource management center 300 is also used to determine whether there is a new industrial computer device 410 connected to it, and if so, the new industrial computer device 410 is added to the resource pool 400.

[0091] Specifically, after receiving the visual analysis request sent by the gateway device 200, the computing resource management center 300 will obtain an industrial computer device 410 with excess computing resources and available from the resource pool 400 to perform visual analysis on the current product image. At the same time, the computing resource management center 300 will maintain a heartbeat connection with each industrial computer device 410 in the resource pool 400 and obtain the load status of each industrial computer device 410. Once an industrial computer device 410 is found to be abnormal, it will be removed from the resource pool 400. Once a new available industrial computer device 410 is connected to the computing resource management center 300, the computing resource management center 300 will be connected to the resource pool 400. When the industrial computer device 410 is connected to the computing resource management center 300, the new industrial computer device 410 will be automatically added to the resource pool 400. By obtaining the heartbeat and load conditions of several industrial computer devices 410 in the resource pool 400, the computing resources of the industrial computer device 410 can be determined in real time according to the heartbeat and load conditions of the industrial computer device 410, so as to manage and allocate the computing resources of the resource pool 400; when it is determined that a new industrial computer device 410 is connected to the computing resource management center 300, the new industrial computer device 410 will be added to the resource pool 400, so that the computing resources of the resource pool 400 can meet the production requirements.

[0092] In one embodiment, the industrial computer device 410 is also used to detect its own operating status according to a preset period, generate a health status according to the operating status, and report the health status to the computing resource management center 300;

[0093] The industrial computer device 410 is also used to automatically repair itself by taking corresponding repair actions according to the operation status when detecting that its own operation status is abnormal;

[0094] After the industrial computer device 410 is connected to the gateway device 200, when it detects that its own operation is abnormal, it sends the detection result to the gateway device 200, so that the gateway device 200 disconnects from the industrial computer device 410, and sends the next product image to another industrial computer device 410 in the resource pool 400 that has excess computing resources and is available through the computing resource management center 300 when the product image is sent next time.

[0095] The industrial computer device 410 regularly checks its own operating status, and when it detects that its own operating status is abnormal, it takes corresponding repair actions to automatically repair itself according to the operating status, so that it can return to normal in a timely and rapid manner, ensuring that the resource pool 400 has enough industrial computer devices 410 in an available state to meet the computing requirements of production.

[0096] In one embodiment, the computing resource management center 300 is also used to determine whether the industrial computer device 410 is abnormal based on the health status of the industrial computer device 410 . If so, the abnormal industrial computer device 410 is removed from the resource pool 400 .

[0097] When it is determined that the industrial computer device 410 is abnormal according to the health status of the industrial computer device 410 , the abnormal industrial computer device 410 is removed from the resource pool 400 to ensure that the industrial computer devices 410 in the resource pool 400 are normally available as much as possible.

[0098] In one embodiment, the industrial computer device 410 is also used to detect its own operating status according to a preset period, including: the industrial computer device 410 is also used to detect the current operating status of its own GPU and / or the current operating status of its own visual inspection model according to a preset period, and the visual inspection model is used to perform visual analysis on product images.

[0099] Specifically, each industrial computer device 410 in the resource pool 400 will check its own health status according to the set period. Once it is found to be in an unhealthy state, it will take corresponding repair actions according to different situations to maintain the availability of the industrial computer device 410 itself. These health checks include two levels. The first level is the hardware level: automatically check the current operating status of the GPU, including whether the GPU is currently online, whether it is available, whether the GPU load is reasonable, whether the GPU temperature exceeds the limit, etc.; once it is found that the GPU is offline and unavailable or the temperature exceeds the limit, the entire industrial computer device will be automatically shut down. The first level is the software level: it automatically checks the current operation of its own visual inspection model. Once an abnormality is found in the current operation, it immediately restarts the software to restore the industrial computer device 410 from the software and model levels. The industrial computer device 410 regularly checks its own operation from both the hardware and software levels, and uses corresponding repair actions to repair it when an abnormality is detected, so that both the hardware and software levels of the industrial computer device 410 are in normal condition.

[0100] In one embodiment, the industrial computer device 410 is used to perform visual analysis on the product image, including:

[0101] The industrial computer device 410 is used to call the visual inspection model to perform visual analysis on the product image. The visual inspection model includes a cropping algorithm and a deep learning model. When the industrial computer device 410 receives the product image to be analyzed, it automatically calls the cropping algorithm and the deep learning model to perform visual analysis on the product image, and after completing the visual analysis, directly sends the visual analysis result to the corresponding gateway device 200.

[0102] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. An industrial visual inspection system based on computing resource sharing, characterized in that: include: Several image acquisition devices, several gateway devices, a computing resource management center, a resource pool and several sorting devices, one of the gateway devices is respectively connected to one or more of the image acquisition devices and one of the sorting devices, several of the gateway devices are respectively connected to the computing resource management center, and the resource pool includes several industrial computer devices; The computing resource management center is used to manage the computing resources of several industrial computer devices in the resource pool; The image acquisition device is used to acquire product images and send the acquired product images to the gateway device; The gateway device is used to receive the product image, and send the product image to an industrial computer device with redundant computing resources and available in the resource pool through the computing resource management center, including: The gateway device is used to receive the product image, generate a visual analysis request according to the product image and send it to the computing resource management center, so that the computing resource management center receives the visual analysis request, obtains an industrial computer device with excess computing resources and available from the resource pool according to the visual analysis request, and generates feedback information to the gateway device; And maintain connection with the industrial computer device to directly send the subsequent product image to the industrial computer device; The industrial computer device is used to perform visual analysis on the product image, and send the visual analysis result as well as its current load condition and health condition to the connected gateway device; The gateway device is also used to receive the visual analysis result and the current load condition and health condition sent by the connected industrial computer device, send the visual analysis result to the sorting device, and judge whether the industrial computer device is abnormal according to the current load condition and health condition of the industrial computer device. If so, disconnect the connection with the industrial computer device, and send the next product image to another industrial computer device with redundant computing resources and available in the resource pool through the computing resource management center when sending the product image next time, including: The gateway device is also used to determine whether the industrial computer device is abnormal according to the current load condition and health condition of the industrial computer device; If so, disconnect the connection with the industrial computer device, generate a switching request when sending the product image next time, and send the switching request to the computing resource management center, so that the computing resource management center receives the switching request, re-acquires an industrial computer device with excess computing resources and available from the resource pool according to the switching request, and generates new feedback information to the gateway device; and maintaining connection with another industrial computer device to directly send subsequent product images to the other industrial computer device; The sorting device is used to sort the products according to the visual analysis results; The gateway device is also used to send the product image to the industrial computer device with excess computing resources and available according to the feedback information, or to send the product image to be sent next time to the industrial computer device with excess computing resources and available according to the new feedback information.

2. The industrial visual inspection system based on computing resource sharing according to claim 1, characterized in that: The industrial computer device is also used to generate a visual analysis abnormality result to the gateway device when there is an abnormality in the visual analysis of the product image; The gateway device is also used to send a switching request to the computing resource management center when the product image fails to be sent to the industrial computer device or the visual analysis abnormal result is received, so that the next product image to be sent can be sent to another industrial computer device in the resource pool with excess computing resources and available through the computing resource management center.

3. The industrial visual inspection system based on computing resource sharing according to claim 1, characterized in that: The computing resource management center is used to manage the computing resources of several industrial computer devices in the resource pool, including: The computing resource management center is used to obtain the heartbeat and load conditions of several industrial computer devices in the resource pool, and determine the computing resources of the industrial computer devices according to the heartbeat and load conditions of the industrial computer devices.

4. The industrial visual inspection system based on computing resource sharing according to claim 3 is characterized in that: The computing resource management center is also used to determine whether there is a new industrial computer device connected to it, and if so, add the new industrial computer device to the resource pool.

5. The industrial visual inspection system based on computing resource sharing according to claim 1, characterized in that: The industrial computer device is also used to detect its own operating status according to a preset period, generate a health status according to the operating status, and report the health status to the computing resource management center; The industrial computer device is also used to automatically repair itself by taking corresponding repair actions according to the operating conditions when detecting that its own operating conditions are abnormal; After the industrial computer device is connected to the gateway device, when it detects that its own operation is abnormal, it sends the detection result to the gateway device, so that the gateway device disconnects from the industrial computer device, and sends the next product image to another industrial computer device with excess computing resources and available in the resource pool through the computing resource management center when the product image is sent next time.

6. The industrial visual inspection system based on computing resource sharing according to claim 5, characterized in that: The computing resource management center is also used to determine whether the industrial computer device is abnormal according to the health status of the industrial computer device. If so, the abnormal industrial computer device is removed from the resource pool.

7. The industrial visual inspection system based on computing resource sharing according to claim 5, characterized in that: The industrial computer device is also used to detect its own operating status according to a preset period, including: the industrial computer device is also used to detect the current operating status of its own GPU and / or the current operating status of its own visual detection model according to a preset period, and the visual detection model is used to perform visual analysis on the product image.

8. The industrial visual inspection system based on computing resource sharing according to claim 1, characterized in that: The industrial computer device is used to perform visual analysis on the product image, including: The industrial computer device is used to call the visual inspection model to perform visual analysis on the product image.

Images