Detection terminal, system and method based on Internet of Things intelligent industrial vision

Abstract

The invention provides a detection terminal, system and method based on Internet of Things intelligent industrial vision, and relates to the field of industrial vision detection. The detection terminal based on the Internet of Things intelligent industrial vision comprises an upper cover and a lower shell, the upper cover and the lower shell are matched with each other to form a shell of the detection terminal, a mounting cavity is formed in the lower shell, a limiting column is rotatably connected with an extension assembly, and two symmetrical adjusting grooves are formed in the side wall of the side, away from the lower shell, of the upper cover. Wherein a binocular camera is rotatably arranged in one adjusting groove, a thermal imaging camera is rotatably arranged in the other adjusting groove, and the binocular camera and the thermal imaging camera are connected through an angle adjusting assembly. The detection terminal has a very flexible adjustment function, and a more accurate feature point pairing relationship can be obtained by aggregating the relationship between the internal feature points of the image and the external feature points of the image through the graph neural network, so that a registration result with a better effect is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial vision detection, and specifically to a detection terminal, system and method based on intelligent industrial vision of the Internet of Things. Background Art

[0002] Products have gradually evolved from single large-scale production to diversified small-batch production, and product inspection has gradually evolved from manual experience selection to inspection using industrial vision systems. Visual inspection is the judgment of machines instead of human eyes. Visual inspection is to transmit the image information of the product to the imaging processing system through machine vision, and judge the image feature signal based on the image information to realize the industrial visual inspection of the product. The existing industrial visual inspection terminals are often fixed devices when in use, and do not have very flexible usage characteristics.

[0003] With the advancement of computer vision technology, the most important object detection technology in computer vision has also entered a new era. As the application field of object detection becomes increasingly widespread, it is difficult for images generated by a single camera to obtain sufficient information to support the completion of object detection tasks. At this time, it is necessary to introduce different devices of the same target to obtain images, which can not only provide rich information, but also overcome the defects of single-modality images themselves. In order to fuse heterogeneous image information, the first thing to do is image registration.

[0004] Image registration is a basic task in computer vision. Its main purpose is to fit the transformation parameters between two images, which reflect the mapping relationship between the images. The images used for registration usually come from cameras taken at different angles in the same scene. In the image registration task, heterogeneous image registration is a special case, where heterogeneity mainly refers to the fact that there are great modal differences between the registered images or they come from different sensors.

[0005] Homologous image registration tasks can achieve good results using artificially designed descriptors such as sift and its variants such as surf. However, due to the characteristics of heterologous images themselves, these traditional feature extraction methods are usually greatly challenged in heterologous image registration. The discriminability of the descriptor greatly affects the performance of registration. For heterologous images with significant appearance differences and low correlation, obtaining feature descriptors with sufficient discriminability is still a difficult problem that needs to be further solved. Compared with the difficulties of geometric deformation, illumination changes, etc. faced in traditional homologous image registration, heterologous image registration also has to solve the problem of significant appearance differences caused by sensors with different imaging mechanisms. Traditional registration methods are limited by artificially designed feature extraction methods, and often cannot obtain discriminative feature descriptors in the more difficult heterologous image registration. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a detection terminal, system and method based on the intelligent industrial vision of the Internet of Things, which solves the problems of inflexible use of existing industrial detection terminals and poor industrial vision detection image registration results.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a detection terminal based on the intelligent industrial vision of the Internet of Things, comprising an upper cover and a lower shell, the upper cover and the lower shell cooperate with each other to form the outer shell of the detection terminal, an installation cavity is arranged in the lower shell, a limit column is arranged in the installation cavity, the limit column is rotatably connected with an extension component, two symmetrical adjustment grooves are opened on the side wall of the upper cover away from the lower shell, a binocular camera is rotatably arranged in one of the adjustment grooves, and a thermal imaging camera is rotatably arranged in the other adjustment groove, and the binocular camera and the thermal imaging camera are connected through an angle adjustment component.

[0008] Preferably, the extension assembly includes a telescopic rod, a first stabilizing sleeve, and a second stabilizing sleeve. The first stabilizing sleeve and the second stabilizing sleeve are telescopically and slidably connected. One end of the telescopic rod is fixed to the bottom of the first stabilizing sleeve, and the other end of the telescopic rod is fixed to the inner wall of the upper cover. The first stabilizing sleeve and the limiting column are rotatably connected to each other, and the end of the second stabilizing sleeve away from the first stabilizing sleeve is fixedly connected to the upper cover.

[0009] Preferably, the outside of the first stabilizing sleeve is fixedly connected with a driven gear, the driven gear is meshingly connected with a driving gear, the driving gear is fixedly connected to the driving end of the first motor, and the first motor is arranged in the installation cavity.

[0010] Preferably, a mounting angle is provided on a side of the lower shell away from the upper cover, and a mounting hole is provided on the mounting angle.

[0011] Preferably, the angle adjustment assembly includes a second motor, a worm, a rotating shaft and a worm wheel, the second motor is arranged in the upper cover, the worm wheel and the worm are meshed with each other, the worm is fixedly connected to the driving end of the second motor, the worm wheel is fixedly connected to the rotating shaft, and the two ends of the rotating shaft are respectively connected to the binocular camera and the thermal imaging camera.

[0012] Preferably, a detection system is a detection system used for a detection terminal based on the intelligent industrial vision of the Internet of Things, comprising a main board, wherein the main board is arranged in an upper cover, and the main board is connected to a CPU, a GPU, an NPU, a memory module, a multimedia module, an interface, a binocular camera, and a thermal imaging camera through a control circuit.

[0013] Preferably, a detection method of a detection system used in a detection terminal based on the intelligent industrial vision of the Internet of Things comprises the following steps:

[0014] S1. Position adjustment: The extension component adjusts the upper cover to the detection height position, and the first motor rotates the upper cover to a certain angle through the driving gear and the driven gear. After that, the second motor drives the worm gear through the worm to drive the binocular camera and the thermal imaging camera to rotate to a certain angle. After that, the detection position is determined, the binocular camera detects the visible light heterogeneous image data, and the thermal imaging camera collects the corresponding original infrared thermal imaging data;

[0015] S2. Data processing: The detection system collects the corresponding raw infrared thermal imaging data and visible light heterogeneous image data, and processes the image data to form an unlabeled data set;

[0016] S3. Feature extraction:

[0017] S3.1. Extract feature points and corresponding descriptors from the input image through a reinforcement learning neural network, and the encoder of the feature point extraction network processes the input image to obtain feature point distribution;

[0018] S3.2. Output the initial descriptor matrix after multiple convolutions of the feature map output by the encoder, expand the initial descriptor matrix by double interpolation, and perform normalization to obtain the descriptor of each pixel, and jointly encode it with the descriptor corresponding to the feature point to obtain a descriptor of uniform length;

[0019] S4. Feature point registration: the feature points and corresponding descriptors obtained in step S3 are processed by a graph neural network, the registration matrix between heterogeneous images is obtained by a neural network algorithm, and the first-stage registration point pairs are output. The first-stage registration point pairs obtained are screened to obtain the common view area, and steps S3.1 and S3.2 are repeated for the common view area to obtain the second-stage registration point pairs;

[0020] S5. Image registration: the obtained first- and second-stage registration point pairs are used to calculate the sampling weights based on the confidence level, and the suboptimal transformation model is estimated based on the sampling weights to obtain the optimal transformation model, thus completing the image registration task, realizing visual detection and outputting the results for display in the multimedia module.

[0021] The present invention provides a detection terminal, system and method based on intelligent industrial vision of the Internet of Things.

[0022] It has the following beneficial effects:

[0023] 1. The present invention adjusts the upper cover to the detection height position through the extension component, the first motor rotates the upper cover to a certain angle through the driving gear and the driven gear, and the second motor drives the worm wheel through the worm to drive the binocular camera and the thermal imaging camera to rotate a certain angle through the rotating shaft. After that, the detection position is determined, so that the detection terminal has a very flexible adjustment function, which is suitable for industrial visual inspection of various categories of products. The binocular camera detects visible light heterogeneous image data, and the thermal imaging camera collects the corresponding original infrared thermal imaging data.

[0024] 2. The present invention uses a graph neural network to aggregate the relationship between internal feature points of the image and external feature points of the image to obtain a more accurate feature point pairing relationship, thereby obtaining a better registration result, achieving optimal registration of industrial visual inspection images and improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a front view of the detection terminal of the present invention;

[0026] Figure 2 is a side view of the detection terminal of the present invention;

[0027] Figure 3 For the present invention Figure 1 The enlarged view of point A in the middle;

[0028] Figure 4 It is a detection system diagram of the detection terminal of the present invention.

[0029] Among them, 1. upper cover; 2. lower shell; 3. installation angle; 4. installation cavity; 5. first motor; 6. driving gear; 7. limit column; 8. driven gear; 9. telescopic rod; 10. first stabilizing sleeve; 11. second stabilizing sleeve; 12. binocular camera; 13. main board; 14. adjustment slot; 15. rotating shaft; 16. worm; 17. second motor; 18. worm gear; 19. thermal imaging camera; 20. GPU; 21. NPU; 22. CPU; 23. memory module; 24. multimedia module; 25. interface. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Embodiment 1:

[0032] like Figure 1-4As shown, an embodiment of the present invention provides a detection terminal based on the intelligent industrial vision of the Internet of Things, including an upper cover 1 and a lower shell 2, the upper cover 1 and the lower shell 2 cooperate with each other to form the outer shell of the detection terminal, a mounting cavity 4 is arranged in the lower shell 2, a limiting column 7 is arranged in the mounting cavity 4, the limiting column 7 is rotatably connected with an extension component, and two symmetrical adjustment grooves 14 are opened on the side wall of the upper cover 1 away from the lower shell 2, a binocular camera 12 is rotatably arranged in one of the adjustment grooves 14, and a thermal imaging camera 19 is rotatably arranged in the other adjustment groove 14, the binocular camera 12 and the thermal imaging camera 19 are connected by an angle adjustment component, aiming at improving the quality and efficiency of operation and inspection, and taking improving the informatization, digitization and intelligence of operation and inspection as a way, and having the characteristics of panoramic state perception, scientific health diagnosis and intelligent operation and maintenance.

[0033] The extension assembly includes a telescopic rod 9, a first stabilizing sleeve 10, and a second stabilizing sleeve 11. The first stabilizing sleeve 10 and the second stabilizing sleeve 11 are telescopically and slidably connected. One end of the telescopic rod 9 is fixed to the bottom of the first stabilizing sleeve 10, and the other end of the telescopic rod 9 is fixed to the inner wall of the upper cover 1. The first stabilizing sleeve 10 and the limiting column 7 are rotatably connected to each other, and the end of the second stabilizing sleeve 11 away from the first stabilizing sleeve 10 is fixedly connected to the upper cover 1.

[0034] The outside of the first stabilizing sleeve 10 is fixedly connected with a driven gear 8 , which is meshedly connected with a driving gear 6 . The driving gear 6 is fixedly connected to a driving end of a first motor 5 , and the first motor 5 is disposed in the mounting cavity 4 .

[0035] A mounting corner 3 is provided on a side of the lower shell 2 away from the upper cover 1 , and a mounting hole is provided on the mounting corner 3 .

[0036] The angle adjustment assembly includes a second motor 17, a worm 16, a rotating shaft 15 and a worm wheel 18. The second motor 17 is arranged in the upper cover 1. The worm wheel 18 and the worm 16 are meshed with each other. The worm 16 is fixedly connected to the driving end of the second motor 17. The worm wheel 18 is fixedly connected to the rotating shaft 15. The two ends of the rotating shaft 15 are respectively connected to the binocular camera 12 and the thermal imaging camera 19.

[0037] Embodiment 2:

[0038] like Figure 1-2 4, an embodiment of the present invention provides a detection system, which is a detection system used by a detection terminal based on the intelligent industrial vision of the Internet of Things, including a main board 13, which is arranged in the upper cover 1, and the main board 13 is connected to the CPU 22, GPU 20, NPU 21, memory module 23, multimedia module 24, interface 24, binocular camera 12 and thermal imaging camera 19 through a control circuit;

[0039] The configurations of the detection system are as follows:

[0040]

[0041]

[0042]

[0043] Embodiment three:

[0044] like Figure 1-4 As shown, the embodiment of the present invention provides a detection method of a detection system used in a detection terminal based on IoT smart industrial vision according to claim 6, comprising the following steps:

[0045] S1. Position adjustment: The extension assembly adjusts the upper cover 1 to the detection height position, and the first motor 5 rotates the upper cover 1 to a certain angle through the driving gear 6 and the driven gear 8. After that, the second motor 17 drives the worm gear 18 through the worm 16 to make the rotating shaft 15 drive the binocular camera 12 and the thermal imaging camera 19 to rotate to a certain angle. After that, the detection position is determined, the binocular camera 12 detects the visible light heterogeneous image data, and the thermal imaging camera 19 collects the corresponding original infrared thermal imaging data;

[0046] S2. Data processing: The detection system collects the corresponding raw infrared thermal imaging data and visible light heterogeneous image data, and processes the image data to form an unlabeled data set;

[0047] S3. Feature extraction:

[0048] S3.1. Extract feature points and corresponding descriptors from the input image through a reinforcement learning neural network, and the encoder of the feature point extraction network processes the input image to obtain feature point distribution;

[0049] S3.2. Output the initial descriptor matrix after multiple convolutions of the feature map output by the encoder, expand the initial descriptor matrix by double interpolation, and perform normalization to obtain the descriptor of each pixel, and jointly encode it with the descriptor corresponding to the feature point to obtain a descriptor of uniform length;

[0050] S4. Feature point registration: the feature points and corresponding descriptors obtained in step S3 are processed by a graph neural network, the registration matrix between heterogeneous images is obtained by a neural network algorithm, and the first-stage registration point pairs are output. The first-stage registration point pairs obtained are screened to obtain the common view area, and steps S3.1 and S3.2 are repeated for the common view area to obtain the second-stage registration point pairs;

[0051] S5. Image registration: the obtained first- and second-stage registration point pairs are used to calculate the sampling weights based on the confidence level, and the suboptimal transformation model is estimated based on the sampling weights to obtain the optimal transformation model, thus completing the image registration task, realizing visual detection and outputting the results for display in the multimedia module.

[0052] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A detection terminal based on the intelligent industrial vision of the Internet of Things, comprising an upper cover (1) and a lower shell (2), characterized in that: The upper cover (1) and the lower shell (2) cooperate with each other to form the outer shell of the detection terminal. The lower shell (2) is provided with an installation cavity (4), and a limit column (7) is provided in the installation cavity (4). The limit column (7) is rotatably connected with an extension component. Two symmetrical adjustment grooves (14) are opened on the side wall of the upper cover (1) away from the lower shell (2), and a binocular camera (12) is rotatably provided in one of the adjustment grooves (14), and a thermal imaging camera (19) is rotatably provided in the other adjustment groove (14). The binocular camera (12) and the thermal imaging camera (19) are connected via an angle adjustment component.

2. According to claim 1, a detection terminal based on IoT intelligent industrial vision is characterized in that: The extension assembly comprises a telescopic rod (9), a first stabilizing sleeve (10), and a second stabilizing sleeve (11); the first stabilizing sleeve (10) and the second stabilizing sleeve (11) are telescopically and slidably connected; one end of the telescopic rod (9) is fixed to the bottom of the first stabilizing sleeve (10), and the other end of the telescopic rod (9) is fixed to the inner wall of the upper cover (1); the first stabilizing sleeve (10) and the limiting column (7) are rotatably connected to each other; and one end of the second stabilizing sleeve (11) away from the first stabilizing sleeve (10) is fixedly connected to the upper cover (1).

3. According to claim 2, a detection terminal based on IoT intelligent industrial vision is characterized in that: The first stabilizing sleeve (10) is fixedly connected to the outside with a driven gear (8), the driven gear (8) is meshingly connected with a driving gear (6), the driving gear (6) is fixedly connected to the driving end of a first motor (5), and the first motor (5) is arranged in the installation cavity (4).

4. The detection terminal based on the intelligent industrial vision of the Internet of Things according to claim 1 is characterized in that: A mounting corner (3) is provided on a side of the lower shell (2) away from the upper cover (1), and a mounting hole is provided on the mounting corner (3).

5. The detection terminal based on the intelligent industrial vision of the Internet of Things according to claim 1 is characterized in that: The angle adjustment assembly comprises a second motor (17), a worm (16), a rotating shaft (15) and a worm wheel (18); the second motor (17) is arranged in the upper cover (1); the worm wheel (18) and the worm (16) are meshed with each other; the worm (16) is fixedly connected to the driving end of the second motor (17); the worm wheel (18) is fixedly connected to the rotating shaft (15); and the two ends of the rotating shaft (15) are respectively connected to the binocular camera (12) and the thermal imaging camera (19).

6. A detection system, the detection system being a detection system used by the detection terminal based on the intelligent industrial vision of the Internet of Things according to any one of claims 1 to 5, comprising a main board (13), characterized in that: The main board (13) is arranged in the upper cover (1), and the main board (13) is connected to a CPU (22), a GPU (20), an NPU (21), a memory module (23), a multimedia module (24), an interface (24), a binocular camera (12), and a thermal imaging camera (19) through a control circuit.

7. A detection method for a detection system used in a detection terminal based on the intelligent industrial vision of the Internet of Things according to claim 6, characterized in that: The following steps are involved: S1. Position adjustment: the extension component adjusts the upper cover (1) to the detection height position, the first motor (5) rotates the upper cover (1) to a certain angle through the driving gear (6) and the driven gear (8), and then the second motor (17) drives the worm wheel (18) through the worm (16) to make the rotating shaft (15) drive the binocular camera (12) and the thermal imaging camera (19) to rotate to a certain angle, and then the detection position is determined, the binocular camera (12) detects visible light heterogeneous image data, and the thermal imaging camera (19) collects the corresponding original infrared thermal imaging data; S2. Data processing: The detection system collects the corresponding raw infrared thermal imaging data and visible light heterogeneous image data, and processes the image data to form an unlabeled data set; S3. Feature extraction: S3.1 extracts feature points and corresponding descriptors from the input image through a reinforcement learning neural network, and the encoder of the feature point extraction network processes the input image to obtain feature point distribution; S3.

2. Output the initial descriptor matrix after multiple convolutions of the feature map output by the encoder, expand the initial descriptor matrix by double interpolation, and perform normalization to obtain the descriptor of each pixel, and jointly encode it with the descriptor corresponding to the feature point to obtain a descriptor of uniform length; S4. Feature point registration: the feature points and corresponding descriptors obtained in step S3 are processed by a graph neural network, the registration matrix between heterogeneous images is obtained by a neural network algorithm, and the first-stage registration point pairs are output. The first-stage registration point pairs obtained are screened to obtain the common view area, and steps S3.1 and S3.2 are repeated for the common view area to obtain the second-stage registration point pairs; S5. Image registration: the obtained first- and second-stage registration point pairs are used to calculate the sampling weights based on the confidence level, and the suboptimal transformation model is estimated based on the sampling weights to obtain the optimal transformation model, thus completing the image registration task, realizing visual detection and outputting the results for display in the multimedia module.

Images