Multi-camera image acquisition method and device

Through multi-camera image acquisition methods and weighted fusion technology, the problem of incomplete image acquisition is solved, image accuracy and product detection accuracy are improved, and quality inspection efficiency and industrialization are improved.

CN113888408BActive Publication Date: 2024-12-03ZHEJIANG SCI-TECH UNIV +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111128099.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-26
Publication Date
2024-12-03
Estimated Expiration
2041-09-26

AI Technical Summary

Technical Problem

In the online detection process based on machine vision, the problem of incomplete image acquisition leads to low product detection accuracy, especially for objects to be tested with less feature information, the existing feature point matching methods are prone to errors.

Method used

By using multi-camera image acquisition method, by transmitting the object to be measured to the image acquisition area, using more than two cameras of equal height to acquire images, and pre-processing and stitching of the acquired images, including adaptive filtering, morphological processing and edges Detection, use the weighted fusion method to fuse the overlapping areas of the stitched image to ensure the integrity and accuracy of the image.

Benefits of technology

It improves the accuracy and integrity of the image of the object to be tested, reduces the error matching rate of feature points, improves the accuracy of product inspection and the work efficiency of enterprise quality inspection, and promotes the industrialization and the development of intelligent manufacturing in the field of quality inspection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113888408B_ABST
    Figure CN113888408B_ABST
Patent Text Reader

Abstract

The present invention provides a multi-camera image acquisition method and device, which are applied to the image acquisition link in the automatic detection of products. The image acquisition method obtains the image of the object to be tested with high precision, which effectively improves the accuracy of product detection. The multi-camera image acquisition method comprises: transferring the object to be tested to the image acquisition area; installing more than two cameras on the shooting frame to collect the image of the object to be tested; performing image preprocessing on the collected multiple images to be tested; splicing the images, and performing image fusion on the overlapping areas of the spliced ​​images to obtain the image of the object to be tested. The present invention uses multiple cameras to collect the image of the object to be tested, which improves the image precision of the object to be tested and ensures the integrity of the object to be tested.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image acquisition, and in particular to a multi-camera image acquisition method and device. Background Art

[0002] With the continuous development of computer technology, more and more companies are using online inspection based on machine vision to detect surface defects of products. The online inspection process based on machine vision uses a camera to obtain the image of the object to be tested, and then obtains the information of the object to be tested through image processing to determine the defects of the object to be tested. Image acquisition is a very important link in the entire quality inspection process. The higher the accuracy of the image acquisition of the object to be tested, the higher the accuracy of product detection.

[0003] When high-precision images are acquired, the problem of incomplete acquisition of the object to be tested may occur. Using multiple cameras to acquire images of the object to be tested, and then stitching the acquired images of the object to be tested, improves the image accuracy of the object to be tested and ensures the integrity of the object to be tested. At present, the commonly used image stitching method is to use image feature point matching for image stitching. The method based on feature point matching requires more features in the matching image. For objects to be tested with less feature information, feature point matching will result in errors. Summary of the invention

[0004] In view of the problems existing in the image acquisition link in the current enterprise quality inspection process, the present invention provides a multi-camera image acquisition method and device, which is applied to the image acquisition link in product automatic detection. The image of the object to be tested obtained by this image acquisition method has high precision and effectively improves the accuracy of product detection. The multi-camera image acquisition method includes: transferring the object to be tested to the image acquisition area; installing more than two cameras on the shooting frame to collect the image of the object to be tested; performing image preprocessing on the collected multiple images to be tested; splicing the images, and performing image fusion on the overlapping areas of the spliced ​​images to obtain the image of the object to be tested.

[0005] The technical solution adopted by the present invention is specifically as follows:

[0006] A multi-camera image acquisition method, comprising:

[0007] Transmitting the object to be measured to the image acquisition area;

[0008] The image of the object to be measured is collected by using two or more cameras located at the same height directly above the image collection area; the image collected by each camera is equal in size, and the object to be measured has a partially overlapping area in the images collected by adjacent cameras.

[0009] Perform image preprocessing on the collected images;

[0010] The images captured by adjacent cameras are stitched together in sequence, and the overlapping areas of the stitched images are fused to obtain a complete image of the object to be tested. The specific process of stitching the images captured by adjacent cameras in sequence is as follows: obtaining the intersection point set of the outline of the object to be tested and the edge of the image in each image, for two images i and j captured by adjacent cameras, searching for the corresponding point set located on the object to be tested in image j according to the height of the intersection point set in image i, obtaining the corresponding point set of the intersection point set in image i and image j according to the pixel trajectory distance between each point in the corresponding point set and the point with the highest proximity in the intersection point set in image j, obtaining the rotation matrix and translation matrix of the stitching according to the intersection point set and the corresponding point set, rotating and translating image j according to the rotation matrix and the translation matrix to complete the stitching of images i and j.

[0011] Furthermore, the collected images are preprocessed, including:

[0012] The image is processed by adaptive filtering to remove image noise and retain edge detail information;

[0013] Perform morphological processing on the image, including corrosion and expansion, and use the threshold-based image segmentation algorithm to extract the image of the object to be tested;

[0014] The edge detection algorithm is used to extract the edge of the object to be detected.

[0015] Furthermore, the first intersection point set includes at least the intersection points where the intersection heights of the contour of the object to be measured and the edge of the image are maximum and minimum.

[0016] Furthermore, image fusion is performed on the overlapping areas of the stitched images, specifically: the pixel values ​​of the overlapping areas of the stitched images are calculated using a weighted fusion method, and the weights of the pixels of the overlapping areas of the stitched images are determined according to the distances of the pixels of the overlapping areas from the image boundaries of the overlapping areas. The farther the pixels of the overlapping areas are from the image boundaries of the overlapping areas, the greater the weight of the image pixels, and vice versa. The pixel values ​​of the overlapping areas of the two images calculated by weights are added together to obtain the pixel values ​​of the stitched image, so that the transition at the seam of the stitched image is more natural, thereby achieving image fusion.

[0017] A multi-camera image acquisition device, comprising:

[0018] A transmission module, used for transmitting the object to be measured to the image acquisition area;

[0019] An image acquisition module, comprising two or more cameras located at the same height directly above the image acquisition area, for acquiring images of the object to be measured;

[0020] An image processing module, used for performing image preprocessing on the collected multiple images;

[0021] The image stitching module stitches the images captured by adjacent cameras in sequence, and performs image fusion on the overlapping areas of the stitched images to obtain the image of the object to be tested. The specific process of stitching the images captured by adjacent cameras in sequence is as follows: obtaining the intersection point set of the outline of the object to be tested and the edge of the image in each image, and for the two images i and j captured by adjacent cameras, searching for the corresponding point set located on the object to be tested in image j according to the height of the intersection point set in image i, obtaining the corresponding point set of the intersection point set in image i and image j according to the pixel trajectory distance between each point in the corresponding point set and the point with the highest proximity in the intersection point set in image j, and obtaining the rotation matrix and translation matrix of the stitching according to the intersection point set and the corresponding point set, and rotating and translating image j according to the rotation matrix and the translation matrix to complete the stitching of images i and j.

[0022] Furthermore, the image acquisition module also includes a shooting frame for fixing cameras, and the cameras are installed on the shooting frame at equal distances horizontally to ensure that there is a partial overlap between the images captured by adjacent cameras, and the cameras are at the same height from the object to be measured and perpendicular to the surface of the object to be measured.

[0023] Furthermore, a shading plate is installed outside the shooting frame, and an LED light source is installed inside the shading plate.

[0024] Furthermore, the image processing module performs image preprocessing on the collected multiple images to be tested by the following method, including:

[0025] Perform adaptive filtering on the image;

[0026] Perform morphological processing on the image, including corrosion and expansion, and use the threshold-based image segmentation algorithm to extract the image of the object to be tested;

[0027] The edge detection algorithm is used to extract the edge of the image of the object to be tested.

[0028] Furthermore, the first intersection point set includes at least the intersection points where the intersection heights of the contour of the object to be measured and the edge of the image are maximum and minimum.

[0029] Furthermore, in the image stitching module, the image fusion is specifically as follows: using a weighted fusion method to calculate the pixel value of the overlapping area of ​​the stitched image, determining the weight of the pixel of the overlapping area of ​​the stitched image according to the distance between the pixel of the overlapping area and the image boundary of the overlapping area, the farther the pixel of the overlapping area is from the image boundary of the overlapping area, the greater the weight of the image pixel, and vice versa. The pixel values ​​of the overlapping areas of the two images calculated by weights are added to obtain the pixel value of the stitched image, so that the transition at the seam of the stitched image is more natural, thereby achieving image fusion.

[0030] The beneficial effects of the method of the present invention are as follows: the present invention uses multiple cameras to collect images of objects to be tested, thereby improving the image accuracy of the objects to be tested and ensuring the integrity of the objects to be tested. The image edge stitching method is used to stitch the collected images, reduce the number of matching point sets, and improve the image matching speed. For objects to be tested with less feature information, the corresponding feature point set can be accurately determined, the feature point error matching rate can be reduced, and the image stitching can be made more accurate. Collecting high-precision images in the quality inspection process can improve the accuracy of product inspection and improve the work efficiency of enterprise quality inspection. Thereby improving the degree of industrialization in the field of quality inspection and promoting the development of intelligent manufacturing in enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings described herein are used to provide further understanding of the present specification and constitute a part of the present specification. The illustrative embodiments of the present specification and their descriptions are used to explain the present specification and do not constitute improper limitations on the present specification.

[0032] Figure 1 is a flow chart of the image acquisition method of the present invention;

[0033] Figure 2 A diagram of an image stitching method provided by an embodiment of the present invention;

[0034] Figure 3 An image fusion diagram provided by an embodiment of the present invention;

[0035] Figure 4 It is a structural schematic diagram of the image acquisition device of the present invention. DETAILED DESCRIPTION

[0036] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the technical solutions of this specification will be fully described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0037] Based on the recognition of the above problems, the embodiments of this specification provide a method and device for collecting multi-camera images in the process of clothing quality inspection. Figure 1 This is a flow chart of the multi-camera image acquisition method of the present invention, which specifically includes the following steps:

[0038] Step S100, transferring the garment to be tested to an image acquisition area.

[0039] Preferably, the garment to be tested is laid flat on a conveyor belt, and the conveyor belt conveys the garment to be tested to an image acquisition area, where the image acquisition area is an area where a camera shoots the object to be tested.

[0040] Step S200: two or more cameras are installed on a shooting frame to collect images of the garment to be tested.

[0041] Preferably, the shooting frame is constructed with a European standard 3030 aluminum profile mounting frame, with five shading plates installed on the outside to prevent uneven illumination of the image caused by interference from external light sources. The shading plates are equipped with LED light sources inside to ensure the same lighting environment when the image is shot. Three Mars4072S-24uc industrial cameras are installed on the shooting frame to collect images of the garment to be tested. The cameras are installed on the shooting frame at equal distances horizontally. The mounting frame where the cameras are located can adjust the height. The camera height is adjusted to ensure that there is a partial overlap between the images collected by adjacent cameras. The cameras are at the same height from the garment to be tested and are perpendicular to the surface of the garment to be tested. When the garment to be tested is transported to the shooting area, the cameras take a group of photos of the garment to be tested.

[0042] Step S300: performing image preprocessing on the collected multiple images to be tested.

[0043] Preferably, the method comprises: performing adaptive filtering processing on the image to remove image noise and retain edge detail information; performing morphological processing on the image to convert the image from RGB color space to HSV color space, so as to obtain the HSV color space distribution of the background of the clothing to be tested, separating the clothing image to be tested from the background, and then performing an opening operation on the clothing image to be tested, first corroding the clothing image to be tested to eliminate small noise points, and then performing an expansion processing to fill small holes in the image and small concave parts at the edge of the image, and finally using a threshold-based image segmentation algorithm to extract the image of the object to be tested; using the Canny operator to perform edge detection on the clothing image to be tested, so as to accurately extract the edge contour of the clothing to be tested.

[0044] Step S400, stitching the images, and fusing the overlapping areas of the stitched images to obtain an image of the garment to be tested.

[0045] Preferably, the image stitching is performed by stitching the image edges of the partially overlapping areas, specifically: Figure 2 As shown, the image captured by camera 1 is image 1, the image captured by camera 2 is image 2, and the image captured by camera 3 is image 3. The images captured by adjacent cameras are stitched together in sequence. Taking the stitching of image 1 and image 2 as an example, the details are as follows: Taking the lower left corner of the image as the coordinate origin, it is known that the highest intersection point A between the left edge of image 2 and the contour height of the garment to be measured and the highest intersection point B between the right edge of image 1 and the contour height of the garment to be measured, Figure 1According to the height of the intersection point A, i.e. the ordinate, the corresponding point A′ is found in image 2 as the intersection point with the contour of the garment to be tested. The pixel track distance d between the coordinates of the corresponding point and the point B with the largest ordinate in the intersection point of the right edge of image 1 and the contour of the object to be tested can be obtained. The pixel track distance d can be found in image 2 Figure 1 Similarly, given the intersection point C between the left edge of image 2 and the outline of the garment to be measured and the intersection point D between the right edge of image 1 and the outline of the garment to be measured, we can Figure 1 Search Figure 2 The corresponding point C′ of the point with the smallest ordinate in the intersection is found in image 2 Figure 1 The corresponding point D' of the minimum ordinate point in the intersection can be obtained through the corresponding four pairs of points A, B, C, D and A', B', C', D' in image 1 and image 2 to obtain the rotation matrix and translation matrix of the splicing. All pixel values ​​in image 2 are rotated and translated accordingly to achieve image splicing. The spliced ​​image and image 3 are spliced ​​and fused in the same way to obtain the complete garment image to be tested.

[0046] Furthermore, the overlapping areas of the stitched images are subjected to image fusion. The image fusion is to calculate the pixel values ​​of the overlapping areas of the stitched images using a weighted fusion method, and determine the weights of the pixels of the overlapping areas of the stitched images according to the distances between the pixels of the overlapping areas and the image boundaries of the overlapping areas. The farther the pixels of the overlapping areas are from the image boundaries of the overlapping areas, the greater the weight of the pixels, and vice versa. The pixel values ​​of the overlapping areas of the two images calculated by weights are added together to obtain the pixel values ​​of the stitched images, so that the transition at the junction of the stitched images is more natural, thereby achieving image fusion. Figure 3 As shown, a fused image of the garment to be tested is obtained. It can be seen that the method of the present invention has a very good splicing effect.

[0047] Figure 4 The invention discloses a multi-camera image acquisition device, comprising: a transmission module, used for transmitting a garment to be tested to an image acquisition area; an image acquisition module, comprising two or more cameras located at the same height just above the image acquisition area, used for acquiring images of the object to be tested; an image processing module, used for performing image preprocessing on the acquired multiple images of the garment to be tested; and an image splicing module, used for splicing the garment images to be tested according to the method of the invention, performing image fusion on the overlapping areas of the spliced ​​images, and obtaining a complete image of the garment to be tested.

[0048] Preferably, the image acquisition module also includes a shooting frame and a plurality of cameras arranged on the shooting frame. A shading plate is installed on the outside of the shooting frame, and an LED light source is installed inside the shading plate. The plurality of cameras capture images of the garment to be tested. The cameras are installed on the shooting frame at equal intervals horizontally to ensure that there is a partial overlapping area between the images captured by adjacent cameras. The cameras are at the same height from the garment to be tested and are perpendicular to the surface of the garment to be tested.

[0049] Preferably, the image processing module performs image preprocessing on the collected multiple clothing images to be tested by the following methods, including: performing adaptive filtering processing on the images; performing morphological processing on the images, including corrosion and expansion, and extracting the clothing images to be tested using a threshold-based image segmentation algorithm; and extracting the edges of the clothing images to be tested using an edge detection algorithm.

[0050] Preferably, the image fusion of the image stitching module is to calculate the pixel value of the overlapping area of ​​the stitched image by weighted fusion method, and the weight of the pixel of the overlapping area of ​​the stitched image is determined according to the distance between the pixel of the overlapping area and the image boundary of the overlapping area. The farther the pixel of the overlapping area is from the image boundary of the overlapping area, the greater the weight of the image pixel, and vice versa. The pixel values ​​of the overlapping areas of the two images calculated by weights are added to obtain the pixel value of the stitched image, so as to realize image fusion.

[0051] The multi-camera image acquisition method and device of the present invention utilize multiple cameras to acquire images of objects to be tested, thereby improving the image accuracy of the objects to be tested; an image edge stitching method is used to stitch the acquired images to ensure the integrity of the objects to be tested. Acquiring high-precision images in the quality inspection process can improve the accuracy of product inspection and improve the work efficiency of enterprise quality inspection. This will improve the degree of industrialization in the field of quality inspection and promote the development of intelligent manufacturing in enterprises.

[0052] The above description is only the solution of the present invention and is only used to explain the present invention and is not used to limit the scope of protection of the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-camera image acquisition method, characterized in that: include: Transmitting the object to be measured to the image acquisition area; The image of the object to be measured is collected by using two or more cameras located at the same height directly above the image collection area; the image collected by each camera is equal in size, and the object to be measured has a partially overlapping area in the images collected by adjacent cameras; Perform image preprocessing on the collected images; The images captured by adjacent cameras are stitched together in sequence, and image fusion is performed on the overlapping areas of the stitched images to obtain a complete image of the object to be measured, wherein the specific process of stitching the images captured by adjacent cameras in sequence is: obtaining the intersection point set of the contour of the object to be measured and the edge of the image in each image, for two images i and j captured by adjacent cameras, searching for a corresponding point set located on the object to be measured in image j according to the height of the intersection point set in image i, obtaining the corresponding point set of the intersection point set in image i and image j according to the pixel trajectory distance between each point in the corresponding point set and the point with a high proximity to the intersection point set in image j, calculating the rotation matrix and translation matrix of the stitching according to the intersection point set and the corresponding point set, rotating and translating image j according to the rotation matrix and the translation matrix to complete the stitching of images i and j.

2. The multi-camera image acquisition method according to claim 1, characterized in that: Perform image preprocessing on the acquired images, including: The image is processed by adaptive filtering to remove image noise and retain edge detail information; Perform morphological processing on the image, including corrosion and expansion, and use the threshold-based image segmentation algorithm to extract the image of the object to be tested; The edge detection algorithm is used to extract the edge of the object to be detected.

3. The multi-camera image acquisition method according to claim 1, characterized in that: The intersection point set includes at least the intersection points where the intersection heights of the contour of the object to be measured and the edge of the image are the largest and the smallest.

4. The multi-camera image acquisition method according to claim 1, characterized in that: Image fusion is performed on the overlapping areas of the stitched images. Specifically, the weighted fusion method is used to calculate the pixel values ​​of the overlapping areas of the stitched images, and the weights of the pixels in the overlapping areas of the stitched images are determined according to the distance between the pixels in the overlapping areas and the image boundary of the overlapping areas. The farther the pixels in the overlapping areas are from the image boundary of the overlapping areas, the greater the weight of the image pixels, and vice versa.

5. A multi-camera image acquisition device, characterized in that: include: A transmission module, used for transmitting the object to be measured to the image acquisition area; An image acquisition module, comprising two or more cameras located at the same height directly above the image acquisition area, for acquiring images of the object to be measured; An image processing module, used for performing image preprocessing on the collected multiple images; The image stitching module stitches the images captured by adjacent cameras in sequence, and performs image fusion on the overlapping areas of the stitched images to obtain the image of the object to be measured, wherein the specific process of stitching the images captured by adjacent cameras in sequence is: obtaining the intersection point set of the outline of the object to be measured and the edge of the image in each image, for two images i and j captured by adjacent cameras, searching for the corresponding point set located on the object to be measured in image j according to the height of the intersection point set in image i, obtaining the corresponding point set of the intersection point set in image i and image j according to the pixel trajectory distance between each point in the corresponding point set and the point with the highest proximity in the intersection point set in image j, calculating the rotation matrix and translation matrix of the stitching according to the intersection point set and the corresponding point set, rotating and translating image j according to the rotation matrix and the translation matrix to complete the stitching of images i and j.

6. The multi-camera image acquisition device according to claim 5, characterized in that: The image acquisition module also includes a shooting frame for fixing cameras, and the cameras are installed on the shooting frame at equal distances horizontally to ensure that there is a partial overlap between the images captured by adjacent cameras. The cameras are at the same height from the object to be measured and are perpendicular to the surface of the object to be measured.

7. The multi-camera image acquisition device according to claim 6, characterized in that: A shading plate is arranged outside the shooting frame, and an LED light source is arranged inside the shading plate.

8. The multi-camera image acquisition device according to claim 5, characterized in that: The image processing module performs image preprocessing on the collected multiple images to be tested by the following methods, including: Perform adaptive filtering on the image; Perform morphological processing on the image, including corrosion and expansion, and use the threshold-based image segmentation algorithm to extract the image of the object to be tested; The edge detection algorithm is used to extract the edge of the image of the object to be tested.

9. The multi-camera image acquisition device according to claim 5, characterized in that: The intersection point set includes at least the intersection points where the intersection heights of the contour of the object to be measured and the edge of the image are the largest and the smallest.

10. The multi-camera image acquisition device according to claim 5, characterized in that: In the image stitching module, the image fusion is specifically as follows: using a weighted fusion method to calculate the pixel value of the overlapping area of ​​the stitched image, and determining the weight of the pixel in the overlapping area of ​​the stitched image according to the distance between the pixel in the overlapping area and the image boundary of the overlapping area. The farther the pixel in the overlapping area is from the image boundary of the overlapping area, the greater the weight of the image pixel, and vice versa.

Citation Information

Patent Citations

  • Cigarette pack lamination layer number measurement method and device by multi-camera array

    CN105574845A

  • Object flatness measuring method and device, storage medium and electronic measuring equipment

    CN109584157A

  • Overhead line system panoramic image splicing method and system based on multiple cameras and equipment

    CN112365404A

  • Image splicing method, apparatus, terminal, and storage medium

    US20190251663A1