Image stitching method and device

Abstract

The present application provides a kind of image stitching method and device, including using multiple cameras to obtain the multiple first to-be-stitched images of calibration board and determine its position in calibration board design drawing, according to this, the multiple first to-be-stitched images are stitched, and the first position relationship of the shooting field of view of multiple cameras relative to operation platform is determined according to the stitching result of the multiple first to-be-stitched images;Multiple cameras are used to obtain multiple second to-be-stitched images of microelectronic components, and the second to-be-stitched images are stitched according to the stitching parameters, to obtain the complete image of microelectronic components, and the position of microelectronic components relative to operation platform is determined according to the first position relationship.The present application determines the stitching parameters of multiple cameras in advance by constructing calibration board, so that multiple partial images of microelectronic components can be directly stitched in the production process of microelectronic components according to the stitching parameters, and the position relationship between microelectronic components and operation platform is determined.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to an image stitching method and apparatus. Background Technology

[0002] Real-time monitoring of the chip component manufacturing process is crucial to ensuring product quality; however, conventional cameras cannot simultaneously guarantee both field of view and resolution when capturing images.

[0003] Specifically, in order to ensure the field of view for the production of microelectronic components, that is, to determine the position of the microelectronic components relative to the operating table in order to carry out production operations on the microelectronic components, a camera is usually used to take an image of the microelectronic components on the operating table. The image includes the complete microelectronic components and the operating table. However, in this way, the image of the microelectronic components occupies a small proportion of the captured image, and it is impossible to obtain high-resolution images of the microelectronic components.

[0004] If multiple high-precision cameras are used to capture partial images of microelectronic components, and then stitching methods widely used in the field of computer vision are used to stitch these partial images together to obtain a complete image of the microelectronic component, on the one hand, the stitching speed of current computer image stitching technology is difficult to meet the speed requirements of real-time monitoring of microelectronic component production; on the other hand, the stitched complete image of the microelectronic component is also difficult to reflect the relative positional relationship between the corresponding microelectronic component and the operating table, thus making it difficult to apply to the monitoring of microelectronic component production and processing.

[0005] Therefore, in the real-time monitoring of microelectronic component production, how to determine the relative position of the microelectronic component and the operating table while acquiring high-resolution images of the microelectronic component has become an urgent problem to be solved. Summary of the Invention

[0006] This invention provides an image stitching method and apparatus to address the shortcomings of existing technologies in ensuring both field of view and resolution during real-time monitoring of microelectronic components. It achieves a high-resolution image stitching method to simultaneously guarantee both field of view and resolution during real-time monitoring of microelectronic components.

[0007] This invention provides an image stitching method, comprising:

[0008] Multiple cameras are used to acquire multiple first images to be stitched on the calibration board, and the position of each first image to be stitched on the calibration board design drawing is determined. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. An ArUco encoding area is provided on the surface of the calibration board.

[0009] Based on the position of each first image to be stitched in the calibration board design drawing, the multiple first images to be stitched are stitched together, and the stitching parameters of the multiple cameras and the first positional relationship of the shooting field of view of the multiple cameras relative to the operating table are determined based on the stitching result of the multiple first images to be stitched together.

[0010] Multiple second images of microelectronic components are acquired using the multiple cameras. The second images are then stitched together according to the stitching parameters to obtain a complete image of the microelectronic components. The position of the microelectronic components relative to the operating table is determined based on the first positional relationship.

[0011] According to an image stitching method provided by the present invention, the step of stitching together multiple first images to be stitched according to their positions in the calibration board design drawing of each first image to be stitched specifically includes:

[0012] Determine the first position of each first image to be stitched in the calibration board design drawing, and cut out a number of cropped images of the corresponding area in the calibration board design drawing according to the first position, wherein the cropped images include at least the image corresponding to each first image to be stitched in the calibration board design drawing;

[0013] Each first image to be stitched is matched with the corresponding plurality of cropped images to determine the matching point of each first image to be stitched.

[0014] The position of each first image to be stitched is determined in the calibration board design based on the matching points.

[0015] According to an image stitching method provided by the present invention, the step of determining the first position of each first image to be stitched in the calibration board design drawing specifically includes:

[0016] The plurality of first images to be stitched together and the calibration board design drawing are binarized, and the coded image corresponding to the ArUco coding region is determined from the plurality of first images to be stitched together and the calibration board design drawing after binarization.

[0017] The encoded image is divided into blocks and the binary code corresponding to each block is determined. The similarity between each block of the encoded image of the multiple first images to be stitched and each block of the encoded image of the calibration board design is calculated based on the binary code.

[0018] The binary codes corresponding to a group of the first images to be stitched together with a similarity greater than a preset similarity threshold and the binary codes corresponding to the calibration board design are determined as valid codes, and the first position is determined based on the valid codes.

[0019] According to an image stitching method provided by the present invention, the step of matching each first image to be stitched with the corresponding plurality of cropped images to determine the matching point of each first image to be stitched specifically includes:

[0020] Each of the first images to be stitched together and the calibration board design drawing is binarized, and multiple feature points are determined in the binarized first images to be stitched together and the calibration board design drawing.

[0021] Calculate the Euclidean distance between each feature point in each of the first images to be stitched and each feature point in the calibration board design, and determine the feature point in each of the first images to be stitched corresponding to the minimum Euclidean distance as the pairing point;

[0022] Based on the random sampling consensus algorithm, a matching point is determined for each of the first images to be stitched together in the pairing points.

[0023] According to an image stitching method provided by the present invention, the step of determining multiple feature points in the first image to be stitched after binarization processing and the calibration board design drawing specifically includes:

[0024] Each pixel in the first image to be stitched and the calibration board design drawing after binarization is compared with the pixel value of the pixel within a preset radius around the pixel. If the pixel value of the pixel is greater than or less than the pixel value of the pixel within the preset radius around the pixel, the pixel is determined as the feature point.

[0025] According to an image stitching method provided by the present invention, the encoding in the ArUco encoding area has repeated encoding, and the calibration board surface is further provided with a character area on one side of the ArUco encoding area. The character area contains multiple non-repeating characters, and the multiple characters are arranged along the stitching direction of the first image to be stitched.

[0026] According to an image stitching method provided by the present invention, the calibration plate is further provided with a checkerboard area on one side of the ArUco encoding area, and before the step of stitching the second image to be stitched according to the stitching parameters, the method further includes:

[0027] The multiple cameras are used to acquire multiple third images of the checkerboard area of ​​the calibration board, and the stitching parameters are used to stitch the third images.

[0028] If the difference between the side length of each chessboard grid in the stitched image of the third image to be stitched and the side length of each chessboard grid in the calibration board design drawing is less than a preset threshold, the stitching parameters are determined to be the stitching parameters when stitching the second image to be stitched.

[0029] The present invention also provides an image stitching device, comprising:

[0030] The acquisition module is used to acquire multiple first images to be stitched using multiple cameras, and to determine the position of each first image to be stitched in the calibration board design drawing. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. An ArUco encoding area is provided on the surface of the calibration board.

[0031] The determining module is used to stitch the plurality of first images to be stitched together according to the position of each first image to be stitched in the calibration board design drawing, and to determine the stitching parameters of the plurality of cameras and the first positional relationship of the shooting field of view of the plurality of cameras relative to the operating table based on the stitching result of the plurality of first images to be stitched together.

[0032] The stitching module is used to acquire multiple second images of microelectronic components using the multiple cameras, stitch the second images of microelectronic components according to the stitching parameters to obtain a complete image of the microelectronic components, and determine the position of the microelectronic components relative to the operating table according to the first positional relationship.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image stitching method described above.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image stitching method as described above.

[0035] The image stitching method and apparatus provided by this invention, by designing and constructing a calibration board, determines the positions of multiple first images to be stitched in a single shot on the calibration board design drawing through registration, stitches the multiple images to be stitched, thereby obtaining stitching parameters for multiple cameras, and determines the first positional relationship between the areas corresponding to the fields of view of multiple cameras and the operation table based on the positional relationship between the calibration board and the operation table. Thus, when using multiple high-precision cameras to acquire second images to be stitched of high-resolution microelectronic components, it is possible to quickly stitch the second images to be stitched according to the predetermined stitching parameters and the first positional relationship, and determine the relative positional relationship between the electronic components and the operation table corresponding to the second images to be stitched. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is one of the flowcharts illustrating the image stitching method provided by the present invention;

[0038] Figure 2 This is a schematic diagram of the calibration plate structure in the image stitching method provided by the present invention;

[0039] Figure 3 This is one of the schematic diagrams of multiple first images to be stitched together in the image stitching method provided by the present invention;

[0040] Figure 4 This is a second schematic diagram of multiple first images to be stitched together in the image stitching method provided by the present invention;

[0041] Figure 5 This is a schematic diagram of each block in the image stitching method provided by the present invention;

[0042] Figure 6 This is a schematic diagram of the image stitching method provided by the present invention, mainly used to display pixels within a preset radius;

[0043] Figure 7 This is a schematic diagram of multiple third images to be stitched together in the image stitching method provided by the present invention;

[0044] Figure 8 This is a schematic diagram of multiple second images to be stitched together in the image stitching method provided by the present invention;

[0045] Figure 9 This is a schematic diagram of the stitching result of multiple second images to be stitched in the image stitching method provided by the present invention;

[0046] Figure 10 This is a schematic diagram of the image stitching device provided by the present invention;

[0047] Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] like Figure 1 As shown, the image stitching method of the present invention includes:

[0050] Step 101: Use multiple cameras to acquire multiple first images to be stitched on the calibration board, and determine the position of each first image to be stitched on the calibration board design drawing. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. The calibration board surface is provided with an ArUco encoding area.

[0051] First, the calibration board design is finalized, and then the calibration board is fabricated based on this design. The calibration board includes an encoding area containing ArUco codes. ArUco codes are square markers with a black background, created by combining a black border and a white pattern, resembling a QR code. The encoding area of ​​the calibration board consists of multiple ArUco codes. The encoding area of ​​the calibration board is shown below. Figure 2 The central region is shown.

[0052] Optionally, the ArUco encoding in the encoding area has no repeating pattern.

[0053] Based on this, before using multiple high-precision cameras to photograph microelectronic components, the stitching parameters of the multiple cameras must first be determined using a calibration board. Specifically, after the multiple cameras are fixedly installed at a determined shooting angle, the calibration board is placed on the operating table, ensuring that the position of the coding area of ​​the calibration board is within the shooting field of view of the multiple cameras. That is, in the photos taken by the multiple cameras used for real-time monitoring of microelectronic component production, each image contains a portion of the image of the coding area of ​​the calibration board, and in the multiple images taken by the multiple cameras at one time, each image has an overlapping portion.

[0054] Multiple high-resolution images of microelectronic components are captured by multiple cameras simultaneously at the same time. The resulting first set of images to be stitched together consists of multiple images captured synchronously by the same cameras. Figure 3 As shown.

[0055] Since there is no duplicate encoding in the encoding area, it is possible to determine which part of the calibration board design drawing each first image to be stitched corresponds to by registering a portion of the first image to be stitched with the encoded portion of the calibration board design drawing, which means determining the position of each first image to be stitched on the calibration board design drawing.

[0056] Step 102: Based on the position of each first image to be stitched in the calibration board design drawing, stitch multiple first images to be stitched together, and determine the stitching parameters of multiple cameras and the first positional relationship of the shooting field of multiple cameras relative to the operating table based on the stitching results of multiple first images to be stitched together.

[0057] For multiple first images to be stitched together obtained from a single shot, the multiple first images to be stitched together are stitched together according to the position of each first image to be stitched together in the calibration board design drawing, so that the complete image of the calibration board obtained after stitching the multiple first images to be stitched together is the same as the calibration board design drawing.

[0058] Based on this, the parameters used when stitching multiple first images to be stitched are recorded and used as stitching parameters for multiple cameras.

[0059] It is understandable that if the angles and positions of multiple cameras remain fixed, the field of view of each camera will not change. Therefore, the parameters used to record and stitch multiple images to be stitched are the stitching parameters corresponding to the field of view of multiple cameras in this case. No matter what kind of microelectronic components the multiple cameras are used to photograph, the same stitching parameters can be used to stitch the multiple images obtained by the multiple cameras simultaneously to obtain the correct stitched image.

[0060] Meanwhile, after stitching together multiple first images to be stitched to obtain a complete calibration board image, its relative position is compared with that of the calibration board design drawing. Combined with the position of the calibration board on the operating table when the first image to be stitched was taken, the relative position between the stitched calibration board image and the operating table can be calculated, which is the first positional relationship between the shooting fields of multiple cameras and the operating table.

[0061] When multiple cameras photograph microelectronic components in production, the positional relationship between the photographed microelectronic components and the operating table is the first positional relationship.

[0062] It should be noted that when the positions or angles of multiple cameras change, the camera fields of view change simultaneously, and the stitching parameters also change accordingly. In this case, a calibration board can be placed on the operating table, and multiple new first images to be stitched can be taken again. Based on the stitching results, new stitching parameters are determined, and a new first positional relationship is also determined for stitching images of microelectronic components.

[0063] Step 103: Use multiple cameras to acquire multiple second images of the microelectronic components to be stitched together, stitch the second images to be stitched together according to the stitching parameters to obtain a complete image of the microelectronic components, and determine the position of the microelectronic components relative to the operating table according to the first positional relationship.

[0064] After determining the splicing parameters and the first positional relationship, the calibration board can be removed, allowing the operating table to be used for the production of microelectronic components, and multiple cameras to be used to monitor the microelectronic components during the production process.

[0065] At this time, multiple cameras acquire multiple second images of microelectronic components to be stitched together. It can be understood that the multiple second images to be stitched together are multiple partial images of microelectronic components acquired by multiple cameras at the same time.

[0066] By directly stitching the obtained second stitched image using pre-determined stitching parameters of multiple cameras, a complete image of the microelectronic component can be obtained. At the same time, based on the pre-determined first positional relationship, the positional relationship between the microelectronic component and the operating table within the field of view of multiple cameras can also be determined.

[0067] This invention designs and constructs a calibration board, and through registration, determines the positions of multiple first images to be stitched in a single shot on the calibration board design drawing. The multiple images are then stitched together to obtain stitching parameters for multiple cameras. Furthermore, based on the positional relationship between the calibration board and the operating platform, a first positional relationship is determined between the areas corresponding to the fields of view of the multiple cameras and the operating platform. Therefore, when using multiple high-precision cameras to acquire second images of high-resolution microelectronic components, the invention enables rapid stitching of the second images based on pre-determined stitching parameters and the first positional relationship, and determines the relative positional relationship between the electronic components and the operating platform corresponding to the second images.

[0068] It is understandable that, since the splicing parameters and the first positional relationship used in the real-time monitoring of microelectronic component production in this application are predetermined by the calibration board, as long as the shooting fields of multiple cameras remain unchanged, the splicing of multiple second images to be spliced ​​can be completed quickly using the predetermined splicing parameters and the first positional relationship. There is no need to use computer matching technology to extract and match the obtained images while monitoring in real time, thereby meeting the requirements of real-time monitoring for image splicing speed.

[0069] In the image stitching method of the present invention, the encoding in the ArUco encoding area has repeated encoding, and a character area is also provided on one side of the calibration board surface on the ArUco encoding area. The character area contains multiple non-repeating characters, and the multiple characters are arranged along the stitching direction of the first image to be stitched.

[0070] Although using non-repeating ArUco encoding in the encoding area of ​​the calibration board during the design of the calibration board can ensure that when stitching multiple first images to be stitched together, each first image to be stitched can find a unique corresponding part in the calibration board design drawing, thereby determining the positional relationship between each first image to be stitched and the calibration board design drawing.

[0071] However, producing unique ArUco codes is difficult and costly. To further reduce the manufacturing cost of the calibration board, duplicate ArUco codes are included in the coding area of ​​the calibration board surface, such as... Figure 2 As shown, the central area of ​​the calibration board in this application is the encoding area, which contains three repeated ArUco codes, with the rightmost code being an incomplete code.

[0072] Based on this, when registering the first image to be stitched with the calibration board design drawing, a first image to be stitched can find multiple corresponding coding regions in the calibration board design drawing. In order to determine the true position of the first image to be stitched in the calibration board design drawing, a character area is also provided on one side of the coding area of ​​the calibration board in this application.

[0073] Optionally, the character area consists of multiple non-repeating characters or patterns, and the extension direction of the characters in the character area is parallel to the splicing direction of the images to be spliced.

[0074] In this application, such as Figure 2 As shown, the character area is decorated with white, non-repeating numbers and letters on a chessboard-like background.

[0075] Based on this, when acquiring the first image to be stitched, it is necessary to ensure that the character area and encoding area of ​​the calibration board are both within the field of view of multiple cameras during the placement of the calibration board. This ensures that each first image to be stitched contains an encoding area image for registration, and also contains a character area image to determine the actual registration area. Therefore, the resulting multiple first images to be stitched are as follows: Figure 4 As shown.

[0076] When registering each first image to be stitched with the calibration board design drawing, by comparing the image corresponding to the character area in the first image to be stitched with the image in the calibration board design drawing, the unique area corresponding to each first image to be stitched obtained in one shooting can be found in the calibration board design drawing, thereby determining the position of each image to be stitched in the calibration board design drawing.

[0077] The image stitching method of the present invention includes the step of stitching multiple first images to be stitched together according to their positions in the calibration board design drawing of each first image to be stitched, specifically comprising:

[0078] Determine the first position of each first image to be stitched in the calibration board design drawing, and cut out several cropped images of the corresponding area in the calibration board design drawing according to the first position. The cropped images include at least the image corresponding to each first image to be stitched in the calibration board design drawing.

[0079] The first position is the position of the image obtained after the initial matching of each first image to be stitched with the calibration board design drawing.

[0080] Optionally, each first image to be stitched and the calibration board design drawing are divided into multiple blocks of the same size. The similarity between each block of the first image to be stitched and each block of the calibration board design drawing is compared, and the blocks with a similarity greater than a preset value are determined as valid blocks. The valid blocks corresponding to each first image to be stitched are determined, and the position of the image corresponding to the valid block in the calibration board design drawing is taken as the first position.

[0081] Then, based on the first position, the corresponding area of ​​the image is cropped from the calibration board design drawing to obtain several cropped images.

[0082] Optionally, if there is no duplicate encoding in the encoding area of ​​the calibration board, multiple valid blocks corresponding to the first image to be stitched are concentrated in the same area. Based on this distribution area, the calibration board design drawing is cropped to obtain a cropped image, which includes all valid blocks in that area.

[0083] Optionally, if there is repetition in the coding of the calibration board coding area, multiple cropped images are determined for each first image to be stitched, wherein the number of cropped images is determined according to the number of repeated coding groups in the calibration board coding area.

[0084] Taking the calibration board provided in this application as an example, the effective blocks corresponding to each first image to be stitched will be concentrated in two or three positions of the calibration board design drawing. Two or three cropped images are obtained by cropping according to the distribution position of each effective block. Each cropped image includes all effective blocks at the concentrated distribution position.

[0085] It should be noted that if there is repetition in the encoding of the calibration board encoding area, each cropped image should also include the character area image above the distribution position of each effective block, so as to determine the actual position of the first image to be stitched in the calibration board design drawing based on the character area image during the subsequent registration process.

[0086] Match each first image to be stitched with several corresponding cropped images to determine the matching points of each first image to be stitched.

[0087] The position of each first image to be stitched in the calibration board design is determined based on the matching points.

[0088] Match each first image to be stitched obtained from a single capture with several corresponding cropped images.

[0089] It should be noted that when there is duplicate encoding in the encoding area of ​​the calibration board, the first image to be stitched and its corresponding multiple cropped images used for this matching all contain images of the character area of ​​the calibration board.

[0090] In one feasible implementation, feature points of multiple first images to be stitched together and the corresponding cropped images are determined.

[0091] Feature points are points in a corresponding image that are salient, unique, and distinguishable.

[0092] Optionally, the feature points include corner points in the first image to be stitched and the corresponding cropped image.

[0093] Optionally, edge detection technology is used to determine the segmentation edges in the local image of the calibration board and the cropped image corresponding to the local image of the calibration board, and the points located at the turning points of the segmentation edges are determined as feature points.

[0094] Optionally, the pixel value of each pixel in the first image to be stitched and the corresponding cropped image is compared with the pixel values ​​of pixels within a preset range around that pixel. Pixels whose pixel values ​​at any point are significantly different from the pixel values ​​of multiple pixels within the preset range around them are used as feature points.

[0095] Optionally, each feature point in the first image to be stitched is compared with multiple feature points in its corresponding cropped image, and the set of feature points with the highest similarity is used as the matching point.

[0096] This method determines the matching points of multiple first images to be stitched together, acquired in a single shooting.

[0097] It is important to note that the matching point corresponds to a feature point in the first image to be stitched together, and a feature point in the calibration board design drawing that has the highest similarity to it.

[0098] Therefore, based on the determined matching points, the matching points in all the first images to be stitched obtained in one shot are moved to the positions of the matching points in the corresponding calibration board design drawing, thus completing the stitching and stitching together all the partial images of the calibration board obtained in one shot into a complete image of the calibration board.

[0099] The stitching parameters are determined based on the coordinate relationship between the matching points of each first image to be stitched and the matching points of its corresponding calibration board design, and these parameters are used as the stitching parameters for multiple cameras.

[0100] The step of determining the first position of each first image to be stitched in the calibration board design drawing in the image stitching method of the present invention specifically includes:

[0101] Binarization processing is performed on multiple first images to be stitched together and calibration board design drawings. The coded image corresponding to the ArUco coding area is determined from the multiple first images to be stitched together and calibration board design drawings after binarization processing.

[0102] Binarization is performed on multiple first images to be stitched together and the calibration board design drawing to convert the ArUco codes in the first images to be stitched together and the ArUco codes in the coding area of ​​the calibration board design drawing into binary codes that can be used for positioning.

[0103] Specifically, by binarizing the above image, the image contains only "0" and "1", where "0" represents a black area and "1" represents a white area. The binarization formula is as follows:

[0104]

[0105] In the formula, n is the pixel value of each pixel, and t is the threshold. Since ArUco encoding is also a black-and-white encoding pattern, 128 is used as the threshold in this application.

[0106] Based on this, the Canny algorithm is used to detect edges in the image, and the detected edges are filtered and sorted to find all quadrilateral regions with similar side lengths. The detected edges are then compared with the data stored in the encoding library, and images belonging to the calibration board's encoding area are selected from the first image to be stitched together. The data stored in the encoding library is predetermined based on the encoding area images in the calibration board design drawing.

[0107] In this way, multiple first stitched images after binarization and the coded images corresponding to the coding areas in the calibration board design drawing are obtained.

[0108] The coded image is divided into blocks and the binary code corresponding to each block is determined. Based on the binary code, the similarity between each block of the first images to be stitched and each block of the calibration board design is calculated.

[0109] Furthermore, the coded image of the calibration board design drawing and the coded image in the first image to be stitched obtained through screening are divided into blocks. The number of blocks is determined according to the number of codes on the calibration board, ensuring that the number of black and white squares in each block is equal. The block division in this application is as follows: Figure 5 As shown, it is an 8x8 square.

[0110] After being divided into blocks, each block can be given a corresponding binary code. In this application, the corresponding 8*8 blocks result in a 64-bit binary code.

[0111] The similarity between each block of the encoded image of each first image to be stitched and each block of the encoded image of the calibration board design drawing is determined by comparing their binary codes using the following formula:

[0112]

[0113] In the formula, n represents the length of the binary code, which is 64 in this application; A represents the binary code vector of each block corresponding to the coded image of the calibration board design drawing; B represents the binary code vector of each block corresponding to the coded image in the first image to be stitched; and s represents the similarity.

[0114] The binary codes corresponding to the first set of images to be stitched together with a similarity greater than a preset similarity threshold and the binary codes corresponding to the calibration board design are determined as valid codes, and the first position is determined based on the valid codes.

[0115] A set of binary codes with a similarity greater than a preset similarity threshold is determined as a valid code. Each set of valid codes corresponds to the binary code of a first image to be stitched together and the binary code of a calibration board design.

[0116] Optionally, the preset similarity threshold is determined based on experience; in this application, the preset similarity threshold is 0.8.

[0117] All valid codes corresponding to the encoding of each first image to be stitched are determined, and the corresponding position in the calibration board design drawing is determined based on the valid codes. This position is used as the first position to achieve further registration between the first image to be stitched and the calibration board design drawing.

[0118] The image stitching method of this invention includes the step of matching each first image to be stitched with several corresponding cropped images to determine the matching points of each first image to be stitched.

[0119] Each first image to be stitched and the calibration board design drawing is binarized, and multiple feature points are determined in the binarized first image to be stitched and the calibration board design drawing.

[0120] It should be noted that, in one feasible implementation, the execution module for determining the cropped image is different from the execution module for determining the matching point. Therefore, when determining the matching point, it is still necessary to perform binarization processing on each first image to be stitched and the cropped image of the calibration board design drawing for subsequent calculations.

[0121] In other feasible implementations, if the cropping module and the matching point determination module use the same execution module, the binarized first image to be stitched and the cropped image can be directly obtained.

[0122] Multiple feature points were identified in the first binarized image to be stitched together and several corresponding binarized cropped images.

[0123] Feature points are pixels whose values ​​are significantly different from those of their surrounding pixels. Feature points are selected from the binarized first image to be stitched together and its corresponding binarized cropped image.

[0124] Calculate the Euclidean distance between each feature point in each first image to be stitched and each feature point in the calibration board design, and determine the feature point in each first image to be stitched corresponding to the minimum Euclidean distance as the pairing point;

[0125] For the multiple feature points obtained through screening, this application designs a novel binary-based feature descriptor based on the characteristics of binarized images. This descriptor is used to calculate the Euclidean distance between each feature point in the first image to be stitched and each feature point in the calibration board design, in order to screen and obtain matching points. The range is further narrowed down based on the feature points to determine the matching points used in actual matching.

[0126] Specifically, for each feature point, a local region with a side length of 2×R is extracted, called a block. Each feature point is located within its corresponding local region, and each feature point has the same relative position within the local region.

[0127] Optionally, R is determined based on the number of pixels contained in the first image to be stitched. In this application, R is set to 48, meaning the side length of each block is 96.

[0128] Furthermore, each block is divided into multiple units on an equal basis. Optionally, the length and width of each unit are determined according to the side length of each block. In this application, each block is divided into multiple square units with a side length of 6 on an equal basis.

[0129] By counting the occurrences of 1 and 0 within each cell, a two-dimensional feature vector can be obtained. For each block, the feature vectors within the 36 cells belonging to that block are counted, ultimately forming a 72-dimensional feature descriptor. This feature descriptor is used as the description vector for the feature points corresponding to that block.

[0130] Feature descriptors for all feature points in multiple first images to be stitched and the calibration board design drawing are constructed sequentially. Then, the Euclidean distance between the feature descriptor of each feature point in the first images to be stitched and the feature descriptor of each feature point in the calibration board design drawing is calculated sequentially using the following formula:

[0131]

[0132] In the formula, L(P) A P B ) represents P A Feature descriptors and P B The Euclidean distance of the feature descriptors, where P A P is a feature descriptor representing the feature points of the first image to be stitched together. A = (k1, k2, k3, ... k) n ), k nP represents the feature descriptor of the feature points in the first image to be stitched in the nth dimension; B It is the feature description set of feature points in the calibration board design drawing, P B = (j1, j2, j3, ... j n ), j n The feature descriptor represents a feature point in the nth dimension calibration board design drawing. In this application, n is 72.

[0133] Based on the above formula, calculate the Euclidean distance between each feature point in the calibration board design drawing and the feature descriptor corresponding to each feature point in the first image to be stitched, and take the pair of feature points with the smallest Euclidean distance as the pairing points.

[0134] Based on the random sampling consensus algorithm, a matching point is determined for each first image to be stitched in the pairing points.

[0135] It should be noted that the feature point with the minimum Euclidean distance can include one or more sets of feature points. For example, for each first image to be stitched and its corresponding cropped images, if there are multiple sets of feature points corresponding to the minimum Euclidean distance, all sets of feature points are determined as paired points; or, for another example, multiple sets of feature points whose difference from the minimum Euclidean distance is within a preset error range are determined as paired points, where the preset error range is determined based on experience.

[0136] Since matching based on Euclidean distance is a brute-force matching method, it is inevitable that incorrect matching points will be obtained during the matching process. Therefore, considering that there is only an affine transformation between the first image to be stitched obtained by taking pictures and the calibration board design, the Random Sample Consensus (RANSAC) algorithm is used based on the affine model to remove incorrect matching points, so as to select matching points for each first image to be stitched from the matching points.

[0137] Specifically, the paired points in each of the first images to be stitched are defined as the paired point set A, and the paired points in the cropped image corresponding to each of the first images to be stitched are defined as the paired point set B. A subset containing at least three pairs of corresponding points is randomly selected from paired point sets A and B. It can be understood that a pair of corresponding points represents the paired point in the first image to be stitched and the paired point in the cropped image with the smallest Euclidean distance to it. These paired points are represented as homogeneous coordinates and shown in matrix form as follows:

[0138]

[0139] Using the least squares method, T = BA(B T B) -1 Obtain the optimal solution and get the transformation matrix T between the first image to be stitched and the calibration board design.

[0140] It should be noted that for each initial image to be stitched, a coordinate system is established with the top-left corner of the image as the origin, the length direction of the image as the positive X-axis, and the width direction of the image as the positive Y-axis, to determine the coordinates of each feature point in the initial image to be stitched. For each cropped image, a coordinate system is established with the top-left corner of the cropped image as the origin, the length direction of the cropped image as the positive X-axis, and the width direction of the cropped image as the positive Y-axis, to determine the coordinates of each feature point in each cropped image.

[0141] Then, the transformation matrix T is used to calculate the paired point position B′ corresponding to point A according to the following formula:

[0142]

[0143] In the above formula, x a y a Let x and y represent the x and y coordinates of the paired point A in set A, respectively. b′ y b′ These represent the calculation point positions B′ corresponding to the paired points calculated based on the transformation matrix T. Here, the calculation point corresponding to A is the coordinate of point A in the calibration plate design drawing.

[0144] Based on this, the Euclidean distance d between the calculated point location B′ and the actual paired point B can be obtained. It can be understood that the actual paired point B is the corresponding point of point A in set B.

[0145]

[0146] x b x b′ Let y represent the x-coordinates of point B and point B′, respectively. b y b′ Let represent the ordinates of points B and B′, respectively. If the calculated d value is less than a set threshold, then the paired point A is determined as an interior point. The set threshold is determined empirically.

[0147] Repeat the above steps, each time randomly selecting three sets of points with the same name from set A and set B to calculate the transformation matrix T, and counting the number of interior points corresponding to each transformation matrix T.

[0148] The T with the most corresponding interior points is taken as the correct transformation matrix, and its corresponding paired point is taken as the matching point.

[0149] Specifically, during the stitching process, multiple matching points in each first image to be stitched are moved to the coordinates of multiple matching points in the corresponding calibration board design drawing, thereby enabling the stitching of multiple first images to be stitched from a single capture.

[0150] It is important to note that since the local coordinates of the first image to be stitched and the cropped image are used when calculating the matching point coordinates, the coordinates of multiple matching points in the calibration board design drawing, that is, the matching point coordinates of each cropped image, need to be normalized before stitching the images. This normalization should be made so that the coordinates of the calibration board design drawing are uniformly set in the calibration board design drawing coordinate system, with the upper left corner of the calibration board design drawing as the origin, the length direction of the calibration board design drawing as the positive X-axis, and the width direction of the calibration board design drawing as the positive Y-axis.

[0151] Specifically, the coordinates (x, y) of the pixel corresponding to the top left corner of each cropped image in the calibration board design coordinate system are determined. The coordinates of the matching point in the calibration board design coordinate system are obtained by adding x and y to the horizontal and vertical coordinates of the matching point in the cropped image, respectively, thus obtaining the actual set of matching points in the calibration board design.

[0152] Furthermore, based on set A and set... The transformation matrix T′ is calculated in the same way as the steps above. It can be understood that the transformation matrix T corresponds to the transformation matrix of each first image to be stitched and its corresponding cropped image, while the transformation matrix T′ corresponds to the transformation matrix of each first image to be stitched and the complete calibration board design. The transformation matrix T′ is determined as the stitching parameter of multiple cameras.

[0153] The transformation matrix T′ can be used to directly transform multiple first images to be stitched into the coordinate system of the calibration board design. After transforming all the first images to be stitched obtained in one shot into the coordinate system of the calibration board design, the stitching of multiple first images to be stitched is completed.

[0154] At the same time, by calculating the coordinates of the (0,0) point after transformation, the coordinates of the upper left pixel of the first images to be stitched together can be obtained in the coordinate system of the calibration board. Thus, based on the placement of the calibration board, its length and width, etc., the relative positions of the first images to be stitched together and the operating table are determined, which means the first positional relationship of the shooting fields of multiple cameras relative to the operating table is determined.

[0155] The image stitching method of this invention includes the step of determining multiple feature points in the first image to be stitched after binarization and the calibration board design drawing, specifically comprising:

[0156] Each pixel in the first image to be stitched and the calibration board design drawing after binarization is compared with the pixel value of the pixel corresponding to the pixel within a preset radius around the pixel. If the pixel value corresponding to the pixel is greater than or less than the pixel value corresponding to the pixel within the preset radius around the pixel, the pixel is determined as a feature point.

[0157] like Figure 6As shown, since the pixel value of each pixel in each image is 0 or 1 after binarization, for the first image to be stitched and the calibration board design drawing after binarization, i.e. the cropped image, for each pixel P to be determined, the pixel value of the point is compared with the pixel value of the surrounding pixels covered by a circle with a preset radius.

[0158] Optionally, the preset radius is determined based on the number of pixels. In this application, the preset radius is 4.

[0159] Point P is identified as a feature point if the pixel value of point P is greater than or less than the pixel values ​​of the 16 pixels covered by a circle with a radius of 4.

[0160] In the image stitching method of the present invention, a checkerboard area is further provided on one side of the ArUco encoding area of ​​the calibration plate. Before the step of stitching the second image to be stitched according to the stitching parameters, the method further includes:

[0161] Multiple cameras are used to acquire a third image of the checkerboard area of ​​the calibration board, and the third image is stitched together using stitching parameters.

[0162] After obtaining the transformation matrix T′, it can be used for image stitching. However, in order to further ensure the stitching accuracy, the accuracy of the transformation matrix T′ needs to be verified before actually using the transformation matrix T′ to stitch multiple second images of microelectronic components.

[0163] Specifically, such as Figure 2 As shown, the calibration board in this application also includes a black and white checkerboard area, which is an array of multiple square checkerboard squares of equal length and width.

[0164] The calibration board is repositioned so that its checkerboard area falls within the field of view of multiple cameras. Multiple images of the calibration board's checkerboard area are captured in a single shot to obtain the third image to be stitched together. Figure 7 As shown, the transformation matrix T′ is used as the stitching parameter to stitch together multiple third images to be stitched.

[0165] If the difference between the side length of each chessboard grid in the stitched image of the third image to be stitched and the side length of each chessboard grid in the calibration board design drawing is less than a preset threshold, the stitching parameters are determined to be the stitching parameters when stitching the second image to be stitched.

[0166] Since the checkerboard grid in the image formed by stitching together the calibration board design and multiple third images is a standard black and white grid, the side length of each individual grid in the image is counted after binarizing the calibration board design and the multiple third images to be stitched together, in order to represent the size of each individual grid.

[0167] If the difference between the side length of each cell in the completed stitching of multiple third images to be stitched and the side length of each cell in the calibration board design is less than the preset threshold t, then the stitching parameters are considered to meet the requirements of the actual application, and thus the stitching parameters are determined to be the stitching parameters used when stitching the second images to be stitched of microelectronic components.

[0168] In this application, the preset threshold t is 1.

[0169] Optionally, when comparing the side length of each cell in the stitched image with the side length of each cell in the calibration board design drawing, the absolute value of the difference is compared with a preset threshold t.

[0170] Based on this, when the splicing parameters in this application are actually used, the second images of multiple microelectronic components to be spliced ​​are obtained as follows: Figure 8 As shown, the image after stitching together multiple second images is as follows: Figure 9 As shown.

[0171] The image stitching apparatus provided by the present invention will be described below. The image stitching apparatus described below can be referred to in correspondence with the image stitching method described above. For example... Figure 10 As shown, the image stitching device includes an acquisition module 1001, a determination module 1002, and a stitching module 1003.

[0172] The acquisition module 1001 is used to acquire multiple first images to be stitched using multiple cameras, and to determine the position of each first image to be stitched in the calibration board design drawing. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. An ArUco encoding area is provided on the surface of the calibration board.

[0173] First, the calibration board design is finalized, and then the calibration board is fabricated based on this design. The calibration board includes an encoding area containing ArUco codes. ArUco codes are square markers with a black background, created by combining a black border and a white pattern, resembling a QR code. The encoding area of ​​the calibration board consists of multiple ArUco codes. The encoding area of ​​the calibration board is shown below. Figure 2 The central region is shown.

[0174] Optionally, the ArUco encoding in the encoding area has no repeating pattern.

[0175] Based on this, before using multiple high-precision cameras to photograph microelectronic components, the stitching parameters of the multiple cameras must first be determined using a calibration board. Specifically, after the multiple cameras are fixedly installed at a determined shooting angle, the calibration board is placed on the operating table, ensuring that the position of the coding area of ​​the calibration board is within the shooting field of view of the multiple cameras. That is, in the photos taken by the multiple cameras used for real-time monitoring of microelectronic component production, each image contains a portion of the image of the coding area of ​​the calibration board, and in the multiple images taken by the multiple cameras at one time, each image has an overlapping portion.

[0176] Multiple high-resolution images of microelectronic components are captured by multiple cameras simultaneously at the same time. The resulting first set of images to be stitched together consists of multiple images captured synchronously by the same cameras. Figure 3 As shown.

[0177] Since there is no duplicate encoding in the encoding area, it is possible to determine which part of the calibration board design drawing each first image to be stitched corresponds to by registering a portion of the first image to be stitched with the encoded portion of the calibration board design drawing, which means determining the position of each first image to be stitched on the calibration board design drawing.

[0178] The determining module 1002 is used to stitch multiple first images to be stitched together according to their positions in the calibration board design drawing of each first image to be stitched together, and to determine the stitching parameters of multiple cameras and the first positional relationship of the shooting fields of multiple cameras relative to the operating table based on the stitching results of the multiple first images to be stitched together.

[0179] For multiple first images to be stitched together obtained from a single shot, the multiple first images to be stitched together are stitched together according to the position of each first image to be stitched together in the calibration board design drawing, so that the complete image of the calibration board obtained after stitching the multiple first images to be stitched together is the same as the calibration board design drawing.

[0180] Based on this, the parameters used when stitching multiple first images to be stitched are recorded and used as stitching parameters for multiple cameras.

[0181] It is understandable that if the angles and positions of multiple cameras remain fixed, the field of view of each camera will not change. Therefore, the parameters used to record and stitch multiple images to be stitched are the stitching parameters corresponding to the field of view of multiple cameras in this case. No matter what kind of microelectronic components the multiple cameras are used to photograph, the same stitching parameters can be used to stitch the multiple images obtained by the multiple cameras simultaneously to obtain the correct stitched image.

[0182] Meanwhile, after stitching together multiple first images to be stitched to obtain a complete calibration board image, its relative position is compared with that of the calibration board design drawing. Combined with the position of the calibration board on the operating table when the first image to be stitched was taken, the relative position between the stitched calibration board image and the operating table can be calculated, which is the first positional relationship between the shooting fields of multiple cameras and the operating table.

[0183] When multiple cameras photograph microelectronic components in production, the positional relationship between the photographed microelectronic components and the operating table is the first positional relationship.

[0184] It should be noted that when the positions or angles of multiple cameras change, the camera fields of view change simultaneously, and the stitching parameters also change accordingly. In this case, a calibration board can be placed on the operating table, and multiple new first images to be stitched can be taken again. Based on the stitching results, new stitching parameters are determined, and a new first positional relationship is also determined for stitching images of microelectronic components.

[0185] The stitching module 1003 is used to acquire multiple second images of microelectronic components using multiple cameras, stitch the second images of microelectronic components according to stitching parameters to obtain a complete image of the microelectronic components, and determine the position of the microelectronic components relative to the operating table according to the first positional relationship.

[0186] After determining the splicing parameters and the first positional relationship, the calibration board can be removed, allowing the operating table to be used for the production of microelectronic components, and multiple cameras to be used to monitor the microelectronic components during the production process.

[0187] At this time, multiple cameras acquire multiple second images of microelectronic components to be stitched together. It can be understood that the multiple second images to be stitched together are multiple partial images of microelectronic components acquired by multiple cameras at the same time.

[0188] By directly stitching the obtained second stitched image using pre-determined stitching parameters of multiple cameras, a complete image of the microelectronic component can be obtained. At the same time, based on the pre-determined first positional relationship, the positional relationship between the microelectronic component and the operating table within the field of view of multiple cameras can also be determined.

[0189] This invention designs and constructs a calibration board, and through registration, determines the positions of multiple first images to be stitched in a single shot on the calibration board design drawing. The multiple images are then stitched together to obtain stitching parameters for multiple cameras. Furthermore, based on the positional relationship between the calibration board and the operating platform, a first positional relationship is determined between the areas corresponding to the fields of view of the multiple cameras and the operating platform. Therefore, when using multiple high-precision cameras to acquire second images of high-resolution microelectronic components, the invention enables rapid stitching of the second images based on pre-determined stitching parameters and the first positional relationship, and determines the relative positional relationship between the electronic components and the operating platform corresponding to the second images.

[0190] It is understandable that, since the splicing parameters and the first positional relationship used in the real-time monitoring of microelectronic component production in this application are predetermined by the calibration board, as long as the shooting fields of multiple cameras remain unchanged, the splicing of multiple second images to be spliced ​​can be completed quickly using the predetermined splicing parameters and the first positional relationship. There is no need to use computer matching technology to extract and match the obtained images while monitoring in real time, thus meeting the needs of real-time monitoring.

[0191] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other through the communications bus 1140. The processor 1110 can call logic instructions in the memory 1130 to execute an image stitching method. This method includes: acquiring multiple first images to be stitched using multiple cameras; determining the position of each first image to be stitched in a calibration board design drawing, wherein the calibration board is pre-constructed according to the calibration board design drawing and placed on an operating table, and an ArUco encoding area is provided on the calibration board surface; stitching the multiple first images to be stitched according to their positions in the calibration board design drawing, and determining the stitching parameters of the multiple cameras and a first positional relationship of the shooting fields of the multiple cameras relative to the operating table based on the stitching results of the multiple first images to be stitched; acquiring multiple second images to be stitched using the multiple cameras; stitching the second images to be stitched according to the stitching parameters to obtain a complete image of the microelectronic component, and determining the position of the microelectronic component relative to the operating table based on the first positional relationship.

[0192] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image stitching method provided by the above methods. The method includes: acquiring multiple first images to be stitched using multiple cameras, determining the position of each first image to be stitched in a calibration board design drawing, wherein the calibration board is pre-constructed according to the calibration board design drawing and placed on an operating table, and an ArUco encoding area is provided on the surface of the calibration board; stitching the multiple first images to be stitched according to the position of each first image to be stitched in the calibration board design drawing, and determining the stitching parameters of the multiple cameras and a first positional relationship of the shooting field of view of the multiple cameras relative to the operating table based on the stitching result of the multiple first images to be stitched; acquiring multiple second images to be stitched using the multiple cameras, stitching the second images to be stitched according to the stitching parameters to obtain a complete image of the microelectronic component, and determining the position of the microelectronic component relative to the operating table based on the first positional relationship.

[0194] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image stitching method, characterized in that, include: Multiple cameras are used to acquire multiple first images to be stitched on the calibration board, and the position of each first image to be stitched on the calibration board design drawing is determined. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. An ArUco encoding area is provided on the surface of the calibration board. Based on the position of each first image to be stitched in the calibration board design drawing, the multiple first images to be stitched are stitched together, and the stitching parameters of the multiple cameras and the first positional relationship of the shooting field of view of the multiple cameras relative to the operating table are determined based on the stitching result of the multiple first images to be stitched together. Multiple second images of the microelectronic components are acquired using the multiple cameras. The second images are then stitched together according to the stitching parameters to obtain a complete image of the microelectronic components. The position of the microelectronic components relative to the operating table is determined according to the first positional relationship. The step of stitching together the multiple first images to be stitched according to their positions in the calibration board design drawing specifically includes: Determine the first position of each first image to be stitched in the calibration board design drawing, and cut out a number of cropped images of the corresponding area in the calibration board design drawing according to the first position, wherein the cropped images include at least the image corresponding to each first image to be stitched in the calibration board design drawing; Each first image to be stitched is matched with the corresponding plurality of cropped images to determine the matching point of each first image to be stitched. The position of each first image to be stitched in the calibration board design is determined based on the matching points; The calibration board also has a checkerboard area on one side of the ArUco encoding area. Before the step of stitching the second image to be stitched according to the stitching parameters, the method further includes: The multiple cameras are used to acquire multiple third images of the checkerboard area of ​​the calibration board, and the stitching parameters are used to stitch the third images. If the difference between the side length of each chessboard grid in the stitched image of the third image to be stitched and the side length of each chessboard grid in the calibration board design drawing is less than a preset threshold, the stitching parameters are determined to be the stitching parameters when stitching the second image to be stitched.

2. The image stitching method according to claim 1, characterized in that, The step of determining the first position of each first image to be stitched in the calibration board design drawing specifically includes: The plurality of first images to be stitched together and the calibration board design drawing are binarized, and the coded image corresponding to the ArUco coding region is determined from the plurality of first images to be stitched together and the calibration board design drawing after binarization. The encoded image is divided into blocks and the binary code corresponding to each block is determined. The similarity between each block of the encoded image of the multiple first images to be stitched and each block of the encoded image of the calibration board design is calculated based on the binary code. The binary codes corresponding to a group of the first images to be stitched together with a similarity greater than a preset similarity threshold and the binary codes corresponding to the calibration board design are determined as valid codes, and the first position is determined based on the valid codes.

3. The image stitching method according to claim 1, characterized in that, The step of matching each first image to be stitched with its corresponding plurality of cropped images to determine the matching point of each first image to be stitched specifically includes: Each of the first images to be stitched together and the calibration board design drawing is binarized, and multiple feature points are determined in the binarized first images to be stitched together and the calibration board design drawing. Calculate the Euclidean distance between each feature point in each of the first images to be stitched and each feature point in the calibration board design, and determine the feature point in each of the first images to be stitched corresponding to the minimum Euclidean distance as the pairing point; Based on the random sampling consensus algorithm, a matching point is determined for each of the first images to be stitched together in the pairing points.

4. The image stitching method according to claim 3, characterized in that, The step of determining multiple feature points in the first image to be stitched after binarization and the calibration board design drawing specifically includes: Each pixel in the first image to be stitched and the calibration board design drawing after binarization is compared with the pixel value of the pixel within a preset radius around the pixel. If the pixel value of the pixel is greater than or less than the pixel value of the pixel within the preset radius around the pixel, the pixel is determined as the feature point.

5. The image stitching method according to any one of claims 1-4, characterized in that, The ArUco encoding area contains repeated codes. The calibration board surface is also provided with a character area on one side of the ArUco encoding area. The character area contains multiple non-repeating characters, and the multiple characters are arranged along the splicing direction of the first image to be spliced.

6. An image stitching apparatus, applied to the image stitching method as described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire multiple first images to be stitched using multiple cameras, and to determine the position of each first image to be stitched in the calibration board design drawing. The calibration board is pre-constructed according to the calibration board design drawing and placed on the operating table. An ArUco encoding area is provided on the surface of the calibration board. The determining module is used to stitch the plurality of first images to be stitched together according to the position of each first image to be stitched in the calibration board design drawing, and to determine the stitching parameters of the plurality of cameras and the first positional relationship of the shooting field of view of the plurality of cameras relative to the operating table based on the stitching result of the plurality of first images to be stitched together. The stitching module is used to acquire multiple second images of microelectronic components using the multiple cameras, stitch the second images of microelectronic components according to the stitching parameters to obtain a complete image of the microelectronic components, and determine the position of the microelectronic components relative to the operating table according to the first positional relationship. The acquisition module is specifically used to determine the first position of each first image to be stitched in the calibration board design drawing, and to cut out a number of cropped images of the corresponding area in the calibration board design drawing according to the first position, wherein the cropped images include at least the image corresponding to each first image to be stitched in the calibration board design drawing. Each first image to be stitched is matched with the corresponding plurality of cropped images to determine the matching point of each first image to be stitched. The position of each first image to be stitched in the calibration board design is determined based on the matching points; The stitching module is further configured such that, before the step of stitching the second image to be stitched according to the stitching parameters, a checkerboard area is provided on one side of the calibration board for the ArUco encoding area, the module further includes: The multiple cameras are used to acquire multiple third images of the checkerboard area of ​​the calibration board, and the stitching parameters are used to stitch the third images. If the difference between the side length of each chessboard grid in the stitched image of the third image to be stitched and the side length of each chessboard grid in the calibration board design drawing is less than a preset threshold, the stitching parameters are determined to be the stitching parameters when stitching the second image to be stitched.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image stitching method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image stitching method as described in any one of claims 1 to 5.

Images