Image reading method and device based on image splicing technology and related assembly

Abstract

The invention discloses an image reading method and device based on an image splicing technology and a related component, and the method comprises the steps: carrying out the local image collection of a target object according to a preset index, and obtaining a local image sequence; acquiring splicing information of each local image in the local image sequence, and splicing and fusing all the local images into a global image according to the splicing information; constructing an image pyramid based on the global image; acquiring historical interaction data of a user, evaluating a reading probability of pixels in an adjacent range in the image pyramid according to the historical reading data, and calculating a prediction reading coefficient of an adjacent image layer of each image layer according to the reading probability; and pre-loading the image of each image layer in the image pyramid according to the predicted reading coefficient, and reading and rendering the pre-loaded image in response to an interaction instruction of a user. According to the invention, the image reading efficiency and precision can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image reading method, device and related components based on image stitching technology. Background Art

[0002] With the rapid development of semiconductor technology, the requirements for detection accuracy and efficiency of oversized objects such as wafers are getting higher and higher. When processing oversized images, traditional automatic optical inspection (AOI) systems are often limited by the field of view and processing capabilities of image acquisition equipment. Single image acquisition often cannot cover the entire inspection object, and multi-image stitching technology faces challenges such as low stitching accuracy and slow processing speed.

[0003] In existing AOI systems, the commonly used method involves dividing a large image into multiple small areas for separate inspection, and then performing complex image stitching processing. This method not only increases the complexity of data processing, but may also lead to error accumulation at the stitching edge, thus affecting the final inspection results. In addition, as the image size increases, the demand for storage and computing resources also increases significantly, which limits the application scope of the AOI system.

[0004] The advancement of image processing and computer vision technology has provided new possibilities for efficient processing of ultra-large images. However, how to apply these advanced technologies to the AOI field to achieve fast and accurate reading and processing of ultra-large images is a problem that technicians in this field need to solve. Summary of the invention

[0005] The embodiments of the present invention provide an image reading method, an apparatus, a computer device and a storage medium based on image stitching technology, aiming to improve the reading efficiency and accuracy of images.

[0006] In a first aspect, an embodiment of the present invention provides an image reading method based on image stitching technology, comprising:

[0007] Collect local images of the target object according to the preset index to obtain a local image sequence;

[0008] Acquire stitching information of each local image in the local image sequence, and stitch and fuse all the local images into a global image according to the stitching information;

[0009] An image pyramid is constructed based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer;

[0010] Acquire user historical interaction data, evaluate the read probability of adjacent pixels in the image pyramid according to the historical read data, and calculate the predicted read coefficient of adjacent layers of each layer according to the read probability;

[0011] The images of each layer in the image pyramid are preloaded according to the predicted reading coefficients, and the preloaded images are read and rendered in response to the user's interactive instructions.

[0012] In a second aspect, an embodiment of the present invention provides an image reading device based on image stitching technology, comprising:

[0013] An acquisition unit, used to acquire local images of the target object according to a preset index to obtain a local image sequence;

[0014] A splicing and fusing unit, used for acquiring splicing information of each local image in the local image sequence, and splicing and fusing all the local images into a global image according to the splicing information;

[0015] A construction unit, configured to construct an image pyramid based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer;

[0016] A prediction unit, used to obtain user historical interaction data, evaluate the read probability of adjacent range pixels in the image pyramid according to the historical read data, and calculate the predicted read coefficient of the adjacent layers of each layer according to the read probability;

[0017] The reading and rendering unit is used to preload the images of each layer in the image pyramid according to the predicted reading coefficients, and read and render the preloaded images in response to user interaction instructions.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image reading method based on image stitching technology as described in the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the image reading method based on the image stitching technology as described in the first aspect is implemented.

[0020] The embodiment of the present invention provides an image reading method, device, computer equipment and storage medium based on image stitching technology, the method comprising: collecting local images of a target object according to a preset index to obtain a local image sequence; obtaining stitching information of each local image in the local image sequence, and stitching and fusing all the local images into a global image according to the stitching information; constructing an image pyramid based on the global image; wherein the image pyramid includes a base layer, an intermediate layer and a top layer; obtaining user historical interaction data, evaluating the read probability of adjacent pixels in the image pyramid according to the historical reading data, and calculating the predicted read coefficient of the adjacent layers of each layer according to the reading probability; preloading the images of each layer in the image pyramid according to the predicted read coefficient, and reading and rendering the preloaded images in response to the user's interactive instructions. The embodiment of the present invention improves the reading speed and stitching accuracy of super-large images through innovative image stitching technology, while reducing the demand for storage and computing resources, and meets the demand for efficient and high-precision image detection in industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a flow chart of an image reading method based on image stitching technology provided by an embodiment of the present invention;

[0023] Figure 2 Another schematic diagram of a flow chart of an image reading method based on image stitching technology provided by an embodiment of the present invention;

[0024] Figure 3 A system block diagram of an image reading method based on image stitching technology provided by an embodiment of the present invention;

[0025] Figure 4 A schematic block diagram of an image reading device based on image stitching technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0028] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0029] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] See below Figure 1 , an embodiment of the present invention provides an image reading method based on image stitching technology, specifically comprising: steps S101 to S105.

[0031] Step S101, collecting local images of the target object according to a preset index to obtain a local image sequence;

[0032] Step S102, obtaining stitching information of each local image in the local image sequence, and stitching and fusing all the local images into a global image according to the stitching information;

[0033] Step S103: constructing an image pyramid based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer;

[0034] Step S104, obtaining user historical interaction data, evaluating the read probability of pixels in a neighboring range in the image pyramid according to the historical read data, and calculating the predicted read coefficient of the neighboring layers of each layer according to the read probability;

[0035] Step S105: preload the images of each layer in the image pyramid according to the predicted reading coefficients, and read and render the preloaded images in response to the user's interactive instructions.

[0036] In this embodiment, combined with Figure 2First, local images of the target object are collected according to the index to obtain multiple local images, and these local images can be combined into a local image sequence according to the index. Then, the image stitching technology is used to stitch and fuse the local images to obtain the global image corresponding to the target object, and then the image pyramid is constructed based on the global image. The reading probability of the neighboring pixels in the image pyramid is evaluated through the user's historical interaction data, and the predicted reading coefficient is further calculated based on the reading probability, so as to preload the image according to the predicted reading coefficient. Finally, based on the preloaded image, the corresponding reading and rendering are performed according to the user's interaction instructions.

[0037] This embodiment improves the reading speed and stitching accuracy of large images through innovative image stitching technology, while reducing the demand for storage and computing resources, meeting the demand for efficient and high-precision image detection in industrial production. In particular, the image reading method provided in this embodiment is particularly suitable for image reading scenarios of large-sized objects such as wafers.

[0038] In an actual application scenario, taking the wafer as an example, when performing local image acquisition of the target object according to the preset index, first prepare a CCD high-definition digital industrial camera for image acquisition, with an acquisition frame rate of 15fps and an acquisition resolution of 4K or 2K. Then place the wafer on the stage of the AOI equipment, determine the initial positioning point of the stage, and set the initial parameters to initialize the acquisition control instructions. Subsequently, image acquisition starts from the left side of the wafer and scans from top to bottom to generate the first column of image arrangement. When the up and down scans are completed, the wafer moves to the left and starts scanning from bottom to top to generate the second column of arrangement images. Whenever the up and down scans are completed, the wafer continues to move to the left and scans from top to bottom or from bottom to top in the same way until the entire wafer is scanned. In this way, the global wafer image is divided into a matrix distributed image sequence of M rows and N columns {I 11 ,…I 1n ,I 21 ,…,I mn}. Each local image I ij (i=1,…,mj=1,…,n) have the same size, with a width of W and a height of H, and are sorted from (1,1) to (M,N) in the acquisition order, numbered in a progressive naming manner of the coordinate array, and a total of M*N local images are acquired, and the wafer local images are saved.

[0039] In one embodiment, the step S102 includes:

[0040] Based on a preset horizontal scaling factor and a preset vertical scaling factor, down-sampling each of the local images using a nearest neighbor interpolation method to obtain a new local image sequence including a new local image;

[0041] Performing image registration processing on adjacent new local images in the new local image sequence;

[0042] Mapping the result of the image registration processing to the local image in the local image sequence, and obtaining the pixel offset parameter of the local image during the image registration processing, and then performing adaptive registration on the local image according to the pixel offset parameter;

[0043] According to the result of the adaptive registration, the overlapping area of ​​adjacent local images is obtained, and the weight of the overlapping area is assigned according to the number of pixels in the overlapping area, and then the local images after the weight assignment are spliced ​​to obtain a spliced ​​image;

[0044] A gradient operator is used to calculate the gradient of the overlapping area in the stitched image, and the gradient of the overlapping area is interpolated, and then a fusion process is performed in the gradient domain to obtain the global image.

[0045] In this embodiment, when the local images are stitched and fused, each local image I is first xy The nearest neighbor interpolation method is used to achieve downsampling. When selecting the new image I' xy When the value of each pixel in the original image I is directly taken xy The value of the nearest pixel in . Define the horizontal scaling factor S w and the vertical scaling factor S h After downsampling, a new set of reduced local image sequences {I′ 11 , ...I′ 1n , I′ 21 , ..., I′ mn}, where the width is W′=S w *W, height is H′=S h *H.

[0046] Then, the new partial images are subjected to image registration processing. Here, the image registration processing of adjacent new partial images in the new partial image sequence includes:

[0047] Determining whether the image illumination condition meets the preset illumination requirement according to the new partial image;

[0048] If it is determined that the illumination condition of the image meets the preset illumination requirement, an absolute error sum algorithm is used to perform image registration processing on the new partial image;

[0049] If it is determined that the image illumination condition does not meet the preset illumination requirement, a normalized product correlation algorithm is used to perform image registration processing on the new local image.

[0050] That is, when the image lighting conditions are good, the Sum of Absolute Differences (SAD) algorithm is used at the pixel level to achieve the new local image sequence {I′ 11 , ...I′ 1n , I′ 21 , ..., I′ mn}Image registration of adjacent images. The specific calculation formula is as follows:

[0051]

[0052] Among them, i, j represent the pixel coordinate position, image I′ xy is the image to be matched of size W′×H′, I′ pq is the desired matching image of size W′×H′, the goal is to be in image I′ xy Find the value corresponding to I′ pq The pixel area to be matched. xy Match the expected image I′ pq The similarity of image matching is defined as the cumulative sum of the absolute values ​​of the pixel gray value differences. The smaller the absolute difference SAD, the more similar it is. The downsampled image I′ is determined based on the similarity. xy and I′ pq The matching pixel range.

[0053] When the image illumination condition is unstable, the Normalized Cross Correlation (NCC) algorithm is used for image registration. The image matching similarity is the result of normalizing the cross correlation sum. The calculation formula is as follows:

[0054]

[0055] Among them, I′ xy (i, j) is the pixel value of the image to be matched, I′ pq (i, j) is the pixel value corresponding to the desired matching image, μ xy is the image I′ xy The average pixel value, μ pq is the image I′ pq The average pixel value, NCC(I′ xy , I′ pq ) has a value range of [-1, 1], where -1 means the two images are completely unrelated and 1 means the two images are completely related. The special property of the normalized correlation coefficient means that the matching results will not be affected by linear illumination changes.

[0056] After the above-mentioned image registration process, the obtained matching range result is mapped to the original image, and the pixel offset parameters of each match are counted. In this way, the matching parameters of the previous image can be referred to when registering images in the same column or row to achieve adaptive image registration. Specifically, the matching results of each matching offset parameter are counted. When performing a new round of matching, the image position number and the statistical results of the previous image matching in the same row or row are referred to, and the range with the highest probability of successful matching is preferentially selected for image registration. If the registration effect is not ideal, continue to adjust the matching range and record the pixel offset parameters for this time. In this way, through continuous image registration, the matching data of the pixel offset parameters can be accumulated, so that adaptive image registration can be achieved by counting the image matching results. When performing image registration of sequence images pairwise, estimate the geometric transformation parameters of each image and the four neighboring images above, below, left and right. In image I' xy Find the same as I' pq The matching pixel area selects n pairs of matching feature points and constructs the geometric transformation parameter matrix A. The specific affine transformation matrix formula is as follows:

[0057]

[0058]

[0059] Among them, (x, y) and (p, q) represent the image I' xy With image I' pq A pair of matching points in i Indicates changes such as image rotation and scaling, (t x ,t y ) represents the image translation change, and the affine geometric change matrix A is estimated using the least squares method.

[0060] After completing the adaptive registration of the images, the relative positions and stitching order of each local image are determined according to the acquisition order of each local image (i.e., the preset index), and the non-essential parts of the local image are cropped and the image geometry is changed according to the above adaptive registration results to achieve the effect of stitching preprocessing. Subsequently, the global coordinate position is mapped according to the coordinate position of the origin on the image after stitching preprocessing, and the coordinates of the overlapping parts are recorded to determine the overlapping area between the images and save the position of the stitching seam.

[0061] According to the determined overlapping area, the number of pixels in the overlapping area is obtained, and weights are assigned to the pixels in the first overlapping area image (i.e., one of the images corresponding to the overlapping area) and the pixels in the second overlapping area image (i.e., the other image corresponding to the overlapping area), and the weighted first overlapping area image and the weighted second overlapping area image are spliced ​​by weighted averaging, so that a fused image generated by two adjacent local images can be obtained.

[0062] Repeat the weighted stitching steps above until all local images are stitched together. At this point, you can get a preliminary global stitching fusion image to be optimized. Continue to find the stitching seam positions recorded in the above steps, calculate and match the image gradients in the overlapping area, and then fuse them in the gradient domain to maintain the continuity of the stitching edges. After completing the global optimization of the stitched image, you will get a complete image G stitched and fused by M*N local images. 0 .

[0063] In one embodiment, the step S103 includes:

[0064] Obtain a pre-established blank pyramid; wherein the blank pyramid includes a base layer, a middle layer and a top layer;

[0065] Using the global image as the image of the base layer in the blank pyramid;

[0066] Performing a first downsampling process on the global image to obtain an intermediate image, and using the intermediate image as an image of an intermediate layer in the blank pyramid;

[0067] A second downsampling process is performed on the intermediate image to obtain a top image, and the top image is used as the image of the top layer.

[0068] When constructing an image pyramid in this embodiment, a blank pyramid is first pre-built. The blank pyramid has three layers, including a top layer at the top of the pyramid, a middle layer of the pyramid, and a bottom layer at the bottom of the pyramid. 0 is the base image at the bottom of the blank pyramid. The global image is downsampled by 4 times to obtain the image G for the middle layer of the blank pyramid. 1 , and then downsample by 4 times to get the image G for the top layer of the blank pyramid 2 , thus constructing an image pyramid containing a base layer, an intermediate layer, and a top layer.

[0069] In one embodiment, the step S104 includes:

[0070] Acquire user historical interaction data, and perform statistics on the user historical interaction data to obtain a read probability of pixels in a neighboring range in the image pyramid;

[0071] The predicted reading coefficient is calculated according to the reading probability according to the following formula:

[0072]

[0073] Wherein, P([u, v]) represents the predicted reading coefficient, p([u, v]) represents the reading probability, and λ represents a custom penalty coefficient.

[0074] After the image pyramid is constructed, the image G is read at the current pyramid level. i When (i=0,1,2), for the pixels in the vicinity of the current layer image, firstly count the historical user interaction data (such as translation, reduction, enlargement, stay and other interaction operations), then refer to the statistical results to calculate the probability p([u,v]) of the pixels in the vicinity of the pixel (u,v) being read, and further calculate the predicted reading weight P([u,v]) according to the probability of being read. The calculation formula is as follows:

[0075]

[0076] Where λ is a custom penalty coefficient, which makes P([u,v]) satisfy that when P([u,v]) approaches 0, the degree is stronger, indicating that the probability of the pixel being read is lower, and when P([u,v]) approaches 1, the degree is stronger, indicating that the probability of the pixel being read is higher.

[0077] In one embodiment, the step S105 includes:

[0078] Based on the predicted reading coefficient, determining whether the probability of corresponding pixels and other image pyramid layers being read meets a preset reading requirement;

[0079] If it is determined that the probability of corresponding pixels and other image pyramid layers being read meets the preset reading requirements, the image of the corresponding range is preloaded;

[0080] If it is determined that the probability of corresponding pixels and other image pyramid layers being read does not meet the preset reading requirement, the preloading of the image in the corresponding range is cancelled.

[0081] Read image G at the current pyramid level iWhen (i=0,1,2), the predicted read weight P([u,v]) of the pixels in the vicinity of the current layer image is recorded. When P([u,v]) approaches 0, it means that the pixel and the corresponding positions of other pyramid layers are less likely to be read, so they do not need to be preloaded into the memory. When P([u,v]) approaches 1, it means that the pixel and the corresponding positions of other pyramid layers may be read, so they need to be preloaded into the memory.

[0082] In one embodiment, the step S105 further includes:

[0083] In response to a translation instruction from a user, updating the coordinates of the top layer of the image pyramid according to the coordinate information corresponding to the translation instruction, and displaying the top layer after the coordinates are updated;

[0084] In response to a zoom-in instruction from a user, updating the coordinates of a base layer of the image pyramid according to coordinate information corresponding to the zoom-in instruction, and displaying the base layer after the coordinates are updated;

[0085] In response to a zoom-out instruction from the user, coordinates of the middle layer and the top layer of the image pyramid are updated according to coordinate information corresponding to the zoom-out instruction, and the middle layer and the top layer after the updated coordinates are displayed.

[0086] When the user browses and interacts, the top-level image preview interface of the image pyramid is first displayed. When the user performs a pan operation, the preview image after the coordinates are updated is viewed; when the user zooms in, the predicted bottom-level image of the pyramid is read, the coordinate system position of the bottom-level image of the pyramid is updated, and the zoomed-in preview image is viewed; when the user zooms out, the predicted middle-level image and top-level image of the pyramid are read, the coordinate system position of the top-level image of the pyramid is updated, and the zoomed-out preview image is viewed. After completing all corresponding function operations, exit the corresponding display interface.

[0087] In a practical application scenario, Figure 3 As shown, Figure 3 This is a system block diagram of the image reading method based on image stitching technology provided in this embodiment. The system specifically includes an image acquisition submodule, an image fusion submodule, an image generation submodule, and an image browsing submodule. Among them,

[0088] The image acquisition submodule is used to acquire wafer surface images using a high-resolution industrial camera in a preset order to obtain an image sequence consisting of multiple ultra-clear images of wafer parts;

[0089] The image fusion submodule is used to perform image scaling, image registration, image stitching and image fusion operations on the images in the image sequence obtained by the image acquisition submodule, and to construct an image pyramid of the global image through the fused global image;

[0090] The image generation submodule is used to store the image pyramid obtained by the image fusion submodule and predict the reading probability of multi-level images based on the user's historical operations;

[0091] The image browsing submodule is used to preload the results of the image generation submodule into the memory, and read and render the wafer image pre-stored in the memory in the operation interface.

[0092] Compared with the prior art, the image reading method based on image stitching technology provided in this embodiment has the following advantages:

[0093] (1) On the one hand, by using a high-resolution industrial camera to capture the wafer surface image, the high definition of the image is ensured. On the other hand, the image is preloaded into the memory to achieve a delay-free and non-stuttering image browsing experience. Based on this, this embodiment can support real-time viewing of 4K high-precision wafer local images, thereby achieving the effect of browsing high-definition images of any local position of the wafer without delay or stuttering.

[0094] (2) The adaptive image registration method can adapt to different image acquisition schemes and equipment precision to ensure the accuracy of image stitching. In addition, this embodiment provides a variety of image similarity calculation methods and adaptive image registration methods to support the processing of images under different conditions, thereby ensuring the reliability of image registration.

[0095] (3) The “predicted reading coefficient” design is adopted to predict the reading statistical probability of multi-level images based on the user’s historical operations and image browsing mode, and calculate the predicted reading weight. This can stop the reading of unnecessary local images in advance and speed up the image pre-reading speed. This intelligent prediction mechanism can also reduce the waste of computing resources and improve the overall reading efficiency.

[0096] Figure 4 A schematic block diagram of an image reading device 400 based on image stitching technology provided in an embodiment of the present invention, the device 400 includes:

[0097] The acquisition unit 401 is used to acquire local images of the target object according to a preset index to obtain a local image sequence;

[0098] A stitching and fusing unit 402 is used to obtain stitching information of each local image in the local image sequence, and stitch and fuse all the local images into a global image according to the stitching information;

[0099] A construction unit 403 is used to construct an image pyramid based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer;

[0100] The prediction unit 404 is used to obtain user historical interaction data, evaluate the read probability of the adjacent range pixels in the image pyramid according to the historical read data, and calculate the predicted read coefficient of the adjacent layers of each layer according to the read probability;

[0101] The reading and rendering unit 405 is used to preload the images of each layer in the image pyramid according to the predicted reading coefficients, and read and render the preloaded images in response to the user's interactive instructions.

[0102] In one embodiment, the splicing and fusion unit 402 includes:

[0103] A downsampling unit, configured to perform downsampling processing on each of the local images by using a nearest neighbor interpolation method based on a preset horizontal scaling factor and a vertical scaling factor, so as to obtain a new local image sequence including a new local image;

[0104] A first registration unit, configured to perform image registration processing on adjacent new local images in the new local image sequence;

[0105] A second registration unit is used to map the result of the image registration processing to the local image in the local image sequence, obtain the pixel offset parameter of the local image during the image registration processing, and then perform adaptive registration on the local image according to the pixel offset parameter;

[0106] A weighted stitching unit, used for obtaining the overlapping area of ​​adjacent local images according to the result of the adaptive registration, assigning weights to the overlapping area according to the number of pixels in the overlapping area, and then stitching the local images after the weight assignment to obtain a stitched image;

[0107] The gradient fusion unit is used to calculate the gradient of the overlapping area in the stitched image by using a gradient operator, perform interpolation processing on the gradient of the overlapping area, and then perform fusion processing in the gradient domain to obtain the global image.

[0108] In one embodiment, the first registration unit comprises:

[0109] An illumination judgment unit, used for judging whether the illumination condition of the image meets the preset illumination requirement according to the new partial image;

[0110] A first determination unit, configured to perform image registration processing on the new partial image by using an absolute error sum algorithm if it is determined that the image illumination condition meets the preset illumination requirement;

[0111] The second determination unit is configured to perform image registration processing on the new partial image by using a normalized product correlation algorithm if it is determined that the image illumination condition does not meet the preset illumination requirement.

[0112] In one embodiment, the construction unit 403 includes:

[0113] A blank acquisition unit, used to acquire a pre-established blank pyramid; wherein the blank pyramid includes a base layer, a middle layer and a top layer;

[0114] A base setting unit, used for using the global image as an image of a base layer in the blank pyramid;

[0115] An intermediate setting unit, used for performing a first down-sampling process on the global image to obtain an intermediate image, and using the intermediate image as an image of an intermediate layer in the blank pyramid;

[0116] The top setting unit is used to perform a second downsampling process on the intermediate image to obtain a top image, and use the top image as the image of the top layer.

[0117] In one embodiment, the prediction unit 404 includes:

[0118] A history statistics unit, used to obtain user history interaction data, and perform statistics on the user history interaction data to obtain a read probability of pixels in a neighboring range in the image pyramid;

[0119] A coefficient calculation unit is used to calculate the predicted reading coefficient according to the reading probability according to the following formula:

[0120]

[0121] Wherein, P([u, v]) represents the predicted reading coefficient, p([u, v]) represents the reading probability, and λ represents a custom penalty coefficient.

[0122] In one embodiment, the reading and rendering unit 405 includes:

[0123] A reading judgment unit, used for judging whether the probability of corresponding pixels and other layers of the image pyramid being read meets a preset reading requirement based on the predicted reading coefficient;

[0124] A third determination unit is used to preload the image in the corresponding range if it is determined that the probability of corresponding pixels and other layers of the image pyramid being read meets the preset reading requirement;

[0125] The fourth determination unit is configured to cancel preloading of the image in the corresponding range if it is determined that the probability of the corresponding pixel and other layers of the image pyramid being read does not meet the preset reading requirement.

[0126] In one embodiment, the reading and rendering unit 405 further includes:

[0127] a first response unit, configured to respond to a translation instruction of a user, update the coordinates of the top layer of the image pyramid according to the coordinate information corresponding to the translation instruction, and display the top layer after the coordinates are updated;

[0128] a second response unit, configured to respond to a zoom-in instruction of the user, update the coordinates of the base layer of the image pyramid according to the coordinate information corresponding to the zoom-in instruction, and display the base layer after the coordinates are updated;

[0129] The third responding unit is used to respond to a zoom-out instruction from the user, update the coordinates of the middle layer and the top layer of the image pyramid according to the coordinate information corresponding to the zoom-out instruction, and display the middle layer and the top layer after the coordinates are updated.

[0130] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0131] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0132] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0133] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0134] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. An image reading method based on image stitching technology, characterized in that: include: Collect local images of the target object according to the preset index to obtain a local image sequence; Acquire stitching information of each local image in the local image sequence, and stitch and fuse all the local images into a global image according to the stitching information; An image pyramid is constructed based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer; Acquire user historical interaction data, evaluate the read probability of adjacent pixels in the image pyramid according to the historical read data, and calculate the predicted read coefficient of adjacent layers of each layer according to the read probability; The images of each layer in the image pyramid are preloaded according to the predicted reading coefficients, and the preloaded images are read and rendered in response to the user's interactive instructions.

2. The image reading method based on image stitching technology according to claim 1, characterized in that: The acquiring the stitching information of each local image in the local image sequence, and stitching and fusing all the local images into a global image according to the stitching information, comprises: Based on a preset horizontal scaling factor and a preset vertical scaling factor, down-sampling each of the local images using a nearest neighbor interpolation method to obtain a new local image sequence including a new local image; Performing image registration processing on adjacent new local images in the new local image sequence; Mapping the result of the image registration processing to the local image in the local image sequence, and obtaining the pixel offset parameter of the local image during the image registration processing, and then performing adaptive registration on the local image according to the pixel offset parameter; According to the result of the adaptive registration, the overlapping area of ​​adjacent local images is obtained, and the weight of the overlapping area is assigned according to the number of pixels in the overlapping area, and then the local images after the weight assignment are spliced ​​to obtain a spliced ​​image; A gradient operator is used to calculate the gradient of the overlapping area in the stitched image, and the gradient of the overlapping area is interpolated, and then a fusion process is performed in the gradient domain to obtain the global image.

3. The image reading method based on image stitching technology according to claim 2, characterized in that: The performing image registration processing on adjacent new partial images in the new partial image sequence comprises: Determining whether the image illumination condition meets the preset illumination requirement according to the new partial image; If it is determined that the illumination condition of the image meets the preset illumination requirement, an absolute error sum algorithm is used to perform image registration processing on the new partial image; If it is determined that the image illumination condition does not meet the preset illumination requirement, a normalized product correlation algorithm is used to perform image registration processing on the new local image.

4. The image reading method based on image stitching technology according to claim 1, characterized in that: The image pyramid is constructed based on the global image; wherein the image pyramid includes a base layer, an intermediate layer and a top layer, including: Obtain a pre-established blank pyramid; wherein the blank pyramid includes a base layer, a middle layer and a top layer; Using the global image as the image of the base layer in the blank pyramid; Performing a first downsampling process on the global image to obtain an intermediate image, and using the intermediate image as an image of an intermediate layer in the blank pyramid; A second downsampling process is performed on the intermediate image to obtain a top image, and the top image is used as the image of the top layer.

5. The image reading method based on image stitching technology according to claim 1, characterized in that: The acquiring of user historical interaction data, evaluating the read probability of adjacent pixels in the image pyramid according to the historical read data, and calculating the predicted read coefficient of adjacent layers of each layer according to the read probability, includes: Acquire user historical interaction data, and perform statistics on the user historical interaction data to obtain a read probability of pixels in a neighboring range in the image pyramid; The predicted reading coefficient is calculated according to the reading probability according to the following formula: Wherein, P([u, v]) represents the predicted reading coefficient, p([u, v]) represents the reading probability, and λ represents a custom penalty coefficient.

6. The image reading method based on image stitching technology according to claim 1, characterized in that: The preloading of images of each layer in the image pyramid according to the predicted reading coefficients, and reading and rendering the preloaded images in response to user interaction instructions, includes: Based on the predicted reading coefficient, determining whether the probability of corresponding pixels and other image pyramid layers being read meets a preset reading requirement; If it is determined that the probability of corresponding pixels and other image pyramid layers being read meets the preset reading requirements, the image of the corresponding range is preloaded; If it is determined that the probability of corresponding pixels and other image pyramid layers being read does not meet the preset reading requirement, the preloading of the image in the corresponding range is cancelled.

7. The image reading method based on image stitching technology according to claim 1, characterized in that: The method of preloading images of each layer in the image pyramid according to the predicted reading coefficients, and reading and rendering the preloaded images in response to user interaction instructions, further includes: In response to a translation instruction from a user, updating the coordinates of the top layer of the image pyramid according to the coordinate information corresponding to the translation instruction, and displaying the top layer after the coordinates are updated; In response to a zoom-in instruction from a user, updating the coordinates of a base layer of the image pyramid according to coordinate information corresponding to the zoom-in instruction, and displaying the base layer after the coordinates are updated; In response to a zoom-out instruction from the user, coordinates of the middle layer and the top layer of the image pyramid are updated according to coordinate information corresponding to the zoom-out instruction, and the middle layer and the top layer after the updated coordinates are displayed.

8. An image reading device based on image stitching technology, characterized in that: include: An acquisition unit, used to acquire local images of the target object according to a preset index to obtain a local image sequence; A splicing and fusing unit, used for acquiring splicing information of each local image in the local image sequence, and splicing and fusing all the local images into a global image according to the splicing information; A construction unit, configured to construct an image pyramid based on the global image; wherein the image pyramid includes a base layer, a middle layer and a top layer; A prediction unit, used to obtain user historical interaction data, evaluate the read probability of adjacent range pixels in the image pyramid according to the historical read data, and calculate the predicted read coefficient of the adjacent layers of each layer according to the read probability; The reading and rendering unit is used to preload the images of each layer in the image pyramid according to the predicted reading coefficients, and read and render the preloaded images in response to the user's interactive instructions.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the image reading method based on the image stitching technology as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image reading method based on the image stitching technology according to any one of claims 1 to 7 is implemented.

Images