Reinforcement learning-based image stitching method, apparatus and device, and storage medium

By using a reinforcement learning-based image stitching method, initial parameters are obtained and calibration parameters are optimized using a calibration board, which solves the problem of low image stitching quality in hardware stitching and achieves high real-time performance and high precision image stitching.

WO2024230054A1PCT designated stage expired Publication Date: 2024-11-14GUANGDONG UNIV OF TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/122590
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-10
Filing Date
2023-09-28
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In existing technologies, hardware stitching methods suffer from poor image stitching quality when the motion platform and camera plane are difficult to keep perfectly parallel, and traditional methods cannot guarantee global stitching quality.

Method used

A reinforcement learning-based approach is adopted to obtain initial calibration parameters through a calibration board. By combining image stitching quality and motion platform position information, a state value function is constructed to optimize the calibration parameters and correct local errors online, thereby improving the image stitching quality.

Benefits of technology

While ensuring real-time stitching speed, it significantly improves image stitching quality and enhances the overall stitching effect of hardware stitching methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023122590_14112024_PF_FP_ABST
    Figure CN2023122590_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application discloses a reinforcement learning-based image stitching method, apparatus and device, and a storage medium. The method comprises: acquiring an initial calibration parameter by means of a calibration plate, and collecting a sample image and position information of a motion platform; taking the image stitching quality and the position information of the motion platform as states, taking the calibration parameter adjustment amount as an action, and setting a negative reward function on the basis of the image stitching quality; acquiring a state set and a negative reward value set according to a randomly generated action set, the initial calibration parameter, the position information of the motion platform, and the negative reward function, to construct a probabilistic dynamics model; constructing a state value function on the basis of a state occurrence probability obtained by the probabilistic dynamics model and a negative reward value, and acquiring an optimal action by optimizing the state value function; and acquiring an optimized calibration parameter by means of the optimal action and the initial calibration parameter, and performing image stitching on the corresponding sample image by means of the optimized calibration parameter. The technical problem in the prior art of low image stitching quality is ameliorated.
Need to check novelty before this filing date? Find Prior Art

Description

Image stitching method, device, equipment and storage medium based on reinforcement learning Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image stitching method, apparatus, device and storage medium based on reinforcement learning. Background Art

[0002] In the field of precision visual inspection, inspecting flexible circuit substrates requires a high-resolution camera device with a telecentric lens. However, the field of view of such a camera is generally much smaller than the area of ​​the inspected object. Therefore, it is necessary to use the camera device to collect small-scale image information at different positions of the inspected object. Then, image stitching technology is used to stitch the images at different positions in the small range into a large-scale image based on the overlapping areas, ultimately achieving large-scale image acquisition and measurement of the inspected object.

[0003] Traditional image stitching methods generally require calibrating external parameters and obtaining physical information about the platform carrying the detection object. By converting the actual physical position into the image position, the two images are stitched together based on the image position information. After calibration, this method can achieve real-time stitching simply by using an affine transformation matrix, resulting in fast stitching speed. However, when using this method for calibration, it is difficult to achieve absolute parallelism between the motion platform plane and the camera plane. This reduces the accuracy of the calibration parameters during extrinsic calibration. The motion platform will cause a certain degree of angular rotation during movement, causing the detection object to rotate slightly at certain moments. Therefore, directly using the initial calibration parameters obtained through calibration for image stitching makes it difficult to ensure global stitching quality.

[0004] Summary of the Invention

[0005] The present application provides a reinforcement learning-based image stitching method, apparatus, device and storage medium, which are used to improve the technical problem of low image stitching quality in the prior art.

[0006] In view of this, the first aspect of the present application provides an image stitching method based on reinforcement learning, comprising:

[0007] Acquiring initial calibration parameters by using a calibration plate provided on the motion platform, and collecting sample images of the test sample on the motion platform at each moment and position information of the motion platform at each moment during the movement of the motion platform;

[0008] The image stitching quality at each moment and the position information of the motion platform at each moment are used as the state at each moment, the calibration parameter adjustment amount at each moment is used as the action at each moment, and a negative reward function is set based on the image stitching quality at each moment;

[0009] Randomly generating an action set, and obtaining a state set and a negative reward value set according to the action set, the initial calibration parameters, the position information of the motion platform at each moment, and the negative reward function;

[0010] Constructing a Markov experience sequence based on the action set, the state set, and the negative reward value set, and constructing a probabilistic dynamic model through the Markov experience sequence, wherein the probabilistic dynamic model is used to predict the probability of occurrence of the state at the next moment based on the state and action at the current moment;

[0011] Constructing a state-value function based on the probability of occurrence of the state at each moment and the negative reward value at each moment, and obtaining the optimal action at each moment by optimizing the state-value function;

[0012] The optimized calibration parameters at each moment are obtained by using the optimal action at each moment and the initial calibration parameters, and the corresponding sample images are spliced ​​using the optimized calibration parameters at each moment.

[0013] Optionally, the image stitching quality is calculated as follows:

[0014] After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image;

[0015] The similarity between the first overlapping image and the second overlapping image is calculated to obtain image stitching quality.

[0016] Optionally, the state value function is:

[0017] Where, is the state value function, s0 is the state at the initial moment, is the action strategy, s t is the state at time t, c(s t ) is the negative reward value corresponding to the state at time t, p(s t ) is the probability of the state at time t, and T is the final time.

[0018] Optionally, performing image stitching on corresponding sample images using the optimized calibration parameters at each moment includes:

[0019] Calculating the platform movement distance of the motion platform at two adjacent moments according to the position information of the motion platform at each moment;

[0020] Calculating the image translation distance of the sample image at each two adjacent moments according to the optimized calibration parameters at each moment, the platform movement distance at each two adjacent moments, and the position information of the motion platform at each moment;

[0021] Image stitching is performed on the sample images at two adjacent moments based on the image translation distance of the sample images at two adjacent moments.

[0022] A second aspect of the present application provides an image stitching device based on reinforcement learning, comprising:

[0023] a calibration and image acquisition unit, configured to obtain initial calibration parameters using a calibration plate provided on the motion platform, and to acquire sample images of the test sample on the motion platform at each moment and position information of the motion platform at each moment during the movement of the motion platform;

[0024] A setting unit, configured to use the image stitching quality at each moment and the position information of the motion platform at each moment as the state at each moment, and the calibration parameter adjustment amount at each moment as the action at each moment, and to set a negative reward function based on the image stitching quality at each moment;

[0025] an acquisition unit, configured to randomly generate an action set, and acquire a state set and a negative reward value set according to the action set, the initial calibration parameters, the position information of the motion platform at each moment, and the negative reward function;

[0026] a model construction unit, configured to construct a Markov experience sequence based on the action set, the state set, and the negative reward value set, and to construct a probabilistic dynamic model based on the Markov experience sequence, wherein the probabilistic dynamic model is configured to predict the probability of occurrence of the state at the next moment based on the state and action at the current moment;

[0027] An optimization unit, configured to construct a state-value function based on the probability of occurrence of the state at each moment and the negative reward value at each moment, and obtain the optimal action at each moment by optimizing the state-value function;

[0028] The stitching unit is used to obtain the optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters, and stitch the corresponding sample images using the optimized calibration parameters at each moment to obtain an optimized stitched image.

[0029] Optionally, the image stitching quality is calculated as follows:

[0030] After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image;

[0031] The similarity between the first overlapping image and the second overlapping image is calculated to obtain image stitching quality.

[0032] Optionally, the state value function is:

[0033] Where, is the state value function, s0 is the state at the initial moment, is the action strategy, s t is the state at time t, c(s t ) is the negative reward value corresponding to the state at time t, p(s t ) is the probability of the state at time t, and T is the final time.

[0034] Optionally, the splicing unit is specifically used to:

[0035] Obtaining optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters;

[0036] Calculating the platform movement distance of the motion platform at two adjacent moments according to the position information of the motion platform at each moment;

[0037] Calculating the image translation distance of the sample image at each two adjacent moments according to the optimized calibration parameters at each moment, the platform movement distance at each two adjacent moments, and the position information of the motion platform at each moment;

[0038] Image stitching is performed on the sample images at two adjacent moments based on the image translation distance of the sample images at two adjacent moments.

[0039] A third aspect of the present application provides an image stitching device based on reinforcement learning, the device comprising a processor and a memory;

[0040] The memory is used to store program code and transmit the program code to the processor;

[0041] The processor is used to execute the image stitching method based on reinforcement learning described in any one of the first aspects according to the instructions in the program code.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, it implements the image stitching method based on reinforcement learning described in any one of the first aspects.

[0043] It can be seen from the above technical solutions that this application has the following advantages:

[0044] The present application provides an image stitching method based on reinforcement learning, comprising: obtaining initial calibration parameters by means of a calibration plate set on a motion platform, and collecting sample images of detection samples on the motion platform at each moment and position information of the motion platform at each moment during the movement of the motion platform; taking the image stitching quality at each moment and the position information of the motion platform at each moment as the state at each moment, taking the calibration parameter adjustment amount at each moment as the action at each moment, and setting a negative reward function based on the image stitching quality at each moment; randomly generating an action set, and calculating the negative reward function based on the action set, the initial calibration parameters, the position information of the motion platform at each moment and the negative reward function. The reward function obtains a state set and a negative reward value set; a Markov experience sequence is constructed based on the action set, state set and negative reward value set, and a probabilistic dynamic model is constructed through the Markov experience sequence. The probabilistic dynamic model is used to predict the probability of the state at the next moment based on the state and action at the current moment; a state value function is constructed based on the probability of the state at each moment and the negative reward value at each moment, and the optimal action at each moment is obtained by optimizing the state value function; the optimized calibration parameters at each moment are obtained through the optimal action at each moment and the initial calibration parameters, and the corresponding sample images are stitched using the optimized calibration parameters at each moment.

[0045] In this application, after obtaining the initial calibration parameters through the calibration board, the image stitching quality and the position information of the motion platform are used as the state, the calibration parameter adjustment amount is used as the action, and a negative reward function is set according to the image stitching quality, thereby constructing a state-value function. By optimizing the state-value function, the calibration parameters at different positions are optimized to achieve online correction of local errors in the calibration parameters. The hardware stitching coefficient is optimized through the reinforcement learning method to achieve the purpose of improving the image stitching quality, thereby improving the technical problem of low image stitching quality in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] FIG1 is a flow chart of an image stitching method based on reinforcement learning provided in an embodiment of the present application;

[0048] FIG2 is a schematic diagram of the structure of an image stitching system provided in an embodiment of the present application;

[0049] FIG3 is a schematic diagram of a checkerboard calibration plate provided in an embodiment of the present application;

[0050] FIG4 is a schematic diagram of a spliced ​​image provided in an embodiment of the present application;

[0051] FIG5 is a schematic diagram of a stitched image obtained by stitching images using optimized calibration parameters according to an embodiment of the present application;

[0052] FIG6 is a schematic diagram of a stitched image obtained by stitching images using initial calibration parameters according to an embodiment of the present application;

[0053] FIG7 is a structural diagram of an image stitching device based on reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] Traditional image stitching methods are generally divided into software and hardware. Software stitching methods typically detect feature points in the overlapping portion of two images, compare them, and transform the position and shape of the two images, fusing the overlapping portions to achieve image stitching. Hardware stitching methods, on the other hand, generally require calibrating external parameters and obtaining physical information about the platform carrying the detected object. The actual physical position is converted into image position, and the two images are stitched based on this image position information. Generally speaking, software stitching methods produce better stitching quality but are time-consuming and cannot achieve real-time stitching. Hardware stitching methods, after calibration, only require affine transformation matrices for real-time stitching, but they place high demands on the platform's hardware and motion control precision. Hardware stitching methods are generally faster, but the stitching quality is lower than that of software stitching methods. Therefore, designing a stitching method and equipment with high real-time performance and high precision is an urgent problem that the industry needs to solve.

[0056] In order to improve the above problems, this application improves the hardware stitching method. Based on the hardware stitching method, the initial calibration parameters are used as prior information, and reinforcement learning is used to optimize the calibration parameters to ensure the stitching speed while improving the stitching quality of the hardware stitching method. Among them, reinforcement learning includes obtaining state information, setting a reward function, and outputting the optimal action based on the state and reward function. For ease of understanding, please refer to Figure 1. The embodiment of this application provides an image stitching method based on reinforcement learning, including:

[0057] Step 101: Acquire initial calibration parameters through a calibration plate set on the motion platform, and collect sample images of the test sample on the motion platform at each moment and position information of the motion platform at each moment during the movement of the motion platform.

[0058] When collecting sample images of the test sample and performing image stitching, a camera, a motion platform, a test sample, a calibration plate and an industrial computer are required. Please refer to Figure 2 (the industrial computer is not shown). The camera is a high-resolution camera with a telecentric lens. The calibration plate and the test sample are fixed on the motion platform. When the motion platform moves, the test sample and the calibration plate move in two dimensions with the motion platform. The camera collects the sample image and sends it to the industrial computer, which performs image stitching.

[0059] Since the distortion coefficient of a high-resolution camera with a telecentric lens is extremely low, there is no need for internal parameter calibration, and it is for two-dimensional motion plane stitching. Therefore, the embodiment of the present application mainly calibrates external parameters when performing parameter calibration, namely scale and angle, which can be obtained through the checkerboard calibration plate shown in Figure 3. Suppose the actual physical length of the selected checkerboard is L, and the checkerboard length of the checkerboard image captured by the camera is l, then the scale ρ = L / l can be calculated; by identifying and connecting the corner points of the squares in the checkerboard image, the checkerboard line f1 is obtained, and the angle θ can be obtained by calculating the angle between the checkerboard line f1 and the horizontal line f2. The scale and angle obtained by calibration of the calibration plate are the initial calibration parameters.

[0060] The position information of the motion platform is obtained by the plane precision detection equipment of the precision motion platform. The motion platform feeds back the position information while it is moving. Let the position information of the motion platform fed back at time t be The position information of the motion platform fed back at time t-1 is: The movement distance between time t and the previous time t-1 is Assume that the two-dimensional motion coordinates of the sample image at time t-1 are (x t-1 ,y t-1 ), and the affine transformation is performed by the scale ρ and angle θ obtained after the above calibration to obtain the two-dimensional motion coordinates (x t ,y t ),Right now:

[0061] After the camera is calibrated, the motion platform moves to the starting position where the camera can measure the test sample. During the time period from time t-1 to time t, the motion platform moves the distance Mapped to image translation distance (Δx t ,Δy t ), based on the image translation distance (Δx t ,Δyt ) can realize image stitching of the sample image at time t-1 and the sample image at time t. However, the embodiment of the present application takes into account that in an actual physical system, it is difficult for the motion platform plane and the camera plane to be absolutely parallel, and the accuracy of the scale and angle will be reduced when performing external parameter calibration; the motion platform will cause a certain degree of angular rotation during the movement process, causing the detection sample to have a small angle rotation at certain moments. Therefore, it is difficult to ensure the global stitching quality of the image by directly using the initial calibration parameters obtained by calibration for image stitching. The embodiment of the present application is based on the existing calibrated external parameter parameters, performs real-time scale compensation and angle compensation according to the structural characteristics and movement conditions of the platform, and then performs image stitching using the calibration parameters obtained by compensation (i.e., the optimized calibration parameters) to improve the global stitching quality of the image.

[0062] Step 102: The image stitching quality at each moment and the position information of the motion platform at each moment are used as the state at each moment, and the calibration parameter adjustment amount at each moment is used as the action at each moment. A negative reward function is set based on the image stitching quality at each moment.

[0063] In order to optimize the global image stitching quality, the embodiment of the present application automatically optimizes the calibration parameters by constructing the state, action and reward function of the intelligent agent, and performs image stitching according to the optimized calibration parameters.

[0064] In the embodiment of the present application, the image stitching quality at each moment and the position information of the motion platform at each moment are used to determine the image stitching quality at each moment. is the state at each moment, that is s t is the state at time t, δ t is the image stitching quality at time t, and the calibration parameter adjustment amount at each time (Δρ t ,Δθ t ) is the action at each moment, that is, a t =[Δρ t ,Δθ t ], a t is the action at time t, Δρ t is the scale adjustment at time t, Δθ t is the angle adjustment at time t. Assume that the initial scale obtained by calibration is ρ0 and the initial angle is θ0. Then the scale used for the spliced ​​image at time t is ρ t =ρ0+Δρ t , the angle used to stitch the image at time t is θ t =θ0+Δθ t ; A negative reward function is set based on the image stitching quality at each moment. In order to facilitate subsequent gradient calculations, the embodiment of this application adopts a negative reward function, and the negative reward function is c t =1-δ t, the goal is to minimize the negative reward function c t , so that δ t maximize.

[0065] In an embodiment of the present application, the initial scale and initial angle obtained from the calibration plate are used as prior information for subsequent scale and angle optimization. This process is to improve the convergence speed of the calibration parameter optimization process. If the initial scale and initial angle obtained from the calibration plate are not used, a reasonable action range cannot be provided for the calibration parameter optimization, which may easily cause the strategy optimization to not converge or the optimization speed to be slow, thereby causing the calibration parameter optimization to fail. The initial calibration parameters obtained from the calibration plate are used as prior information, and then the calibration parameters are optimized based on the prior information. This is better than simply using reinforcement learning to output the calibration parameters, and solves the instability of pure reinforcement learning.

[0066] Furthermore, in the embodiment of the present application, the image stitching quality is calculated as follows:

[0067] After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image;

[0068] The similarity between the first overlapping image and the second overlapping image is calculated to obtain the image stitching quality.

[0069] Please refer to Figure 4. Image 1 and Image 2 in Figure 4 are sample images at two adjacent moments. After image 1 and image 2 are stitched together, there will be an overlapping area, that is, the location of the rectangular box in Figure 4. By intercepting the overlapping area, a first overlapping image and a second overlapping image, that is, image A and image B in Figure 4, can be obtained. By calculating the similarity between image A and image B, the image stitching quality of the stitched image obtained by stitching image 1 and image 2 can be obtained. Specifically, the similarity between the two images can be obtained by calculating the structural similarity index between the two images. Of course, other methods can also be used to calculate the similarity between the two images. The embodiment of the present application is explained by taking the calculation of the structural similarity index between images to calculate the similarity between images as an example. By calculating the structural similarity index between image A and image B, the similarity value between image A and image B (i.e., SSIM value) is obtained, and the image stitching quality (i.e., SSIM value) after the splicing of image 1 and image 2 is obtained. The calculation formula of the structural similarity index SSIM(A,B) between image A and image B is: SSIM(A,B)=[l(A,B)] α [c(A,B)] β [s(A,B)] γ ;

[0070] Where l(A,B) is the brightness comparison result between image A and image B, c(A,B) is the contrast comparison result between image A and image B, s(A,B) is the structure comparison result between image A and image B, and μ A 、μ B are the average pixel values ​​of image A and image B, σ A , σ B are the pixel standard deviations of image A and image B, σ AB is the pixel covariance between images A and B; c1, c2, and c3 are constants; α, β, and γ are the importance coefficients for brightness, contrast, and structure, respectively. SSIM(A,B) ranges from 0 to 1. Higher SSIM(A,B) values ​​indicate greater similarity between images A and B and higher image stitching quality.

[0071] Step 103: randomly generate an action set, and obtain a state set and a negative reward value set based on the action set, initial calibration parameters, position information of the motion platform at each moment, and the negative reward function.

[0072] After defining the state, action, and negative reward function, we can adjust the amount Δρ according to the scale t , Angle adjustment amount Δθ t The range of values ​​randomly generates an action set {a1,a2,...,a w}, w is the total number of randomly generated actions; according to the generated action set, initial calibration parameters, the position information of the motion platform at each moment and the negative reward function, the corresponding state set {s1,s2,...,s w} and the corresponding negative reward value set {c1,c2,...,c w}. Among them, the scale adjustment amount Δρ t , Angle adjustment amount Δθ t The value range can be determined according to the camera's setting parameters. Specifically, the scale adjustment amount and angle adjustment amount can be set according to the ratio of the camera's resolution to the physical size of the object being measured, the accuracy required for the stitching operation, and the camera's temperature drift stability. Because different stitching requires different accuracy and camera performance also has different differences, it is necessary to select a suitable value range based on the actual stitching operation.

[0073] Step 104: construct a Markov experience sequence based on the action set, the state set, and the negative reward value set, and construct a probabilistic dynamic model through the Markov experience sequence. The probabilistic dynamic model is used to predict the probability of the state at the next moment based on the state and action at the current moment.

[0074] According to the action set, state set and negative reward value set, a set of Markov experience sequences {(s1,a1,s′1,c1),(s2,a2,s′2,c2),…,(s w ,a w ,s′ w ,c w )}, where s′1 is the state at the next moment after state s1, s′2 is the state at the next moment after state s2, and s′ w For state s w The state of the next moment after the current moment.

[0075] According to the above Markov empirical sequence, the probabilistic dynamic model s′ can be constructed i =f(s i ,a i ), the probabilistic dynamic model can predict the state at the next moment based on the current state and the current action. The probabilistic dynamic model can be fitted by deep learning methods with the state-action set {(s1,a1),(s2,a2),…,(s i ,a i ),…,(s w ,a w )} is the input data of the neural network, and the state at the next moment {s′1,s′2,...,s′ i ,…,s′ w} is the label of the neural network. The neural network is trained to obtain a probabilistic dynamic model. When the state s(t-1) and action a(t-1) at time t-1 are input to the probabilistic dynamic model, it will automatically output the probability p(s) of the state s(t) at the next moment. t ).

[0076] Step 105: Construct a state-value function based on the probability of occurrence of the state at each moment and the negative reward value at each moment, and obtain the optimal action at each moment by optimizing the state-value function.

[0077] The probability of the state at each moment can be predicted by the probabilistic dynamic model. t ), multiplying the probability of the state by the negative reward value can obtain the state value function, that is:

[0078] Where, is the state value function, s0 is the state at the initial moment, is the action strategy, For action a t The policy function can be a linear policy function or a nonlinear policy function, s t is the state at time t, c(s t) is the negative reward value corresponding to the state at time t, p(s t ) is the probability of the state at time t, and T is the final time.

[0079] When optimizing the state value function, the value of the state value function is minimized to achieve the maximum reward. The probability p(s t ) is substituted into the state value function, and the gradient of the state value function is obtained to obtain the minimum value, and then the optimal action strategy at time t is obtained. Thus we can get the optimal action at time t The embodiment of the present application performs model fitting based on the data, obtains the state value function through strategy evaluation based on the probabilistic dynamic model obtained by fitting, and then uses the state value function minimization to achieve strategy optimization, output the current optimal action, compensate the calibration parameters online, maximize the image stitching quality, and improve the global image stitching quality. In the process of hardware image stitching, the embodiment of the present application obtains initial calibration parameters through hardware system calibration to ensure the convergence of calibration parameter optimization, and compensates the local errors of the initial calibration parameters obtained by the hardware system through calibration parameter optimization, realizing the organic combination of software and hardware, and improving the image stitching quality while ensuring the real-time performance of image stitching; the embodiment of the present application first constructs a probabilistic dynamic model in view of the characteristic of temperature drift of the camera under long-term operation, and uses model-based reinforcement learning to avoid the problem of low data efficiency utilization existing in model-free reinforcement learning.

[0080] Step 106 : Obtain optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters, perform image stitching on the corresponding sample images through the optimized calibration parameters at each moment, and obtain an optimized stitched image.

[0081] After obtaining the optimal action at time t Finally, the optimized calibration parameters at time t are obtained through the optimal action at time t and the initial calibration parameters Then, the sample image at time t and the sample image at time t+1 can be stitched together using the optimized calibration parameters at time t. The platform movement distance of the motion platform at each adjacent time can be calculated based on the position information of the motion platform at each time; the image translation distance of the sample image at each adjacent time can be calculated based on the optimized calibration parameters at each time, the platform movement distance at each adjacent time, and the position information of the motion platform at each time; and the sample images at each adjacent time can be stitched together based on the image translation distance of the sample images at each adjacent time.

[0082] Taking the inspection of flexible circuit boards as an example, the optimized calibration parameters obtained by the above calibration parameter optimization are used for image stitching, and the stitched image shown in Figure 5 is obtained. The initial calibration parameters obtained by calibration are directly used for image stitching without calibration parameter optimization, and the stitched image shown in Figure 6 is obtained. By comparing the image stitching effects of the two methods, it can be found that the image stitching quality of Figure 5 is significantly better than that of Figure 6.

[0083] In an embodiment of the present application, after obtaining the initial calibration parameters through the calibration board, the image stitching quality and the position information of the motion platform are used as the state, the calibration parameter adjustment amount is used as the action, and a negative reward function is set according to the image stitching quality, thereby constructing a state value function. By optimizing the state value function, the calibration parameters at different positions are optimized to achieve online correction of local errors in the calibration parameters. The hardware stitching coefficient is optimized through the reinforcement learning method to achieve the purpose of improving the image stitching quality, thereby improving the technical problem of low image stitching quality in the existing technology.

[0084] The above is an embodiment of an image stitching method based on reinforcement learning provided by the present application. The following is an embodiment of an image stitching device based on reinforcement learning provided by the present application.

[0085] Referring to FIG. 7 , an image stitching device based on reinforcement learning is provided in an embodiment of the present application, including:

[0086] The calibration and image acquisition unit is used to obtain initial calibration parameters through the calibration plate set on the motion platform, and to collect the sample image of the test sample on the motion platform at each moment and the position information of the motion platform at each moment during the movement of the motion platform;

[0087] A setting unit is configured to set a negative reward function based on the image stitching quality at each moment and the position information of the motion platform at each moment as the state at each moment, and the calibration parameter adjustment amount at each moment as the action at each moment;

[0088] An acquisition unit, configured to randomly generate an action set, and acquire a state set and a negative reward value set based on the action set, initial calibration parameters, position information of the motion platform at each moment, and a negative reward function;

[0089] A model building unit is used to build a Markov experience sequence based on the action set, state set, and negative reward value set, and to build a probabilistic dynamic model based on the Markov experience sequence. The probabilistic dynamic model is used to predict the probability of the state at the next moment based on the state and action at the current moment;

[0090] The optimization unit is used to construct a state-value function based on the probability of occurrence of the state at each moment and the negative reward value at each moment, and obtain the optimal action at each moment by optimizing the state-value function;

[0091] The stitching unit is used to obtain the optimized calibration parameters at each moment through the optimal action and initial calibration parameters at each moment, and stitch the corresponding sample images using the optimized calibration parameters at each moment to obtain an optimized stitched image.

[0092] As a further improvement, the image stitching quality is calculated as follows:

[0093] After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image;

[0094] The similarity between the first overlapping image and the second overlapping image is calculated to obtain the image stitching quality.

[0095] As a further improvement, the state value function is:

[0096] Where, is the state value function, s0 is the state at the initial moment, is the action strategy, s t is the state at time t, c(s t ) is the negative reward value corresponding to the state at time t, p(s t ) is the probability of the state at time t, and T is the final time.

[0097] As a further improvement, the splicing unit is specifically used to:

[0098] Obtain optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters;

[0099] Calculate the platform movement distance of the motion platform at each two adjacent moments according to the position information of the motion platform at each moment;

[0100] Calculating the image translation distance of the sample image between two adjacent moments according to the optimized calibration parameters at each moment, the platform movement distance between two adjacent moments, and the position information of the motion platform at each moment;

[0101] Image stitching is performed on the sample images at two adjacent moments based on the image translation distance of the sample images at two adjacent moments.

[0102] In an embodiment of the present application, after obtaining the initial calibration parameters through the calibration board, the image stitching quality and the position information of the motion platform are used as the state, the calibration parameter adjustment amount is used as the action, and a negative reward function is set according to the image stitching quality, thereby constructing a state value function. By optimizing the state value function, the calibration parameters at different positions are optimized to achieve online correction of local errors in the calibration parameters. The hardware stitching coefficient is optimized through the reinforcement learning method to achieve the purpose of improving the image stitching quality, thereby improving the technical problem of low image stitching quality in the existing technology.

[0103] The embodiment of the present application further provides an image stitching device based on reinforcement learning, the device including a processor and a memory;

[0104] The memory is used to store program codes and transmit the program codes to the processor;

[0105] The processor is configured to execute the image stitching method based on reinforcement learning in the aforementioned method embodiment according to instructions in the program code.

[0106] An embodiment of the present application further provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, the image stitching method based on reinforcement learning in the aforementioned method embodiment is implemented.

[0107] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0108] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0109] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program code.

[0114] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image stitching method based on reinforcement learning, characterized in that: include: Acquire initial calibration parameters by means of a calibration plate arranged on the motion platform, and collect sample images of the detection sample on the motion platform at each time and position information of the motion platform at each time during the movement of the motion platform; The image stitching quality at each moment and the position information of the motion platform at each moment are taken as the state at each moment, the calibration parameter adjustment amount at each moment is taken as the action at each moment, and a negative reward function is set based on the image stitching quality at each moment; Randomly generate an action set, and obtain a state set and a negative reward value set according to the action set, the initial calibration parameters, the position information of the motion platform at each time, and the negative reward function; Constructing a Markov experience sequence according to the action set, the state set and the negative reward value set, and constructing a probabilistic dynamic model through the Markov experience sequence, wherein the probabilistic dynamic model is used to predict the probability of occurrence of the state at the next moment according to the state and action at the current moment; A state value function is constructed based on the probability of occurrence of the state at each moment and the negative reward value at each moment, and the optimal action at each moment is obtained by optimizing the state value function; The optimized calibration parameters at each moment are obtained through the optimal action at each moment and the initial calibration parameters, and the corresponding sample images are stitched through the optimized calibration parameters at each moment.

2. The image stitching method based on reinforcement learning according to claim 1, characterized in that: The calculation process of the image stitching quality is: After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image; The similarity between the first overlapping image and the second overlapping image is calculated to obtain the image stitching quality.

3. The image stitching method based on reinforcement learning according to claim 1, characterized in that: The state value function is: In the formula, is the state value function, s0 is the state at the initial moment, is the action strategy, s t is the state at time t, c(s t ) is the negative reward value corresponding to the state at time t, p(s t ) is the state at time t The probability of occurrence, T is the final moment.

4. The image stitching method based on reinforcement learning according to claim 1, characterized in that: The image stitching of the corresponding sample images by using the optimized calibration parameters at each moment includes: Calculate the platform movement distance of the motion platform at two adjacent moments according to the position information of the motion platform at each moment; Calculating the image translation distance of the sample image at each two adjacent moments according to the optimized calibration parameters at each moment, the platform movement distance at each two adjacent moments, and the position information of the motion platform at each moment; Image stitching is performed on the sample images at two adjacent moments based on the image translation distance of the sample images at two adjacent moments.

5. An image stitching device based on reinforcement learning, characterized in that: include: A calibration and image acquisition unit, used to obtain initial calibration parameters through a calibration plate arranged on the motion platform, and to acquire sample images of the detection sample on the motion platform at each moment and position information of the motion platform at each moment during the movement of the motion platform; A setting unit, used to set a negative reward function based on the image stitching quality at each moment, taking the image stitching quality at each moment and the position information of the motion platform at each moment as the state at each moment, taking the calibration parameter adjustment amount at each moment as the action at each moment; An acquisition unit, used for randomly generating an action set, and acquiring a state set and a negative reward value set according to the action set, the initial calibration parameters, the position information of the motion platform at each moment, and the negative reward function; A model building unit, used to build a Markov experience sequence according to the action set, the state set and the negative reward value set, and to build a probabilistic dynamic model through the Markov experience sequence, wherein the probabilistic dynamic model is used to predict the probability of occurrence of the state at the next moment according to the state and action at the current moment; An optimization unit, used to construct a state value function based on the occurrence probability of the state at each moment and the negative reward value at each moment, and obtain the optimal action at each moment by optimizing the state value function; The stitching unit is used to obtain the optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters, and stitch the corresponding sample images through the optimized calibration parameters at each moment to obtain the optimized stitched image.

6. The image stitching device based on reinforcement learning according to claim 5, characterized in that: The calculation process of the image stitching quality is: After stitching the sample images at two adjacent moments, the overlapping area of ​​the stitched images is intercepted to obtain a first overlapping image and a second overlapping image; The similarity between the first overlapping image and the second overlapping image is calculated to obtain the image stitching quality.

7. The image stitching device based on reinforcement learning according to claim 5, characterized in that: The state value function is: In the formula, is the state value function, s0 is the state at the initial moment, is the action strategy, s t is the state at time t, c(s t ) is the negative reward value corresponding to the state at time t, p(s t ) is the probability of the state at time t, and T is the final time.

8. The image stitching device based on reinforcement learning according to claim 5, characterized in that: The splicing unit is specifically used for: Obtaining optimized calibration parameters at each moment through the optimal action at each moment and the initial calibration parameters; Calculate the platform movement distance of the motion platform at two adjacent moments according to the position information of the motion platform at each moment; Calculating the image translation distance of the sample image at each two adjacent moments according to the optimized calibration parameters at each moment, the platform movement distance at each two adjacent moments, and the position information of the motion platform at each moment; Image stitching is performed on the sample images at two adjacent moments based on the image translation distance of the sample images at two adjacent moments.

9. An image stitching device based on reinforcement learning, characterized in that: The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the image stitching method based on reinforcement learning according to any one of claims 1 to 4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and when the program code is executed by the processor, the image stitching method based on reinforcement learning according to any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Panoramic splicing method based on multi-camera calibration, panoramic splicing system

    CN109064404A

  • Unmanned aerial vehicle image capturing method based on reinforcement learning image processing technology

    CN113689439A

  • Unmanned aerial vehicle obstacle avoidance method based on deep reinforcement learning

    CN114089776A

  • Image splicing method and device based on reinforcement learning, equipment and storage medium

    CN116228549A

  • Systems and methods for user monitoring

    US20210068745A1