Visual perception-based free forging online size detection and feedback method and system

Abstract

The application discloses a free forging online size detection and feedback method and system based on visual perception, relates to the free forging processing detection technical field, and comprises the following steps: acquiring first image data, thermal field data and optical observation data corresponding to a target forging; generating offset data representing the expected displacement vector of each pixel point in the first image data according to the thermal field data and a preset refractive index mapping relationship; calculating optical observation residual data according to the optical observation data and the offset data; judging whether the mean square value of the optical observation residual data exceeds a preset mean square threshold value, and if yes, correcting the refractive index mapping relationship and regenerating the offset data; reversely correcting the first image data according to the offset data to obtain second image data; extracting the forging size data of the target forging according to the second image data and judging whether a feedback instruction is generated; and the beneficial effects are that the optical measurement precision in the high-temperature free forging processing environment can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of free forging processing detection, and particularly relates to a free forging online size detection and feedback method and system based on visual perception. BACKGROUND

[0002] In the free forging production process, online size detection is an important link to ensure product quality. Optical measurement is a widely used measurement method in the free forging process, which uses industrial cameras, lasers or infrared sensors for non-contact measurement, can realize high measurement speed and automation degree, and is suitable for continuous online detection and measurement of complex curved surface forgings.

[0003] However, in the high-temperature free forging processing environment, there is a large temperature gradient on the surface of the forging and the surrounding air, which will cause the refractive index distribution of the air to change significantly, which may cause the optical measurement result to deviate. The current optical measurement method is usually based on two-dimensional image information or a single light path for analysis, so it is difficult to fully reflect the influence of the three-dimensional space temperature gradient on the optical refraction in the measurement process. In this environment, although it can achieve size detection to a certain extent, there is still room for improvement in measurement accuracy and stability for the area with greater thermal disturbance or complex curved surface.

[0004] Therefore, a free forging online size detection and feedback method and system based on visual perception are proposed. SUMMARY

[0005] In view of the above prior art, the present application is proposed. The embodiments of the present application provide a free forging online size detection and feedback method and system based on visual perception, which can improve the optical measurement accuracy in the high-temperature free forging processing environment.

[0006] According to an aspect of the present application, a visual perception-based online free forging size detection and feedback method is provided, comprising: acquiring first image data, thermal field data and optical observation data corresponding to a target forging; generating refractive index field data representing optical refraction characteristics of air at different spatial positions in the first image data according to the thermal field data and a preset refractive index mapping relationship; generating offset data representing expected displacement vectors of each pixel point in the first image data under air refraction interference according to the refractive index field data; calculating optical observation residual data representing differences between predicted offsets and measured offsets according to the optical observation data and the offset data; judging whether a mean square value of the optical observation residual data exceeds a preset mean square threshold, and if so, modifying the refractive index mapping relationship in a direction of reducing the mean square value to not more than the mean square threshold, and regenerating the offset data according to the modified refractive index mapping relationship; reversely modifying each pixel of the first image data according to the offset data to obtain second image data; extracting forging size data of the target forging according to the second image data; and judging whether a deviation of the forging size data from preset target size data is greater than a preset deviation threshold, and if so, generating a feedback instruction.

[0007] According to another aspect of the present application, a visual perception-based online free forging size detection and feedback system is provided, comprising: a data acquisition module for acquiring first image data, thermal field data and optical observation data corresponding to a target forging; a refractive index field generation module for generating refractive index field data representing optical refraction characteristics of air at different spatial positions in the first image data according to the thermal field data and a preset refractive index mapping relationship; an offset generation module for generating offset data representing expected displacement vectors of each pixel point in the first image data under air refraction interference according to the refractive index field data; a residual calculation module for calculating optical observation residual data representing differences between predicted offsets and measured offsets according to the optical observation data and the offset data; a mapping modification module for judging whether a mean square value of the optical observation residual data exceeds a preset mean square threshold, and if so, modifying the refractive index mapping relationship in a direction of reducing the mean square value to not more than the mean square threshold, and regenerating the offset data according to the modified refractive index mapping relationship; an image modification module for reversely modifying each pixel of the first image data according to the offset data to obtain second image data; a size extraction module for extracting forging size data of the target forging according to the second image data; and a feedback module for judging whether a deviation of the forging size data from preset target size data is greater than a preset deviation threshold, and if so, generating a feedback instruction.

[0008] According to another aspect of the present application, an electronic device is provided, comprising a memory for storing computer executable instructions, and a processor for executing the computer executable instructions, which, when executed by the processor, implement the steps of the method as described above.

[0009] According to another aspect of the present application, a computer storage medium is provided, having stored thereon computer executable instructions, which, when executed by a processor, implement the steps of the method as described above.

[0010] Compared with the prior art, the free forging online size detection and feedback method and system based on visual perception according to the embodiments of the present application can compensate the influence of the change of air refractive index under high temperature environment on optical measurement by dynamically correcting the refractive index mapping relationship and correcting the image data reversely, thereby having the advantage of improving the measurement accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application taken in conjunction with the accompanying drawings. The drawings provided herein are for illustrative purposes only and, therefore, are not to be construed as being to scale. The following detailed description of embodiments of the present application is provided for the purpose of fully disclosing embodiments of the present application, and is not intended to limit the scope of the application.

[0012] Figure 1 Flow chart of the free forging online size detection and feedback method based on visual perception of the present application.

[0013] Figure 2 Block diagram of the free forging online size detection and feedback system based on visual perception of the present application.

[0014] Figure 3 Diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0015] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are merely a part of the embodiments of the present application, and are not intended to limit the present application, and that the present application can be implemented in various ways. The embodiments will be preferably described in the order of the following examples.

[0016] Exemplary method:

[0017] In traditional high-temperature free forging environments, when optical measurement systems acquire images of the forging surface using industrial cameras or laser sensors, the surface of the forging and the surrounding air form a non-uniform temperature field due to high-temperature thermal radiation, resulting in a three-dimensional spatial gradient distribution of the air refractive index. Existing methods, based on two-dimensional image information or refraction correction models for single light paths, cannot accurately characterize the cumulative refraction effect of the three-dimensional temperature gradient on the light propagation path. This causes a deviation between the predicted and actual observed values ​​of the image coordinate offset of key feature points on the forging surface, directly affecting the accuracy of forging dimensional data extraction.

[0018] If the above problems are not addressed, the forging dimensional data output by the optical measurement system will not accurately reflect the actual machining state, leading to delayed or incorrect adjustment of process parameters. During continuous machining, the cumulative effect of dimensional errors may cause the forging geometry to deviate from tolerances, resulting in insufficient machining allowances or material waste in subsequent processes.

[0019] To address the aforementioned challenges, this application first considers how to establish the relationship between three-dimensional spatial temperature gradients and light refraction effects. Existing methods, relying solely on two-dimensional temperature compensation or single-path refraction correction, cannot resolve the continuous refraction trajectory of light in three-dimensional space. To resolve this, this application attempts to perform a three-dimensional mapping between thermal field data and refractive index distribution. By constructing refractive index field data covering the image field of view, the light offset effect at different spatial locations is quantified. Simultaneously, this application finds that relying solely on theoretical refractive index is insufficient to adapt to dynamically changing temperature fields, necessitating the introduction of optical observation data for real-time model calibration. By comparing the residuals of predicted and measured offsets, the refractive index mapping relationship is dynamically corrected, effectively eliminating the discrepancy between theory and actual environment.

[0020] In response, this application proposes a method for online dimensional detection and feedback of free forgings based on visual perception, such as... Figure 1As shown, the process includes: acquiring first image data, thermal field data, and optical observation data corresponding to the target forging; generating refractive index field data characterizing the optical refraction characteristics of air at different spatial positions in the first image data based on the thermal field data and a preset refractive index mapping relationship; generating offset data representing the expected displacement vector of each pixel in the first image data under air refraction interference based on the refractive index field data; calculating optical observation residual data characterizing the difference between the predicted offset and the measured offset based on the optical observation data and the offset data; determining whether the mean square value of the optical observation residual data exceeds a preset mean square threshold; if so, correcting the refractive index mapping relationship in the direction of reducing the mean square value to not exceed the mean square threshold, and regenerating the offset data based on the corrected refractive index mapping relationship; performing reverse correction on each pixel of the first image data based on the offset data to obtain second image data; extracting the forging size data of the target forging based on the second image data; determining whether the deviation between the forging size data and the preset target size data is greater than a preset deviation threshold; if so, generating a feedback instruction.

[0021] The first image data refers to two-dimensional or three-dimensional image data containing the surface morphology of the target forging, which is acquired by an industrial camera. Specifically, it can be achieved using a visible light camera, an infrared thermal imager, or a multispectral imaging device, and is used to obtain the geometric features and optical properties of the forging surface.

[0022] Among them, thermal field data refers to the three-dimensional spatial temperature field information that reflects the temperature distribution of the air around the target forging. Specifically, it can be realized by using thermocouple arrays, infrared thermometers or heat flow sensors, and is used to characterize the thermodynamic basis of changes in air refractive index.

[0023] Among them, optical observation data refers to the actual imaging coordinates of optical markers at known spatial locations collected under high-temperature conditions. Specifically, it can be achieved using laser markers, reflective spheres, or coded markers, and is used to establish a reference for the measured offset vector.

[0024] The refractive index mapping relationship refers to the mathematical correspondence between air temperature and air refractive index. Specifically, this mapping relationship can be expressed using the Glaston-Dale formula or an empirical fitting formula.

[0025] ;

[0026] in, For temperature The refractive index of air below, Reference temperature The standard refractive index is approximately 1.000293. It is a first-order temperature coefficient (approximately -1.0 × 10⁻⁶). / ° ), It is a second-order temperature coefficient (approximately -5×). / ° ), used to correct nonlinear effects under high temperature conditions;

[0027] In practical applications, the refractive index mapping relationship is represented as a function model containing adjustable parameters, such as... , Dynamic corrections are made using optical observation data to adapt to the complex temperature field distribution and air composition changes in actual high-temperature environments.

[0028] Among them, refractive index field data refers to a three-dimensional distribution matrix describing the refractive index values ​​of air at different spatial locations. Specifically, it can be generated by combining the temperature-refractive index empirical formula with spatial interpolation algorithms to quantify the impact of air thermal disturbance on the light propagation path.

[0029] The offset data refers to the set of pixel position offset vectors caused by the refractive index field. Specifically, it can be calculated by combining ray tracing algorithm with refractive index gradient integral, and is used to predict the degree of image distortion caused by thermal disturbance.

[0030] Among them, optical observation residual data refers to the difference between the predicted offset vector and the measured offset vector. Specifically, it can be calculated using Euclidean distance or vector differential mode operation, and is used to evaluate the accuracy of the refractive index mapping relationship.

[0031] The mean square threshold refers to the preset upper limit of the residual statistics, which can be set according to historical experimental data or error tolerance standards, and is used to trigger the iterative correction conditions of the refractive index mapping relationship.

[0032] Inverse correction refers to performing inverse coordinate transformation on image pixels based on offset data. Specifically, it can be implemented using affine transformation, perspective transformation, or optical flow field compensation algorithms to eliminate image distortion caused by thermal disturbance.

[0033] Among them, the forging size data refers to the set of geometric parameters extracted from the corrected image. Specifically, it can be implemented using edge detection, point cloud registration or 3D reconstruction algorithms, and is used to quantify the deviation between the actual size of the forging and the design value.

[0034] Feedback instructions refer to control signals that trigger adjustments to process parameters. These can be implemented through a PLC controller or an industrial IoT platform to correct deformation errors in the forging process in real time.

[0035] The core innovation of this application lies in achieving thermal disturbance compensation by dynamically correcting the refractive index mapping relationship, constructing a closed-loop correction system by combining thermal field data and optical observation data, and continuously improving the image correction accuracy by using a residual-driven iterative optimization mechanism. Ultimately, it achieves more accurate detection of forging dimensions compared to existing technologies under high temperature and strong gradient environments.

[0036] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0037] On the free forging production line, high-resolution industrial cameras and infrared thermal imagers are installed to acquire visible light images and temperature distribution images of the forging surface, respectively. Simultaneously, multiple optical markers are placed around the forging to monitor the impact of air refraction on the light propagation path in real time.

[0038] First, an industrial camera captures a high-resolution image of the forging surface as the first image data, an infrared thermal imager captures the temperature distribution of the forging and the surrounding air as thermal field data, and the actual position coordinates of the optical markers are used as optical observation data.

[0039] Next, based on the thermal field data and the pre-calibrated temperature-refractive index correspondence, three-dimensional refractive index field data covering the entire image field of view is generated. This data characterizes the optical refractive properties of air at different spatial locations.

[0040] Then, based on the ray tracing algorithm, the propagation trajectory of each ray from the camera to the surface of the forging in the refractive index field is calculated to obtain the expected displacement vector of each pixel in the first image data affected by air refraction, forming offset data.

[0041] Furthermore, the actual positions of the optical markers in the image are compared with the positions predicted based on the offset data to calculate the optical observation residual data. If the mean square value of the residual exceeds a preset threshold, the refractive index mapping relationship is adjusted through an optimization algorithm to minimize the deviation between the predicted and actual positions.

[0042] Subsequently, the first image data was reversed at the pixel level using the corrected offset data to obtain the second image data with the refractive distortion eliminated.

[0043] Finally, edge detection and dimensional measurement algorithms are applied to the second image data to extract key dimensional data of the forging. The measured dimensions are compared with the target dimensions, and if the deviation exceeds a preset threshold, a corresponding process parameter adjustment command is generated and fed back to the forging control system.

[0044] Through the above scheme, this application achieves the modeling and correction of complex three-dimensional refractive effects in the high-temperature free forging processing environment. By introducing real-time optical observation data to dynamically correct the theoretical refractive index mapping relationship, the accuracy of refractive index field prediction is improved. Independent inverse correction is performed on each pixel in the image, eliminating local distortion and improving the accuracy of forging dimension measurement.

[0045] In some of the above-described solutions of this application, generating refractive index field data characterizing the optical refraction properties of air at different spatial locations in the first image data includes: extracting temperature data of each spatial sampling point based on thermal field data; calculating the air refractive index value corresponding to each spatial sampling point based on the mapping relationship between temperature data and refractive index; performing spatial continuity processing on the air refractive index value based on a preset spatial interpolation function to generate initial three-dimensional refractive index field data covering the field of view corresponding to the first image data; and performing smoothing constraint processing on the initial three-dimensional refractive index field data to ensure that the gradient change of refractive index at the edge of the target forging does not exceed a preset gradient change threshold, thereby obtaining the refractive index field data.

[0046] Among them, the spatial interpolation function can be cubic spline interpolation or radial basis function interpolation, which is used to extend the refractive index data of discrete sampling points into a continuous three-dimensional field distribution.

[0047] Specifically, the smoothing constraint process involves introducing a Laplace smoothing operator or anisotropic diffusion equation to suppress drastic changes in local refractive index caused by abrupt temperature gradient changes in the initial refractive index field.

[0048] The preset gradient change threshold is pre-calibrated based on the temperature dependence of the thermal expansion coefficient of the forging material and the air refractive index.

[0049] Specifically, near the surface of a high-temperature forging, the spatial resolution of the thermal field data acquisition may be insufficient to accurately reflect the temperature gradient distribution. Therefore, discrete temperature data is converted into a continuous refractive index field, i.e., initial three-dimensional refractive index field data, using a spatial interpolation function. However, in the edge region of the forging, the initial three-dimensional refractive index field data may exhibit abnormally large refractive index gradients due to sparse temperature sensor placement or measurement noise. Therefore, smoothing constraint processing adjusts the refractive index field through mathematical optimization methods, ensuring that the refractive index gradient near the forging surface conforms to physical laws. For example, when using anisotropic diffusion equations, the gradient change along the forging normal direction is limited to a threshold range, while the tangential gradient is allowed to vary appropriately. This process ensures that when calculating the light propagation path, the gradient change of the refractive index field in the edge region of the forging will not introduce non-physical offsets due to data interpolation errors, thereby improving the reliability of subsequent optical observation residual calculations.

[0050] Through the above technical solution, this application can accurately obtain the air refractive index field distribution, providing a foundation for subsequent calculation of light propagation paths. This improves the accuracy and continuity of the refractive index field data, thereby enhancing the accuracy of overall dimensional detection. Simultaneously, the smoothing constraint processing avoids abnormal light deflection caused by excessive refractive index gradients at the edges of forgings, enhancing the stability of the measurement results.

[0051] In some of the solutions described above in this application, generating offset data to represent the expected displacement vector of each pixel in the first image data under atmospheric refraction interference includes the following steps:

[0052] First, for each pixel of the first image data, the light propagation path from the image plane to the surface of the target forging is determined. Specifically, the propagation path of light from the camera lens to the forging surface can be simulated using a ray tracing algorithm. For example, an iterative approach can be used, starting from the optical center of the camera and moving along the direction of the light in small steps until it intersects with the surface of the forging.

[0053] Secondly, based on the refractive index field data, the refractive index gradient at each spatial location along the propagation path of each ray is calculated. Specifically, the spatial gradient of the refractive index can be calculated in three-dimensional space using the central difference method. For example, for a certain point in space... Its refractive index gradient It can be represented as:

[0054] ;

[0055] in Point The refractive index at that point , , for , , A tiny displacement in a certain direction.

[0056] Finally, numerical integration is performed along each ray propagation path using the corresponding refractive index gradient to obtain the offset data representing the expected displacement vector of each pixel under air refraction interference. Furthermore, the Runge-Kutta method can be used for numerical integration. For example, a segment of the ray path can be divided into equal parts... Then, integrate the smaller segments using the fourth-order Runge-Kutta formula:

[0057] ;

[0058] The coefficients are defined as follows:

[0059] ;

[0060] in The integration step size is... The function to be integrated is specifically represented in this application as the function of the influence of the refractive index gradient on the direction of light propagation. This represents the function value at the i-th integration step (i.e., the accumulated ray offset). This represents the function value at the (i+1)th integration step; This represents the coordinates of the ray propagation path corresponding to the i-th integration step; , , , These are the intermediate coefficients for the fourth-order Runge-Kutta method, corresponding to the function derivatives at the starting, midpoint (twice), and ending points of the integration step, respectively. The expected offset vector for that pixel can be obtained by integrating over the entire ray path.

[0061] Through the above technical solution, this application can accurately simulate the propagation path of light in a non-uniform refractive index field and precisely calculate the image distortion caused by air refraction. This effectively eliminates the influence of thermal disturbance on optical measurements, improving the accuracy and reliability of free forging dimensional detection. Furthermore, this method considers the refractive index distribution in three-dimensional space, which, compared to traditional two-dimensional analysis methods, can more comprehensively reflect optical refraction phenomena under complex thermal environments, thus demonstrating better applicability in the measurement of forgings with large temperature gradients or complex curved surfaces.

[0062] In some of the above-described schemes of this application, the optical observation residual data for calculating the difference between the predicted offset and the measured offset includes: extracting the measured position coordinates of each observation point in the optical observation data; calculating the measured offset vector of each observation point based on the measured position coordinates and the preset reference position coordinates, where the reference position coordinates are the optical marker positions calibrated within the field of view of the first image data under conditions without thermal disturbance; extracting the predicted offset vector corresponding to the pixel positions of each observation point based on the offset data; and calculating the difference between the measured offset vector and the predicted offset vector of each observation point to obtain the optical observation residual data.

[0063] Among them, the optical markers can be laser dot matrix or laser speckle projected onto the surface of the forging.

[0064] The measured offset vector is calculated using the vector difference between the current coordinates and the reference coordinates, reflecting the actual displacement caused by thermal disturbance.

[0065] Through the above technical solution, this application can quantify the difference between predicted and measured offsets. This allows for the assessment of the accuracy of the refractive index field data, providing a basis for subsequent corrections to the refractive index mapping relationship. Specifically, by calculating the difference between the measured and predicted offsets, the degree of agreement between the current refractive index field model and actual observation results can be reflected. This method considers actual observation data and can better capture the influence of thermal disturbances on optical measurements, thereby improving the accuracy and reliability of free forging dimensional detection.

[0066] In some of the schemes described above in this application, the refractive index mapping relationship is corrected in the direction of reducing the mean square value to no more than the mean square threshold, which is achieved through the following iterative process:

[0067] The first step is to calculate the three-dimensional spatial temperature gradient field covering the field of view of the first image data based on the thermal field data. Specifically, the finite difference method can be used to numerically differentiate the thermal field data to obtain the temperature gradient components in the x, y, and z directions, and then synthesize the three-dimensional temperature gradient vector field.

[0068] The second step is to divide the spatial region corresponding to the refractive index field data into a high gradient region and a low gradient region based on the three-dimensional spatial temperature gradient field.

[0069] This application further proposes two optional methods for dividing high-gradient and low-gradient regions. In practical applications, the appropriate division method can be selected based on the specific measurement scenario and computational resource conditions: The first method is a gradient threshold-based division method, suitable for scenarios with relatively regular temperature field distributions, limited computational resources, or high real-time requirements. This method has low computational complexity and short single division time, but its adaptability to complex temperature distributions is relatively weak. The second method is a region expansion method based on local maxima, suitable for scenarios with multiple local heat sources and complex temperature field distributions. This method can more accurately identify the spatial structure characteristics of temperature gradients and improve the accuracy of refractive index field prediction in complex scenarios compared to the first method, but the computation time is longer. In forging dimension inspection with high accuracy requirements, the second method is preferred; in applications with high response speed requirements, the first method can be used. The core function of both division methods is to identify regions with drastic changes in air refractive index and improve the targeting of refractive index mapping relationship correction through differentiated weight allocation. The specific division methods are as follows:

[0070] In the first approach: First, the temperature gradient magnitude of each spatial point in the corresponding spatial region is extracted based on the three-dimensional spatial temperature gradient field data. Specifically, the partial derivatives of the temperature field in the x, y, and z directions can be calculated using the finite difference method, and then the magnitude of the temperature gradient vector is obtained as the temperature gradient magnitude. Then, spatial points with temperature gradient magnitudes greater than a preset gradient threshold are classified as high gradient regions, and regions outside the high gradient regions are classified as low gradient regions.

[0071] Furthermore, a preset gradient threshold is set to 50°C / m. Spatial points with a temperature gradient amplitude greater than 50°C / m are classified as high gradient regions, and spatial points with a temperature gradient amplitude less than 50°C / m are classified as low gradient regions. For example, for a spatial point, if its temperature gradient amplitude is 60°C / m, it is classified as a high gradient region.

[0072] In the second approach: local maxima are extracted from the three-dimensional spatial temperature gradient field. Local maxima are spatial points with the largest temperature gradient amplitude within a predefined neighborhood. Based on each local maxima, the region is expanded according to a preset spatial distance rule and a preset gradient amplitude threshold. The connected regions obtained from the region expansion are divided into high gradient regions. Regions outside the high gradient regions are divided into low gradient regions.

[0073] The neighborhood range can be set as a cube-shaped region centered on the current point, with a side length of 3 to 5 spatial sampling intervals. During the region expansion process, the spatial distance rule can be set to start from the local maximum point and gradually expand along the decreasing direction of the temperature gradient until the gradient magnitude is lower than the preset gradient magnitude threshold or the maximum expansion distance is reached. The gradient magnitude threshold can be set to 60% to 80% of the initial local maximum to ensure that the gradient change amplitude within the expansion region is within a reasonable range.

[0074] Specifically, by traversing each spatial point in the three-dimensional temperature gradient field, the temperature gradient magnitude of all points within its neighborhood is compared. If the current point has the largest gradient magnitude, it is marked as a local maximum. Starting from each local maximum, the region is expanded along the decreasing direction of the temperature gradient. During each expansion, it is checked whether the gradient magnitude of adjacent points is higher than the gradient magnitude threshold and does not exceed the maximum expansion distance. If the conditions are met, it is included in the high gradient region. If adjacent points have been expanded and covered by other local maximum points during the expansion process, the region to which they belong is determined according to the priority of the gradient magnitude. Finally, all connected regions formed by the expansion are merged into high gradient regions, and the remaining regions are automatically classified as low gradient regions. The temperature changes drastically in high gradient regions, while the temperature changes relatively gently in low gradient regions.

[0075] Returning to the iterative process, the third step is to extract the observation deviation vector for each observation point based on the optical observation residual data.

[0076] Furthermore, the optical observation residual data can be represented as a two-dimensional vector field, with each observation point corresponding to an observation bias vector. It should be understood that this vector field is defined for pixel locations with observation data, and for pixel locations without observation data, its value can be obtained through interpolation or other methods.

[0077] The fourth step involves assigning a first weight and a second weight to the observation bias vectors of each observation point in the high-gradient and low-gradient regions, respectively, with the first weight being greater than the second weight. For example, the first weight can be set to 0.8 and the second weight to 0.2 to emphasize the influence of the high-gradient region.

[0078] The fifth step is to construct an objective function based on the weighted observation bias vectors. The objective function is the sum of the squares of the magnitudes of the weighted observation bias vectors.

[0079] Step 6: Based on the objective function, update the refractive index mapping relationship using the gradient descent algorithm. Specifically, let the adjustable parameter of the refractive index mapping relationship be... (For example, the first-order temperature coefficient in the aforementioned empirical formula for temperature-refractive index) and second-order temperature coefficient The objective function is expressed as:

[0080] ;

[0081] in, Here is the weight of the i-th observation point (0.8 in the high gradient region and 0.2 in the low gradient region). Based on current parameters The difference between the calculated predicted offset vector and the measured offset vector.

[0082] Update parameters using gradient descent:

[0083] ;

[0084] in, Indicates the number of iterations. This is the learning rate (typically ranging from 0.001 to 0.01). For the objective function with respect to parameters The gradient is calculated using numerical differentiation, i.e.:

[0085] ;

[0086] in, For small perturbations (typically taking values ​​of...) (0.1% of the current value).

[0087] Stochastic gradient descent can be used to accelerate convergence. In each iteration, a batch of observation points is randomly selected (the batch size is 10%-30% of the total number of observation points), the gradient of the objective function of the batch is calculated, and the parameters are updated in the opposite direction of the gradient until the objective function converges or the mean square value meets the threshold requirement.

[0088] The seventh step is to recalculate the mean square value based on the updated refractive index mapping. Specifically, this involves regenerating the refractive index field data and offset data using the updated refractive index mapping, and calculating the mean square value of the new optical observation residual data.

[0089] Step 8: Determine if the mean square value does not exceed the mean square threshold. If it does, stop the iteration. For example, the mean square threshold can be set to 0.1 pixels, and the iteration process will terminate when the mean square value is less than this threshold.

[0090] Exemplary system:

[0091] Figure 2 The figure illustrates a vision-based online dimensional detection and feedback system for free forgings according to an embodiment of this application, comprising: a data acquisition module for acquiring first image data, thermal field data, and optical observation data corresponding to the target forging; a refractive index field generation module for generating refractive index field data characterizing the optical refraction characteristics of air at different spatial positions in the first image data based on the thermal field data and a preset refractive index mapping relationship; an offset generation module for generating offset data representing the expected displacement vector of each pixel in the first image data under air refraction interference based on the refractive index field data; and a residual calculation module for calculating the difference between the predicted offset and the measured offset based on the optical observation data and the offset data. The system includes: optical observation residual data of the difference between offsets; a mapping correction module, used to determine whether the mean square value of the optical observation residual data exceeds a preset mean square threshold; if so, the refractive index mapping relationship is corrected in the direction of reducing the mean square value to not exceed the mean square threshold, and the offset data is regenerated based on the corrected refractive index mapping relationship; an image correction module, used to reverse correct each pixel of the first image data based on the offset data to obtain the second image data; a size extraction module, used to extract the forging size data of the target forging based on the second image data; and a feedback module, used to determine whether the deviation between the forging size data and the preset target size data is greater than a preset deviation threshold; if so, a feedback instruction is generated.

[0092] In one example, the refractive index field generation module generates refractive index field data characterizing the optical refractive properties of air at different spatial locations in the first image data, including: extracting temperature data for each spatial sampling point based on thermal field data; calculating the air refractive index value corresponding to each spatial sampling point based on the mapping relationship between temperature data and refractive index; performing spatial continuity processing on the air refractive index value based on a preset spatial interpolation function to generate initial three-dimensional refractive index field data covering the field of view corresponding to the first image data; and performing smoothing constraint processing on the initial three-dimensional refractive index field data to ensure that the gradient change of refractive index at the edge of the target forging does not exceed a preset gradient change threshold, thereby obtaining the refractive index field data.

[0093] In one example, the offset generation module generates offset data representing the expected displacement vector of each pixel in the first image data under air refraction interference by: determining the light propagation path from the image plane to the surface of the target forging for each pixel in the first image data; calculating the refractive index gradient of the spatial location traversed by each light propagation path based on the refractive index field data; and numerically integrating the corresponding refractive index gradient along each light propagation path to obtain the offset data representing the expected displacement vector of each pixel under air refraction interference.

[0094] In one example, the residual calculation module calculates the optical observation residual data, which represents the difference between the predicted offset and the measured offset, by: extracting the measured position coordinates of each observation point in the optical observation data; calculating the measured offset vector of each observation point based on the measured position coordinates and the preset reference position coordinates, where the reference position coordinates are the optical marker positions calibrated within the field of view of the first image data under conditions without thermal disturbance; extracting the predicted offset vector corresponding to the pixel positions of each observation point based on the offset data; and calculating the difference between the measured offset vector and the predicted offset vector of each observation point to obtain the optical observation residual data.

[0095] In one example, the mapping correction module corrects the refractive index mapping relationship in the direction of reducing the mean square value to no more than the mean square threshold. This is achieved through the following iterative process: 1) Calculate the three-dimensional spatial temperature gradient field covering the field of view of the first image data based on the thermal field data; 2) Divide the spatial region corresponding to the refractive index field data into high-gradient and low-gradient regions based on the three-dimensional spatial temperature gradient field; 3) Extract the observation deviation vector of each observation point based on the optical observation residual data; 4) Assign a first weight and a second weight to the observation deviation vector of each observation point in the high-gradient and low-gradient regions, respectively, with the first weight being greater than the second weight; 5) Construct an objective function based on the weighted observation deviation vectors, where the objective function is the sum of the squares of the magnitudes of the weighted observation deviation vectors; 6) Update the refractive index mapping relationship based on the objective function using a gradient descent algorithm; 7) Recalculate the mean square value based on the updated refractive index mapping relationship; 8) Determine whether the mean square value does not exceed the mean square threshold; if so, stop the iteration.

[0096] In one example, the mapping correction module divides the spatial region corresponding to the refractive index field data into high gradient region and low gradient region based on the three-dimensional spatial temperature gradient field. This includes: extracting the temperature gradient amplitude of each spatial point in the spatial region corresponding to the refractive index field data based on the three-dimensional spatial temperature gradient field; classifying spatial points with temperature gradient amplitude greater than a preset gradient threshold as high gradient region; and classifying the region outside the high gradient region as low gradient region.

[0097] In another example, the mapping correction module divides the spatial region corresponding to the refractive index field data into high gradient regions and low gradient regions based on the three-dimensional spatial temperature gradient field. This includes: extracting local maxima in the three-dimensional spatial temperature gradient field, where local maxima are spatial points with the largest temperature gradient amplitude within a predefined neighborhood; expanding the region based on each local maxima according to preset spatial distance rules and preset gradient amplitude thresholds, and dividing the connected regions obtained from the region expansion into high gradient regions; and dividing the region outside the high gradient regions into low gradient regions.

[0098] Exemplary electronic device:

[0099] Figure 3 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0100] like Figure 3 As shown, the electronic device includes one or more processors and memory.

[0101] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0102] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0103] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0104] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device may include any other suitable components depending on the specific application.

[0105] Exemplary computer-readable media:

[0106] Embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps described in the "Exemplary Methods" section above according to the various embodiments of this application.

[0107] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0108] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0109] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0110] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0111] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0112] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for online dimensional detection and feedback of free forgings based on visual perception, characterized in that, include: Acquire the first image data, thermal field data, and optical observation data corresponding to the target forging; Based on the thermal field data and the preset refractive index mapping relationship, refractive index field data characterizing the optical refractive properties of air at different spatial locations in the first image data is generated. Based on the refractive index field data, offset data is generated to represent the expected displacement vector of each pixel in the first image data under air refraction interference; Based on the optical observation data and the offset data, calculate the optical observation residual data that characterizes the difference between the predicted offset and the measured offset; Determine whether the mean square value of the optical observation residual data exceeds a preset mean square threshold. If so, correct the refractive index mapping relationship in the direction of reducing the mean square value to no more than the mean square threshold, and regenerate the offset data according to the corrected refractive index mapping relationship. Based on the offset data, each pixel of the first image data is reverse-corrected to obtain the second image data; Extract the forging dimension data of the target forging based on the second image data; Determine whether the deviation between the forging size data and the preset target size data is greater than a preset deviation threshold. If so, generate a feedback instruction.

2. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 1, characterized in that, The refractive index field data that characterizes the optical refractive properties of air at different spatial locations in the first image data includes: Temperature data of each spatial sampling point is extracted based on the thermal field data; Calculate the air refractive index value corresponding to each spatial sampling point based on the temperature data and the refractive index mapping relationship; The air refractive index values ​​are spatially continuous based on a preset spatial interpolation function to generate initial three-dimensional refractive index field data covering the field of view corresponding to the first image data. The initial three-dimensional refractive index field data is subjected to smoothing constraint processing to ensure that the gradient change of the refractive index at the edge of the target forging does not exceed a preset gradient change threshold, thereby obtaining the refractive index field data.

3. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 1, characterized in that, The offset data used to generate the expected displacement vector of each pixel in the first image data under air refraction interference includes: For each pixel of the first image data, determine the light propagation path from the image plane to the surface of the target forging; Based on the refractive index field data, calculate the refractive index gradient at the spatial location along each of the light propagation paths; By numerically integrating the corresponding refractive index gradient along each of the light propagation paths, the offset data representing the expected displacement vector of each pixel under air refraction interference is obtained.

4. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 1, characterized in that, The optical observation residual data used to calculate the difference between the predicted and measured offsets includes: Extract the measured position coordinates of each observation point from the optical observation data; The measured offset vector of each observation point is calculated based on the measured position coordinates and the preset reference position coordinates. The reference position coordinates are the optical marker positions calibrated within the field of view of the first image data under the condition of no thermal disturbance. Based on the offset data, the predicted offset vector corresponding to the pixel position of each observation point is extracted; The difference between the measured offset vector and the predicted offset vector at each observation point is calculated to obtain the optical observation residual data.

5. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 1, characterized in that, The correction of the refractive index mapping relationship in the direction of reducing the mean square value to no more than the mean square threshold is achieved through the following iterative process: Based on the thermal field data, calculate the three-dimensional spatial temperature gradient field covering the field of view of the first image data; Based on the three-dimensional spatial temperature gradient field, the spatial region corresponding to the refractive index field data is divided into a high gradient region and a low gradient region; Extract the observation deviation vector for each observation point based on the optical observation residual data; A first weight and a second weight are assigned to the observation bias vector of each observation point in the high gradient region and the low gradient region, respectively, wherein the first weight is greater than the second weight; An objective function is constructed based on the weighted observation bias vectors, wherein the objective function is the sum of the squares of the magnitudes of the weighted observation bias vectors; Based on the objective function, the refractive index mapping relationship is updated using a gradient descent algorithm; The mean square value is recalculated based on the updated refractive index mapping relationship; Determine whether the mean square value does not exceed the mean square threshold; if so, stop the iteration.

6. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 5, characterized in that, The step of dividing the spatial region corresponding to the refractive index field data into a high-gradient region and a low-gradient region based on the three-dimensional spatial temperature gradient field includes: Based on the three-dimensional spatial temperature gradient field, the temperature gradient amplitude of each spatial point in the corresponding spatial region of the refractive index field data is extracted; Spatial points with temperature gradient magnitudes greater than a preset gradient threshold are classified as high gradient regions; The region outside the high gradient region is designated as the low gradient region.

7. The method for online dimensional detection and feedback of free forgings based on visual perception according to claim 5, characterized in that, The step of dividing the spatial region corresponding to the refractive index field data into a high-gradient region and a low-gradient region based on the three-dimensional spatial temperature gradient field includes: Local maxima are extracted from the three-dimensional spatial temperature gradient field. The local maxima are the spatial points with the largest temperature gradient amplitude within a predefined neighborhood. Centered on each of the local maxima, the region is expanded according to a preset spatial distance rule and a preset gradient magnitude threshold, and the connected region obtained by the region expansion is divided into a high gradient region. The region outside the high gradient region is designated as the low gradient region.

8. A vision-based online dimensional detection and feedback system for free forgings, characterized in that, include: The data acquisition module is used to acquire the first image data, thermal field data, and optical observation data corresponding to the target forging. The refractive index field generation module is used to generate refractive index field data that characterizes the optical refraction properties of air at different spatial locations in the first image data, based on the thermal field data and a preset refractive index mapping relationship. The offset generation module is used to generate offset data based on the refractive index field data to represent the expected displacement vector of each pixel in the first image data under air refraction interference. The residual calculation module is used to calculate optical observation residual data, which represents the difference between the predicted offset and the measured offset, based on the optical observation data and the offset data. The mapping correction module is used to determine whether the mean square value of the optical observation residual data exceeds a preset mean square threshold. If so, the refractive index mapping relationship is corrected in the direction of reducing the mean square value to no more than the mean square threshold, and the offset data is regenerated according to the corrected refractive index mapping relationship. An image correction module is used to reverse correct each pixel of the first image data according to the offset data to obtain the second image data; The size extraction module is used to extract the forging size data of the target forging based on the second image data; The feedback module is used to determine whether the deviation between the forging size data and the preset target size data is greater than a preset deviation threshold. If so, a feedback instruction is generated.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method as described in any one of claims 1 to 7.

10. A computer storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 7.

Images