Machine learning-based correlation imaging information extraction technology and parallel sampling system

By employing machine learning-based correlation imaging information extraction technology and a parallel sampling system, the problems of low matching accuracy and low computational efficiency in multi-source, multi-temporal, and multi-view image data processing by traditional methods have been solved. This has enabled high-precision image registration and deep-level correlation information mining, thereby improving the system's real-time processing capabilities and analytical capabilities in complex scenarios.

CN120431137BActive Publication Date: 2025-10-28NAT UNIV OF DEFENSE TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510565190.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-04-30
Publication Date
2025-10-28
Estimated Expiration
2045-04-30

AI Technical Summary

Technical Problem

Traditional correlation imaging information extraction techniques suffer from problems such as low matching accuracy, low computational efficiency, insufficient adaptability, difficulty in mining deep correlation information, and insufficient real-time processing capability when processing multi-source, multi-temporal, and multi-view image data.

Method used

The system employs machine learning-based correlation imaging information extraction technology and a parallel sampling system, including modules for data collection, feature point detection, feature matching and image registration, and correlation information analysis. It uses parallel sampling technology to collect image data, detects feature points through the difference of Gaussian pyramid, matches feature descriptors using the nearest neighbor distance ratio method, and combines convolutional neural networks to construct a correlation information model, thereby achieving end-to-end learning and feature point correspondence prediction.

Benefits of technology

It improves image matching accuracy and registration precision, optimizes the image analysis process, enhances the system's automation and intelligence, improves processing capabilities in complex scenarios, and enables real-time image analysis and decision support.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120431137B_ABST
    Figure CN120431137B_ABST
Patent Text Reader

Abstract

This invention relates to the field of correlated imaging information extraction technology, specifically to a machine learning-based correlated imaging information extraction technology and parallel sampling system. It includes a data collection module, a feature point detection module, a feature matching and image registration module, and a correlated information analysis module. The data collection module collects historical multi-image data and uses parallel sampling technology to collect real-time multi-image data. The feature point detection module extracts features using the difference of Gaussian pyramid. The feature matching and image registration module achieves high-precision matching and registration using the nearest neighbor distance ratio method. The correlated information analysis module uses a convolutional neural network to build a model, taking the registration data as input and feature point pairs as targets, to achieve fast and accurate prediction of feature point correspondences in real-time images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of correlated imaging information extraction technology, specifically to correlated imaging information extraction technology and parallel sampling system based on machine learning. Background Technology

[0002] Traditional correlation imaging information extraction techniques face numerous challenges when processing multi-source, multi-temporal, and multi-view image data.

[0003] First, in the feature matching stage, commonly used methods struggle to handle significant variations between images and complex scenes, resulting in low matching accuracy and impacting the accuracy of subsequent analysis. Second, during image registration, traditional algorithms are computationally inefficient when processing large amounts of image data and lack adaptability to non-rigid deformations, making it difficult to achieve high-precision alignment of multiple images. Furthermore, in terms of extracting correlation information, traditional methods often rely on manually designed features and rules, making it difficult to fully explore deep-level correlations between images, especially when dealing with high-dimensional, non-linear relationships. Traditional techniques also lack effective end-to-end learning mechanisms; each processing stage is relatively independent, making global optimization difficult.

[0004] Furthermore, traditional methods lack quantum correlation constraint mechanisms in few-shot learning, resulting in a lack of physical basis for attention weight adjustment and difficulty in effectively utilizing the correlation characteristics of multiple detectors; existing real-time detection networks lack multi-scale feature fusion, channel attention lacks quantum statistical verification, small target localization is easily affected by noise, and the detection results lack verification of quantum correlation characteristics, resulting in a high false detection rate in complex scenarios.

[0005] Finally, when dealing with real-time data streams, traditional methods lack sufficient processing speed and adaptability, making it difficult to meet the real-time analysis needs of rapidly changing scenarios. These problems severely limit the effectiveness and scope of traditional technologies in complex environments, necessitating new technological solutions to overcome these limitations. Summary of the Invention

[0006] The purpose of this invention is to provide a machine learning-based correlation imaging information extraction technology and a parallel sampling system to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a machine learning-based correlation imaging information extraction technology and parallel sampling system, comprising a data collection module, a feature point detection module, a feature matching and image registration module, and a correlation information analysis module, wherein:

[0008] The data collection module uses parallel sampling technology to collect real-time multi-image data; it also collects historical multi-image data with the same structure as the real-time multi-image data from a public database.

[0009] The feature point detection module is used to detect feature points on multiple historical image data. It constructs a Gaussian difference pyramid on each image in the multiple image data to detect extreme points and generate feature descriptors.

[0010] The feature matching and image registration module performs feature descriptor matching based on the feature descriptors generated for each image in the historical multi-image data, using the nearest neighbor distance ratio method to generate feature point pairs corresponding to the matched feature descriptors; based on the feature point pairs corresponding to the matched feature descriptors and the historical multi-image data of the corresponding matched image, it calculates the transformation matrix between the images, and applies the calculated transformation matrix to align the historical multi-image data of the matched image to the same coordinate system, generating registered multi-image data;

[0011] The association information analysis module uses a convolutional neural network algorithm to construct an association information model. It uses the feature point pairs corresponding to the matched feature descriptors as the target features of the association information model and registers multiple image data as the input features of the association information model to construct the association information model. The constructed association information model is used to predict the correspondence between feature points in real-time multiple image data.

[0012] As a further improvement to this technical solution, the data collection module uses parallel sampling technology to collect real-time multiple image data. Parallel sampling technology is a method of acquiring data from multiple data sources and multiple sampling points simultaneously. In image acquisition, multiple synchronized image sensors are used to capture different perspectives of the scene at the same time, generating image data from multiple perspectives.

[0013] As a further improvement to this technical solution, the feature point detection module constructs a Gaussian difference pyramid on each image in multiple image data sets to detect extreme points and generate feature descriptors, specifically including:

[0014] For each image in the historical image data, multiple Gaussian blurred images are created using different σ values ​​to generate a scale space, which is the representation of the image at different resolutions. Adjacent Gaussian blurred images are subtracted to obtain a scale layer. This process is repeated to form a Gaussian difference pyramid, where σ is the standard deviation of the Gaussian blur, used to control the degree of Gaussian blur.

[0015] At each scale level of the Difference of Gaussian pyramid, each pixel is compared with its surrounding pixels, and also with the corresponding pixels at the previous and next scale levels. If the pixel is a local extremum in this 3x3x3 cube, it is marked as a potential feature point.

[0016] Calculate the gradient magnitude and direction of the region surrounding the potential feature point, create a 360-degree orientation histogram, find the main peak of the histogram, and assign its direction to the potential feature point as the main direction.

[0017] A fixed-size neighborhood is selected around the feature point, and this neighborhood is divided into multiple sub-regions. A gradient direction histogram is calculated for each sub-region. The histogram information of all sub-regions is concatenated into a vector, and the vector is normalized to generate a feature descriptor. The feature descriptor includes the corresponding feature point, and the feature point includes the coordinate position, scale layer, and principal direction.

[0018] As a further improvement to this technical solution, the feature matching and image registration module includes a feature matching unit. The process by which the feature matching unit generates feature point pairs corresponding to matched feature descriptors specifically includes:

[0019] Based on the feature descriptors generated from each image in historical multi-image data, the nearest neighbor distance ratio method is used for feature descriptor matching. For each feature descriptor in one image, the nearest and second nearest neighbor feature descriptors are found in another image, and the ratio of the nearest neighbor distance to the second nearest neighbor distance is calculated. If the ratio is less than a preset ratio threshold, it means that the pair of feature descriptors is successfully matched, and a matched pair of feature descriptors is generated. The determination of the nearest and second nearest neighbors is obtained by calculating the distance between feature descriptors in different images. The nearest neighbor represents the smallest distance, and the second nearest neighbor represents the smallest distance found again among all distances after removing the nearest neighbor.

[0020] For each pair of matching feature descriptors, extract their corresponding feature point information to generate feature point pairs, each pair containing corresponding feature points from the two images.

[0021] As a further improvement to this technical solution, the feature matching and image registration module includes an image registration unit. The image registration unit calculates the transformation matrix between images based on the feature point pairs corresponding to the matched feature descriptors and historical image data of the corresponding matched images. Specifically, this includes:

[0022] Based on the matched feature point pairs and corresponding historical image data, a robust estimation method is used to calculate the transformation matrix between images. The minimum number of feature point pairs is randomly selected, where the minimum number of feature point pairs is determined according to the transformation model, which includes, but is not limited to, affine transformation, similarity transformation, and perspective transformation. Based on the minimum number of feature point pairs and the corresponding transformation model, the transformation matrix is ​​calculated. The error of all feature point pairs is calculated based on the transformation matrix, and the number of interior points that conform to the transformation model is counted. This step is repeated, and the transformation matrix with the largest number of interior points is selected as the transformation matrix between images.

[0023] As a further improvement to this technical solution, the image registration unit applies the calculated transformation matrix to align multiple historical image data of the matching image to the same coordinate system, generating registered multiple image data, specifically including:

[0024] A transformation matrix is ​​applied to each matching image to perform a geometric transformation on the image. The transformed image is then resampled to the target coordinate system, which is the smallest rectangular region that can contain all images. The target coordinate system is determined by calculating the average position and orientation of all transformed images. The set of images that are aligned to the same coordinate system is used as the registration data for multiple images.

[0025] As a further improvement to this technical solution, the association information analysis module utilizes a convolutional neural network algorithm to construct an association information model. This involves using the feature point pairs corresponding to the matched feature descriptors as the target features of the association information model, and registering multiple image data as the input features of the association information model, thereby constructing the association information model. Specifically, this includes:

[0026] The registered multi-image data generated by the image registration unit is used as the input features of the association information model, and the feature point pairs corresponding to the matched feature descriptors generated by the feature matching unit are used as the target features of the association information model.

[0027] Design a convolutional neural network architecture that can process multiple image data for registration and output predicted feature point correspondences. The convolutional neural network structure includes an input layer, convolutional layers, pooling layers, and fully connected layers. The input layer receives multiple image data for registration; the convolutional layers extract local features of the images; the pooling layers reduce the spatial dimension of the feature maps; and the fully connected layers synthesize high-level features to generate the final prediction.

[0028] Multiple registered image data are input into a convolutional neural network. Feature point pairs corresponding to the matched feature descriptors are used as supervision information. A loss function is defined to measure the difference between the predicted feature point correspondence and the real feature point pairs. Backpropagation algorithm and optimizer are used to update network parameters. The training process is iterated until the model converges or reaches the preset number of training rounds.

[0029] The constructed association information model is used to determine the correspondence between feature points in real-time multi-image data. By inputting real-time multi-image data into the constructed association information model, the model outputs the predicted correspondence between feature points.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] 1. This machine learning-based correlation imaging information extraction technology and parallel sampling system generates accurate feature point pairs by matching feature descriptors using the nearest neighbor distance ratio method. This not only improves the accuracy of image matching but also lays a solid foundation for subsequent image registration. Based on these matched feature point pairs, the system can calculate accurate image transformation matrices, thus achieving high-quality alignment of multiple historical image data. It effectively solves the geometric differences between images taken at different times and from different perspectives, enabling multiple images to be compared and analyzed in the same coordinate system. Furthermore, by generating registered multiple image data, it provides a consistent data foundation for subsequent image analysis and processing tasks, greatly improving the reliability and accuracy of the analysis results. This multiple application of feature descriptors not only optimizes the image registration process but also provides high-quality input data for subsequent correlation information analysis.

[0032] 2. The machine learning-based correlation imaging information extraction technology and the correlation information analysis module in the parallel sampling system cleverly utilize convolutional neural network algorithms to realize a powerful and flexible correlation information model. This model uses registered multi-image data generated by the feature matching and image registration module as input features, and uses the feature point pairs corresponding to the matched feature descriptors as target features, thereby establishing an end-to-end learning system. The multi-layer structure of the convolutional neural network can automatically learn and extract complex nonlinear relationships between images, effectively process multi-view image data, and integrate registration information. Through convolution and pooling operations, convolutional neural networks can not only capture the spatial relationships of images but also learn the correspondences between feature points, while possessing invariance to image transformations. This deep learning method greatly improves the system's ability to process large-scale, high-dimensional image data, and can uncover deep-level correlation information that is difficult to discover using traditional methods. The trained model has strong generalization and real-time prediction capabilities, and can quickly and accurately predict the correspondences of feature points in new real-time multi-image data, providing strong support for real-time image analysis and decision-making. It not only improves the system's automation and intelligence level but also greatly enhances the system's ability to handle complex scenes and dynamic changes, opening up new avenues for the correlation analysis of multi-source images.

[0033] 3. The machine learning-based correlation imaging information extraction technology and parallel sampling system dynamically adjust the attention distribution through the quantum correlation matrix to enhance the physical rationality of feature interaction; the multi-branch detection architecture integrates spatial pyramid and quantum verification mechanism to improve the utilization rate of multi-scale features; normalized distribution distance regression optimizes the localization accuracy of small targets, and combined with quantum statistical verification, it effectively screens out abnormal detection results. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall modules of the present invention;

[0035] Figure 2 This is a schematic diagram of the feature matching and image registration module unit of the present invention.

[0036] In the diagram: 100, Data collection module; 200, Feature point detection module; 300, Feature matching and image registration module; 301, Feature matching unit; 302, Image registration unit; 400, Association information analysis module. Detailed Implementation

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0039] The following are some of the terminology definitions:

[0040] Real-time multi-image data: refers to a set of image data acquired simultaneously from multiple data sources or multiple sampling points using parallel sampling technology. These images typically contain different perspectives of the same scene.

[0041] Historical multi-image data: refers to a set of previously acquired image data collected from public databases that has the same structural type as real-time multi-image data;

[0042] Feature points: These are points in an image that have unique properties and are identifiable, usually corner points, edge points, or other significant features in the image;

[0043] Feature descriptor: A mathematical description of the region surrounding a feature point. It is usually a multi-dimensional vector used to represent the unique properties of the feature point, which facilitates subsequent feature matching.

[0044] Feature point pair: refers to two feature points that are identified as corresponding points in different images, representing the same physical point in their respective images;

[0045] Transformation matrix: A mathematical matrix that describes the spatial relationship between two images and is used to transform a point in one image to its corresponding position in another image;

[0046] Registration of multiple image data: A set of multiple images that are aligned to the same coordinate system after spatial transformation, so that corresponding points in these images are consistent in space;

[0047] Association Information Model: A machine learning model based on convolutional neural networks, used to analyze registered multiple image data and predict the correspondence between feature points in the images.

[0048] Next, please refer to Figure 1-Figure 2 The present invention provides a technical solution: a machine learning-based correlation imaging information extraction technology and parallel sampling system, including a data collection module 100, a feature point detection module 200, a feature matching and image registration module 300, and a correlation information analysis module 400.

[0049] The data collection module 100 uses parallel sampling technology to collect real-time multi-image data. Parallel sampling technology is a method of acquiring data from multiple data sources and multiple sampling points simultaneously. In image acquisition, multiple synchronized image sensors (such as cameras) are used to capture different perspectives of the scene at the same time (or very close to the same time) to ensure that images are captured at the same time and generate image data from multiple perspectives to identify features that may be ignored in a single perspective.

[0050] The data collection module 100 collects historical multi-image data with the same structure as the real-time multi-image data from a public database.

[0051] The feature point detection module 200 is used to detect feature points on multiple historical image datasets. It constructs a Gaussian difference pyramid on each image in the dataset to detect extreme points and generates feature descriptors, specifically including:

[0052] For each image in the historical image dataset, multiple Gaussian blurred images are created using different σ values ​​to generate a scale space, which is the representation of the image at different resolutions. Adjacent Gaussian blurred images are subtracted to obtain a scale layer. This process is repeated to form a Gaussian difference pyramid, where σ is the standard deviation of the Gaussian blur, which is used to control the degree of Gaussian blur. A larger σ value will produce a stronger blur effect, thereby generating image representations at different scales.

[0053] At each scale level of the Difference of Gaussian pyramid, each pixel is compared with its surrounding pixels, and also with the corresponding pixels at the previous and next scale levels. If the pixel is a local extremum in this 3x3x3 cube (the current pixel and its 26 neighboring pixels), it is marked as a potential feature point.

[0054] Calculate the gradient magnitude and direction of the region surrounding the potential feature point, create a 360-degree orientation histogram, find the main peak of the histogram, and assign its direction to the potential feature point as the main direction.

[0055] A fixed-size neighborhood (e.g., 16x16 pixels) is selected around the feature point. This neighborhood is divided into multiple sub-regions (e.g., 4x4). A gradient direction histogram is calculated for each sub-region. The histogram information of all sub-regions is concatenated into a vector, usually 128-dimensional (4x4x8, where 8 is the number of direction bins). This vector is then normalized to generate a feature descriptor, which includes the corresponding feature point. The feature point includes its coordinate position, scale layer, and principal direction.

[0056] The feature matching unit 301 in the feature matching and image registration module 300 performs feature descriptor matching based on the feature descriptors generated for each image in the historical multi-image data, using the nearest neighbor distance ratio method, to generate feature point pairs corresponding to the matched feature descriptors. Specifically, this includes:

[0057] Based on the feature descriptors generated from each image in historical multi-image data, the nearest neighbor distance ratio method is used for feature descriptor matching. For each feature descriptor in one image, the nearest and second nearest neighbor feature descriptors are found in another image, and the ratio of the nearest neighbor distance to the second nearest neighbor distance is calculated. If the ratio is less than a preset ratio threshold (usually 0.7 or 0.8), it means that the pair of feature descriptors is successfully matched, and a matched pair of feature descriptors is generated. The determination of the nearest and second nearest neighbors is obtained by calculating the distance between feature descriptors in different images. The nearest neighbor represents the smallest distance, and the second nearest neighbor represents the smallest distance among all distances after removing the nearest neighbor.

[0058] For each pair of matching feature descriptors, extract their corresponding feature point information to generate feature point pairs, each pair containing corresponding feature points from the two images.

[0059] The image registration unit 302 in the feature matching and image registration module 300 calculates the transformation matrix between images based on the feature point pairs corresponding to the matched feature descriptors and the historical multiple image data of the corresponding matched image; applying the calculated transformation matrix, it aligns the historical multiple image data of the matched image to the same coordinate system to generate registered multiple image data, specifically including:

[0060] Based on the matched feature point pairs and the corresponding historical image data, a robust estimation method is used to calculate the transformation matrix between images. The minimum number of feature point pairs is randomly selected, where the minimum number of feature point pairs is determined according to the transformation model, which includes but is not limited to affine transformation, similarity transformation, and perspective transformation. Based on the minimum number of feature point pairs and the corresponding transformation model, the transformation matrix is ​​calculated. The error of all feature point pairs is calculated based on the transformation matrix, and the number of interior points that conform to the transformation model is counted. This step is repeated, and the transformation matrix with the most interior points is selected as the transformation matrix between images.

[0061] A transformation matrix is ​​applied to each matching image, and the image is geometrically transformed (such as translation, rotation, scaling, etc.) using the transformation matrix. The transformed image is then resampled to the target coordinate system, which is the smallest rectangular region that can contain all images. The target coordinate system is determined by calculating the average position and orientation of all transformed images. The set of images that are aligned to the same coordinate system is used as the registration data for multiple images.

[0062] The association information analysis module 400 utilizes a convolutional neural network algorithm to construct an association information model. It uses the feature point pairs corresponding to matched feature descriptors as the target features of the association information model and registers multiple image data as input features to construct the model. Specifically, this includes:

[0063] The registered multi-image data generated by the image registration unit 302 is used as the input features of the association information model, and the feature point pairs corresponding to the matched feature descriptors generated by the feature matching unit 301 are used as the target features of the association information model.

[0064] Design a convolutional neural network architecture that can process multiple image data for registration and output predicted feature point correspondences. The convolutional neural network structure includes an input layer, convolutional layers, pooling layers, and fully connected layers. The input layer receives multiple image data for registration; the convolutional layers extract local features from the images; the pooling layers reduce the spatial dimension of the feature maps to improve computational efficiency; and the fully connected layers synthesize high-level features to generate the final prediction.

[0065] Multiple registered image data are input into a convolutional neural network. Feature point pairs corresponding to the matched feature descriptors are used as supervision information. A loss function, such as mean squared error or cross entropy, is defined to measure the difference between the predicted feature point correspondence and the true feature point pair. Backpropagation algorithm and optimizer (such as stochastic gradient descent) are used to update the network parameters. The training process is iterated until the model converges or reaches the preset number of training rounds.

[0066] The constructed association information model is used to determine the correspondence between feature points in real-time multi-image data. By inputting real-time multi-image data into the constructed association information model, the model outputs the predicted correspondence between feature points.

[0067] The correlation information analysis module (400) also includes a few-shot learning unit and a real-time detection unit. The few-shot learning unit adopts a visual transformer network architecture and introduces a quantum correlation constraint mechanism in the multi-head attention mechanism. The attention weight distribution is dynamically adjusted through the quantum correlation matrix. The quantum correlation matrix is ​​obtained by calculating the intensity distribution of the two detectors. By synchronously collecting the light intensity distribution data of the two detectors, the average light intensity ensemble of each pixel is calculated. The quantum correlation matrix is ​​generated by the difference between the ensemble average of the product of the light intensity of the two detectors and the average of their respective independent ensembles.

[0068] The real-time detection unit includes an improved target detection network, whose detection head contains four parallel processing branches:

[0069] Spatial pyramid pooling branch is used for multi-scale feature extraction;

[0070] Channel attention branch, used to enhance effective feature channels;

[0071] Normalized distance regression branch is used to improve the localization accuracy of small targets;

[0072] The quantum correlation feature verification branch is used to screen detection results that conform to quantum statistical properties.

[0073] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A machine learning-based correlation imaging information extraction technology and parallel sampling system, characterized in that, The system includes a data collection module (100), a feature point detection module (200), a feature matching and image registration module (300), and a correlation information analysis module (400). Specifically: the data collection module (100) uses parallel sampling technology to collect real-time multi-image data; it also collects historical multi-image data with the same structure as the real-time multi-image data from a public database; the feature point detection module (200) detects feature points in the historical multi-image data by constructing a Gaussian difference pyramid on each image in the historical multi-image data to detect extreme points and generate feature descriptors; the feature matching and image registration module (300) uses the nearest neighbor distance ratio method based on the feature descriptors generated for each image in the historical multi-image data. The algorithm performs feature descriptor matching to generate feature point pairs corresponding to the matched feature descriptors; based on the feature point pairs corresponding to the matched feature descriptors and the historical multiple image data of the corresponding matched image, it calculates the transformation matrix between the images, and applies the calculated transformation matrix to align the historical multiple image data of the matched image to the same coordinate system to generate registered multiple image data; the association information analysis module (400) uses the convolutional neural network algorithm to construct the association information model, and uses the feature point pairs corresponding to the matched feature descriptors as the target features of the association information model, and the registered multiple image data as the input features of the association information model to construct the association information model; the constructed association information model is used to predict the correspondence of feature points in real-time multiple image data.

2. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 1, characterized in that, The data collection module (100) uses parallel sampling technology to collect real-time multi-image data. Parallel sampling technology is a method of acquiring data from multiple data sources and multiple sampling points at the same time. In image acquisition, multiple synchronized image sensors are used to capture different perspectives of the scene at the same time to generate image data from multiple perspectives.

3. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 1, characterized in that, The feature point detection module (200) constructs a Gaussian difference pyramid on each image in the multi-image data to detect extreme points and generate feature descriptors. Specifically, this includes: creating multiple Gaussian blurred images for each image in the historical multi-image data using different σ values ​​to generate a scale space, where the scale space is the representation of the image at different resolutions; subtracting adjacent Gaussian blurred images to obtain scale layers, repeating this process to form a Gaussian difference pyramid, where σ is the standard deviation of the Gaussian blur, used to control the degree of Gaussian blur; and comparing each pixel with its surrounding pixels in each scale layer of the Gaussian difference pyramid, while also comparing the pixel with the previous and next scale layers. If a pixel at a given layer location is a local extremum within the 3x3x3 cube, it is marked as a potential feature point. The gradient magnitude and direction of the region surrounding the potential feature point are calculated, and a 360-degree histogram of orientations is created. The dominant peak of the histogram is identified, and its direction is assigned to the potential feature point as the dominant orientation. A fixed-size neighborhood is selected around the feature point, and this neighborhood is divided into multiple sub-regions. A gradient orientation histogram is calculated for each sub-region. The histogram information from all sub-regions is concatenated into a vector, which is then normalized to generate a feature descriptor. The feature descriptor includes the corresponding feature point, which includes its coordinate position, scale layer, and dominant orientation.

4. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 3, characterized in that, The feature matching and image registration module (300) includes a feature matching unit (301). The process of generating feature point pairs corresponding to the matched feature descriptors by the feature matching unit (301) specifically includes: based on the feature descriptors generated for each image in historical multi-image data, the nearest neighbor distance ratio method is used to perform feature descriptor matching. For each feature descriptor in one image, the nearest neighbor and the second nearest neighbor feature descriptors are found in another image, and the ratio of the nearest neighbor distance to the second nearest neighbor distance is calculated. If the ratio is less than a preset ratio threshold, it indicates that the pair of feature descriptors is successfully matched, and a matched feature descriptor pair is generated. The determination of the nearest neighbor and the second nearest neighbor is obtained by calculating the distance between the feature descriptors of different images. The nearest neighbor represents the smallest distance, and the second nearest neighbor represents the smallest distance found again among all distances after removing the nearest neighbor. For each pair of matched feature descriptors, the corresponding feature point information is extracted to generate a feature point pair. Each pair contains corresponding feature points from the two images.

5. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 4, characterized in that, The feature matching and image registration module (300) includes an image registration unit (302). The image registration unit (302) calculates the transformation matrix between images based on the feature point pairs corresponding to the matched feature descriptors and the historical multiple image data of the corresponding matched images. Specifically, it calculates the transformation matrix between images using a robust estimation method based on the matched feature point pairs and the corresponding historical multiple image data. It randomly selects the minimum number of feature point pairs, where the minimum number of feature point pairs is determined according to the transformation model. The transformation model includes, but is not limited to, affine transformation, similarity transformation, and perspective transformation. Based on the minimum number of feature point pairs and the corresponding transformation model, it calculates the transformation matrix. It calculates the error of all feature point pairs based on the transformation matrix and counts the number of interior points that conform to the transformation model. It repeats this step and selects the transformation matrix with the most interior points as the transformation matrix between images.

6. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 5, characterized in that, The image registration unit (302) applies the calculated transformation matrix to align the historical multiple image data of the matching image to the same coordinate system and generate registered multiple image data. Specifically, it includes: applying the transformation matrix to each matching image, performing geometric transformation on the image using the transformation matrix, resampling the transformed image to the target coordinate system, wherein the target coordinate system is the smallest rectangular area that can contain all images, which is determined by calculating the average position and direction of all transformed images, and using the set of images that align all images to the same coordinate system as the registered multiple image data.

7. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 1, characterized in that, The association information analysis module (400) constructs an association information model using a convolutional neural network algorithm. It uses the feature point pairs corresponding to matched feature descriptors as the target features of the association information model and registers multiple image data as the input features. Specifically, this includes: using registered multiple image data generated by the image registration unit (302) as the input features of the association information model; using the feature point pairs corresponding to matched feature descriptors generated by the feature matching unit (301) as the target features of the association information model; and designing a convolutional neural network architecture capable of processing registered multiple image data and outputting predicted feature point correspondences. The convolutional neural network structure includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The system consists of an input layer that receives and registers multiple image data; a convolutional layer that extracts local features from the images; a pooling layer that reduces the spatial dimension of the feature maps; and a fully connected layer that synthesizes high-level features to generate the final prediction. The registered image data is input into the convolutional neural network, and the feature point pairs corresponding to the matched feature descriptors are used as supervision information. A loss function is defined to measure the difference between the predicted feature point correspondences and the true feature point pairs. Backpropagation and an optimizer are used to update the network parameters, and the training process is iterated until the model converges or reaches a preset number of training epochs. The constructed association information model is used to determine the feature point correspondences of real-time multiple image data. By inputting real-time multiple image data into the constructed association information model, the model outputs the predicted feature point correspondences.

8. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 1, characterized in that, The association information analysis module (400) includes a few-shot learning unit and a real-time detection unit: the few-shot learning unit adopts a visual transformer network architecture, introduces a quantum association constraint mechanism in the multi-head attention mechanism, and dynamically adjusts the attention weight distribution through the quantum association matrix; the real-time detection unit includes an improved target detection network, whose detection head includes four parallel processing branches: a spatial pyramid pooling branch for multi-scale feature extraction; and a channel attention branch for enhancing effective feature channels. Normalized distance regression branch is used to improve the localization accuracy of small targets; The quantum correlation feature verification branch is used to screen detection results that conform to quantum statistical properties.

9. The machine learning-based correlation imaging information extraction technology and parallel sampling system according to claim 8, characterized in that, The quantum correlation matrix is ​​obtained by calculating the intensity distribution of the dual detectors. The specific process includes: synchronously acquiring the light intensity distribution data of the two detectors; calculating the average light intensity ensemble of each pixel; and generating the quantum correlation matrix by the difference between the ensemble average of the product of the light intensities of the two detectors and the average of their respective independent ensembles.

Citation Information

Patent Citations

  • Fundus image feature point matching method based on improved feature descriptor and KNN search

    CN117975546A

  • Multiple Hypotheses Segmentation-Guided 3D Object Detection and Pose Estimation

    US20180144458A1