A system for improving resolution of optical measurements of large moving objects

Abstract

The application discloses a system for improving optical measurement resolution of a large moving object, comprising an optical device, a measurement space, an image acquisition module and an image processing module; the optical device comprises two complex cameras, which synchronously acquire target measurement region data and image; the measurement space is provided with a light source and a target measurement region, the light source is used for generating light rays in a specific wavelength range; the target measurement region is a large moving object to be measured and a motion range thereof; the image acquisition module is used for receiving photoelectric signals sent by an image sensor in real time and converting the photoelectric signals into M*N pixel pictures; the image processing module performs stereo vision calibration, three-dimensional reconstruction and data fusion on the M*N pixel pictures and outputs three-dimensional motion information of the large moving object. The application solves the problems that in the prior art, a measurement field of view and resolution are restricted to each other and three-dimensional high-speed measurement technology has limited resolution when measuring a large moving object.

Description

Technical Field

[0001] This invention belongs to the field of photogrammetry technology, specifically relating to a system for improving the optical measurement resolution of large moving objects. Background Technology

[0002] The technical term field of view (FOV) represents the maximum range that an optical system can observe, usually expressed in angles. A larger FOV means a larger observation range. The technical term resolution specifically refers to the resolution of an optical system, describing its ability to resolve object details. When the image sensor is a CCD or CMOS, if the maximum pixel count of the image sensor is M×N pixels, the image size captured using it is P×Q millimeters. 2 The resolution of an optical system is typically measured by its ability to resolve details in a specific direction, P / M (millimeters per pixel) and Q / N (millimeters per pixel). Higher P / M and Q / N values ​​generally indicate lower resolution. Traditional optical measurements show that a larger field of view results in lower resolution; the measurement field of view and resolution are mutually restrictive.

[0003] The technical term speckle pattern (SP) refers to random, small patterns with high black-and-white contrast. These patterns can be created on the surface of the object being measured using the texture itself, black-and-white paint, fluorescent solvents, etc., and are commonly used in 3D measurement technology for feature marking.

[0004] Large-scale motion refers to an object whose range of motion is at least S (S≥2) times its own size.

[0005] Traditional high-speed 3D measurement technology, developed based on binocular stereo vision and high-speed photogrammetry, is currently one of the better techniques for measuring large moving objects. However, due to the relatively small measurement target and the relatively large range of motion, measuring the detailed changes of the object itself during its motion remains a significant challenge.

[0006] To measure large moving objects, the measuring device must be able to dynamically track objects over a wide range while maintaining high resolution. However, traditional optical measurement techniques are limited by the trade-off between field of view and resolution. To simultaneously increase the measurement field of view and improve resolution, more high-speed cameras are needed, forming camera arrays, image sensor arrays, or lens arrays. Such arrays independently image each sub-field of view target within the measurement range onto its respective image sensor or a local area of ​​the image sensor. The sensors (local areas) are arranged side-by-side without superposition. This type of device improves resolution by increasing the number of pixels in the image sensor, and generally has a large size and high cost. Summary of the Invention

[0007] In view of the shortcomings of the prior art, the purpose of this invention is to provide a system for improving the optical measurement resolution of large moving objects, so as to solve the problems of mutual constraints between the measurement field of view and resolution and the limited resolution of three-dimensional high-speed measurement technology when measuring large moving objects.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] The present invention provides a system for improving the optical measurement resolution of large moving objects, comprising: an optical device, a measurement space, an image acquisition module, and an image processing module; wherein,

[0010] The optical device includes two complex cameras assembled at an angle of 15° to 45°, and the two complex cameras synchronously acquire data of the target measurement area and image it; each complex camera consists of a complex lens, an image sensor, and a light combining component, and performs independent imaging of the target measurement area.

[0011] A multi-lens system, consisting of multiple fixed-focus sub-lenses with different focal lengths and filters, is used to converge light into the target measurement area and fine-tune the image sharpness.

[0012] The light combining component is used to combine the light passing through the composite lens into the same image sensor after reflection and refraction, so that the image of each fixed-focus sub-lens occupies the total number of pixels of the image sensor.

[0013] An image sensor is used to receive and convert light signals. It has M×N photosensitive units distributed in an array, that is, M×N pixels. The maximum number of pixels in an image sensor is M×N.

[0014] The measurement space includes a light source and a target measurement area. The light source generates light within a specific wavelength range. The target measurement area is the large moving object to be measured and its range of motion. The large moving object is marked with randomly distributed speckle patterns.

[0015] The image acquisition module is used to receive photoelectric signals sent by the image sensor in real time and convert them into an M×N pixel image for output to the image processing module.

[0016] The image processing module is used to perform stereo vision calibration, 3D reconstruction and data fusion on the received M×N pixel image, and output the 3D motion information of the large moving object.

[0017] Furthermore, the field of view of the complex camera corresponds to the target measurement area, namely the large moving object to be measured and its range of motion; the target measurement area is divided into multiple sub-regions, and the field of view of each sub-lens of the complex lens is a sub-field of view, which corresponds to one sub-region of the target measurement area; the size of the large moving object to be measured should be less than or equal to the size of any sub-region; the edges of adjacent sub-regions partially overlap, and the overlapping part does not exceed 5% of the sub-region.

[0018] Furthermore, in the complex camera, a sub-field of view or a sub-region of the target measurement area is imaged as a sub-image in the image sensor, and the number of sub-images is S, where S≥2; each sub-image occupies all pixels of the image sensor, i.e., M×N pixels;

[0019] The image from the image sensor is a combined effect of superimposing the gray levels of each sub-image, meaning that the entire target measurement area is imaged onto the image sensor, and the final image is M×N pixels.

[0020] The actual reconstructed pixels of the target measurement area are approximately equal to M×N×S pixels.

[0021] Furthermore, the light source is a light source within a specific wavelength range, corresponding one-to-one with the speckle pattern marked on the surface of the large moving object under test, the filter of the multi-lens camera, and the light combining component (i.e., the optical characteristics of the speckle pattern marked on the surface of the large moving object under test, the filter, and the light combining component correspond to the wavelength changes of the light source); when the light source is in the off state, the measurement space is a dark field with no light; when the light source is in the on state, only the light from the speckle pattern on the surface of the large moving object can enter the multi-lens camera in the measurement space.

[0022] Among them, speckle allows the light entering the lens from the surface of a moving object to have different wavelength characteristics, which can be achieved by depending on the material of the object under test, the black / white coating or fluorescent solvent used to create the speckle.

[0023] Among them, a large moving object refers to a smaller moving object that moves within a relatively large range; the range of motion is S times the size of the moving object, where S≥2.

[0024] Furthermore, the image acquisition module converts the photoelectric signals sent by each image sensor into an M×N pixel image, and uses a hard-triggered synchronization controller to enable the two multiple cameras (two image sensors) to synchronously output their respective M×N pixel images in time.

[0025] Furthermore, the stereo vision calibration of the image processing module refers to the calibration of the internal parameters, external parameters, and relative relationships of the external parameters between the S sub-fields of view of the compound camera. Each sub-field of view is calibrated step-by-step according to Zhang Zhengyou's calibration method; the coordinates O of the i-th sub-field of view are denoted as... i -x i y i z iWith world coordinate system O w -x w y w z w The rotation and translation matrix [Rt] i-w .

[0026] The three-dimensional reconstruction of the image processing module is specifically as follows: based on the three-dimensional digital image correlation (3D-DIC) method, the image of M×N pixels is calculated, and the three-dimensional motion information of large moving objects in the sub-field of view is output.

[0027] The data fusion of the image processing module specifically involves: based on the motion characteristics of the three-dimensional data, using an encoding method to unify the data of S sub-regions to the same world coordinate system, and fusing the overlapping points between the sub-regions;

[0028] Encoding is based on the initial state and motion information of the large moving object. Without loss of generality, it is assumed that the sub-image of the initial state (time t0, encoded as #0) originates from the i-th sub-field of view, and the 3D coordinates of the object under this sub-field of view are reconstructed. When the object moves, if sub-image separation occurs, the sub-field of view needs to be transformed. When the sub-image moves forward, it is encoded as #1. The 3D coordinates of the separated part are reconstructed using the calibration result of the (i+1)-th sub-field of view, and the rotation and translation matrix [Rt] is used. (i+1)-w The 3D coordinates are transformed to a unified world coordinate system; when the sub-image moves negatively, it is encoded as #-1. The 3D coordinates of the separated part are reconstructed using the calibration results of the (i-1)th sub-viewfield, and the rotation and translation matrix [R t] is used. (i-1)-w Transform the 3D coordinates to a unified world coordinate system until the image separation state disappears and the encoding is reset to zero.

[0029] The beneficial effects of this invention are:

[0030] If the image sensor of the complex camera has M×N pixels and the complex camera has S sub-lenses (S≥2), in the dark field environment set by this invention, when measuring a small object (size Q, P / Q≥S) with a certain range of motion (target area size P), the reconstructed complex camera image resolution is approximately M×N×S pixels. If a traditional M×N resolution camera is used to directly image the range of motion P, the resolution of object Q on the image sensor is approximately only M×N×Q / P, which is nearly 1 / S lower than the resolution of the complex camera of this invention.

[0031] This invention adjusts the wavelength range of light entering the lens of a multi-camera system within the measurement environment to overlay images of the large moving object in the sub-field of view onto an image sensor. Since the dark environment remains bright after overlapping with a bright target, the motion of objects in the sub-field of view and even the entire measurement field of view is reconstructed through image separation of the moving object. This allows the measurement method to improve the measurement resolution of smaller targets in a relatively large measurement area without sacrificing or increasing the number of pixels in the image sensor. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the device structure of the present invention.

[0033] Figure 2 This is a schematic diagram of the light combining component structure in this invention.

[0034] Figure 3 This is a schematic diagram illustrating the relationship between field of view and resolution in this invention. Detailed Implementation

[0035] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0036] Reference Figure 1 As shown, a system for improving the optical measurement resolution of large moving objects according to the present invention includes: an optical device, a measurement space, an image acquisition module, and an image processing module; wherein,

[0037] The optical device includes two complex cameras assembled at an angle of 15° to 45°, and the two complex cameras synchronously acquire data of the target measurement area and image it; each complex camera consists of a complex lens, an image sensor, and a light combining component, and performs independent imaging of the target measurement area.

[0038] A multi-lens system, consisting of multiple fixed-focus sub-lenses with different focal lengths and filters, is used to converge light into the target measurement area and fine-tune the image sharpness.

[0039] The light combining component is used to combine the light passing through the composite lens into the same image sensor after reflection and refraction, so that the image of each fixed-focus sub-lens occupies the total number of pixels of the image sensor.

[0040] An image sensor is used to receive and convert light signals. It has M×N photosensitive units distributed in an array, that is, M×N pixels. The maximum number of pixels in an image sensor is M×N.

[0041] The measurement space includes a light source and a target measurement area. The light source generates light within a specific wavelength range. The target measurement area is the large moving object to be measured and its range of motion. The large moving object is marked with randomly distributed speckle patterns.

[0042] The image acquisition module is used to receive photoelectric signals sent by the image sensor in real time and convert them into an M×N pixel image for output to the image processing module.

[0043] The image processing module is used to perform stereo vision calibration, 3D reconstruction and data fusion on the received M×N pixel image, and output the 3D motion information of the large moving object.

[0044] Example, refer to Figures 1-3 As shown:

[0045] (1) The optical device includes:

[0046] Multiple cameras 1 and 2 are assembled at an angle of 15° to 45°. The two multiple cameras simultaneously observe and image the target measurement area. Each multiple camera consists of a multiple lens 10, an image sensor 20, and a light combining component 30, which are assembled to independently image the target measurement area.

[0047] Photosynthetic module 30 (e.g.) Figure 2 As shown, it includes multiple reflectors 301 and light combining prisms 302, which are assembled with the composite lens 10 and image sensor 20 through a dustproof adapter tube 303. The light passing through the composite lens 10 is reflected and refracted and then combined onto the same image sensor, so that the image of each fixed-focus sub-lens basically occupies the entire resolution of the image sensor.

[0048] The compound lens 10 includes three fixed-focus sub-lenses with different focal lengths and filters;

[0049] The image sensor 20 has a resolution of M×N pixels.

[0050] (2) The target measurement area 40 includes:

[0051] The fields of view of dual cameras 1 and 2 correspond to the target measurement area 40, which is the flight range of the insect being measured.

[0052] like Figure 3 As shown, the sub-lens fields of view of multiple cameras 1 and 2 are set to correspond one-to-one with a certain sub-region in the target measurement area, and the edges of adjacent sub-regions overlap by a very small part, that is, the overlapping part between sub-regions does not exceed 5% of the sub-regions. All sub-regions are continuously superimposed to form the target measurement area.

[0053] (3) The measurement space includes:

[0054] The measurement space contains only light source 50 and insects 401 marked with speckle patterns; there are no other light sources such as sunlight. When the light source is off, the entire measurement environment is dark, with no light. When the light source is on, only light from the surface of the object being measured can enter the complex camera within the entire measurement space.

[0055] The light source 50 is a light source with a specific wavelength range, corresponding one-to-one with the speckle pattern on the surface of the insect 401 marked with speckle, the filter used in the compound lens 10, and the light combining component 30. Specifically, there are two schemes, A and B:

[0056] Option A: The light source 50 is an ultraviolet lamp with a wavelength of 425nm. The speckle on the surface of the insect 401 marked by speckle is a fluorescent speckle. The fluorescent speckle emits visible light in the range of 440nm to 740nm under the illumination of the ultraviolet lamp. The light intensity is approximately the same after passing through the filter of the composite lens 10 and the light combining component 30, that is, the light intensity of the measured object reaching the image sensor after passing through different sub-regions of the target measurement area 40 is approximately the same.

[0057] Option B: The wavelength range of the light source 50 is 440nm to 740nm white light source. The speckle on the surface of the insect 401 (the measurement object) marked by speckle is black and white speckle. The black and white speckle diffusely reflects the white light source through the compound lens 10 and the light combining component 30, and the light intensity is approximately the same. That is, the light intensity of the measurement object reaching the image sensor through different sub-regions of the target measurement area 40 is approximately the same.

[0058] (4) Sub-image separation and resolution:

[0059] In this example, the three sub-fields of view of the compound lens 10 have three sub-images in the image sensor 20, and the resolution of each sub-image is the total resolution of the image sensor 20, i.e., M×N pixels.

[0060] The multi-camera in this example is used in a set measurement space, and the image acquisition module 60 synchronously acquires images from the two image sensors 20. The actual target measurement area 40 of each multi-camera is three times the size of a single sub-region. When the sub-region is in a dark field, the image grayscale value is approximately 0, and the resolution of each dark field sub-image is M×N. When the sub-image is a target speckle image, the image grayscale value is the speckle grayscale value f, and the resolution of the speckle sub-image is M×N.

[0061] The image from each image sensor 20 is ultimately a combined effect of the grayscale superposition of three sub-images. That is, the grayscale value of the final image originates from the grayscale superposition of an image with a resolution of M×N×3, f+0+0=f; therefore, the actual resolution of the target measurement area is approximately equivalent to M×N×3 pixels.

[0062] (5) Image processing:

[0063] According to Zhang Zhengyou's calibration method, the internal and external parameters of the three sub-fields of view of the multi-camera are calibrated to establish the stereoscopic visual relationship between sub-lens 1 and sub-image 1 of multi-camera 1 and sub-lens 1 and sub-image 1 of multi-camera 2; similarly, the stereoscopic visual relationship between sub-lens 2 and sub-image 2 of multi-camera 1 and sub-lens 2 and sub-lens 3 and sub-lens 3 of multi-camera 1 and sub-lens 3 and sub-image 3 of multi-camera 2 is established; the grayscale of the sub-image is determined by the sub-image separation and resolution in (4) above.

[0064] Then, based on the 3D Digital Image Correlation (3D-DIC) method, the 3D motion information of each of the three pairs of sub-images of multi-camera 1 and multi-camera 2 is reconstructed.

[0065] Finally, the data fusion based on the image processing module 70 is to unify the data of the three sub-regions to the same world coordinate system by encoding according to the motion characteristics of the three-dimensional data, and to fuse the overlapping points between the sub-regions.

[0066] Most image sensors have similar aspect ratios, i.e., M≈N. Figure 3 As shown, in the dark environment set in this example, a unidirectional large-motion / flying object (only along the M direction or only along the N direction) is measured. The resolution of each sub-image is M×N pixels, and the resolution of the reconstructed multi-camera image is approximately M×N×3. If a traditional M×N resolution camera is used to image the flight area, the resolution of the object on the image sensor is approximately only M×N×1 / 9.

[0067] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A system for improving the optical measurement resolution of large moving objects, characterized in that, include: Optical device, measurement space, image acquisition module and image processing module; The optical device includes two complex cameras assembled at an angle of 15° to 45°, and the two complex cameras synchronously acquire data of the target measurement area and image it; each complex camera consists of a complex lens, an image sensor, and a light combining component, and performs independent imaging of the target measurement area. A multi-lens system, consisting of multiple fixed-focus sub-lenses with different focal lengths and filters, is used to converge light into the target measurement area and fine-tune the image sharpness. The light combining component is used to combine the light passing through the composite lens into the same image sensor after reflection and refraction, so that the image of each fixed-focus sub-lens occupies the total number of pixels of the image sensor. An image sensor is used to receive and convert light signals. It has M×N photosensitive units distributed in an array, that is, M×N pixels. The maximum number of pixels in an image sensor is M×N. The measurement space includes a light source and a target measurement area. The light source generates light within a specific wavelength range. The target measurement area is the large moving object to be measured and its range of motion. The large moving object is marked with randomly distributed speckle patterns. The image acquisition module is used to receive photoelectric signals sent by the image sensor in real time and convert them into an M×N pixel image for output to the image processing module. The image processing module is used to perform stereo vision calibration, 3D reconstruction and data fusion on the received M×N pixel image, and output the 3D motion information of the large moving object.

2. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The field of view of the complex camera corresponds to the target measurement area, namely the large moving object to be measured and its range of motion; the target measurement area is divided into multiple sub-regions, and the field of view of each sub-lens of the complex camera is a sub-field of view, which corresponds to one of the sub-regions in the target measurement area; the size of the large moving object to be measured should be less than or equal to the size of any sub-region; the edges of adjacent sub-regions partially overlap, and the overlapping part does not exceed 5% of the sub-region.

3. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, In the complex camera, a sub-field of view or a sub-region of the target measurement area is imaged as a sub-image in the image sensor, and the number of sub-images is S, where S≥2; each sub-image occupies all the pixels of the image sensor, i.e., M×N pixels; The image from the image sensor is a combined effect of superimposing the gray levels of each sub-image, meaning that the entire target measurement area is imaged onto the image sensor, and the final image is M×N pixels. The actual reconstructed pixels of the target measurement area are approximately equal to M×N×S pixels.

4. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The light source is a light source with a specific wavelength range, which corresponds one-to-one with the speckle pattern marked on the surface of the large moving object to be measured, the filter of the multi-lens camera, and the light combining component; when the light source is in the off state, the measurement space is a dark field with no light; when the light source is in the on state, only the light from the speckle pattern on the surface of the large moving object can enter the multi-lens camera in the measurement space.

5. The system for improving the optical measurement resolution of large moving objects according to claim 4, characterized in that, Speckles cause the light entering the lens from the surface of a moving object to have different wavelength characteristics, which is achieved by the material of the object being tested and the black / white paint or fluorescent solvent used to create the speckle.

6. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The image acquisition module converts the photoelectric signals sent by each image sensor into an M×N pixel image and uses a hard-triggered synchronization controller to enable the two multiple cameras to synchronously output their respective M×N pixel images in time.

7. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The stereo vision calibration of the image processing module refers to the calibration of the internal parameters, external parameters, and relative relationships of the external parameters between the S sub-fields of view of the camera. Each sub-field of view is calibrated step-by-step according to Zhang Zhengyou's calibration method; the coordinates O of the i-th sub-field of view are denoted as follows: i -x i y i z i With world coordinate system O w -x w y w z w The rotation and translation matrix [Rt] i-w .

8. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The 3D reconstruction of the image processing module specifically involves: calculating the 3D motion information of large moving objects in the sub-field of view based on 3D digital image correlation methods.

9. The system for improving the optical measurement resolution of large moving objects according to claim 1, characterized in that, The data fusion of the image processing module specifically involves: based on the motion characteristics of the three-dimensional data, using an encoding method to unify the data of S sub-regions to the same world coordinate system, and fusing the overlapping points between the sub-regions; Encoding is based on the initial state and motion information of the large moving object. Without loss of generality, it is assumed that the sub-image of the initial state originates from the i-th sub-viewfield, and the 3D coordinates of the object under that sub-viewfield are reconstructed. When the object moves, if sub-image separation occurs, the sub-viewfield needs to be transformed. When the sub-image moves forward, it is encoded as #1. The 3D coordinates of the separated part are reconstructed using the calibration result of the (i+1)-th sub-viewfield, and the rotation and translation matrix [Rt] is used. (i+1)-w The 3D coordinates are transformed to a unified world coordinate system; when the sub-image moves negatively, it is encoded as #-1. The 3D coordinates of the separated part are reconstructed using the calibration results of the (i-1)th sub-viewfield, and the rotation and translation matrix [R t] is used. (i-1)-w Transform the 3D coordinates to a unified world coordinate system until the image separation state disappears and the encoding is reset to zero.

Images