Oblique scanning imaging method and system, electronic equipment and storage medium
By rotating the linear array camera to achieve oblique scanning, the problem of imaging depth of field loss when the linear array camera is improved is solved, and high-resolution images are acquired while maintaining the imaging depth of field and image quality.
Patent Information
- Application Number
- CN202510149272.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-11
- Publication Date
- 2025-05-16
AI Technical Summary
In the process of improving image resolution, existing linear array cameras tend to lose the imaging depth of field, resulting in a decrease in image quality of objects with height differences to the surface.
By rotating the linear array camera, it can be switched from the positive scan state to the oblique scan state, and adjust the scanning angle to achieve oblique scan, thereby significantly improving the image resolution.
Without losing the imaging depth of field and introducing additional noise, the resolution of line scan images can be effectively improved and image quality can be improved.
Smart Images
Figure CN120017762A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine vision technology, and more specifically to an oblique scanning imaging method, system, electronic equipment and storage medium. Background Art
[0002] In industrial production, linear array cameras are often used to scan and image products that are transported at a constant speed on the assembly line. Unlike the matrix field of view of the area array camera, the linear array camera only images in a narrow field of view, which is called the scanning line of the linear array camera. When the object to be measured passes through the scanning line of the linear array camera at a constant speed, the linear array camera continuously exposes and accumulates several scanning line data at high speed, and obtains a complete scanned image of the object to be measured after splicing. In the above process, the scanning line must be perpendicular to the direction of the moving speed of the object to be measured, otherwise mis-cutting distortion will occur in the scanned image.
[0003] Linear array cameras achieve scanning imaging through continuous exposure line by line. Increasing the line scanning frequency can improve the resolution of the scanned image in the speed direction; increasing the scanning point density within the scanning line can improve the resolution of the scanned image perpendicular to the speed direction. However, this method of improving image resolution may cause loss of imaging depth of field. When capturing images of objects with height differences on their surfaces, for example, when the target object is a conveyor belt, the scanned surface of the target object is the groove surface of the conveyor belt facing the linear array camera and the lens, and the imaging depth of field should cover the range from the bottom of the conveyor belt groove to the high edge of the outer edge of the conveyor belt; when the target object is a rack, lead screw, etc., the scanned plane is a toothed curved surface, and the imaging depth of field should cover the curved surface range from the top of the tooth to the bottom of the tooth; when the target object is the arc surface on one side of a cylinder, the imaging depth of field should cover a certain height range from the top edge of the arc. In the above cases, the loss of imaging depth of field will lead to a decrease in the quality of the acquired image.
[0004] In view of this, the present invention is proposed. Summary of the invention
[0005] The present invention is proposed in view of the above problems. According to one aspect of the present invention, an oblique scanning imaging method is provided, which is applied to an oblique scanning imaging system, wherein the oblique scanning imaging system is used to collect images of a target object transported along a straight line; the oblique scanning imaging system comprises a linear array camera, wherein the linear array camera is located above the target object, and its imaging axis faces the scanned surface of the target object, and the scanned surface of the target object has a height difference; The method comprises: Adjust the line array camera to rotate clockwise by a preset angle from a normal scanning state to put the line array camera in an oblique scanning state, wherein when the line array camera is in the normal scanning state, the scanning sector of the line array camera is perpendicular to the moving speed direction of the target object; When the line array camera is in an oblique scanning state, the line array camera is used to collect an initial image of the target object; the initial image includes multiple lines of pixel data collected in sequence by the line array camera, each line of pixel data in the multiple lines of pixel data includes multiple pixel points, and the time interval between any two adjacent lines of pixel data collected by the line array camera in the multiple lines of pixel data is a preset time length; Establishing an interpolation point array covering the pixel point array of the initial image on the initial image, wherein the interpolation point array is distributed along a direction orthogonal to a horizontal axis and a vertical axis of a spatial coordinate system, and the pixel point array includes the plurality of lines of pixel data; Determining a pixel value of each interpolation point in the interpolation point array based on the pixel point array of the initial image; Based on the interpolation point array, a target image is generated.
[0006] Exemplarily, before establishing the interpolation point array covering the pixel point array of the initial image on the initial image, the method further comprises: Projecting each pixel point of the initial image from the image coordinate system to the space coordinate system to obtain the coordinates of each pixel point in the initial image in the space coordinate system; Among them, the horizontal coordinate of each interpolation point in the interpolation point array is within the horizontal coordinate range of the pixel point array of the initial image, and the vertical coordinate of each interpolation point in the interpolation point array is within the vertical coordinate range of the pixel point array of the initial image.
[0007] Exemplarily, the distance between two adjacent interpolation points in the interpolation point array is a preset spacing, and the preset spacing is a smaller value between a row spacing and a column spacing in the pixel point array; or, The distance between two adjacent interpolation points in the same row in the interpolation point array is the row spacing in the pixel point array; the distance between two adjacent interpolation points in the same column in the interpolation point array is the column spacing in the pixel point array; The row spacing is the spacing between any two adjacent pixels in any row of pixel data in the pixel array; the column spacing is the spacing between any two adjacent pixels in any column of pixel data in the pixel array.
[0008] Exemplarily, the determining the pixel value of each interpolation point in the interpolation point array based on the pixel point array of the initial image includes: For any interpolation point in the interpolation point array, a pixel value of the interpolation point is determined based on pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array.
[0009] Exemplarily, when the distance between two adjacent interpolation points in the same row in the interpolation point array is the same as the row spacing in the pixel point array, for any interpolation point in the interpolation point array, the at least preset number of pixel points are pixel points in the pixel point array that are in the same column as the interpolation point; When the distance between two adjacent interpolation points located in the same row in the interpolation point array is different from the row spacing in the pixel point array, for any interpolation point in the interpolation point array, the at least preset number of pixel points are at least preset number of pixel points in the pixel point array that are closest in spatial distance to the interpolation point.
[0010] Exemplarily, when the at least preset number of pixel points are pixel points in the pixel array that are in the same column as the interpolation point, the distance between each pixel point in the at least preset number of pixel points and the interpolation point satisfies the following condition: ≤ dy' ,in, represents the distance between any pixel point among the at least preset number of pixel points and the interpolation point, dy' represents the distance between two adjacent interpolation points in the same column in the interpolation point array; When the at least preset number of pixel points are at least preset number of pixel points in the pixel array that are closest in spatial distance to the interpolation point, the distance between each pixel point in the at least preset number of pixel points and the interpolation point satisfies the following condition: ,in, represents the distance between any pixel point among the at least preset number of pixel points and the interpolation point, dy' represents the distance between two adjacent interpolation points in the same column in the interpolation point array; dx' Represents the distance between two adjacent interpolation points in the same row in the interpolation point array.
[0011] Exemplarily, determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array includes: The pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array are solved by using the nearest neighbor interpolation method to determine the pixel value of the interpolation point.
[0012] Exemplarily, before determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array, the method further includes: Determine the number of pixel points adjacent to the interpolation point in the pixel point array; The step of determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array is performed when the number of pixel points adjacent to the interpolation point in the pixel point array is not 0; When the number of pixel points adjacent to the interpolation point in the pixel point array is 0, the pixel value of the interpolation point is determined to be an invalid value.
[0013] According to another aspect of the present invention, there is provided an oblique scanning imaging system, which is used to capture images of a target object transported along a straight line; the oblique scanning imaging system comprises a linear array camera and a controller, the linear array camera being located above the target object, the linear array camera facing the scanned surface of the target object along an imaging axis, the scanned surface of the target object having a height difference; the controller is used to implement the method of any of the above examples.
[0014] According to another aspect of the present invention, there is provided an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the computer program to implement the above method.
[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program / instruction, and the computer program / instruction implements the above method when executed by a processor.
[0016] In the above technical solution, the spacing between adjacent scanning points in the horizontal direction of the spatial coordinate system is cleverly reduced by rotating the line array camera, thereby significantly improving the resolution of the captured image. This method does not require adjusting the original parameters of the line array camera, such as the scanning frequency, aperture size, and sensor gain. It only relies on changing the scanning angle of the line array camera to achieve oblique scanning, thereby achieving the purpose of resolution enhancement. Thanks to this method, the solution can effectively improve the resolution of line scan images under the premise of existing lighting technology and sensor photosensitivity, without losing imaging depth of field, and without introducing additional noise.
[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0019] Figure 1 A schematic flow chart showing an oblique scanning imaging method according to an embodiment of the present invention; Figure 2 An axial schematic diagram showing the relative positions of a line array camera and a target object according to an embodiment of the present invention; Figure 3 Show Figure 2 A three-dimensional schematic diagram of the relative positions shown; Figure 4 Show Figure 2 A top view of the target object; Figure 5 A schematic diagram showing a linear array camera in an oblique scanning state according to an embodiment of the present invention; Figure 6 Show Figure 5 A top view of the target object; Figure 7 A schematic diagram showing the relative position of a line array camera and a target object when the line array camera is in an oblique scanning state according to another embodiment of the present invention; Figure 8 A schematic diagram showing a plurality of lines of pixel data in an oblique scanning state according to an embodiment of the present invention; Fig. 9 A schematic diagram showing an interpolation point array according to an embodiment of the present invention; Fig.10 A schematic diagram showing an interpolation point array according to another embodiment of the present invention; Fig.11 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.
[0021] According to the above description, the related art usually improves the image resolution by increasing the line scanning frequency and the scanning point density within the scanning line. However, the inventors found through research that the line array camera realizes scanning imaging by continuous exposure line by line, which is affected by the exposure time of the camera and the aperture parameters of the optical lens, resulting in an exclusive relationship between the imaging depth of field and the imaging resolution performance. Specifically, the depth of field formula can be expressed as:
[0022] in, Indicates the imaging depth of field; F Indicates the aperture value. The smaller the aperture value, the larger the aperture diameter. Represents the resolvable spatial scale, that is, the distance between two adjacent scanning points in the scanning line of the linear array camera. The smaller it is, the higher the image resolution; f Indicates the focal length value; L Indicates the object distance value.
[0023] It can be seen from the above depth of field formula that, when other parameters remain unchanged, the higher the resolution, the smaller the imaging depth of field. Therefore, the increase in scanning point density will lead to a loss of imaging depth of field.
[0024] In addition, due to the inverse relationship between the scanning frequency and the exposure time, increasing the scanning frequency often causes the problem of underexposure. Since natural light conditions cannot increase the lighting power, even if a high-brightness LED light source is used, the limitations of installation and heat dissipation conditions will also restrict the lighting power. Although increasing the sensor gain can solve the problem of underexposure, this method will increase the noise in the image signal. In view of this, in the related art, while increasing the scanning frequency, the aperture is usually increased to compensate for the exposure. However, according to the above-mentioned depth of field formula, when other parameters remain unchanged, the aperture increases (that is, the aperture value decreases), the imaging depth of field decreases. In other words, increasing the aperture to enhance the exposure will inevitably sacrifice a certain imaging depth of field.
[0025] In the actual application scenarios of linear array cameras, high depth of field imaging is often indispensable. Especially when capturing images of objects with height differences on the surface, ensuring sufficient depth of field is the key to maintaining imaging quality. However, as mentioned above, in the pursuit of improved image resolution, we often face the challenge of reduced depth of field. In other words, there is currently no technology that can effectively improve the resolution of line scan images without sacrificing imaging depth of field or introducing additional noise under existing lighting technology and sensor sensitivity. In view of this, the present invention provides an oblique scanning imaging method, system, electronic device and storage medium. This solution can effectively improve the resolution of line scan images without sacrificing imaging depth of field or introducing additional noise under existing lighting technology and sensor sensitivity. The method, system, electronic device and storage medium are described in detail below.
[0026] According to one aspect of an embodiment of the present invention, an oblique scanning imaging method is provided. The method is applied to an oblique scanning imaging system, which is used to collect images of a target object transported along a straight line; the oblique scanning imaging system includes a linear array camera, which is located above the target object. The linear array camera faces a scanned surface of the target object along an imaging axis, and the scanned surface of the target object has a height difference.
[0027] In this article, the scanned surface of the target object has a height difference, that is, the scanned surface of the target object captured by the line array camera has different heights. In some embodiments, the target object may be an object with a height difference on the surface. For example, the target object may be an object with a curved surface to be scanned. Specifically, for example, it may be a rod, a pipe, a special-shaped shell, etc. conveyed along the axial direction. The scanned surfaces of these objects are all curved surfaces with an arc, and there is a height difference between the center and the edge of the curved surface. In other embodiments, the number of target objects may also be multiple, and the multiple target objects have different heights (in this case, the scanned surfaces of the target objects have a height difference, that is, the scanned surfaces of different target objects have different heights). Multiple target objects may be conveyed along a straight line and pass under the line array camera in sequence. In this case, the line array camera is used to image multiple target objects at one time. For example, the multiple target objects may be multiple packaging boxes with different heights conveyed in sequence. Since the heights of the packaging boxes are different, it is necessary to make the imaging depth of the line array camera cover the lowest height and the highest height of these packaging boxes to ensure that the imaging of packaging boxes of different heights has better imaging quality.
[0028] In this article, the target object is transported along a straight line. The transport method includes but is not limited to a conveyor belt, an assembly line, etc., which will not be described in detail. It can be understood that the number of target objects in this article can be multiple. For example, multiple objects are placed on a conveyor belt, and a linear array camera can capture an image of any of the multiple objects when the object passes below.
[0029] In this article, the lens of the line scan camera can be selected according to actual needs. In a specific embodiment, the lens of the line scan camera can be a fixed-focus low-distortion lens.
[0030] Figure 1 FIG. 2 is a schematic flow chart of an oblique scanning imaging method according to an embodiment of the present invention. Figure 1 As shown, the method may include step S110, step S120, step S130, step S140 and step S150.
[0031] In step S110, the line array camera is controlled to rotate clockwise by a preset angle from a normal scanning state to put the line array camera in an oblique scanning state, wherein when the line array camera is in the normal scanning state, the scanning sector of the line array camera is perpendicular to the moving speed direction of the target object.
[0032] In this example, when the linear scan camera is in the positive scanning state, the imaging depth of field of the linear scan camera covers the scanned surface of the target object at different heights. For example, when the target object is a conveyor belt, the scanned surface of the target object is the groove surface of the conveyor belt facing the linear scan camera and the lens, and the imaging depth of field should cover the range from the bottom of the conveyor belt groove to the high edge of the conveyor belt outer edge. When the target object is a rack, lead screw, etc., the scanned plane is a toothed curved surface, and the imaging depth of field should cover the curved surface range from the top of the tooth to the bottom of the tooth.
[0033] In some embodiments, when the line scan camera is in a forward scanning state, the object distance plane of the line scan camera is located at the middle of the range of the scanned surface. It can be understood that since the parameters of the line scan camera and the distance from the scanned surface do not change during the rotation process, the imaging axis is still along Z The object distance plane is still located in the middle of the scanned surface range, and the imaging depth of field still covers the scanned surface at different heights of the target object. Its world coordinate system is consistent with that in the positive scanning state.
[0034] Figure 2 An axial schematic diagram showing the relative positions of a line array camera and a target object according to an embodiment of the present invention. Figure 3 Show Figure 2 A three-dimensional schematic diagram of the relative positions shown. Figure 4 Show Figure 2 In this embodiment, the linear array camera and its lens can be firstly arranged along the imaging axis. Z The scanning sector is installed facing the scanned surface of the target object. XZ and speed (Fig. v ) direction. At the same time, the lens object distance can be adjusted so that the object distance plane (i.e. Figure 2-4 The dotted box in the figure is located on the plane) XYThe camera is located in the middle of the scanned surface and a suitable aperture is selected so that the imaging depth of field of the linear array camera covers the scanned surface at different heights of the target object, that is, the imaging depth of field covers the farthest to the nearest range of the scanned surface. Figure 2 In the embodiment shown, the world coordinate system XYZ of Y Axis and speed v Parallel and reverse. The scanning lines of the line array camera are located in the scanning sector XZ Distance plane XY The intersection of X superior.
[0035] In this example, the linear array camera can be controlled to switch from a normal scanning state to an oblique scanning state through rotation. In some embodiments, the linear array camera can be controlled to rotate manually. In other embodiments, the oblique scanning imaging system further includes a rotating motor connected to the linear array camera. The rotating motor can rotate clockwise by a preset angle so that the linear array camera and its lens rotate around the imaging axis. Z Rotate to a preset angle. The preset angle can be selected as needed and will not be described in detail.
[0036] Figure 5 A schematic diagram showing a line array camera in an oblique scanning state according to an embodiment of the present invention. Figure 6 Show Figure 5 The top view of the target object in the image. Figure 5 , 6 As shown in the figure, when the line scan camera is in the oblique scanning state, the scanning line of the line scan camera P and X The angle between the axes is the preset angle θ .
[0037] Figure 7 FIG. 2 is a schematic diagram showing the relative position of a linear array camera and a target object when the linear array camera is in an oblique scanning state according to another embodiment of the present invention. Figure 7 In the illustrated embodiment, the scanned surface of the target object is a plane, and the scanned surfaces of the plurality of target objects have different heights.
[0038] In step S120, when the line array camera is in an oblique scanning state, an initial image of the target object is collected by the line array camera; the initial image includes multiple lines of pixel data collected sequentially by the line array camera, each line of pixel data in the multiple lines of pixel data includes multiple pixel points, and the time interval between any two adjacent lines of pixel data collected by the line array camera in the multiple lines of pixel data is a preset time length.
[0039] In this example, the line array camera can obtain multiple lines of pixel data by continuous exposure line by line, and the multiple lines of pixel data are combined to obtain an initial image.
[0040] It can be understood that compared with the normal scanning state, the image resolution obtained in the oblique scanning state is higher. Figure 8 A schematic diagram showing multiple lines of pixel data in an oblique scanning state according to an embodiment of the present invention is shown. Figure 8 As shown, the scanning point spacing is δ In the positive scanning state, each scanning point is X The projection spacing in the direction is δ In the oblique scanning state, each scanning point is X The projection spacing in the direction (that is, the distance between any two adjacent pixels in any row of pixel data) is . The oblique scanning state is X The resolution of the direction is improved to .
[0041] exist Figure 8 middle, dy Indicates that adjacent scanning points are in the oblique scanning state. Y The projection spacing in the direction, i.e. Y Directional resolution . u is the horizontal axis direction of the image coordinate system, t is the longitudinal direction of the image coordinate system. In this embodiment, the scanned surface moves along with the target object at a speed v along Y Axis conveyor, line scan camera at time intervals dt = dy / v Scan. Y Directional spacing can realize the switching of under-scan mode, equal-scan mode and over-scan mode.
[0042] Optionally, the scanning mode selected by the linear array camera when acquiring the initial image of the target object may be any one of an underscan mode, an equal scan mode, and an overscan mode. Y Direction scanning distance ,at this time Y The directional resolution is low, and there are gaps between the exposure areas. It is suitable for situations where the Y-direction resolution requirement is low or the data flow is limited; in overscan mode Y Direction scanning distance ,at this time Y High directional resolution and overlapping exposure areas, suitable for situations where details need to be scanned without omission and data flow is high; in equal scanning mode Y Direction scanning distance ,at this time YModerate directional resolution, continuous exposure area, and moderate flow rate. In actual scenarios, you can select a suitable scanning mode as needed. For example, in scenarios where the speed direction resolution requirement is not high and the data bandwidth is limited, you can select the underscan mode; in scenarios where the speed direction resolution requirement is high and the data bandwidth is sufficient, you can select the overscan mode; in scenarios where the speed direction resolution requirement is moderate and the data bandwidth is moderate, you can use the equal scan mode.
[0043] In step S130, an interpolation point array covering the pixel point array of the initial image is established on the initial image. The interpolation point array is distributed along the orthogonal directions of the horizontal axis and the vertical axis of the spatial coordinate system. The pixel point array includes multiple lines of pixel data.
[0044] As described above, each row of pixel data includes a plurality of pixel points. The pixel points included in the plurality of rows of pixel data may form a pixel point array. The pixel point array may be in the form of a matrix Im Indicates that the matrix column number corresponds to the scanning point number 1 ~m , the matrix row number corresponds to the scanning time of each row 1 ~n .
[0045] After the initial image is obtained, an interpolation point array covering the pixel point array of the initial image can be established on the initial image. The interpolation point array is along the horizontal axis of the spatial coordinate system (i.e. X axis) and the vertical axis (i.e. Y The interpolation points are distributed in the orthogonal directions of the axis. X Axis direction upper edge X The axis is arranged in the direction of extension, and Y Axis direction upper edge Y The axis extension directions are arranged sequentially.
[0046] In step S140 , based on the pixel point array of the initial image, the pixel value of each interpolation point in the interpolation point array is determined.
[0047] After establishing the interpolation point array, the pixel value of each interpolation point in the interpolation point array can be determined according to the pixel value of the pixel point in the pixel point array. For example, for any interpolation point, the pixel value of the pixel point closest to the interpolation point can be used as the pixel value of the interpolation point. For another example, the pixel value of the interpolation point can be determined using the pixel values of multiple pixel points adjacent to the interpolation point.
[0048] In step S150, a target image is generated based on the interpolation point array.
[0049] The inventors have discovered that when the scanning line of the linear array camera is not perpendicular to the moving direction of the target object, the image captured by the linear array camera will produce shear distortion. In the solution of this example, the initial image captured under the oblique scanning state is corrected for shear distortion by using an interpolation point array. Specifically, after obtaining the pixel of each interpolation point, the target image can be generated according to the interpolation point array of known pixel values. Since the interpolation point array is distributed along the orthogonal directions of the horizontal axis and the vertical axis of the spatial coordinate system, there is no shear distortion at each interpolation point in the interpolation point array. Therefore, the interpolation point array of known pixel values is the image after the shear distortion correction, that is, the target image.
[0050] The above technical solution cleverly reduces the spacing between adjacent scanning points in the horizontal direction of the spatial coordinate system by rotating the line array camera, thereby significantly improving the resolution of the acquired image. This method does not require adjusting the original parameters of the line array camera, such as the scanning frequency, aperture size, and sensor gain. It only relies on changing the scanning angle of the line array camera to achieve oblique scanning, thereby achieving the purpose of resolution enhancement. Thanks to this method, the solution can effectively improve the resolution of line scan images under the premise of existing lighting technology and sensor photosensitivity, without losing imaging depth of field, and without introducing additional noise.
[0051] Exemplarily, before establishing an interpolation point array covering the pixel point array of the initial image on the initial image, the method also includes: projecting each pixel point of the initial image from the image coordinate system to the spatial coordinate system to obtain the coordinates of each pixel point in the initial image in the spatial coordinate system; wherein the horizontal coordinate of each interpolation point in the interpolation point array is within the horizontal coordinate range of the pixel point array of the initial image, and the vertical coordinate of each interpolation point in the interpolation point array is within the vertical coordinate range of the pixel point array of the initial image.
[0052] It can be understood that the coordinates of the pixel points in the acquired initial image are image coordinates.
[0053] In such Figure 8 In the embodiment shown, the image horizontal coordinate corresponding to the imaging axis can be measured by calibration as follows: u c Optionally, projecting each pixel point of the initial image from the image coordinate system to the space coordinate system to obtain the coordinates of each pixel point in the initial image in the space coordinate system includes: Each pixel point included in the first row of pixel data in the multiple rows of pixel data is projected to the spatial coordinate system by the following formula to obtain the static coordinates of each pixel point in the spatial coordinate system [ y 1 , x ]:
[0054] The pixel points included in each line of pixel data except the first line in the multiple lines of pixel data are projected to the spatial coordinate system by the following formula to obtain the static coordinates of the pixel points included in each line of pixel data except the first line in the spatial coordinate system [ y, x ]:
[0055] in, δ Indicates the distance between two adjacent scanning points included in the scanning line of the line array camera, that is, the scanning point spacing; is the preset angle; is the horizontal coordinate of the pixel point in the image coordinate system; is the image abscissa corresponding to the imaging axis; The preset duration.
[0056] After obtaining the coordinates of each pixel in the pixel array in the spatial coordinate system, the scanning point array [ P ], which is the image matrix during scanning Im Each pixel point in the spatial coordinate system XY The projection point on the object distance plane can also be regarded as the projection point on the object distance plane. Figure 8 As shown, each point in the array P t,u The pixel value is the matrix value Im t,u , the image coordinates are [ t , u ], the spatial coordinates are [ y, x ].
[0057] Next, an interpolation point array covering the pixel point array can be established in the spatial coordinate system [ Q ] range. In this example, the horizontal coordinate of each interpolation point in the interpolation point array is within the horizontal coordinate range of the pixel point array of the initial image, and the vertical coordinate of each interpolation point in the interpolation point array is within the vertical coordinate range of the pixel point array of the initial image. That is:
[0058] in, , are the scanning point arrays [ P ]Central coordinates y The minimum and maximum values of , is the scanning point array[ P ]Central coordinates x The minimum and maximum values of dx' is the interpolation point array[ Q ]along X The spacing of the directions, dy'is the interpolation point array[ Q ]along Y Direction spacing.
[0059] The above scheme can facilitate the establishment of an interpolation point array by determining the coordinates of the pixel points in the spatial coordinate system, thereby helping to more accurately correct the shear distortion of the initial image.
[0060] Exemplarily, the distance between two adjacent interpolation points in the interpolation point array is a preset spacing, and the preset spacing is the smaller value of the row spacing and the column spacing in the pixel point array; or, the distance between two adjacent interpolation points in the same row in the interpolation point array is the row spacing in the pixel point array; the distance between two adjacent interpolation points in the same column in the interpolation point array is the column spacing in the pixel point array; wherein the row spacing is the spacing between any two adjacent pixel points in any row of pixel data in the pixel point array; and the column spacing is the spacing between any two adjacent pixel points in any column of pixel data in the pixel point array.
[0061] Optionally, the distance between two adjacent interpolation points in the interpolation point array is a preset spacing, and the preset spacing is the smaller value of the row spacing and the column spacing in the pixel point array. dx' = dy' =min( dx, dy ), at this time, the target image formed by the interpolation points is the same shape as the actual image without distortion. This method of obtaining interpolation points can be called a distortion-free difference strategy. The interpolation point array is as follows Fig. 9 This scheme can obtain a target image with optimal resolution while keeping the image shape distortion-free (no shear distortion, and no stretching / compression distortion), which helps to improve the image quality of the obtained image.
[0062] Optionally, the distance between two adjacent interpolation points in the same row in the interpolation point array is the row spacing in the pixel point array; the distance between two adjacent interpolation points in the same column in the interpolation point array is the column spacing in the pixel point array. That is, dx' = dx , dy' = dy .like Fig.10As shown, at this time, the interpolation point array is aligned column by column with the pixel point array of the initial image (i.e., the amount of data in each row is the same), and the row interval is the same (i.e., the number of rows acquired per unit time is the same). It can be understood that the scan data flow is the product of the row data volume and the number of rows acquired per unit time. Therefore, in this embodiment, the invalid pixels in the hypotenuse part are excluded, and the number of valid interpolation points of the interpolated image is the same as the number of pixels in the initial image. This way of obtaining the number of interpolation points can be called an equal flow difference strategy. This scheme can obtain an interpolated image with the same flow as the original scanned image, and the interpolated image can better correct the shear distortion of the oblique scanned initial image, thereby helping to improve the image quality of the target image finally obtained.
[0063] Exemplarily, based on the pixel array of the initial image, the pixel value of each interpolation point in the interpolation point array is determined, including: for any interpolation point in the interpolation point array, based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel array, the pixel value of the interpolation point is determined. The preset number may be a determined number, for example, it may be 1, 2, 3, 4, etc. The preset number may also be an uncertain number, for example, the preset number corresponding to each interpolation point may be different. The preset number of pixel points corresponding to each interpolation point may be all pixel points whose distance from the interpolation point does not exceed the preset distance. This scheme can determine the pixel value of each interpolation point more accurately, thereby helping to accurately correct image distortion and improve the image quality of the generated target image.
[0064] Exemplarily, when the distance between two adjacent interpolation points located in the same row in the interpolation point array is the same as the row spacing in the pixel point array, for any interpolation point in the interpolation point array, at least a preset number of pixel points are pixel points in the pixel point array that are in the same column as the interpolation point; when the distance between two adjacent interpolation points located in the same row in the interpolation point array is different from the row spacing in the pixel point array, for any interpolation point in the interpolation point array, at least a preset number of pixel points are at least a preset number of pixel points in the pixel point array that are closest in spatial distance to the interpolation point.
[0065] For example, if dx > dy , when the distortion-free interpolation strategy is adopted, the distance between two adjacent interpolation points in the same row in the interpolation point array is dx' = dy .like Fig. 9 As shown, at this time, at least a preset number of pixel points (4 in the figure) that are closest to the interpolation point in space can be used to determine the pixel value of the interpolation point. When the equal flow difference strategy is adopted, the distance between two adjacent interpolation points in the same row in the interpolation point array is dx' = dx .like Fig.10As shown, at this time, the pixel value of the interpolation point can be determined by using at least a preset number of pixel points (2 in the figure) in the same column as the interpolation point.
[0066] The above technical solution can determine the pixel value of each interpolation point more accurately, thereby helping to accurately correct image distortion and improve the image quality of the generated target image.
[0067] Exemplarily, when at least a preset number of pixel points are pixel points in the pixel array that are in the same column as the interpolation point, the distance between each pixel point in at least the preset number of pixel points and the interpolation point satisfies the following condition: ≤ dy' ,in, represents the distance between any pixel point among at least a preset number of pixels and the interpolation point, dy' represents the distance between two adjacent interpolation points in the same column in the interpolation point array; when at least a preset number of pixels are at least a preset number of pixels in the pixel array that are closest in spatial distance to the interpolation point, the distance between each pixel in at least the preset number of pixels and the interpolation point satisfies the following conditions: ,in, represents the distance between any pixel point among at least a preset number of pixels and the interpolation point, dy' Represents the distance between two adjacent interpolation points in the same column in the interpolation point array; dx' Represents the distance between two adjacent interpolation points in the same row of the interpolation point array.
[0068] In the above embodiment where the preset number of pixel points corresponding to each interpolation point may be all pixel points whose distance from the interpolation point does not exceed the preset distance, when at least the preset number of pixel points are pixel points in the pixel array that are in the same column as the interpolation point and whose distance from the interpolation point is not more than the preset distance, at least the preset number of pixel points are pixel points in the same column as the interpolation point and whose distance from the interpolation point is not more than the preset distance. ≤ dy' When at least a preset number of pixels are at least a preset number of pixels in the pixel array that are closest to the interpolation point in spatial distance, at least a preset number of pixels are at least a preset number of pixels in the pixel array that are closest to the interpolation point in spatial distance. All pixels of .
[0069] The above technical solution can relatively accurately determine the pixel points used to determine the pixel value of each interpolation point. By using the pixel points determined by the solution, the pixel value of each interpolation point can be relatively accurately determined.
[0070] Exemplarily, the pixel value of the interpolation point is determined based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array, including: using the nearest neighbor interpolation method to solve the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array to determine the pixel value of the interpolation point.
[0071] In some embodiments, the nearest neighbor interpolation method is used to solve the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array, including solving the pixel value of each interpolation point by the following formula:
[0072] in, is the pixel value of the interpolation point, k is at least a preset number of pixels, is the coordinates of at least a preset number of pixel points in the spatial coordinate system, is the pixel value of at least a preset number of pixels, is the interpolation function.
[0073] In the above technical solution, the nearest neighbor interpolation method can be used to solve the pixel value of each interpolation point, which is simple to calculate and highly efficient.
[0074] Exemplarily, before determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array, the method also includes: determining the number of pixel points adjacent to the interpolation point in the pixel point array; wherein, the step of determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array is performed when the number of pixel points adjacent to the interpolation point in the pixel point array is not 0; wherein, when the number of pixel points adjacent to the interpolation point in the pixel point array is 0, the pixel value of the interpolation point is determined to be an invalid value.
[0075] In the solution of this example, the number of pixels adjacent to the interpolation point in the pixel array can be determined first. For example, when at least a preset number of pixels are pixels in the same column as the interpolation point in the pixel array, it can be determined that the distance between each interpolation point satisfies the condition ≤ dy' When at least a preset number of pixels are at least a preset number of pixels in the pixel array that are closest to the interpolation point in spatial distance, it can be determined that the distance between each interpolation point satisfies the condition The number of interpolation points. When the number of interpolation points is 0, the pixel value of the interpolation point cannot be determined by the pixel values of the neighboring pixels. In this case, the pixel value of the interpolation point can be determined to be an invalid value. For example, the pixel value of the interpolation point can be 0 or Null.
[0076] The above scheme determines the number of neighboring pixels of each interpolation point in advance, and can directly determine the pixel value of the interpolation point as an invalid value when the number of neighboring pixels is 0, thereby avoiding the execution of meaningless operations and improving imaging efficiency.
[0077] According to another aspect of an embodiment of the present invention, an oblique scanning imaging system is provided, which is used to capture images of a target object transported along a straight line; the oblique scanning imaging system includes a linear array camera and a controller, the linear array camera is located above the target object, the linear array camera faces a scanned surface of the target object along an imaging axis, and the scanned surface of the target object has a height difference; the controller is used to implement the method of any of the above embodiments.
[0078] According to yet another aspect of the embodiments of the present invention, an electronic device is provided. Fig.11 FIG. 2 shows a schematic block diagram of an electronic device according to an embodiment of the present invention. Fig.11 As shown, the electronic device 1100 includes: a processor 1110 and a memory 1120. The memory 1120 stores a computer program, and the processor 1110 is used to execute the computer program to implement the above method.
[0079] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided. The storage medium stores a computer program / instruction, and the computer program / instruction implements the above method when executed by a processor. The storage medium may include, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
[0080] A person skilled in the art can easily understand the implementation structure, working principle and beneficial effects of the oblique scanning imaging system, electronic device and computer-readable storage medium by reading the above method. For the sake of brevity, no further description is given here.
[0081] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.
[0082] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0083] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
[0084] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0085] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.
[0086] Those skilled in the art will understand that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.
[0087] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.
[0088] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the oblique scanning imaging system and electronic device according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0089] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.
[0090] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. An oblique scanning imaging method, characterized in that: Applied to an oblique scanning imaging system, the oblique scanning imaging system is used to collect an image of a target object transported along a straight line; the oblique scanning imaging system comprises a line array camera, the line array camera is located above the target object, and its imaging axis faces the scanned surface of the target object, and the scanned surface of the target object has a height difference; The method comprises: Adjust the line array camera to rotate clockwise by a preset angle from a normal scanning state to put the line array camera in an oblique scanning state, wherein when the line array camera is in the normal scanning state, the scanning sector of the line array camera is perpendicular to the moving speed direction of the target object; When the line array camera is in an oblique scanning state, the line array camera is used to collect an initial image of the target object; the initial image includes multiple lines of pixel data collected in sequence by the line array camera, each line of pixel data in the multiple lines of pixel data includes multiple pixel points, and the time interval between any two adjacent lines of pixel data collected by the line array camera in the multiple lines of pixel data is a preset time length; Establishing an interpolation point array covering the pixel point array of the initial image on the initial image, wherein the interpolation point array is distributed along a direction orthogonal to a horizontal axis and a vertical axis of a spatial coordinate system, and the pixel point array includes the plurality of lines of pixel data; Determining a pixel value of each interpolation point in the interpolation point array based on the pixel point array of the initial image; Based on the interpolation point array, a target image is generated.
2. The oblique scanning imaging method according to claim 1, characterized in that: Before establishing an interpolation point array covering the pixel point array of the initial image on the initial image, the method further comprises: Projecting each pixel point of the initial image from the image coordinate system to the space coordinate system to obtain the coordinates of each pixel point in the initial image in the space coordinate system; Among them, the horizontal coordinate of each interpolation point in the interpolation point array is within the horizontal coordinate range of the pixel point array of the initial image, and the vertical coordinate of each interpolation point in the interpolation point array is within the vertical coordinate range of the pixel point array of the initial image.
3. The oblique scanning imaging method according to claim 1, characterized in that: The distances between two adjacent interpolation points in the interpolation point array are both preset intervals, and the preset intervals are the smaller value of the row interval and the column interval in the pixel point array; or, The distance between two adjacent interpolation points in the same row in the interpolation point array is the row spacing in the pixel point array; the distance between two adjacent interpolation points in the same column in the interpolation point array is the column spacing in the pixel point array; The row spacing is the spacing between any two adjacent pixels in any row of pixel data in the pixel array; the column spacing is the spacing between any two adjacent pixels in any column of pixel data in the pixel array.
4. The oblique scanning imaging method according to any one of claims 1 to 3, characterized in that: The step of determining the pixel value of each interpolation point in the interpolation point array based on the pixel point array of the initial image comprises: For any interpolation point in the interpolation point array, a pixel value of the interpolation point is determined based on pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array.
5. The oblique scanning imaging method according to claim 4, characterized in that: When the distance between two adjacent interpolation points in the same row in the interpolation point array is the same as the row spacing in the pixel point array, for any interpolation point in the interpolation point array, the at least preset number of pixel points are pixel points in the pixel point array that are in the same column as the interpolation point; When the distance between two adjacent interpolation points in the same row in the interpolation point array is different from the row spacing in the pixel point array, for any interpolation point in the interpolation point array, the at least preset number of pixel points are at least preset number of pixel points in the pixel point array that are closest in spatial distance to the interpolation point; Preferably, when the at least preset number of pixel points are pixel points in the pixel array that are in the same column as the interpolation point, the distance between each pixel point in the at least preset number of pixel points and the interpolation point satisfies the following condition: ≤ dy' ,in, represents the distance between any pixel point among the at least preset number of pixel points and the interpolation point, dy' represents the distance between two adjacent interpolation points in the same column in the interpolation point array; When the at least preset number of pixel points are at least preset number of pixel points in the pixel array that are closest in spatial distance to the interpolation point, the distance between each pixel point in the at least preset number of pixel points and the interpolation point satisfies the following condition: ,in, represents the distance between any pixel point among the at least preset number of pixel points and the interpolation point, dy' represents the distance between two adjacent interpolation points in the same column in the interpolation point array; dx' Represents the distance between two adjacent interpolation points in the same row in the interpolation point array.
6. The oblique scanning imaging method according to claim 4, characterized in that: The step of determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array comprises: The pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array are solved by using the nearest neighbor interpolation method to determine the pixel value of the interpolation point.
7. The oblique scanning imaging method according to claim 4, characterized in that: Before determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array, the method further includes: Determine the number of pixel points adjacent to the interpolation point in the pixel point array; The step of determining the pixel value of the interpolation point based on the pixel values of at least a preset number of pixel points adjacent to the interpolation point in the pixel point array is performed when the number of pixel points adjacent to the interpolation point in the pixel point array is not 0; When the number of pixel points adjacent to the interpolation point in the pixel point array is 0, the pixel value of the interpolation point is determined to be an invalid value.
8. An oblique scanning imaging system, characterized in that: The oblique scanning imaging system is used to capture images of a target object transported along a straight line; the oblique scanning imaging system includes a linear array camera and a controller, the linear array camera is located above the target object, the linear array camera faces the scanned surface of the target object along an imaging axis, and the scanned surface of the target object has a height difference; the controller is used to implement the method described in any one of claims 1 to 7.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.