Image processing method, device, equipment and computer readable storage medium

Abstract

The present invention relates to the field of photography, and discloses an image processing method, device, equipment and computer-readable storage medium. The method comprises: obtaining a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, and a straight line for height difference correction is provided on the calibration plate; grayscale processing is performed on the left-eye image and the right-eye image to obtain a left-eye grayscale image and a right-eye grayscale image; determining the position of the straight line in the left-eye grayscale image and the right-eye grayscale image, and calculating the height difference between the left-eye grayscale image and the right-eye grayscale image according to the position; and setting a cropping rule and a splicing rule based on the left-eye image and the right-eye image according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting. The present invention can solve the misalignment problem of binocular cameras during image splicing and improve the visual effect of captured images.

Description

Technical Field

[0001] The present invention relates to the field of camera technology, and in particular to an image processing method, device, equipment and a computer-readable storage medium. Background Art

[0002] With the popularity of cameras, people's demand for monitoring range is gradually increasing. If you want to increase the monitoring range, you can generally use a wide-angle lens to increase the monitoring angle of view, but the wide-angle lens will cause great distortion of the image around it; another method is to use a binocular camera, which uses two lenses with ordinary viewing angles (moderate focal length, small distortion) and two image acquisition systems to splice the two images collected left and right to increase the monitoring field of view.

[0003] For binocular cameras, due to production process and installation errors, the left and right image acquisition systems are usually not strictly in the same horizontal position. In other words, even if the camera is installed horizontally, the line connecting the optical centers of the two cameras is not horizontal. This will cause the left and right images to be misaligned in height when the images are stitched, thereby affecting the visual effect of the captured image. Summary of the invention

[0004] The main purpose of the present invention is to propose an image processing method, device, equipment and computer-readable storage medium, aiming to solve the misalignment problem of binocular cameras during image stitching and improve the visual effect of captured images.

[0005] A first aspect of the present invention provides an image processing method, the image processing method comprising:

[0006] Acquire a left-eye image and a right-eye image obtained by photographing a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate;

[0007] Performing grayscale processing on the left-eye image and the right-eye image to obtain a left-eye grayscale image and a right-eye grayscale image;

[0008] Determine the position of the straight line in the left eye grayscale image and the right eye grayscale image, and calculate the height difference between the left eye grayscale image and the right eye grayscale image according to the position;

[0009] A cropping rule and a splicing rule based on the left-eye image and the right-eye image are set according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting.

[0010] Optionally, the step of determining the position of the straight line in the left-eye grayscale image and the right-eye grayscale image, and calculating the height difference between the left-eye grayscale image and the right-eye grayscale image according to the position includes:

[0011] Determine a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0012] An absolute value of a difference between the number of rows of the first pixel row and the number of rows of the second pixel row is calculated, and the absolute value is used as a height difference between the left-eye grayscale image and the right-eye grayscale image.

[0013] Optionally, the step of performing grayscale processing on the left-eye image and the right-eye image to obtain the left-eye grayscale image and the right-eye grayscale image includes:

[0014] When the left-eye image and the right-eye image are in YUV encoding format, obtaining Y component data of the left-eye image and the right-eye image;

[0015] A left-eye grayscale image and a right-eye grayscale image are correspondingly generated according to the Y component data.

[0016] Optionally, the step of determining a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image comprises:

[0017] Determine a first stitching position in the left-eye grayscale image, and determine a second stitching position in the right-eye grayscale image;

[0018] For each row of pixels in the left-eye grayscale image, starting from the first stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of m Y component values, and comparing the magnitudes of the sums of the m Y components to determine the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image;

[0019] For each row of pixels in the right-eye grayscale image, starting from the second stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of n Y component values, and comparing the magnitudes of the sums of the n Y components to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0020] Wherein, m is the number of pixel rows of the left-eye grayscale image, and n is the number of pixel rows of the right-eye grayscale image.

[0021] Optionally, the step of setting a cropping rule and a splicing rule based on the left-eye image and the right-eye image according to the height difference includes:

[0022] Comparing the heights of the left-eye grayscale image and the right-eye grayscale image;

[0023] If the height of the left grayscale image is greater than the height of the right grayscale image, the cropping rule is set as follows: the last g rows of the left grayscale image are cropped, and the first g rows of the right grayscale image are cropped; if the height of the left grayscale image is less than the height of the right grayscale image, the cropping rule is set as follows: the first g rows of the left grayscale image are cropped, and the last g rows of the right grayscale image are cropped; wherein g is the height difference;

[0024] The stitching rule is set as follows: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and adjusting the stitched image back to a preset pixel height using a preset image interpolation algorithm.

[0025] A second aspect of the present invention provides an image processing device, the image processing device comprising:

[0026] An acquisition module, used for acquiring a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate;

[0027] A processing module, used for performing grayscale processing on the left-eye image and the right-eye image to obtain a left-eye grayscale image and a right-eye grayscale image;

[0028] A determination module, used for determining the position of the straight line in the left eye grayscale image and the right eye grayscale image, and calculating the height difference between the left eye grayscale image and the right eye grayscale image according to the position;

[0029] A setting module is used to set a cropping rule and a splicing rule based on the left-eye image and the right-eye image according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting.

[0030] Optionally, the determining module is further used for:

[0031] Determine a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0032] An absolute value of a difference between the number of rows of the first pixel row and the number of rows of the second pixel row is calculated, and the absolute value is used as a height difference between the left-eye grayscale image and the right-eye grayscale image.

[0033] Optionally, the processing module is further used for:

[0034] When the left-eye image and the right-eye image are in YUV encoding format, obtaining Y component data of the left-eye image and the right-eye image;

[0035] A left-eye grayscale image and a right-eye grayscale image are correspondingly generated according to the Y component data.

[0036] Optionally, the determining module is further used for:

[0037] Determine a first stitching position in the left-eye grayscale image, and determine a second stitching position in the right-eye grayscale image;

[0038] For each row of pixels in the left-eye grayscale image, starting from the first stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of m Y component values, and comparing the magnitudes of the sums of the m Y components to determine the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image;

[0039] For each row of pixels in the right-eye grayscale image, starting from the second stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of n Y component values, and comparing the magnitudes of the sums of the n Y components to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0040] Wherein, m is the number of pixel rows of the left-eye grayscale image, and n is the number of pixel rows of the right-eye grayscale image.

[0041] Optionally, the setting module is further used to:

[0042] Comparing the heights of the left-eye grayscale image and the right-eye grayscale image;

[0043] If the height of the left grayscale image is greater than the height of the right grayscale image, the cropping rule is set as follows: the last g rows of the left grayscale image are cropped, and the first g rows of the right grayscale image are cropped; if the height of the left grayscale image is less than the height of the right grayscale image, the cropping rule is set as follows: the first g rows of the left grayscale image are cropped, and the last g rows of the right grayscale image are cropped; wherein g is the height difference;

[0044] The stitching rule is set as follows: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and adjusting the stitched image back to a preset pixel height using a preset image interpolation algorithm.

[0045] A third aspect of the present invention provides an image processing device, comprising: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory so that the image processing device executes the above-mentioned image processing method.

[0046] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned image processing method.

[0047] The present invention obtains a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate; grayscale processing is performed on the left-eye image and the right-eye image to obtain a left-eye grayscale image and a right-eye grayscale image; the position of the straight line in the left-eye grayscale image and the right-eye grayscale image is determined, and the height difference between the left-eye grayscale image and the right-eye grayscale image is calculated according to the position; and a cropping rule and a splicing rule based on the left-eye image and the right-eye image are set according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting. The present invention pre-calculates the height difference of the binocular images by a calibration method, and sets corresponding cropping rules and splicing rules based on the height difference to perform height difference correction on the binocular images obtained by subsequent shooting, thereby solving the misalignment problem of the binocular camera during image splicing and improving the visual effect of the shot image. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of an embodiment of an image processing method of the present invention;

[0049] Figure 2 A schematic diagram of a calibration plate in an embodiment of the image processing method of the present invention;

[0050] Figure 3 A schematic diagram of splicing a left-eye image and a right-eye image in an embodiment of the image processing method of the present invention;

[0051] Figure 4 This is a schematic diagram of a left-eye grayscale image and a right-eye grayscale image before cropping in an embodiment of the image processing method of the present invention;

[0052] Figure 5 It is a schematic diagram of cutting and then splicing the left-eye grayscale image and the right-eye grayscale image in an embodiment of the image processing method of the present invention;

[0053] Figure 6 A schematic diagram of modules of an embodiment of an image processing device of the present invention;

[0054] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The embodiments of the present invention provide an image processing method, apparatus, device and computer-readable storage medium, which pre-calculate the height difference of binocular images by calibration, and set corresponding cropping rules and stitching rules based on the height difference to correct the height difference of binocular images obtained by subsequent shooting, thereby solving the misalignment problem of binocular cameras during image stitching and improving the visual effect of the captured images.

[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0057] For ease of understanding, the specific process of the image processing method embodiment of the present invention is described below.

[0058] Reference Figure 1 , Figure 1 The following is a flow chart of an embodiment of an image processing method of the present invention, wherein the method comprises:

[0059] Step 101, obtaining a left-eye image and a right-eye image obtained by photographing a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate;

[0060] The execution subject of the image processing method of this embodiment may be a binocular camera, or a server or other terminal device with image processing function. The following description will be made by taking a binocular camera as an example.

[0061] In this embodiment, a calibration plate is firstly provided. The calibration plate may be rectangular and has a straight line for height difference correction. For the convenience of shooting, the straight line may be provided in the middle of the calibration plate, such as on the horizontal axis of symmetry.

[0062] Reference Figure 2 , Figure 2The figure is a schematic diagram of a calibration plate in an embodiment of the image processing method of the present invention. To reduce color interference and facilitate straight line recognition, the calibration plate can be a white plate with a horizontal black straight line drawn in the middle, and the straight line should have smooth edges without misalignment. The size of the calibration plate and the length and width of the straight line can be flexibly set according to the shooting range of the binocular camera. For example, the length of the calibration plate can be set to 2.2m and the height to 1.6m, the length of the straight line is slightly shorter than the length of the calibration plate, and the width of the straight line is 3-6cm.

[0063] During specific implementation, the calibration board can be placed vertically first, and then the binocular camera can be placed at a certain distance in front of the calibration board, such as about 5m, and the lens height of the binocular camera can be made basically consistent with the height of the straight line on the calibration board. The binocular camera shoots the calibration board to obtain the left eye image and the right eye image, in which the straight line is roughly in the middle of the left and right images.

[0064] Reference Figure 3 , Figure 3 The figure is a schematic diagram of the splicing of the left and right images in the embodiment of the image processing method of the present invention. Since the left and right image acquisition systems of the binocular camera are usually not strictly at the same horizontal position, when the images are spliced, the straight lines in the left and right images will be misaligned at the splicing point.

[0065] Step 102, performing grayscale processing on the left eye image and the right eye image to obtain a left eye grayscale image and a right eye grayscale image;

[0066] In this step, the binocular camera performs grayscale processing on the captured left and right images, and obtains the left and right grayscale images accordingly. The grayscale processing algorithm can be flexibly set according to different image encoding formats. For example, if the captured image is in YUV format, the chrominance components U and V can be stripped, and the remaining brightness component Y can generate the corresponding grayscale image. If the captured image is in RGB format, the average value of the three components R, G, and B can be calculated, and then the corresponding grayscale image is generated according to the average value.

[0067] Step 103, determining the position of the straight line in the left eye grayscale image and the right eye grayscale image, and calculating the height difference between the left eye grayscale image and the right eye grayscale image according to the position;

[0068] After obtaining the left eye grayscale image and the right eye grayscale image, the binocular camera further determines the position of the straight line in the left eye grayscale image and the right eye grayscale image, thereby calculating the height difference between the two images. Since the straight line itself has a certain width in the vertical direction, the height difference between the two images can be calculated based on the upper edge of the straight line in the two images, or the height difference between the two images can be calculated based on the lower edge of the straight line in the two images, which can be flexibly set during implementation.

[0069] When calculating the height difference between two images based on the upper edge of the straight line in the two images, step 103 above may further include: determining the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and the second pixel row where the upper edge of the straight line is located in the right-eye grayscale image; calculating the absolute value of the difference in the number of rows between the first pixel row and the second pixel row, and taking the absolute value as the height difference between the left-eye grayscale image and the right-eye grayscale image.

[0070] Here, assume that the pixel row where the upper edge of the straight line is located in the left-eye grayscale image is the i-th row, and the pixel row where the upper edge of the straight line is located in the right-eye grayscale image is the j-th row. Calculate the value of g = |i - j|, and this g value is the height difference between the left-eye grayscale image and the right-eye grayscale image.

[0071] Step 104, setting a cropping rule and a stitching rule based on the left-eye image and the right-eye image according to the height difference, where the cropping rule and the stitching rule are used to correct the height difference of the binocular images obtained by subsequent shooting.

[0072] In this step, the step of setting a cropping rule and a stitching rule based on the left-eye image and the right-eye image according to the height difference may further include: comparing the heights of the left-eye grayscale image and the right-eye grayscale image; if the height of the left-eye grayscale image is greater than the height of the right-eye grayscale image, then set the cropping rule as: cropping the last g rows of the left-eye grayscale image and cropping the first g rows of the right-eye grayscale image; if the height of the left-eye grayscale image is less than the height of the right-eye grayscale image, then set the cropping rule as: cropping the first g rows of the left-eye grayscale image and cropping the last g rows of the right-eye grayscale image; where g is the height difference; set the stitching rule as: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and using a preset image interpolation algorithm to adjust the stitched image back to the preset pixel height.

[0073] Specifically, the heights of the left-eye image and the right-eye image can be judged by comparing the sizes of i and j. If i < j, it means that the height of the left-eye grayscale image is greater than the height of the right-eye grayscale image. At this time, set the cropping rule as: cropping the last g rows of the left-eye grayscale image and cropping the first g rows of the right-eye grayscale image. On the contrary, if i > j, it means that the height of the left-eye grayscale image is less than the height of the right-eye grayscale image. At this time, set the cropping rule as: cropping the first g rows of the left-eye grayscale image and cropping the last g rows of the right-eye grayscale image.

[0074] The stitching rule corresponding to the above cropping rule is: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and then using a preset image interpolation algorithm to adjust the stitched image back to the original pixel height, wherein the image interpolation algorithm can refer to the existing image interpolation algorithm, which will not be described in detail here.

[0075] Reference Figure 4 , Figure 4 The figure is a schematic diagram of the left eye grayscale image and the right eye grayscale image before cropping in the embodiment of the image processing method of the present invention. The left eye grayscale image and the right eye grayscale image are represented in the form of a pixel array, and each data in the pixel array corresponds to the grayscale value of the pixel point at that location. The pixel with a grayscale value of 253 represents the calibration plate background, and the pixel with a grayscale value of 12 represents the straight line. It can be seen from the position of the straight line in the left eye grayscale image and the right eye grayscale image that the height of the left eye grayscale image before cropping is greater than the height of the right eye grayscale image, and the height difference is 4 lines. At this time, the last 4 lines of the left eye grayscale image need to be cropped, and the first 4 lines of the right eye grayscale image need to be cropped.

[0076] Reference Figure 5 , Figure 5 The schematic diagram of cropping and splicing the left grayscale image and the right grayscale image in the embodiment of the image processing method of the present invention. Through cropping, the left grayscale image and the right grayscale image are each reduced by g rows of pixels, and the cropped left grayscale image and the right grayscale image are spliced ​​left and right, and then the image interpolation algorithm is used to pull the image back to the original pixel height.

[0077] When the binocular camera captures images subsequently, the above-mentioned cropping rules and stitching rules are called to process the captured images, so that height difference correction can be achieved. It should be noted that compared with the prior art method of post-processing and correcting captured images through complex artificial intelligence algorithms, this embodiment pre-calculates the height difference of the binocular images and sets corresponding cropping rules and stitching rules based on the height difference to perform height difference correction on the binocular images obtained by subsequent capture. This method has very low requirements on the computing performance of the device. While solving the problem of image misalignment, it will not cause problems affecting the image acquisition frame rate and image quality due to insufficient device performance.

[0078] In this embodiment, a left-eye image and a right-eye image are obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate; the left-eye image and the right-eye image are gray-scaled to obtain a left-eye gray-scale image and a right-eye gray-scale image; the position of the straight line in the left-eye gray-scale image and the right-eye gray-scale image is determined, and the height difference between the left-eye gray-scale image and the right-eye gray-scale image is calculated according to the position; and a cropping rule and a splicing rule based on the left-eye image and the right-eye image are set according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting. In this embodiment, the height difference of the binocular images is pre-calculated by a calibration method, and corresponding cropping rules and splicing rules are set based on the height difference to perform height difference correction on the binocular images obtained by subsequent shooting, thereby solving the misalignment problem of the binocular camera during image splicing and improving the visual effect of the captured image.

[0079] Furthermore, based on the first embodiment of the image processing method in the present invention, a second embodiment of the image processing method in the present invention is proposed.

[0080] In this embodiment, the above step 102 may include: when the left eye image and the right eye image are in YUV encoding format, obtaining Y component data of the left eye image and the right eye image; and generating a left eye grayscale image and a right eye grayscale image according to the Y component data.

[0081] In the YUV encoding format, Y is the brightness component, U and V are the chrominance components, and each pixel corresponds to a Y value. The value range of Y is 0-255. The smaller the value, the darker the grayscale, and the larger the value, the brighter the grayscale. The binocular camera extracts the Y component data from the YUV encoding of the left image and the right image, and the image generated according to the Y component data is a grayscale image stripped of color.

[0082] By stripping off the chromaticity component, not only can the amount of height difference calculation be reduced, but also the interference of color on the judgment of straight lines can be reduced. Because there are many interfering colors in the actual test and calibration environment, it is not an ideal situation. This also reduces the requirements of the calibration environment and the difficulty of calibration.

[0083] Furthermore, the step of determining the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and the second pixel row where the upper edge of the straight line is located in the right-eye grayscale image may include:

[0084] A first stitching position is determined in the left-eye grayscale image, and a second stitching position is determined in the right-eye grayscale image; for each row of pixels in the left-eye grayscale image, starting from the first stitching position, Y component data of a preset number of row pixel points are summed to obtain a sum of m Y component values, and the sizes of the sums of the m Y components are compared to determine a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image; for each row of pixels in the right-eye grayscale image, starting from the second stitching position, Y component data of a preset number of row pixel points are summed to obtain a sum of n Y component values, and the sizes of the sums of the n Y components are compared to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image; wherein m is the number of pixel rows of the left-eye grayscale image, and n is the number of pixel rows of the right-eye grayscale image.

[0085] Specifically, for each row of pixels in the left grayscale image, starting from the first splicing position, the Y component data of a preset number of row pixels are summed to obtain the sum of m Y component values ​​X1 to X2. m , and then use the bubble algorithm to find X1~X m The first minimum value X in i , where 1≤i≤m, i means that the upper edge of the straight line is located in the i-th row in the left grayscale image; for each row of pixels in the right grayscale image, starting from the second splicing position, the Y component data of a preset number of row pixels are summed to obtain the sum of n Y component values ​​X1~X n , and then use the bubble algorithm to find X1~X n The first minimum value X in j , where 1≤j≤n, j means that the upper edge of the line is located in the jth row in the left grayscale image.

[0086] It should be noted that the first stitching position and the second stitching position are the stitching positions when the left-eye grayscale image and the right-eye grayscale image are stitched, which can be represented by pixel columns. For example, when the left-eye grayscale image and the right-eye grayscale image are stitched left and right, the last column of the left-eye grayscale image can be determined as the first stitching position, and the first column of the right-eye grayscale image can be determined as the second stitching position; in addition, the number of pixels participating in the summation in each row of pixels can be flexibly set, such as 10, 20, etc.

[0087] In this embodiment, by summing the Y component data of the preset row of pixels and comparing the summation results to determine the position of the upper edge of the straight line in the left-eye grayscale image and the right-eye grayscale image, the position of the straight line can be quickly determined and the accuracy of the height difference calculation results can be guaranteed.

[0088] The embodiment of the present invention also provides an image processing device.

[0089] Reference Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a module of an embodiment of an image processing device of the present invention. In this embodiment, the image processing device includes:

[0090] An acquisition module 601 is used to acquire a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate;

[0091] A processing module 602 is used to perform grayscale processing on the left eye image and the right eye image to obtain a left eye grayscale image and a right eye grayscale image;

[0092] A determination module 603 is used to determine the position of the straight line in the left eye grayscale image and the right eye grayscale image, and calculate the height difference between the left eye grayscale image and the right eye grayscale image according to the position;

[0093] The setting module 604 is used to set a cropping rule and a splicing rule based on the left eye image and the right eye image according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting.

[0094] Optionally, the determining module 603 is further configured to:

[0095] Determine a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image and a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0096] An absolute value of a difference between the number of rows of the first pixel row and the number of rows of the second pixel row is calculated, and the absolute value is used as a height difference between the left-eye grayscale image and the right-eye grayscale image.

[0097] Optionally, the processing module 602 is further used for:

[0098] When the left-eye image and the right-eye image are in YUV encoding format, obtaining Y component data of the left-eye image and the right-eye image;

[0099] A left-eye grayscale image and a right-eye grayscale image are correspondingly generated according to the Y component data.

[0100] Optionally, the determining module 603 is further configured to:

[0101] Determine a first stitching position in the left-eye grayscale image, and determine a second stitching position in the right-eye grayscale image;

[0102] For each row of pixels in the left-eye grayscale image, starting from the first stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of m Y component values, and comparing the magnitudes of the sums of the m Y components to determine the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image;

[0103] For each row of pixels in the right-eye grayscale image, starting from the second stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of n Y component values, and comparing the magnitudes of the sums of the n Y components to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image;

[0104] Wherein, m is the number of pixel rows of the left-eye grayscale image, and n is the number of pixel rows of the right-eye grayscale image.

[0105] Optionally, the setting module 604 is further used to:

[0106] Comparing the heights of the left-eye grayscale image and the right-eye grayscale image;

[0107] If the height of the left grayscale image is greater than the height of the right grayscale image, the cropping rule is set as follows: the last g rows of the left grayscale image are cropped, and the first g rows of the right grayscale image are cropped; if the height of the left grayscale image is less than the height of the right grayscale image, the cropping rule is set as follows: the first g rows of the left grayscale image are cropped, and the last g rows of the right grayscale image are cropped; wherein g is the height difference;

[0108] The stitching rule is set as follows: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and adjusting the stitched image back to a preset pixel height using a preset image interpolation algorithm.

[0109] The functional implementation and beneficial effects of each module in the above-mentioned image processing device correspond to each step in the above-mentioned image processing method embodiment, and will not be repeated here.

[0110] The image processing apparatus in the embodiment of the present invention is described in detail above from the perspective of modular functional entities. The image processing device in the embodiment of the present invention is described in detail below from the perspective of hardware processing.

[0111] Reference Figure 7 , Figure 7A schematic diagram of the structure of an image processing device provided in an embodiment of the present invention. The image processing device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 may be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the image processing device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the image processing device 700.

[0112] The image processing device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The structure of the image processing device shown does not constitute a limitation on the image processing device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0113] The present invention also provides an image processing device, which includes a memory and a processor. The memory stores instructions, and when the instructions are executed by the processor, the processor executes the steps of the image processing method in the above embodiments.

[0114] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein instructions are stored in the computer, and when the instructions are executed on a computer, the computer executes the steps of the image processing method in the above-mentioned embodiments.

[0115] Those skilled in the art can understand that the above-mentioned integrated modules or units, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0116] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: The image processing method comprises the following steps: Acquire a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate, and the left-eye image and the right-eye image are in a YUV encoding format; acquire Y component data of the left-eye image and the right-eye image, and generate a left-eye grayscale image and a right-eye grayscale image according to the Y component data; Determine a first stitching position in the left-eye grayscale image, and determine a second stitching position in the right-eye grayscale image; For each row of pixels in the left-eye grayscale image, starting from the first stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of m Y component values, and comparing the magnitudes of the sums of the m Y components to determine the first pixel row where the upper edge of the straight line is located in the left-eye grayscale image; For each row of pixels in the right-eye grayscale image, starting from the second stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of n Y component values, and comparing the magnitudes of the sums of the n Y components to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image; Wherein, m is the number of pixel rows of the left-eye grayscale image, and n is the number of pixel rows of the right-eye grayscale image; Calculating an absolute value of a difference between the number of the first pixel row and the number of the second pixel row, and using the absolute value as a height difference between the left-eye grayscale image and the right-eye grayscale image; A cropping rule and a splicing rule based on the left-eye image and the right-eye image are set according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting.

2. The image processing method according to claim 1, characterized in that: The step of setting a cropping rule and a splicing rule based on the left-eye image and the right-eye image according to the height difference comprises: Comparing the heights of the left-eye grayscale image and the right-eye grayscale image; If the height of the left grayscale image is greater than the height of the right grayscale image, the cropping rule is set as follows: the last g rows of the left grayscale image are cropped, and the first g rows of the right grayscale image are cropped; if the height of the left grayscale image is less than the height of the right grayscale image, the cropping rule is set as follows: the first g rows of the left grayscale image are cropped, and the last g rows of the right grayscale image are cropped; wherein g is the height difference; The stitching rule is set as follows: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and adjusting the stitched image back to a preset pixel height using a preset image interpolation algorithm.

3. An image processing device, characterized in that: The image processing device comprises: An acquisition module is used to acquire a left-eye image and a right-eye image obtained by shooting a preset calibration plate with a binocular camera, wherein a straight line for height difference correction is provided on the calibration plate, and the left-eye image and the right-eye image are in a YUV encoding format; A processing module, used for acquiring Y component data of the left eye image and the right eye image, and correspondingly generating a left eye grayscale image and a right eye grayscale image according to the Y component data; A determination module, used to determine a first stitching position in the left-eye grayscale image and a second stitching position in the right-eye grayscale image; for each row of pixels in the left-eye grayscale image, starting from the first stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of m Y component values, and comparing the sizes of the sums of the m Y components to determine a first pixel row where the upper edge of the straight line is located in the left-eye grayscale image; for each row of pixels in the right-eye grayscale image, starting from the second stitching position, summing the Y component data of a preset number of row pixel points to obtain a sum of n Y component values, and comparing the sizes of the sums of the n Y components to determine a second pixel row where the upper edge of the straight line is located in the right-eye grayscale image; wherein m is the number of pixel rows in the left-eye grayscale image, and n is the number of pixel rows in the right-eye grayscale image; calculating the absolute value of the difference between the number of the first pixel row and the second pixel row, and using the absolute value as the height difference between the left-eye grayscale image and the right-eye grayscale image; A setting module is used to set a cropping rule and a splicing rule based on the left-eye image and the right-eye image according to the height difference, and the cropping rule and the splicing rule are used to perform height difference correction on the binocular images obtained by subsequent shooting.

4. The image processing device according to claim 3, wherein: The setting module is also used for: Comparing the heights of the left-eye grayscale image and the right-eye grayscale image; If the height of the left grayscale image is greater than the height of the right grayscale image, the cropping rule is set as follows: the last g rows of the left grayscale image are cropped, and the first g rows of the right grayscale image are cropped; if the height of the left grayscale image is less than the height of the right grayscale image, the cropping rule is set as follows: the first g rows of the left grayscale image are cropped, and the last g rows of the right grayscale image are cropped; wherein g is the height difference; The stitching rule is set as follows: stitching the cropped left-eye grayscale image and the right-eye grayscale image to obtain a stitched image, and adjusting the stitched image back to a preset pixel height using a preset image interpolation algorithm.

5. An image processing device, characterized in that: The image processing device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the image processing device to perform the image processing method according to any one of claims 1 to 2.

6. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the image processing method according to any one of claims 1 to 2 is implemented.

Images