A real-time distance measuring modeling system using dual cameras

Abstract

The application discloses a kind of real-time ranging modeling system using double camera, it is related to visual ranging technical field, including: calibration acquisition module, calibration first camera focal length and two camera baseline length, acquisition first camera image and second camera image, generate processing area, establish image cache queue;Static correction module extracts static anchor point in processing area, constructs triangular mesh element, with first camera image as reference, according to triangular mesh element to the position correction of second camera image, form second camera image after position correction;Contour ranging module extracts target contour position based on first camera image, forms target contour circumscribed rectangle, matches the same sequence contour position in second camera image after position correction, generates contour parallax, and according to contour parallax conversion target distance value, forms target contour proximal depth boundary, target contour mid-depth boundary and target contour distal depth boundary.

Description

Technical Field

[0001] This invention relates to the field of visual ranging technology, and in particular to a real-time ranging modeling system using dual cameras. Background Technology

[0002] Dual-camera real-time ranging modeling technology belongs to the field of visual ranging technology and is widely used in intelligent vehicles, mobile robots, experimental measurement platforms, and near-range target perception scenarios. Conventional methods typically rely on the geometric relationship of dual-camera imaging to perform disparity analysis, distance conversion, and spatial position representation of the same scene target. They also combine target boundary, contour, or region information to construct a local geometric description, achieving collaborative processing of target distance perception and scene modeling. This type of method combines the advantages of non-contact measurement, rich imaging information, and moderate hardware cost, making it an important implementation path for visual ranging modeling.

[0003] However, existing methods still have two limitations: First, during continuous sampling, local imaging differences between dual-camera images can easily cause matching position drift, affecting the stability of cross-cycle ranging; second, the target position range and depth range formed in the current sampling cycle are usually not transformed into constraint information for the next sampling cycle, resulting in a wider search range and an increased probability of mismatch. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a real-time ranging modeling system using dual cameras, which solves the problems of easy drift in matching position, wide search range and insufficient stability of cross-cycle ranging during continuous sampling by dual cameras.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a real-time ranging modeling system using dual cameras, comprising: a calibration and acquisition module for calibrating the focal length of the first camera and the baseline lengths of the two cameras, acquiring images from the first and second cameras, generating a processing area, and establishing an image cache queue; a static correction module for extracting static anchor points within the processing area, constructing triangular mesh units, and using the first camera image as a reference to perform position correction on the second camera image according to the triangular mesh units, forming a position-corrected second camera image; a contour ranging module for extracting the target contour position based on the first camera image, forming the target contour circumscribed rectangle, matching the position of the same contour in the position-corrected second camera image, generating contour disparity, and calculating the target distance value based on the contour disparity, forming the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour; and a constraint management module for using the target contour circumscribed rectangle as a constraint rectangle, calculating the lower and upper disparity boundaries for the next sampling period based on the near and far depth boundaries of the target contour, and forming a constraint record together with the target distance value and the median depth boundary of the target contour.

[0008] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the calibration of the focal length of the first camera and the baseline length of the two cameras includes: completing the installation and fixing of the first camera and the second camera; arranging a planar calibration plate within the common visible range of the first camera and the second camera; and simultaneously acquiring multiple sets of calibration images of the first camera and multiple sets of calibration images of the second camera at different positions and in different postures.

[0009] The focal length of the first camera is obtained by calibrating and solving the calibration point position and calibration point spacing parameters in multiple sets of first camera calibration images.

[0010] Based on the pairing relationship of the dual-camera calibration points and the calibration point spacing parameters at the same acquisition time, the baseline length of the two cameras is obtained.

[0011] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the steps of acquiring images from the first camera and the second camera, generating a processing area, and establishing an image cache queue include: during the operation phase, controlling the first camera and the second camera to synchronously acquire images within the same sampling period using a unified clock trigger signal to form the first camera image and the second camera image; delineating effective imaging areas in the first camera image and the second camera image respectively, and performing overlap calculations on the effective imaging areas of the first camera and the second camera to obtain an overlapping field of view; writing the overlapping field of view into the processing area during the initial sampling period; performing overlap clipping on the constraint rectangle of the previous sampling period and the overlapping field of view during non-initial sampling periods, and writing the overlap clipped area into the processing area; and writing the first camera image and the second camera image into the image cache queue in the order of the sampling period.

[0012] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the step of extracting static anchor points within the processing area includes: dividing the processing area into anchor point search areas, and extracting candidate anchor point positions within each anchor point search area to form a candidate anchor point position sequence; based on the candidate anchor point position sequence, extracting local image blocks from the first camera and the second camera, and combining the image cache queue to obtain the anchor point position corresponding to the second camera in the current sampling period, the matching position of the first camera in the previous sampling period, and the matching position of the second camera in the previous sampling period; obtaining the static discrimination value corresponding to each candidate anchor point position based on the anchor point position corresponding to the second camera in the current sampling period, the matching position of the first camera in the previous sampling period, and the matching position of the second camera in the previous sampling period, and taking the candidate anchor point positions whose static discrimination values ​​are not greater than the lower quartile of the current sequence of all static discrimination values ​​as the static anchor point position sequence.

[0013] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the construction of triangular mesh units includes: performing triangulation on the first camera image plane based on a static anchor point position sequence to form a triangular mesh unit sequence; extracting the three vertex positions of each triangular mesh unit in the first camera image and the three corresponding vertex positions in the second camera image according to a unified vertex arrangement order to form a first camera vertex homogeneous position matrix and a second camera vertex homogeneous position matrix; determining whether the corresponding triangular mesh unit is invertible based on the first camera vertex homogeneous position matrix and the second camera vertex homogeneous position matrix, calculating the position correction matrix for invertible triangular mesh units, and removing irreversible triangular mesh units to form a non-degenerate triangular mesh unit sequence.

[0014] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the step of correcting the position of the second camera image according to triangular mesh units to form a position-corrected second camera image includes: extracting all pixel positions within each non-degenerate triangular mesh unit in the second camera image according to a non-degenerate triangular mesh unit sequence, and transforming each pixel position into a corrected pixel position based on the position correction matrix; extracting grayscale assignment positions based on the corrected pixel positions, and assigning the grayscale value of the corresponding pixel position in the second camera image to the grayscale assignment position; after all non-degenerate triangular mesh units have completed grayscale assignment, a position-corrected second camera image is formed.

[0015] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the step of extracting the target contour position based on the first camera image and forming the target contour circumscribed rectangle includes: extracting the boundary grayscale change intensity within the processing area of ​​the first camera image and retaining the pixel positions that pass the boundary screening as strong boundary positions; connecting the strong boundary positions into a closed boundary according to the pixel connectivity relationship, and retaining the target boundary in combination with the constraint rectangle of the previous sampling period; extracting the boundary positions point by point along the target boundary in the boundary connectivity order to form a target contour position sequence; and extracting the left boundary, right boundary, upper boundary, and lower boundary of the target contour according to the target contour position sequence to form the target contour circumscribed rectangle.

[0016] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the step of generating contour disparity by matching the sequential contour positions in the second camera image after position correction includes: for each target contour position in the target contour position sequence, a square pixel region is extracted from the first camera image, and the grayscale distribution in the square pixel region is used as the contour matching basis to extract the corresponding sequential contour position in the second camera image after position correction; the difference between the horizontal position of the target contour position in the first camera image and the horizontal position of the sequential contour position in the second camera image after position correction is used as the contour disparity, and contour disparities with a difference greater than zero are retained to form a contour disparity sequence; an effective contour disparity sequence is formed based on the contour disparity sequence and the constraint record of the previous sampling period, and the target disparity value is extracted from the effective contour disparity sequence.

[0017] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the step of converting the target distance value based on contour disparity to form the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour includes: converting the target distance value based on the focal length of the first camera, the baseline length of the two cameras, and the target disparity value; converting the contour depth value based on the disparity of each contour in the effective contour disparity sequence; sorting all contour depth values; and extracting the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour.

[0018] As a preferred embodiment of the dual-camera real-time ranging modeling system described in this invention, the constraint record includes a constraint rectangle, target distance value, near-end depth boundary of the target contour, median depth boundary of the target contour, far-end depth boundary of the target contour, lower disparity boundary of the next sampling period, and upper disparity boundary of the next sampling period, and is written into the constraint record queue in the order of sampling period.

[0019] The beneficial effects of this invention are as follows: by extracting static anchor points within the processing area, constructing triangular mesh units, and performing position correction on the second camera image, the spatial position consistency between the second camera image and the first camera image after position correction is achieved, enhancing the stability of sequential contour position matching, contour disparity generation, and target distance value conversion; by using the bounding rectangle of the target contour as a constraint rectangle, and combining the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour to form a constraint record, continuous constraints on the processing area and contour disparity retention range of the next sampling period are achieved, reducing mismatches and improving the continuity and stability of cross-period real-time ranging modeling. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a real-time ranging and modeling system using dual cameras.

[0022] Figure 2 This is a schematic diagram of the camera calibration and image acquisition process.

[0023] Figure 3 This is a flowchart of an image position correction method based on static anchor points.

[0024] Figure 4 Flowchart for target contour disparity matching and depth calculation. Detailed Implementation

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0027] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0028] Reference Figures 1-4 This is one embodiment of the present invention, which provides a real-time ranging modeling system using dual cameras, including the following steps:

[0029] The calibration acquisition module calibrates the focal length of the first camera and the baseline length of the two cameras, acquires images from the first camera and the second camera, generates a processing area, and establishes an image cache queue.

[0030] Furthermore, during the calibration preparation phase, the first and second cameras are installed and fixed, and the planar calibration plate is placed within the common visibility range of the first and second cameras.

[0031] The first camera and the second camera are respectively mounted on two mounting positions of the same rigid bracket. The rigid bracket is mounted on a vehicle structural component, a rigid base of a test bench, or other mounting base with rigid constraint capabilities. The form of the mounting base includes, but is not limited to, a vehicle front compartment crossbeam, a vehicle bumper inner bracket, a vehicle front mounting frame, and a test bench metal profile base. After installation, the spatial positional relationship between the first camera and the second camera remains stable, and the optical axis directions of the first camera and the second camera both point towards the calibration area.

[0032] The planar calibration plate is fixed on a bracket or tripod. The surface of the planar calibration plate is arranged with a regular corner dot array pattern, which includes a black and white checkerboard pattern or a black and white dot array pattern. The corner dot spacing between any two adjacent corner dots in the regular corner dot array pattern remains consistent throughout the entire calibration plate. The corner dot spacing represents the actual distance between the centers of adjacent corner dots in the regular corner dot array pattern on the plane of the calibration plate. The calibration dot spacing parameter is obtained by reading the printed marking information or manufacturing inspection information of the planar calibration plate, and the calibration dot spacing parameter is used as the geometric constraint parameter of the planar calibration plate.

[0033] The planar calibration plate is moved along the front-back, left-right, and up-down directions, and the tilt angle of the planar calibration plate relative to the first and second cameras is adjusted so that the planar calibration plate is in multiple different positions and multiple different postures within the common imaging range of the first and second cameras. After each position adjustment and posture adjustment is completed, a unified clock trigger signal is used to control the first and second cameras to synchronously acquire an image, resulting in a set of first camera calibration images and second camera calibration images. The position adjustment, posture adjustment, and synchronous acquisition are repeated to obtain multiple sets of first camera calibration images and multiple sets of second camera calibration images. The multiple sets of first camera calibration images and multiple sets of second camera calibration images are arranged in the order of acquisition time to form a calibration image group.

[0034] Furthermore, for each set of first camera calibration images and second camera calibration images, the image area where the planar calibration board is located is first read, and then the outer boundary of the regular corner point array pattern is identified within the image area, and the board area surrounded by the outer boundary is separated from the background.

[0035] After the board area is separated, when the regular corner dot array pattern is a black and white checkerboard pattern, first identify the dividing line of the black and white squares, and then extract the calibration point position at the intersection of the horizontal dividing line and the vertical dividing line; when the regular corner dot array pattern is a black and white dot array pattern, first identify the outline of each dot, then calculate the center position of the dot inside each dot outline, and use the center position of the dot as the calibration point position.

[0036] After obtaining the positions of all calibration points, take the calibration point at the upper left corner of the board area as the starting calibration point, and number all calibration points sequentially from left to right and from top to bottom. Record the calibration point position of each numbered calibration point in the first camera calibration image and the calibration point position of each numbered calibration point in the second camera calibration image.

[0037] After numbering, each numbered calibration point is written to its actual position on the calibration plate plane according to the number of rows, columns, and calibration point spacing parameters of the planar calibration plate, thereby forming the calibration point position record of the first camera calibration image and the calibration point position record of the second camera calibration image.

[0038] For a set of first camera calibration images and second camera calibration images acquired at the same time, the positions of the two sets of calibration points are recorded and paired point by point according to the same numbering order to form a dual-camera calibration point pairing record.

[0039] Furthermore, based on the calibration point positions and calibration point spacing parameters of multiple sets of first camera calibration images, the focal length of the first camera is obtained.

[0040] In each set of first camera calibration images, the horizontal and vertical image spacing of adjacent calibration points are extracted according to the calibration point numbering order. The actual spacing of adjacent calibration points on the planar calibration plate is then extracted by combining the calibration point spacing parameters. The calibration point position changes, adjacent calibration point image spacing changes, and calibration point arrangement relationships under different positions and postures of the planar calibration plate are calibrated and solved to gradually reduce and stabilize the overall position difference between the image position of each numbered calibration point in the first camera calibration image and the projected position after conversion of the actual position of the planar calibration plate. After fitting, the horizontal focal length parameters and the vertical focal length parameters are obtained.

[0041] The average focal length value of the horizontal focal length parameter and the vertical focal length parameter is taken as the focal length of the first camera, and the focal length of the first camera is denoted as... , is a fixed single numerical value obtained during the calibration phase, and is represented in pixels in the distance conversion, used to characterize the imaging scale of the first camera.

[0042] Furthermore, based on the calibration point positions and calibration point spacing parameters of multiple sets of second camera calibration images, the calibration parameters of the second camera are obtained.

[0043] In each set of second camera calibration images, the horizontal and vertical image spacing of adjacent calibration points are extracted according to the calibration point numbering order. The actual spacing of adjacent calibration points on the planar calibration plate is then extracted by combining the calibration point spacing parameters. The calibration point position changes, adjacent calibration point image spacing changes, and calibration point arrangement relationships under different positions and orientations of the planar calibration plate are calibrated and solved to gradually reduce and stabilize the overall position difference between the image position of each numbered calibration point in the second camera calibration image and the projected position after conversion of the actual position on the planar calibration plate. After fitting, the horizontal focal length parameters and the vertical focal length parameters of the second camera are obtained.

[0044] Furthermore, based on the focal length of the first camera, the calibration parameters of the second camera, the pairing relationship of the dual-camera calibration points at the same acquisition time, and the calibration point spacing parameters, the baseline length of the two cameras is obtained.

[0045] The positions of each numbered calibration point in the first camera calibration image are paired point by point with the positions of the same numbered calibration points in the second camera calibration image. Combined with the actual positions of each numbered calibration point on the planar calibration board, the image positions of each numbered calibration point in the first camera calibration image are converted into projection relationships under the first camera's viewpoint based on the first camera's focal length. Based on the second camera's calibration parameters, the image positions of each numbered calibration point in the second camera calibration image are converted into projection relationships under the second camera's viewpoint. The spatial positional relationships of the planar calibration board under the first camera's viewpoint and under the second camera's viewpoint are then restored respectively. Finally, based on the relative translation relationship between the two sets of spatial positional relationships in the same set of calibration images, the spatial displacement vector between the optical centers of the first and second cameras is obtained.

[0046] The magnitude of the spatial displacement vector is taken as the baseline length between the two cameras, and the baseline length between the two cameras is denoted as... , is a fixed single numerical value obtained during the calibration phase, and is expressed in units of length, used to characterize the actual spatial distance between the optical centers of the first camera and the second camera.

[0047] Furthermore, during the operation phase, a unified clock trigger signal controls the first and second cameras to complete image acquisition within the same sampling period, forming the first camera image and the second camera image.

[0048] The first camera image records the forward scene image content from the first camera's perspective, and the second camera image records the same forward scene image content from the second camera's perspective; for the same target, the target position in the first camera image and the target position in the second camera image have different perspectives.

[0049] After the data acquisition is completed, the sampling period identifier and sampling time identifier are written into the first camera image and the second camera image respectively, and the first camera image and the second camera image with the same sampling period identifier and sampling time identifier are stored together.

[0050] Furthermore, effective imaging areas are defined in the first camera image and the second camera image respectively. The effective imaging area refers to the pixel area retained after removing invalid pixel bands at the image edge.

[0051] The effective imaging area in the first camera image is written as the effective imaging area of ​​the first camera, and the effective imaging area in the second camera image is written as the effective imaging area of ​​the second camera. An overlap calculation is performed on the effective imaging areas of the first camera and the effective imaging areas of the second camera, and the common coverage part is written into the overlapping field of view area.

[0052] In the initial sampling period, the overlapping field of view is directly written into the processing area; in non-initial sampling periods, the constraint rectangle of the previous sampling period and the overlapping field of view are overlapped and clipped, and the overlapped and clipped area is written into the processing area; the constraint rectangle of the previous sampling period is represented in the form of a rectangular bounding box, and the upper left and lower right corners of the constraint rectangle form the rectangular bounding box.

[0053] Furthermore, the images from the first camera and the second camera are written into the image buffer queue in sequence according to the sampling period.

[0054] The image cache queue includes a first camera image cache queue and a second camera image cache queue. The first camera image cache queue stores the first camera images for each sampling period, and the second camera image cache queue stores the second camera images for each sampling period.

[0055] During image storage, the sampling period order of the first camera image cache queue and the second camera image cache queue is kept consistent, and the first camera images and second camera images of multiple consecutive sampling periods are retained; when the number of images in the image cache queue is greater than the cache capacity, the first camera image and second camera image of the earliest sampling period are removed from the image cache queue, while the first camera image and second camera image of the most recent consecutive sampling period are retained; thus forming dual-camera images arranged in the sampling period order in the image cache queue.

[0056] The static correction module extracts static anchor points within the processing area, constructs triangular mesh units, and uses the first camera image as a reference to perform position correction on the second camera image according to the triangular mesh units, thus forming the position-corrected second camera image.

[0057] Furthermore, the anchor point search area is divided into regular grids within the processing area. The anchor point search area refers to a local rectangular pixel area used to search for a candidate anchor point position.

[0058] During the partitioning process, the processing area is divided into several rows and several columns along the horizontal and vertical directions, respectively, so that each anchor point search area has the same side length. For example, when the horizontal pixel length of the processing area is 640 pixels and the vertical pixel length is 480 pixels, it can be divided into 20 columns along the horizontal direction and 15 rows along the vertical direction, thereby forming 300 anchor point search areas. At this time, the horizontal side length of each anchor point search area is 32 pixels and the vertical side length is 32 pixels.

[0059] For each anchor point search area, read the grayscale value of all pixel positions one by one. Taking each pixel position as the center, calculate the horizontal grayscale difference between left and right adjacent pixels and the vertical grayscale difference between top and bottom adjacent pixels. The sum of the absolute values ​​of the horizontal and vertical grayscale differences is used as the grayscale change intensity of the current position.

[0060] The intensity of grayscale changes of all pixels within the same anchor point search area is sorted, and the pixel position corresponding to the largest grayscale change intensity after sorting is taken as the candidate anchor point position of the current anchor point search area; the same process is repeated for all anchor point search areas to form a sequence of candidate anchor point positions.

[0061] Furthermore, after the candidate anchor point position sequence is formed, the pixel distance between each candidate anchor point position and its nearest neighbor is calculated, and the pixel distance from each candidate anchor point position to the left, right, upper, and lower boundaries of the processing area is also calculated.

[0062] The pixel distance between the nearest adjacent candidate anchor points is compared with the pixel distance to each boundary of the processing area. The smallest pixel distance is used as the limit distance for the side length of the local image patch. Within the limit distance range, the largest odd side length is selected as the side length of the local image patch. Centered on the candidate anchor point, the local image patch of the first camera is cropped in the first camera image of the current sampling period according to the side length of the local image patch. The local image patch of the second camera is cropped in the second camera image of the current sampling period.

[0063] A local image patch is a square pixel region centered on the candidate anchor point, with equal horizontal and vertical side lengths.

[0064] Furthermore, in the second camera image of the current sampling period, the search range is taken as the neighboring pixel region corresponding to the candidate anchor point position of the first camera in the current sampling period within the processing area of ​​the second camera image of the current sampling period. The local image block of the first camera in the current sampling period is translated position by position within the search range, and the sum of the absolute values ​​of grayscale difference at each translation position is calculated. The translation position with the smallest sum of absolute values ​​of grayscale difference is determined as the anchor point position of the second camera in the current sampling period.

[0065] Once the anchor point position corresponding to the second camera in the current sampling period is determined, the first camera image and the second camera image from the previous sampling period are extracted from the image cache queue.

[0066] In the first camera image of the previous sampling period, the first camera search region is defined with the current candidate anchor point position as the center. The side length of the first camera search region is the smallest odd side length obtained by expanding the side length of the local image block by one pixel. The local image block of the first camera in the current sampling period is translated position by position within the first camera search region. The sum of the absolute values ​​of the gray level difference at each translation position is calculated, and the translation position with the smallest sum of the absolute values ​​of the gray level difference is determined as the first camera matching position of the previous sampling period. The first camera matching position is used to characterize the corresponding position of the candidate anchor point position in the current sampling period in the first camera image of the previous sampling period.

[0067] In the second camera image of the previous sampling period, the second camera search area is defined with the anchor point position of the second camera in the current sampling period as the center. The side length of the second camera search area is the smallest odd side length obtained by expanding the side length of the local image block by one pixel. The local image block of the second camera in the current sampling period is translated position by position in the second camera search area. The sum of the absolute values ​​of the gray level difference at each translation position is calculated. The translation position with the smallest sum of the absolute values ​​of the gray level difference is determined as the second camera matching position in the previous sampling period. The second camera matching position is used to characterize the corresponding position of the anchor point position of the second camera in the current sampling period in the second camera image of the previous sampling period.

[0068] The horizontal and vertical position differences between the candidate anchor point position in the current sampling period and the matching position of the first camera in the previous sampling period constitute the displacement of the first camera. The horizontal and vertical position differences between the corresponding anchor point position of the second camera in the current sampling period and the matching position of the second camera in the previous sampling period constitute the displacement of the second camera.

[0069] Furthermore, for each candidate anchor point position, the displacement length of the first camera is calculated based on the right-angle displacement synthesis relationship between the lateral and longitudinal position differences in the displacement of the first camera; the displacement length of the second camera is calculated based on the right-angle displacement synthesis relationship between the lateral and longitudinal position differences in the displacement of the second camera; the displacement lengths of the first and second cameras are added together to obtain the static discrimination value, expressed as:

[0070] ;

[0071] in, The sampling period number is The moment The static discriminant value of each candidate anchor point position. The sampling period number is The moment The candidate anchor point positions are located horizontally in the first camera image. The sampling period number is The moment The candidate anchor point positions are located vertically in the first camera image. The sampling period number is The moment The candidate anchor point positions are located horizontally in the first camera image. The sampling period number is The moment The candidate anchor point positions are located vertically in the first camera image. The sampling period number is The moment The candidate anchor points are located at their horizontal positions in the second camera image. The sampling period number is The moment The candidate anchor point positions are located vertically in the second camera image. The sampling period number is The moment The candidate anchor points are located at their horizontal positions in the second camera image. The sampling period number is The moment The candidate anchor point positions are located vertically in the second camera image. Number the candidate anchor points. This is the sampling period number.

[0072] The static discriminant value can take values ​​greater than or equal to zero. The closer the static discriminant value is to zero, the better the value of the first static discriminant. The smaller the positional change of each candidate anchor point in adjacent sampling periods.

[0073] After all static discrimination values ​​are generated, they are sorted from smallest to largest. Candidate anchor point positions whose static discrimination values ​​are not greater than the quartile values ​​of the current static discrimination value sequence are taken as the static anchor point position sequence. The smaller the static discrimination value, the smaller the cross-cycle position change of the candidate anchor point position in the first camera image and the second camera image, and the closer the candidate anchor point position is to the static background position.

[0074] It should be noted that the candidate anchor positions whose static discriminant value is not greater than the lower quartile value of the current static discriminant value sequence are used as the static anchor position sequence because the lower quartile value comes from the sorted distribution of the current static discriminant values. The candidate anchor positions whose static discriminant value is not greater than the lower quartile value belong to a group of candidate anchor positions with relatively small changes in current position. Retaining the group of candidate anchor positions with relatively small changes in current position as the static anchor position sequence can, on the one hand, exclude candidate anchor positions with large changes in position from entering the static anchor position sequence, and on the other hand, retain a sufficient number of static anchor positions to support triangulation and position correction within the processing area.

[0075] Furthermore, the static anchor point position sequence is triangulated within the first camera image plane to form a triangular mesh unit sequence. Each triangular mesh unit contains three vertex positions, and the vertex arrangement order is as follows: the vertex with the smallest vertical position is placed first; among the other two vertices, the vertex with the smaller horizontal position is placed second, and the vertex with the larger horizontal position is placed third; when the vertical positions of two vertices are the same, the vertex with the smaller horizontal position is placed first. According to this vertex arrangement order, three vertex positions are extracted from the first camera image, and three corresponding vertex positions with the same anchor point number are extracted from the second camera image.

[0076] The positions of the three vertices in the first camera image are filled with constant 1 and arranged in the order of first, second, and third to form the homogeneous position matrix of the first camera vertex; the positions of the three corresponding vertices in the second camera image are filled with constant 1 and arranged in the same order to form the homogeneous position matrix of the second camera vertex; each column of the homogeneous position matrix of the first camera vertex consists of the horizontal position, vertical position, and constant 1 of a vertex, and each column of the homogeneous position matrix of the second camera vertex consists of the horizontal position, vertical position, and constant 1 of the corresponding vertices with the same number.

[0077] Based on the position transformation relationship between the homogeneous position matrices of the first and second cameras, we first determine whether the homogeneous position matrix of the second camera is invertible. If the three corresponding vertices in the second camera image do not have duplicate vertices and are not on the same straight line, we determine that the homogeneous position matrix of the second camera is invertible, and obtain the position correction matrix of the triangular mesh element corresponding to the current position, expressed as:

[0078] ;

[0079] in, The sampling period number is The moment Position correction matrix of each triangular mesh cell The sampling period number is The moment The homogeneous position matrix of the three vertices of a triangular mesh unit in the first camera image. The sampling period number is The moment The homogeneous position matrix of the three vertices of a triangular mesh unit in the second camera image. Number the triangular mesh cells.

[0080] When the homogeneous position matrix of the second camera vertex is irreversible, the current triangular mesh unit is determined to be a degenerate triangular mesh unit, and the current triangular mesh unit is removed from the position correction processing object. The position correction matrix is ​​not calculated for the current triangular mesh unit. The remaining triangular mesh units after removal are arranged in the order of triangular mesh unit number to form a non-degenerate triangular mesh unit sequence.

[0081] Furthermore, in the second camera image, all pixel positions within each non-degenerate triangular mesh unit are extracted sequentially according to the non-degenerate triangular mesh unit sequence.

[0082] Arrange the horizontal position, vertical position, and constant of each pixel location to form the original homogeneous pixel position. Based on the position transformation relationship between the position correction matrix of the current triangular mesh cell and the original homogeneous pixel position, obtain the corrected pixel position, expressed as:

[0083] ;

[0084] in, The sampling period number is The moment Corrected pixel position The sampling period number is The moment The original pixel homogeneous position Number the pixel positions within the triangular mesh unit.

[0085] The original pixel homogeneous position consists of the original pixel position's horizontal position, vertical position, and a constant. The first component of the corrected pixel position represents the corrected horizontal position, and the second component represents the corrected vertical position.

[0086] Furthermore, after the corrected pixel position is generated, the closest integer positions of the corrected horizontal position and the corrected vertical position are taken as the grayscale assignment positions, and the grayscale value of the current position in the second camera image is assigned to the grayscale assignment positions.

[0087] After all pixels within a triangular mesh cell have been assigned grayscale values, the process moves to the next triangular mesh cell and continues with the same procedure. Once all triangular mesh cells have been assigned grayscale values, the position-corrected second camera image is formed.

[0088] After position correction, the second camera image retains the grayscale information in the second camera image and makes the spatial position in the processing area consistent with the spatial position in the first camera image.

[0089] Within the same non-degenerate triangular mesh unit, a unified positional correspondence is established between the local spatial position in the first camera image and the local spatial position in the second camera image after position correction, so that the local longitudinal positional shift caused by the difference in the dual camera perspective is corrected; after position correction, the remaining positional difference of the same scene contour position in the two images is mainly manifested as a lateral positional difference.

[0090] The contour ranging module extracts the target contour position based on the first camera image, forms the target contour bounding rectangle, matches the position of the same contour in the second camera image after position correction, generates contour parallax, and calculates the target distance value based on the contour parallax, forming the near-end depth boundary, the median depth boundary, and the far-end depth boundary of the target contour.

[0091] Furthermore, within the processing area of ​​the first camera image, grayscale values ​​are extracted for each pixel location one by one. Taking each pixel location as the center, the horizontal grayscale difference between left and right adjacent pixels is calculated, and the vertical grayscale difference between top and bottom adjacent pixels is calculated. The sum of the absolute values ​​of the horizontal and vertical grayscale differences is used as the boundary grayscale change intensity at the current location.

[0092] All boundary grayscale change intensities are sorted from smallest to largest, and the value at the 75th percentile is taken as the upper quartile value. Pixels with boundary grayscale change intensities reaching or above the upper quartile value are retained as strong boundary locations. After retaining the strong boundary locations, adjacent strong boundary locations are connected into boundary segments according to pixel connectivity, and then the end-to-end boundary segments are merged into closed boundaries. The area of ​​the enclosed region of each closed boundary is calculated. When the sampling period number is greater than one, it is first determined whether there is an overlap between the enclosed region of each closed boundary and the constraint rectangle of the previous sampling period, and the closed boundaries with overlapping regions are retained as candidate target boundaries. When the number of candidate target boundaries is greater than zero, the closed boundary with the largest enclosed region among the candidate target boundaries is taken as the target boundary. When the number of candidate target boundaries is zero, the closed boundary with the largest enclosed region among all closed boundaries is taken as the target boundary. When the sampling period number is one, the closed boundary with the largest enclosed region among all closed boundaries is taken as the target boundary. The boundary positions are extracted point by point along the target boundary according to the boundary connectivity order to form a target contour position sequence.

[0093] The starting position of the target contour position sequence is the boundary position with the smallest vertical position and the smallest horizontal position in the target boundary. After the starting position is determined, all boundary positions are arranged sequentially along the connection direction of the target boundary to keep the target contour position sequence in a unique order.

[0094] Sort all horizontal positions in the target contour position sequence by numerical value, and take the smallest horizontal position as the left boundary of the target contour and the largest horizontal position as the right boundary of the target contour; sort all vertical positions in the target contour position sequence by numerical value, and take the smallest vertical position as the upper boundary of the target contour and the largest vertical position as the lower boundary of the target contour.

[0095] The left boundary, upper boundary, right boundary, and lower boundary of the target contour together form the bounding rectangle of the target contour.

[0096] Furthermore, for each target contour position in the target contour position sequence, in the first camera image, with the current position as the center, the same number of pixels are extended to both sides in the horizontal and vertical directions to extract a square pixel region with an odd side length; the extension range is limited to not exceeding the boundary of the processing area, and the largest square region that satisfies the boundary constraints is selected.

[0097] Using the grayscale distribution within the square pixel region as the basis for contour matching at the current position, since the local region where the current position is located has been corrected, the second camera image and the first camera image form a corresponding relationship at the same vertical position after position correction, the corresponding contour position at the current position only retains the horizontal position difference; accordingly, in the second camera image after position correction, the vertical position of the current position is used as the matching vertical position, and the image moves horizontally position by position, and at each moving position, a pixel region with the same side length as the square pixel region is extracted, and the sum of the absolute values ​​of the grayscale difference between the two pixel regions is calculated; the position with the smallest sum of the absolute values ​​of the grayscale difference is taken as the corresponding contour position.

[0098] The difference between the horizontal position of the target contour in the first camera image and the horizontal position of the same contour in the second camera image after position correction is taken as the contour disparity. The contour disparity represents the difference in horizontal position between corresponding contour points at the same vertical position after position correction. Only contour disparities with a difference greater than zero are retained as valid candidate disparities in the contour disparity sequence, and all retained contour disparities are arranged in the order of the target contour position to form the contour disparity sequence.

[0099] Furthermore, at the moment when the sampling period number is one, all contour disparities greater than zero in the contour disparity sequence are taken as the valid contour disparity sequence; when the number of contour disparities greater than zero is zero, the target distance value and depth boundary of the current sampling period are not generated, and contour matching is re-executed in the next sampling period.

[0100] When the sampling period number is greater than one, extract the lower boundary and upper boundary of the disparity of the next sampling period from the constraint record of the previous sampling period, and use them as the lower boundary and upper boundary of the current disparity, respectively.

[0101] Contour disparities that are greater than zero and located between the lower and upper boundaries of the current disparity are retained as valid contour disparity sequences. When the number of contour disparities located between the lower and upper boundaries of the current disparity is zero, all contour disparities greater than zero in the contour disparity sequence are re-established as valid contour disparity sequences. When the number of all contour disparities greater than zero is still zero, the constraint record of the previous sampling period is directly used as the constraint record of the current sampling period, and the processing of the current sampling period ends.

[0102] After the effective contour disparity sequence is formed, all effective contour disparities are sorted in ascending order of value, and the contour disparity in the middle position after sorting is taken as the target disparity value, which is then written as... When the number of effective contour disparities is even, the average of the two middle contour disparities is taken as the target disparity value.

[0103] Furthermore, the target disparity value is the lateral position difference between the positions of the same contour after position correction. Based on the focal length of the first camera, the baseline length of the two cameras, and the target disparity value, the target distance value is calculated and expressed as:

[0104] ;

[0105] in, The sampling period number is The target distance value at that moment is greater than zero. The sampling period number is The target parallax value at any given time. The focal length of the first camera obtained during the calibration phase. The baseline lengths of the two cameras are obtained during the calibration phase.

[0106] Each contour disparity in the effective contour disparity sequence is converted into a contour depth value, expressed as follows:

[0107] ;

[0108] in, The sampling period number is The moment The contour depth value at each target contour location; the contour depth value is greater than zero. The sampling period number is The moment Contour disparity at each target contour location, Number the location of the target contour.

[0109] All contour depth values ​​are sorted in ascending order. The lower quartile value is used as the near depth boundary of the target contour, the median value is used as the median depth boundary of the target contour, and the upper quartile value is used as the far depth boundary of the target contour.

[0110] The proximal depth boundary, median depth boundary, and distal depth boundary of the target contour are all greater than zero, and the proximal depth boundary of the target contour is not greater than the median depth boundary of the target contour, and the median depth boundary of the target contour is not greater than the distal depth boundary of the target contour.

[0111] It should be noted that the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour are three boundary values ​​extracted after arranging all contour depth values ​​corresponding to the target contour position sequence in numerical order. The near-end depth boundary represents the depth boundary on the side closer to the camera, the median depth boundary represents the middle boundary of all contour depth values, and the far-end depth boundary represents the depth boundary on the side farther from the camera.

[0112] Both target disparity and contour disparity are represented in pixels. Target distance, contour depth, near-end depth boundary of target contour, median depth boundary of target contour, and far-end depth boundary of target contour are all represented in the same length unit as the baseline length of the two cameras.

[0113] The constraint management module uses the bounding rectangle of the target contour as the constraint rectangle. It calculates the lower and upper disparity boundaries for the next sampling period based on the near and far depth boundaries of the target contour, and forms a constraint record together with the target distance value and the median depth boundary of the target contour.

[0114] Furthermore, the bounding rectangle of the target contour formed in the current sampling period is directly used as the constraint rectangle of the current sampling period.

[0115] The left, top, right, and bottom boundaries of the bounding rectangle of the target contour remain unchanged and are used as the left, top, right, and bottom boundaries of the constraint rectangle, respectively.

[0116] Furthermore, combining the focal length of the first camera and the baseline lengths of the two cameras obtained during the calibration phase, and based on the near-end depth boundary and far-end depth boundary of the target contour in the current sampling period, the lower and upper disparity boundaries for the next sampling period are calculated, as follows:

[0117] ;

[0118] ;

[0119] in, The sampling period number is The lower boundary of the parallax in the next sampling period at that time. The sampling period number is The upper boundary of disparity in the next sampling period at that time. The sampling period number is The depth boundary of the target outline at that moment. The sampling period number is The near-end depth boundary of the target profile at any given moment.

[0120] The lower boundary and upper boundary of disparity in the next sampling period are both greater than zero, and the lower boundary of disparity in the next sampling period is not greater than the upper boundary of disparity in the next sampling period.

[0121] Furthermore, the constraint rectangle, target distance value, near-end depth boundary of target contour, median depth boundary of target contour, far-end depth boundary of target contour, lower disparity boundary of the next sampling period, and upper disparity boundary of the next sampling period are combined into the constraint record of the current sampling period.

[0122] The constraint rectangle in the constraint record is used to define the processing area of ​​the next sampling period. The lower boundary of the disparity of the next sampling period and the upper boundary of the disparity of the next sampling period are used to define the contour disparity retention range of the next sampling period. The target distance value and the median depth boundary of the target contour are used to characterize the center distance state of the current sampling period. The near-end depth boundary of the target contour and the far-end depth boundary of the target contour are used to characterize the depth distribution range of the current sampling period.

[0123] The current sampling period constraint record is stored in the constraint record queue in the order of sampling period. After the sampling period number is incremented, the constraint rectangle in the constraint record of the previous sampling period is used as the source of the rectangle constraint for the processing area clipping. The lower boundary and upper boundary of disparity in the constraint record of the previous sampling period are used as the source of the boundary for contour disparity filtering.

[0124] In summary, this invention achieves spatial consistency between the second camera image and the first camera image after position correction by extracting static anchor points within the processing area, constructing triangular mesh units, and performing position correction on the second camera image. This enhances the stability of sequential contour position matching, contour disparity generation, and target distance value conversion. By using the bounding rectangle of the target contour as a constraint rectangle and combining it with the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour to form a constraint record, continuous constraints are achieved on the processing area and contour disparity retention range of the next sampling period. This reduces mismatches and improves the continuity and stability of cross-period real-time ranging modeling.

[0125] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time distance measuring modeling system using dual cameras, characterized in that, include: The calibration acquisition module calibrates the focal length of the first camera and the baseline length of the two cameras, acquires images from the first camera and the second camera, generates a processing area, and establishes an image cache queue. The static correction module extracts static anchor points within the processing area, constructs triangular mesh units, uses the first camera image as a reference, and performs position correction on the second camera image according to the triangular mesh units to form the position-corrected second camera image. The contour ranging module extracts the target contour position based on the first camera image, forms the target contour bounding rectangle, matches the position of the same contour in the second camera image after position correction, generates contour parallax, and calculates the target distance value based on the contour parallax, forming the near-end depth boundary, the median depth boundary, and the far-end depth boundary of the target contour. The constraint management module uses the bounding rectangle of the target contour as the constraint rectangle. It calculates the lower and upper disparity boundaries for the next sampling period based on the near and far depth boundaries of the target contour, and forms a constraint record together with the target distance value and the median depth boundary of the target contour.

2. The dual camera real-time distance measuring modeling system of claim 1, wherein, The calibration of the focal length of the first camera and the baseline length of the two cameras includes completing the installation and fixing of the first camera and the second camera, arranging a planar calibration plate within the common visible range of the first camera and the second camera, and simultaneously acquiring multiple sets of calibration images of the first camera and multiple sets of calibration images of the second camera at different positions and in different postures. The focal length of the first camera is obtained by calibrating and solving the calibration point position and calibration point spacing parameters in multiple sets of first camera calibration images. Based on the pairing relationship of the dual-camera calibration points and the calibration point spacing parameters at the same acquisition time, the baseline length of the two cameras is obtained.

3. The dual camera real-time distance measuring modeling system of claim 1, wherein, The process of acquiring images from the first camera and the second camera, generating a processing area, and establishing an image cache queue includes, during the operation phase, controlling the first camera and the second camera to synchronously acquire images within the same sampling period through a unified clock trigger signal, thereby forming the first camera image and the second camera image. Effective imaging regions are defined in the first camera image and the second camera image respectively, and overlap calculation is performed on the effective imaging regions of the first camera and the second camera to obtain the overlapping field of view region. In the initial sampling period, the overlapping field of view region is written into the processing region. In non-initial sampling periods, the constraint rectangle of the previous sampling period and the overlapping field of view region are overlapped and clipped, and the overlapped and clipped region is written into the processing region. The images from the first camera and the second camera are written into the image buffer queue in order of their sampling periods.

4. The real-time ranging and modeling system using dual cameras as described in claim 3, characterized in that, The step of extracting static anchor points within the processing area includes dividing the processing area into anchor point search areas and extracting candidate anchor point positions within each anchor point search area to form a candidate anchor point position sequence. Based on the candidate anchor point position sequence, local image blocks of the first camera and the second camera are extracted, and combined with the image cache queue, the anchor point position of the second camera in the current sampling period, the matching position of the first camera in the previous sampling period, and the matching position of the second camera in the previous sampling period are obtained. Based on the anchor point position of the second camera in the current sampling period, the matching position of the first camera in the previous sampling period, and the matching position of the second camera in the previous sampling period, the static discrimination value corresponding to each candidate anchor point position is obtained, and the candidate anchor point positions whose static discrimination value is not greater than the lower quartile value of the current sequence of all static discrimination values ​​are taken as the static anchor point position sequence.

5. The real-time ranging and modeling system using dual cameras as described in claim 4, characterized in that, The construction of the triangular mesh unit includes performing triangulation within the first camera image plane based on a static anchor point position sequence to form a triangular mesh unit sequence. Extract the three vertex positions of each triangular mesh unit in the first camera image and the three corresponding vertex positions in the second camera image according to a unified vertex arrangement order, to form the first camera vertex homogeneous position matrix and the second camera vertex homogeneous position matrix. Based on the homogeneous position matrix of the first camera vertex and the homogeneous position matrix of the second camera vertex, determine whether the corresponding triangular mesh unit is invertible, calculate the position correction matrix for the invertible triangular mesh unit, and remove the non-invertible triangular mesh unit to form a non-degenerate triangular mesh unit sequence.

6. The real-time ranging and modeling system using dual cameras as described in claim 5, characterized in that, The step of performing position correction on the second camera image according to the triangular mesh unit to form the position-corrected second camera image includes: extracting all pixel positions inside each non-degenerate triangular mesh unit in the second camera image according to the non-degenerate triangular mesh unit sequence, and transforming each pixel position into the corrected pixel position based on the position correction matrix. The grayscale assignment position is extracted based on the corrected pixel position, and the grayscale value of the corresponding pixel position in the second camera image is assigned to the grayscale assignment position. After all non-degenerate triangular mesh units have completed the grayscale assignment, the position-corrected second camera image is formed.

7. The real-time ranging and modeling system using dual cameras as described in claim 1, characterized in that, The step of extracting the target contour position based on the first camera image and forming the target contour bounding rectangle includes extracting the intensity of boundary grayscale change within the processing area of ​​the first camera image and retaining the pixel positions that pass the boundary screening as strong boundary positions. Strong boundary locations are connected into closed boundaries according to pixel connectivity, and the target boundary is preserved by combining the constraint rectangle of the previous sampling period; Extract the boundary positions point by point along the target boundary in the order of boundary connectivity to form a sequence of target contour positions; Extract the left boundary, right boundary, upper boundary, and lower boundary of the target contour based on the target contour position sequence, and form the bounding rectangle of the target contour.

8. The real-time ranging and modeling system using dual cameras as described in claim 7, characterized in that, The process of generating contour parallax in the second camera image after position correction includes: for each target contour position in the target contour position sequence, a square pixel region is extracted from the first camera image, and the grayscale distribution in the square pixel region is used as the contour matching basis to extract the corresponding parallel contour position in the second camera image after position correction. The difference between the horizontal position of the target contour in the first camera image and the horizontal position of the same contour in the second camera image after position correction is taken as the contour disparity, and the contour disparities with a difference greater than zero are retained to form a contour disparity sequence. An effective contour disparity sequence is formed based on the contour disparity sequence and the constraint record of the previous sampling period, and the target disparity value is extracted from the effective contour disparity sequence.

9. The real-time ranging and modeling system using dual cameras as described in claim 8, characterized in that, The step of converting the target distance value based on the contour parallax to form the near-end depth boundary, the median depth boundary, and the far-end depth boundary of the target contour includes converting the target distance value based on the focal length of the first camera, the baseline length of the two cameras, and the target parallax value. Based on the disparity of each contour in the effective contour disparity sequence, the contour depth value is converted, all contour depth values ​​are sorted, and the near-end depth boundary, median depth boundary, and far-end depth boundary of the target contour are extracted.

10. The real-time ranging and modeling system using dual cameras as described in claim 1 or 9, characterized in that, The constraint record includes a constraint rectangle, target distance value, near-end depth boundary of target contour, median depth boundary of target contour, far-end depth boundary of target contour, lower disparity boundary of the next sampling period, and upper disparity boundary of the next sampling period, and is written into the constraint record queue in the order of sampling period.

Images