190 results about "Vanishing point" patented technology

The invention relates to the technical field of computer vision and virtual reality, and discloses a three-dimensional video fusion method based on camera self-calibration and projection texture mapping, and the method comprises the following steps: S1, obtaining video data, and collecting at least one frame of two-dimensional image in a to-be-fused video stream; optionally, the two-dimensional image is preprocessed; and S2, calibrating internal reference of the camera, detecting linear features in the two-dimensional image by using an image processing algorithm, estimating the position of a vanishing point by using a least square method or other optimization algorithms based on the detected linear segment, and calculating an internal reference matrix of the virtual camera according to the optimized vanishing point position. The perspective relation between the video image and the surface of the three-dimensional model is determined through the vanishing point detection technology, accurate fusion of the video image and the three-dimensional model is achieved, the sense of reality of a virtual scene is improved, and the tedious camera calibration process and the complex three-dimensional reconstruction process based on a calibration plate are avoided.

In various examples, a visualization of an environment may be generated using a Panini projection that is optimized based on detected scene content. For example, image data of an environment may be perspective projected (e.g., using a rectilinear projection) to generate a reference projection image, which may be analyzed to detect the presence of vanishing points and / or horizontal lines (e.g., in a central region). The image data of the environment may be projected using a Panini projection that is optimized based on distances to detected objects, the absence of a detected vanishing point, and / or the presence of a detected horizontal line to generate a Panini projection image. In some embodiments, vertical compression is applied to the Panini projection image to correct for distortion of horizontal lines (e.g., based on the presence of a detected horizontal line).

The invention discloses a building outer wall falling risk detection method and system based on deep learning, and the method comprises the steps: obtaining an original surface image, extracting vertical face geometric features, constructing a homography matrix based on vanishing points, carrying out the geometric correction, and reconstructing an orthographic projection image; performing enhancement processing on the orthographic projection image, extracting an effective detection area and intercepting a detection image; inputting the detection image into a deep learning segmentation model into which edge weight constraint is introduced, and performing semantic segmentation, binaryzation and optimization to obtain a falling region; a mapping relation is established according to the actual physical size of the building outer wall, and pixel information of the falling area is converted into multi-dimensional parameters such as the actual physical area, the elevation, the width and the height; and combining the defect intensity input quantification risk assessment model to calculate a comprehensive risk index, determining a risk level and outputting a result. Automatic identification and quantitative evaluation can be realized, distortion and background interference are eliminated, accurate conversion from pixels to a multi-dimensional physical space is realized, and a scientific basis is provided for safety investigation.

The invention discloses an unmanned aerial vehicle pose determination method and device, electronic equipment and a storage medium, and belongs to the field of unmanned aerial vehicle pose determination, and the method comprises the steps: synchronously obtaining a visible light image, a depth image and inertial data of an unmanned aerial vehicle, and extracting a vanishing point set from the visible light image, extracting a plane normal vector set and a feature point set from the depth image; based on geometric constraint conditions that the vanishing point direction is aligned with a preset orthogonal direction and the plane normal vector is perpendicular to the orthogonal direction, constructing a rotation matrix solving model, and optimizing a rotation matrix by taking minimization of a three-dimensional map point projection error as a target; and meanwhile, combining feature point depth information and an inertia pre-integration error, constructing a translation vector solution model for joint optimization, and finally fusing a rotation matrix and a translation vector to output the pose of the unmanned aerial vehicle. Therefore, by implementing the method, the problem of pose drift caused by lack of geometric constraints in the prior art can be solved.

A method includes obtaining, using at least one processing device, image data associated with a scene. The method also includes identifying, using the at least one processing device, multiple line segments based on the image data. The method further includes identifying, using the at least one processing device, one or more boundaries around one or more objects detected in the image data. In addition, the method includes estimating, using the at least one processing device, a position of a vanishing point associated with the image data based on multiple collections of the line segments while excluding, from the multiple collections, one or more of the line segments that overlap with or that are included within the one or more boundaries.

A method includes obtaining, using at least one processing device, a vanishing point and a boundary based on image data associated with a scene, where the boundary is associated with a detected object within the scene. The method also includes repeatedly, during multiple iterations and using the at least one processing device, (i) identifying multiple patches within the boundary and (ii) determining a similarity of the image data contained within the multiple patches. The method further includes identifying, using the at least one processing device, a modification to be applied to the boundary based on the identified patches and the determined similarities. In addition, the method includes generating, using the at least one processing device, a refined boundary based on the modification, where the refined boundary identifies a specified portion of the detected object.

Disclosed are methods, devices, and computer-readable media for detecting lanes and objects in image frames of a monocular camera. In one embodiment, a method is disclosed comprising receiving a sample set of image frames; detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN); fitting lines based on the plurality of markers; detecting a plurality of vanishing points based on the lines; identifying a best fitting horizon for the sample set of image frames via a RANSAC algorithm; computing an inverse perspective mapping (IPM) based on the best fitting horizon; and computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the parallel line fitting.

A method includes obtaining multiple images of a scene using an imaging sensor associated with a vehicle, where the images of the scene capture lane marking lines associated with a traffic lane in which the vehicle is traveling. The method also includes identifying, in each of at least some of the images, a vanishing point based on the lane marking lines captured in the image. The method further includes identifying an average position of the vanishing points in the at least some of the images. In addition, the method includes determining one or more extrinsic calibration parameters of the imaging sensor based on the average position of the vanishing points. The one or more extrinsic calibration parameters of the imaging sensor may include a pitch angle and a yaw angle of the imaging sensor.

The invention belongs to the technical field of computer vision and intelligent traffic, and particularly relates to a roadside long-distance three-dimensional target detection method based on prior geometric information guidance. According to the method, key modules such as vanishing point clipping, three-dimensional query generation and Range Position Embedding are introduced, so that the problems of sparse target pixels, unstable depth estimation and the like in a long-distance scene can be effectively solved, and accurate detection and positioning of multiple types of targets (vehicles, pedestrians, riders and the like) on the road side in a long-distance range are realized. The method can be widely applied to the fields of intelligent traffic infrastructure perception, vehicle-road cooperation, automatic driving and the like, and the intelligence and reliability of road monitoring and safety protection are improved.

A display video correction device includes: a feature extractor that extracts a feature from a camera image obtained from an in-vehicle camera; a Hough transformer that performs Hough transform on the feature extracted; a straight line detector that detects a plurality of straight lines in the camera image, based on a transform result of the Hough transform performed; a vanishing point calculator that calculates, based on the plurality of straight lines detected, first coordinates indicating coordinates of a vanishing point in the camera image; a difference calculator that calculates a difference between the first coordinates calculated and predetermined second coordinates; and a position corrector that corrects, based on the difference calculated, a position of a display area in the camera image to be displayed on a display device.

A method of performing a camera to ground alignment for a camera system on a vehicle. The method includes determining if enabling conditions have occurred and estimating a location of a vanishing point in a source image. Ground lines are selected based on a source image. A lane line detection is performed based on clustering of the ground lines to determine lane lines in the source image. At least one of pitch, yaw, or roll of the vehicle are estimated from the source image. A cost function based on estimates of pitch, yaw, and roll is minimized to obtain an optimal pitch value, an optimal yaw value, an optimal roll value, and lane lines from the source image. A sliding window-based refinement is performed on source images. Alignment results are broadcast to a downstream application or it is determined if the camera system on the vehicle is misaligned.

To provide an image recognition device and image recognition method which confirm whether a vanishing point exists at a correct position and detect an installation abnormality of an on-vehicle camera even when a white line cannot be detected during vehicle travel.SOLUTION: An image recognition device 10 comprises: an image acquisition unit 11 that acquires an image of a scene ahead of a vehicle imaged by an on-vehicle camera 5; and a vanishing point calculation unit (second vanishing point calculation unit 14) that sets a predetermined region in the image as a detection frame and calculates a vanishing point on the basis of movements of feature points within the detection frame. The vanishing point calculation unit sets the size of the detection frame on the basis of at least a traveling speed of the vehicle and a time interval for capturing the image, and calculates the vanishing point on the basis of a position of a feature point detected within the detection frame in a first image captured at a first time and a position of the feature point detected within the detection frame in a second image captured at a second time after the time interval has elapsed from the first time.SELECTED DRAWING: Figure 1

The invention discloses a power transmission line external damage hidden danger identification method based on a vanishing point driving area. The method comprises the following steps: S1, constructing a YOLOv8 network model based on a deep learning convolutional network; s2, acquiring a target image of external damage of the power transmission line, preprocessing the target image to obtain the preprocessed target image, constructing a data set according to the preprocessed target image, and dividing the data set into a training set and a verification set; s3, the YOLOv8 model is trained through the training set, and the trained YOLOv8 model is obtained; s4, optimizing the target images in the verification set to obtain a plurality of optimized target images; s5, slicing the plurality of optimized target images to obtain a plurality of slice images, and performing target detection on the plurality of slice images based on the trained YOLOv8 model to obtain a corresponding low-missing-report-rate target detection result; and S6, carrying out validity identification detection on the low-missing-report-rate target detection result, and sending an alarm signal according to the detection result.

The invention relates to the technical field of image recognition, in particular to an unmanned aerial vehicle relative pose resolving method, which comprises the following steps of: constructing a mapping relation from a world coordinate system to an image pixel coordinate system based on an imaging model and internal and external parameters of a camera; the attitude angles, including a yaw angle, a pitch angle and a roll angle, of the unmanned aerial vehicle relative to the airport runway are estimated by using the pixel coordinates of the left and right sidelines and the vanishing points of the runway in the image in combination with the airport runway structure model; based on the attitude angle estimation result and the camera projection geometrical relationship, establishing an equation set, and solving the vertical height and lateral deviation position parameters of the unmanned aerial vehicle relative to the runway by combining a sideline pixel linear equation in the image and a three-dimensional linear equation of the runway boundary. According to the method, a unified mapping model from a world coordinate system to an image pixel coordinate system is established, an obtained attitude angle is combined with a linear relation between images of left and right side lines of a runway and the world, a linear equation set is constructed, and the vertical height and the lateral deviation distance of the unmanned aerial vehicle are solved.

The invention provides a camera calibration method and device. The method comprises the following steps: calculating internal and external parameters of an initial camera and a first vanishing point coordinate of a lane line in an image coordinate system according to an initial lane line image collected by a to-be-calibrated camera; obtaining a lane line point set under the image coordinate system according to a real-time lane line image collected by a to-be-calibrated camera, and if parallel target lane lines exist and vanishing points exist between the target lane lines, calculating second vanishing point coordinates of the target lane lines under the image coordinate system according to the lane line point set and internal and external parameters of the initial camera; and when the number of the second node coordinates is greater than a threshold value, calculating a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates, and updating initial camera external parameters in the initial camera internal and external parameters. According to the method and the device, the vehicle-mounted camera can be automatically calibrated and updated, so that the calibration precision and efficiency of the vehicle-mounted camera are guaranteed, and meanwhile, the flexibility and the universality of camera parameter calibration are improved.

The invention discloses a video three-dimensional reconstruction key frame extraction method and device based on mutual verification weighted matching, and the method comprises the steps: obtaining an input video stream, and carrying out the pre-classification of an image scene in the input video stream; calculating image feature points and descriptors by using an SIFT (Scale Invariant Feature Transform) detection algorithm, capturing dynamic changes of matching points of adjacent frames based on FLANN matching and pyramid LK optical flow calculation, and screening according to an intersection of FLANN matching and pyramid LK optical flow calculation results to obtain strong matching feature points; and based on the strong matching feature points, calculating vanishing points, endowing surrounding points within a preset range of the vanishing points with weights, constructing a robust index of the similarity of adjacent frames, obtaining the similarity of the adjacent frames, and realizing screening of the key frames of the video stream. According to the invention, the detection precision and stability are improved, and the robustness of the algorithm in diversified scenes is ensured.

The invention relates to the technical field of road engineering, highway engineering and machine vision, in particular to a method and system for sensing subsidence and jolt of a road surface, and the method comprises the steps: obtaining a video collected by an operating vehicle driving recorder, carrying out the frame extraction of the video, and obtaining an image sequence; performing single-frame identification on the image sequence based on a pre-trained neural network model, extracting driving information of an operating vehicle in each frame of image, vanishing point coordinates, a target object and a scene category, and performing summarization and sorting to obtain time sequence data; slicing the time sequence data, analyzing the position change condition of the vanishing point in the running process of the operating vehicle based on the sliced data, and judging abnormal bumping according to the position change condition of the vanishing point; and grading the abnormal jolts, establishing association between the related identification data and the abnormal jolts based on the time sequence data, and storing the associated identification data and the abnormal jolts in a database. According to the invention, the image analysis technology is utilized to realize bumping analysis of the patrol vehicle, so that pavement subsidence and abnormal bumping points can be sensed.

The invention relates to a visual inertial positioning method and device for a railway train fused with scene information, and the method comprises the steps: building four geometric constraint models through combining a visual inertial odometer with the geometric features of a railway environment; the constraint models are respectively a sleeper advancing distance constraint model, a camera-rail surface absolute height constraint model, a rail vanishing point constraint model and a non-integrity constraint model. And the scene prior information is deeply fused with the visual inertial odometer. In a sliding window optimization framework, an IMU pre-integration residual error, a visual reprojection residual error and a geometric constraint residual error are jointly optimized through a dynamic marginalization strategy, multi-source information collaborative pose estimation is realized, high-precision visual inertial positioning of a train is realized, and particularly, excellent robustness is shown in an environment in which GNSS signals are shielded.

An information processing apparatus acquires an image taken by a camera and estimates a vanishing point by inputting the image to a trained model, the trained model being generated by subjecting a learning model for pose estimation to machine learning using a heatmap indicative of a true value for the vanishing point as training data.

The invention relates to an online calibration method and device for a vehicle, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an image acquired by a camera of a vehicle and laser point cloud data acquired by a laser radar; performing calibration feature extraction on the image, predicting a vanishing point position in the image according to the extracted camera calibration feature, and calculating a camera external parameter of the current online calibration according to the vanishing point position; converting the laser point cloud data into a depth map according to relative external parameters between the camera and the laser radar obtained through previous online calibration, and performing image feature extraction on the depth map to obtain point cloud features; and calculating relative external parameters between the current online calibration camera and the laser radar based on correlation features between the image features and the point cloud features, and calculating external parameters of the current online calibration laser radar according to the external parameters of the current online calibration camera and the relative external parameters between the camera and the laser radar. And an online calibration result can be quickly and accurately obtained.

The invention discloses a non-motor vehicle road network construction method based on streetscape images. The non-motor vehicle road network construction method comprises the following steps of S1, obtaining a plurality of streetscape images and metadata and track sequence data of the streetscape images; s2, extracting a motor vehicle lane area and a non-motor vehicle lane area from each streetscape image, identifying road infrastructures and states related to non-motor vehicle lanes, and classifying tracks; s3, fusing the obtained road area, road infrastructure and state and track sequence data to obtain non-motor vehicle geometric road section information, and constructing a topological relation between non-motor vehicle road sections; s4, determining the actual width of the non-motor vehicle lane according to the vanishing points in the directions of the motor vehicle lane and the non-motor vehicle lane and the vanishing points in the horizontal direction; and S5, attaching the attributes of the non-motor vehicle lanes to the road network geometric data to form a non-motor vehicle road network. According to the method, the non-motor vehicle road network containing geometric road section information, a road section topological relation and lane attribute information is constructed.

An object detection device includes a processor. The processor is configured to apply a Hough transform to coordinate points on a UD map to detect a straight line having a predetermined length, and detect a target disparity corresponding to the detected straight line having the predetermined length as a disparity corresponding to an object parallel to a direction of travel of a stereo camera. In the UD map, a target disparity satisfying a predetermined condition among disparities acquired from a captured image is associated with coordinate points, each of the coordinate points having two-dimensional coordinates formed by a first direction and a direction corresponding to a magnitude of a disparity. The processor is configured to convert, in the Hough transform, a straight line passing through coordinate points associated with the target disparity and a predetermined range based on a vanishing point into a Hough space.

The invention relates to the technical field of tunnel image data processing, in particular to a tunnel smoke image recognition method and system based on multi-scale feature fusion. The method comprises the following steps: obtaining the spatial weight of each pixel point, wherein the spatial weight is in negative correlation with the distance from the pixel point to a vanishing point; obtaining an enhanced gradient of each pixel point, wherein the enhanced gradient is in positive correlation with the closeness degree between the second characteristic value and the first characteristic value of the pixel point, the first-order gradient module value and the spatial weight; taking the enhanced gradient greater than a gradient threshold as a seed point to perform connected domain analysis to obtain a candidate cluster in the tunnel image; according to the method, the extinction coefficient of the candidate cluster where the seed point is located is obtained, the pixel point corresponding to the candidate cluster is obtained from the historical image sequence of the current tunnel image, time domain accumulation is carried out on the enhancement gradient and the extinction coefficient of the pixel point, the smoke judgment score of the current tunnel image is obtained, tunnel smoke is recognized, and the accuracy of the recognition result is effectively improved.

Systems and methods for image dense field based view calibration are provided. In one embodiment, an input image is applied to a dense field machine learning model that generates a vertical vector dense field (VVF) and a latitude dense field (LDF) from the input image. The VVF comprises a vertical vector of a projected vanishing point direction for each of the pixels of the input image. The latitude dense field (LDF) comprises a projected latitude value for the pixels of the input image. A dense field map for the input image comprising the VVF and the LDF can be directly or indirectly used for a variety of image processing manipulations. The VVF and LDF can be optionally used to derive traditional camera calibration parameters from uncontrolled images that have undergone undocumented or unknown manipulations.

Provided is an image processing device comprising: a detection unit that detects another vehicle present in an image captured by an image sensor mounted to an own vehicle; and a correction unit that corrects a vanishing point of the image. The correction unit corrects the vanishing point on the basis of a comparison result between a first width of the detected other vehicle and a second width of the other vehicle estimated using the vanishing point.

An information processing apparatus estimates a plurality of vanishing points and an intrinsic parameter of a camera by inputting an image to a learning model trained by machine learning in advance for estimating a vanishing point and an intrinsic parameter of the camera; projects the estimated vanishing points onto a unit sphere in a world coordinate system on the basis of the intrinsic parameter; and calculates a rotation angle indicative of a posture of the camera on the basis of errors between the projected vanishing points and a plurality of reference vanishing points projected onto the unit sphere in advance.

The invention provides a point and line feature-based pose estimation method and electronic equipment. The method comprises the following steps of extracting point features and line features from an environment image; calculating one or more vanishing points in the environment image based on the extracted line features; according to the geometrical relationship between each line feature and the vanishing point, grouping the line features into different dominant direction sets, and removing the line features which do not belong to any dominant direction set to obtain an optimized line feature set; and fusing the point features and the optimized line feature set, taking the fused point features and the optimized line feature set as input together, performing pose estimation through a visual SLAM algorithm, and outputting pose information of the camera. According to the method, the weak texture environment is effectively dealt with, the point features and the line features are fused, and the line features are optimized by using the vanishing points, so that the key problems that the traditional visual SLAM which only depends on the point features is rare in features and easy to fail in tracking in the low-texture structured environment are solved, and stable and high-precision positioning in the severe environment is realized.

The invention provides a vehicle-mounted camera external parameter calibration method, electronic equipment and a program product, and relates to the technical field of vehicles. The method comprises the following steps: when a camera external parameter dynamic calibration starting condition is satisfied, acquiring a road image sequence corresponding to each camera of a vehicle; each group of road image sequence comprises at least one image of a structured road and a corresponding acquisition moment; determining vanishing point positions and horizon line positions corresponding to the cameras by utilizing an end-to-end vanishing point / horizon line detection model based on each group of road image sequences; wherein the end-to-end vanishing point / horizon line detection model is used for outputting the vanishing point position in the road image sequence, or outputting the vanishing point position and the horizon line position in the road image sequence; and according to the vanishing point position and the horizon line position corresponding to each camera, external parameters of each camera are determined. According to the invention, an efficient and simple solution is provided for external parameter calibration of the vehicle-mounted camera.

This invention proposes a camera focal length and attitude calibration method based on camera position and dual vanishing points, belonging to the fields of computer vision, SLAM, and photogrammetry. This invention only requires knowledge of two sets of parallel lines in space, measuring the camera position and the unit direction vectors of the parallel lines, capturing feature images of the parallel lines, obtaining the imaging relationship in the camera, and extracting the corresponding vanishing points. Utilizing the important geometric phenomenon that the angle between two straight lines in the camera coordinate system is equal to the angle between the unit direction vectors of the two sets of parallel lines, a method for quickly solving the camera focal length is established. Finally, the correspondence of unit direction vectors is converted into the correspondence of spatial points between two coordinate systems with the same origin to solve for the camera attitude. This invention reduces the number of vanishing points required for camera calibration, shortens the calibration time, provides a unique solution, and can simultaneously solve for focal length and attitude, among other advantages.