617 results about "Fish eye" patented technology

The invention relates to a wharf sensing method and system based on data fusion of a laser radar and a camera, and the method comprises the steps: firstly employing a fisheye camera to shoot a checkerboard calibration plate at multiple angles, employing a Zhang's calibration method to calculate an internal reference matrix and a distortion coefficient of the fisheye camera, and synchronously collecting the data of the laser radar and the camera in real time; the method comprises the following steps: performing distortion removal processing on a camera image, calculating an external parameter matrix between a laser radar and a camera through a feature extraction algorithm, a feature point matching algorithm and a nonlinear optimization algorithm, and performing data fusion through an internal parameter matrix, the external parameter matrix and a bilinear interpolation method to generate fusion point cloud data with color information; and finally, identifying a key target of the wharf by marking and training a CNN model, and calculating three-dimensional parameters of the key target by adopting a specific calculation method, so that a sailor can quickly distinguish the target in the fused point cloud data, the accuracy and efficiency of wharf environment perception are remarkably improved, and accurate environment perception and decision support are provided for ship berthing and departing operation.

The invention relates to the technical field of automatic driving, and discloses a parking assisting method and system for automatic driving. The method comprises the following steps: cascading and splicing a first fisheye image acquired by a main camera in a vehicle body and a second fisheye image acquired by a slave camera to obtain a panoramic image; performing parking space boundary identification and obstacle analysis on the panoramic image to obtain parking space boundary information and obstacle position information, and synchronously performing distance calculation on a radar echo signal of an ultrasonic sensor to obtain distance measurement data; and a parking environment map is created according to the parking space boundary information, the obstacle position information and the distance measurement data, path planning is carried out based on the parking environment map, a parking execution instruction is obtained, and the parking execution instruction is used for vehicle parking auxiliary control. According to the method, the view coverage range and the flexibility of the observation angle are expanded, and the execution precision and safety of parking auxiliary control are improved.

The invention discloses a calibration method, system and device for a vehicle around-view camera and a computer medium, and relates to the technical field of image processing, and the method comprises the steps: collecting a calibration plate image obtained after a calibration plate is shot by the around-view camera on a vehicle; projecting the calibration plate image to a world coordinate system to obtain a distorted image; selecting an initial angular point in the distorted image according to the calibration plate; projecting the initial angular point to a fisheye image coordinate system of a look-around camera to obtain a target angular point; according to the target angular point, generating an initial external parameter of the look-around camera; in the calibration plate image, extracting overlapped corner point parameters in the overlapped shooting area; and based on the initial external parameter, generating a target external parameter which enables the error between the overlapping angular point parameters to be minimum, and taking the target external parameter as the external parameter of the surround-view camera. According to the method, the extraction precision and robustness of the angular points are improved, the continuity and smoothness of the images shot by the surround view camera are improved, splicing gaps and dislocation phenomena are reduced, and the calibration result of the vehicle surround view camera is more accurate.

The invention provides an image correction method and system based on a nonlinear distortion distribution elliptical fisheye lens, and a medium. The method comprises the following steps: establishing a nonlinear mixed distortion imaging model based on a mapping relation between image coordinates and a spherical direction; defining a distortion vector, and performing fitting processing on the distortion vector based on a two-dimensional B-spline function to obtain a distortion function; the method comprises the following steps: acquiring images in a five-surface three-dimensional angle calibration space based on an elliptical fisheye lens, analyzing lens imaging parameters, and generating parameter values of a distortion function; analyzing parameter values of the distortion function based on a nonlinear mixed distortion imaging model to generate calibration parameters, and remapping pixel points in the distorted image based on the calibration parameters; performing image reconstruction on the remapped distorted image to obtain a corrected image; improving the sampling density of a target area by constructing a nonlinear mixed distortion imaging model; the system adapts to complex monitoring environments such as galleries and entrances and exits; and spatial distortion is flexibly expressed through a B spline, so that the modeling capability is improved.

The invention provides a GNSS data quality prediction method and device based on a fisheye camera, and a medium, and belongs to the technical field of data processing, and the method comprises the steps: determining the satellite orbit information of a future time period with a to-be-evaluated observation station as a reference point based on ephemeris prediction; according to the fisheye sky image shot at the to-be-evaluated observation station and the satellite orbit information, determining satellite feature data of a future time period and the occlusion porosity of the future time period; respectively inputting the occlusion porosity of the future time period and at least part of satellite feature data of the future time period into a general prediction model and an observation station prediction model corresponding to the observation station to be evaluated for data quality evaluation, and obtaining a first data quality evaluation index and a second data quality evaluation index; and determining a data quality prediction result based on the first data quality evaluation index and the second data quality evaluation index. A general prediction model and an observation station exclusive prediction model are introduced to carry out double-model evaluation, and accurate prediction of GNSS observation data quality is realized.

The invention provides a robot-based blade interior autonomous crawling sensing method and system, and the method comprises the steps: carrying a multi-sensor array on a robot, the multi-sensor array comprises a binocular fisheye camera array and a ToF sensor array, controlling the robot to start from a path travel starting point, obtaining a panoramic depth image based on the multi-sensor array, and transmitting the panoramic depth image to the robot; recognizing an obstacle based on the panoramic depth image, constructing a three-dimensional point cloud map based on the obstacle, avoiding the obstacle based on the three-dimensional point cloud map, generating a feasible path in the blade, and enabling the robot to move based on the feasible path until the robot reaches a path travel terminal point or a minimum space in which the robot cannot pass. Autonomous crawling perception of the robot in the blade can be ensured, movement of the robot in the blade is ensured, and detection is realized.

The invention discloses a coal mining machine positioning and visual analysis system under a multi-algorithm optimized fisheye camera, and relates to the field of coal mining machine positioning and visual analysis. The deployment acquisition module is installed according to an optimization algorithm, has an intelligent function and acquires images in a multispectral manner; the preprocessing module corrects distortion by using an improved model and enhances an image in combination with multiple algorithms; the identification calculation module integrates multiple technologies to identify features of the coal mining machine and accurately position the coal mining machine; the tracking and positioning module fuses multi-algorithm tracking, and deep learning is used for predicting a trajectory during shielding; the visual analysis module analyzes the state of the coal mining machine through multiple technologies; and the transmission interaction module transmits data by using a fusion technology and provides multiple interaction modes. The method is accurate in positioning, the fisheye camera and the correction technology improve the positioning accuracy, and accurate tracking can be achieved when shielding is carried out; visual analysis is deep, and the state of the coal mining machine and environmental hidden dangers are monitored; data transmission is safe and efficient, and interaction is convenient; the system has strong anti-interference and self-optimization capabilities, can adapt to an underground environment, and improves the coal mining operation efficiency and safety.

A method to fine-tune extrinsic parameters of a fisheye lens applied to surround view splicing is provided. Fisheye images are obtained from multiple fisheye lenses. Both sides of the fisheye images contain calibration boards. Initial extrinsic parameters are obtained based on the fisheye images. Multiple corner points of the calibration boards in the fisheye images are detected. The corner points are arranged so that the corner points are connected into a quadrilateral in sequence. The initial extrinsic parameters are multiplied by a fine-tuning value to obtain intermediate extrinsic parameters. The corner points are projected to a top-view projection by using the intermediate extrinsic parameters. The intermediate extrinsic parameters are adjusted according to the shape, size, and overlapping effect of the area surrounded by the corner points in the top-view projection.

The invention relates to an intelligent ship berthing and leaving self-positioning method and system based on texture point clouds, and the method comprises the steps: firstly employing a fisheye camera to shoot a checkerboard calibration plate at multiple angles, employing a Zhang's calibration method to calculate an internal reference matrix and a distortion coefficient of the fisheye camera, and synchronously collecting the data of a laser radar and the camera in real time; the method comprises the following steps: carrying out orthodontic processing on a camera image, calculating an external parameter matrix between a laser radar and a camera through a feature extraction algorithm, a feature point matching algorithm and a nonlinear optimization algorithm, and carrying out data fusion through an internal parameter matrix, the external parameter matrix and a bilinear interpolation method to generate texture point cloud data with color information; the method comprises the following steps: performing texture point cloud registration through an improved NDT-color algorithm in cooperation with open-loop mileage estimation to preliminarily determine the position change of a ship, optimizing a graph model by using a closed-loop detection and graph optimization technology, generating an optimized global displacement vector, and further generating a wharf high-precision map containing color texture information. And real-time self-positioning of the ship in a wharf high-precision map is realized.

The invention discloses a wafer nanoscale defect detection system and method based on multi-mode and feature fusion, the system comprises a multi-band LED light source group, a CMOS camera, a telecentric imaging lens and a displacement platform, the multi-band LED light source group is arranged at the uppermost part, and the CMOS camera, the telecentric imaging lens and the displacement platform are sequentially arranged below the multi-band LED light source group; the multiband LED light source group comprises an angle gt; an incident light LED with the angle of 70 degrees, an incident light LED with the angle of 10-30 degrees, a coherent light source and a phase shift module; the CMOS camera shoots a wafer picture, and the telecentric imaging lens captures details on the surface of the wafer; the displacement platform bears the wafer and carries out displacement. According to the method, the limitation of two-dimensional detection is broken through, and on the premise of not sacrificing the detection speed, high-sensitivity, high-precision and quantifiable online automatic detection of fisheye, pinhole and segment difference defects is realized through an innovative optical system design, a quasi-three-dimensional morphology reconstruction technology and an advanced deep learning network architecture.

The invention relates to the technical field of vehicle visual perception, and particularly discloses a multi-fisheye camera aerial view construction method and system for automatic driving, and the method comprises the steps: carrying out the offline calibration of each fisheye camera based on a polynomial fisheye imaging model, constructing a pixel back projection lookup table, and carrying out the offline calibration of each fisheye camera; performing table lookup type distortion removal processing on the acquired fisheye image; extracting multi-region checkerboard angular points, and establishing a cross-view-angle corresponding point set; solving an initial homography matrix from each fisheye camera to the aerial view reference plane, constructing a joint objective function including a checkerboard re-projection error, an overlapping region alignment error and a physical scale constraint, and performing global joint optimization on all the homography matrixes; generating each path of aerial view sub-graph by adopting a reverse mapping strategy; according to the method, the calibration process is visual, the geometric accuracy, the real-time performance and the engineering transportability are high, and the method is suitable for automatic parking, a vehicle looking-around system and automatic driving environment perception.

The invention discloses an anti-halo interference fisheye image effective area detection method, which performs modeling simulation on halo interference possibly existing in a fisheye image shooting process aiming at halo intensity, fuzzy degree, color, position and size factors, and constructs a fisheye image effective area detection model based on a deep neural network. The input of the model is a halo interference fisheye image, the output is the center coordinate and size of an effective area, and the anti-halo interference fisheye image effective area detection model comprises a fisheye image type identification module, an effective area identification module and an effective area parameter extraction module. The fisheye image effective area detection method provided by the invention has stronger robustness and higher detection precision for various halo interferences, can effectively solve the problems existing in the traditional method, and further improves the distortion parameter estimation precision and the distortion correction effect of the fisheye image.

The embodiment of the invention provides a multi-view-angle-based stage fusion panoramic look-around image generation method and device, equipment and a computer readable storage medium. The method comprises the following steps: acquiring a target image through a target fisheye camera; constructing a multi-view spherical projection model according to the target image; and according to the spherical projection model, generating a target visual angle panoramic spliced image. In this way, the problems of geometric distortion, obvious edge seams, inconsistent brightness, texture information loss and the like existing in a traditional double-fisheye image splicing method are solved.

The invention relates to the technical field, and discloses a fisheye test needle group for testing a circuit board, which comprises a test board, the test board comprises a heating pressing block, an insulating component and a base, a test sample is placed between the heating pressing block and the insulating component, the test sample comprises a sample body and fisheye needles extending from pins on the sample body, and the fisheye needles are arranged on the base. A plurality of groups of double-end probes and elastic sheets are mounted in the base and are in butt joint with fisheye needles, the fisheye needles, the double-end probes and the elastic sheets form a test needle group, when the test sample sinks, the fisheye needles are in butt joint with the double-end probes, the elastic sheets are in butt joint with the fisheye needles to form two pairs of contacts, and when the test sample sinks to guide a first pair of contacts to appear, the first pair of contacts are in butt joint with the fisheye needles. And the wedge-shaped surface in the sinking direction guides the elastic sheet to incline so as to form the second pair of contacts, so that the current and voltage required by Kelvin testing can be met, the two pairs of contacts and a plurality of testing points are integrated in a single structure, and the use requirement is met by the appearance of lateral force while vertical downward force exists.

The present invention generally relates to a technology for recognizing objects in a Around View Monitoring (AVM) video for a vehicle. In particular, the present invention relates to a DNN-based object recognition method for multi-channel fisheye images of a vehicle AVM apparatus, by projecting multi-channel unit images of fisheye-lens cameras of the vehicle AVM apparatus onto a cylindrical projection plane so as to obtain multi-channel unit projection images, combining the multi-channel unit projection images on a single image so as to obtain a composite projection image, and inputting the composite projection image to a Deep Neural Network (DNN) model so as to obtain object recognition output. The invention has an advantage that AVM apparatus can effectively assist in safe driving by monitoring objects around the vehicle. Further, the present invention has an advantage that object recognition may be effectively achieved by the insufficient computing resources of vehicles.

The invention discloses an image retrieval method and device, electronic equipment, a storage medium and a product. The method comprises the following steps: constructing text description according to a text category; according to the text description, performing retrieval in a database by using a text encoder after fine tuning training to obtain a first target image; wherein the database comprises a plurality of fisheye images and image feature vectors of the fisheye images, the image feature vectors of the fisheye images are obtained through the image encoder after fine-tuning training, and the image encoder after fine-tuning training is obtained by performing fine-tuning training on the image encoder based on the fisheye images. According to the method, the image encoder is subjected to fine tuning training based on the fisheye image, so that the image encoder subjected to fine tuning training has the capability of accurately identifying the image feature vector in the fisheye image, the accuracy of text description and image matching can be improved, and the accuracy of image retrieval can be improved.

A 3D measurement system, a laser scanner and a measurement device are provided. The system includes a 3D measurement device and a 360 degree image acquisition system coupled in a fixed relationship to the 3D measurement device. The 360 degree image acquisition system includes a first photosensitive array operably coupled to a first lens, the first lens having a first optical axis in a first direction, the first lens being configured to provide a first field of view greater than 180 degrees. The image acquisition system further includes a second photosensitive array operably coupled to a second lens, the second lens having a second optical axis in a second direction, the second direction is opposite the first direction, the second lens being configured to provide a second field of view greater than 180 degrees. Wherein the first field of view at least partially overlaps with the second field of view.

The invention discloses a dirt detection method and device for a fisheye camera, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: firstly obtaining a to-be-detected image sequence shot by the fisheye camera, carrying out the feature extraction of the to-be-detected image sequence, obtaining a plurality of first features, and carrying out the multi-scale extraction; performing frequency domain processing on a target feature in the plurality of first features to obtain a frequency domain attention feature map, performing time domain processing on the plurality of first features to obtain a time domain attention feature map, and fusing the frequency domain attention feature map and the time domain attention feature map to obtain a fused feature map; the output mask is processed by the decoder to obtain an output mask, a dark light compensation factor can be determined in combination with the brightness mean value and the expected brightness, and the output mask is corrected to be a target mask; and finally, obtaining a detection result according to the mask, and triggering cleaning when the dirt proportion reaches a preset threshold value. The method can improve the accuracy of dirt detection, enhance the robustness of a sensing algorithm, and guarantee the reliability of an automatic driving low-speed scene.

The invention discloses an environment perception method based on a binocular camera and an all-round fisheye camera, and the method comprises the steps: collecting the three-dimensional information of an image, carrying out the stereo vision obstacle detection, generating a disparity map, obtaining the three-dimensional information through the position deviation of a corresponding point after the comparison with a reference condition, carrying out the target detection based on deep learning, and carrying out the semantic segmentation. The method comprises the following steps: detecting a drivable area, inputting a sensing result into a decision planning module for track planning, judging whether each path point on a track is occupied at a time point or not, adjusting a plan to avoid risks, and inputting the adjusted track into a vehicle control module. And the control instruction is input into a vehicle platform drive-by-wire module to drive an original vehicle execution mechanism to complete execution of an acceleration / steering angle instruction. According to the method, the binocular camera is used for collecting images to calculate the depth information of the target pixel points, the distance information of the obstacle is obtained, in other words, depth information measurement and calculation are completed on the premise that obstacle target recognition is not completed, and the intelligent driving function of driving and parking is achieved.

An apparatus comprising an interface and a processor. The interface may be configured to receive pixel data corresponding to a wide angle field of view from a lens. The processor may be configured to receive a distortion specification corresponding the lens, generate pixel data arranged as a single calibration image in response to the wide angle field of view of the lens, perform calibration operations in response to the distortion specification and the single calibration image, and determine a distortion curve configured to correct a distortion of the lens in response to the calibration operations. A calibration scene may be captured in the single calibration image. The calibration scene may comprise three planes with a dot pattern. The calibration operations may comprise calculating an optical center of the lens and calculating the distortion curve in response to the dot pattern.

In the field of piping inspection technology, a fisheye lens unit and a non-destructive piping inspection device are provided. The system comprises a support plate 1, a connecting sleeve 2, a fixed plate 3, a connecting plate, and a fisheye lens, with an arc-shaped groove 6 drilled in the support plate. A first motor 8 realizes circumferential rotation scanning of the fisheye lens through the meshing transmission of a gear 10 and an internal gear 9, and a second motor 13 of the movement mechanism drives a chain transmission to realize axial movement of the fisheye lens. The two motor control systems operate independently, and a multi-degree-of-freedom flexible support system ensures that the fisheye lens can adapt to different pipe diameters and curved piping environments.

The invention relates to the technical field of camera on-orbit calibration, in particular to a fisheye camera on-orbit calibration method based on a starlight vector, which comprises the following steps of: S1, acquiring a plurality of star maps observed by a fisheye camera on orbit, and extracting and denoising star points to form a star point set; s2, according to fisheye imaging distortion distribution, a star point located in an imaging center area is selected from a star point set, and a starlight vector is constructed based on image point coordinates of the star point in combination with star catalogue data subjected to space-time deviation and propagation path deviation correction; and S3, performing local geometrical relationship star point matching on the starlight vector in the step S2. According to the method, the multiple star maps observed by the fisheye camera in orbit are obtained and processed to form the star point set, so that the problems of poor in-orbit calibration applicability and low precision caused by the fact that most traditional fisheye camera calibration methods adopt a ground pre-calibration or space reference object arrangement mode are solved.

The invention relates to an AprilTag-based panoramic camera 3D calibration method and system and a medium, and the method comprises the steps: M1, obtaining the original images of a front vehicle-mounted fisheye camera, a rear vehicle-mounted fisheye camera, a left vehicle-mounted fisheye camera and a right vehicle-mounted fisheye camera, carrying out the fisheye distortion correction of each image, and obtaining the data information of a distortionless image; and M2, based on the data information of the distortionless image, detecting an AprilTag mark in each distortionless image, extracting a detected angular point coordinate as an image point set, querying a three-dimensional world coordinate corresponding to a predefined calibration plate physical size, constructing a matched three-dimensional point set, and obtaining the data information of the matched three-dimensional point set of the image. According to the method, flexible adjustment of the height and view parameters of the aerial view camera is achieved, repeated calibration is avoided, the parameters of the virtual camera are dynamically corrected by fusing the data of the vehicle attitude sensor, and view angle compensation under the vehicle body inclination state is achieved.

The invention discloses a binocular fisheye stereo matching method constructed by fusing monocular depth prior and multi-scale spherical projection cost. The binocular fisheye stereo matching method comprises the following specific steps: S1, image input and feature extraction; s2, monocular depth priori estimation; s3, constructing a cost matrix of multi-scale spherical projection scanning; s4, the deep attention network is matched; s5, performing initial parallax estimation; s6, obtaining a final fine disparity map based on a ConvGRU disparity optimization network, and the method can accurately and efficiently carry out stereo matching without image correction, and improves the matching precision and enhances the robustness of an algorithm to a distorted image by introducing monocular depth estimation prior and a multi-scale spherical projection scanning strategy; the method also avoids the quality loss caused by image correction, retains the original view field information of the fisheye image, gives consideration to the precision and efficiency through a multi-scale spherical scanning strategy, adapts to fisheye distortion characteristics, introduces an attention mechanism and a GRU structure, and improves the expression capability of a parallax estimation network.

Provided are a depth map generation method, a device and a storage medium, which belong to the technical field of image processing. The method includes: generating, according to a first spherical image acquired by a first fisheye lens and a second spherical image acquired by a second fisheye lens, a first disparity map of a spatial region where a terminal device is located; generating a second disparity map of the spatial region according to depth information of the spatial region acquired by an active depth sensor; and generating a target depth map of the spatial region according to the first disparity map and the second disparity map.

The present invention generally relates to a technology for recognizing objects in a Around View Monitoring (AVM) video for a vehicle. In particular, the present invention relates to a DNN-based object recognition method for multi-channel fisheye images of a vehicle AVM apparatus, by projecting multi-channel unit images of fisheye-lens cameras of the vehicle AVM apparatus onto a cylindrical projection plane so as to obtain multi-channel unit projection images, combining the multi-channel unit projection images on a single image so as to obtain a composite projection image, and inputting the composite projection image to a Deep Neural Network (DNN) model so as to obtain object recognition output. The invention has an advantage that AVM apparatus can effectively assist in safe driving by monitoring objects around the vehicle. Further, the present invention has an advantage that object recognition may be effectively achieved by the insufficient computing resources of vehicles.

The invention discloses a multi-view fisheye camera real-time look-around image splicing method and device, and relates to the technical field of image processing, the method generates a lookup table (LUT) based on calibration parameters, converts a fisheye image distortion removal correction process into a table lookup operation, avoids real-time calculation one by one pixel, significantly reduces the operation overhead, and improves the image quality. And the system can keep low-delay processing under the condition of high-resolution input. According to the method, self-adaptive feathering weight distribution is introduced into a seam band, local energy matching and a multi-band fusion method are combined, and the width and the weight of a splicing area are dynamically adjusted according to scene brightness and contrast difference, so that ghosting and brightness abrupt change at the seam are effectively reduced, and the smoothness of splicing transition is ensured.

The application discloses a fisheye image correction method and system based on a geometric model and an effective area extraction algorithm, and relates to the field of image processing.The method comprises the following steps: scanning pixel points from the center of a picture outward, judging the edge condition of a fisheye image where the current row and column are located according to the scanning result, adaptively adjusting the scanning step length, and further shortening the scanning time; and the ratio of the width and height of the fisheye image to the whole picture can be used to distinguish the light spot and the fisheye image, so that the accurate fisheye image radius can be obtained in real time in different devices and various measured pictures, and the image correction based on the geometric model is carried out by using the parameter, so that the accuracy of the extracted fisheye image is ensured, and the fisheye effective area can be extracted in different devices and various measured pictures.

The invention provides a multi-level domain adaptive pedestrian detection method and system based on a teacher model, and the method comprises the steps: training a target detector through employing source domain perspective image data with a pedestrian label, and obtaining an initial target detector; constructing a target domain teacher model and a cross-domain student model; and performing multi-level domain adaptive adversarial training by using the cross-domain student model, updating parameters of the target domain teacher model through index moving average of the weight of the cross-domain student model, generating a pseudo label to supervise the multi-level domain adaptive adversarial training of the cross-domain student model, and optimizing the cross-domain student model in combination with a designed loss function. Obtaining a final cross-domain student model; and detecting pedestrians in the overlooking fisheye image to be detected based on the final cross-domain student model. According to the technical scheme of the invention, the requirements of people flow statistics, safety monitoring and people flow guidance on rapid and accurate detection of pedestrians in a scene can be met.

The invention relates to a method and a system for automatically parking a trolley, the trolley comprises a three-layer frame and a double-layer plate structure arranged on the three-layer frame, the double-layer plate structure comprises a visual perception layer and a power supply layer, and the three-layer frame comprises an expansion interface layer, a main control layer and a driving execution layer; the automatic car parking method comprises the following steps: acquiring aerial view information by using a four-way fisheye camera, and acquiring obstacle information by using a depth camera; parking space information of the target parking space is detected and obtained through a multi-shape mark point detection algorithm according to the aerial view information; according to the obstacle information and the parking space information of the target parking space, determining a driving route of the car through a mixed path planning algorithm; and controlling the trolley to automatically drive to the target parking space position according to the driving route by using a double-node control algorithm. Compared with the prior art, automatic parking in a narrow or unstructured scene can be realized, and an efficient, safe and reusable verification platform is provided for intelligent driving and industrial automation.