436 results about "Single camera" patented technology

The invention relates to a common-field-of-view-free camera external parameter calibration method and system based on a laser tracker. The method comprises the following steps: firstly, setting a laser tracker and defining a global coordinate system based on calibration of external parameters of a common-field-of-view-free camera of the laser tracker; for each single camera, if no common view exists between the single camera and the tracker, the tracker is transferred to the position with the common view, a temporary calibration plate is arranged, and the pose relation between the temporary calibration plate and the global coordinate system is calculated; when a public view exists, a calibration plate with a target seat is arranged in the public view, a laser tracker is used for measuring coordinates of a target ball, a camera is used for measuring coordinates of a conversion ball, a target ball coordinate system and a conversion ball coordinate system are respectively established and unified, and therefore the pose relation between the camera and the calibration plate and the pose relation between the calibration plate and the global coordinate system are calculated; and finally, obtaining external parameters of the common-field-of-view-free camera by combining all pose relationships. Compared with the prior art, the method has the advantages that the external parameters of the camera can be accurately and efficiently acquired when the camera system without the public view field is processed, the anti-interference capability is strong, the efficiency is high and the like.

The invention provides a method for acquiring the position and attitude of a model through a single camera, and belongs to the technical field of wind tunnel measurement. The method comprises the following specific steps: S1, installing a plurality of to-be-tested coding points on a model; s2, the model is adjusted to an initial attitude, so that the incidence angle, the roll angle and the sideslip angle of the model are all zero; s3, acquiring three-dimensional coordinates of a to-be-measured coding point in the wind tunnel coordinate system, shooting an image of the model by using the main camera, and acquiring two-dimensional pixel coordinates of the to-be-measured coding point in the image; s4, calculating a rotation matrix and a translation vector from a two-dimensional pixel coordinate to a three-dimensional coordinate; s5, moving the model, repeating the step S3, and calculating a rotation matrix and a translation vector from a two-dimensional pixel coordinate to a three-dimensional coordinate; s6, on the basis of the data in the step S4 and the step S5, calculating and acquiring pose information of the model and outputting the pose information; and S7, repeating the steps S5 and S6, and obtaining and outputting the pose information of the model in real time. The data obtained by the method provided by the invention is high in precision and more accurate.

The invention is suitable for the field of image processing, and discloses a fast fusion method for point cloud data of multiple 3D cameras, terminal equipment and a storage medium. The multi-3D-camera point cloud data fast fusion method comprises the steps that calibration data of a plurality of 3D cameras are collected, the calibration data comprise an RGB image, a depth image and an internal reference matrix, and a calibration plate needs to appear in the view fields of the 3D cameras at the same time; generating a three-dimensional point cloud coordinate point set according to the calibration data, and calculating a transformation matrix between the reference camera coordinate system and the non-reference camera coordinate system according to the three-dimensional point cloud coordinate point set; and according to the transformation matrix, registering the three-dimensional point cloud coordinate point set to a preset reference camera coordinate system to obtain merged target point cloud data, the target point cloud data including RGB color values. According to the invention, multi-view complementation is realized, a physical blind area of a single camera can be covered, and the problem of point cloud missing caused by insufficient view field and shielding is solved.

The invention provides a time-space correlation multi-camera personnel tracking method and system, and the method comprises the steps: obtaining video streams of all cameras in a scene, carrying out the frame-by-frame detection of the video streams through a detection module based on a deep learning model, and obtaining pedestrian detection samples under different cameras at the current moment and corresponding appearance features. Associating the detection sample with a historical tracking trajectory to obtain a trajectory in each single camera; and carrying out inter-camera association on the trajectory in the single camera by combining a pedestrian re-identification ReID technology and homography-based spatial constraint to obtain a complete tracking trajectory of the current frame. According to the method, in the aspect of position representation of the trajectory, 2D Kalman prediction is used in an image plane, meanwhile, the 3D space position of the trajectory is predicted and tracked, and the trajectory position can be accurately drawn and output.

The invention discloses an automatic cattle checking method and system based on two-dimensional panoramic vision, and belongs to the technical field of computer vision and intelligent breeding, and the method comprises the following steps: reasonably arranging a plurality of fixed cameras in a cattle farm area, obtaining the internal reference and distortion coefficient of each camera through employing a camera calibration technology, and obtaining the internal reference and distortion coefficient of each camera; performing real-time distortion correction on the collected video frames; on the basis of side-by-side splicing or homography matrix perspective transformation, all paths of corrected images are fused into a complete two-dimensional panoramic image, and a view blind area of a single camera is eliminated; calling a lightweight target detection model on the panorama, and extracting bounding boxes and confidence coefficients of all cattle at one time; sorting according to the confidence from high to low, performing IOU non-maximum suppression de-duplication on the detection frame, and ensuring that the same cattle only counts once in an overlapping region; and the whole process is completed through the edge computing node. According to the invention, efficient, real-time and accurate on-site cattle checking of a large-scale cattle farm can be realized.

A method and system for eliminating 2D features from planar surfaces in 2D images. Three digital images are taken, from a single camera at a fixed position, of a subject such as a pallet of boxes. One image (IA) is taken with ambient lighting, one image (I1) has ambient lighting plus a first added light source, and one image (I2) has ambient lighting plus a second added light source. An output image Q is then computed by Q=(I1−IA) / (I2−IA). Subtracting the ambient image removes ambient diffuse and specular reflections. Division eliminates all variations in the output image caused by color. The only variations that remain are those due to the angle between each surface point's normal direction and the direction from the light to that point. The output image Q, devoid of all colors and 2D features, is well suited for computing a robot grasp of an object in the image.

The invention provides a physical driving measurement method for monocular three-dimensional dynamic displacement of a rotary machine, and belongs to the technical field of crossing of computer vision and industrial state monitoring. According to the method, a high-speed dynamic visual acquisition system is constructed, and a time sequence video stream is obtained by using a cooperative marker; constructing a'Gaussian + motion blur 'composite gradient model taking the motion blur width as an endogenous variable, and jointly resolving sub-pixel edge coordinates and ambiguity by adopting a nonlinear optimization algorithm; decoupling the pixel displacement of the radial X axis, the radial Y axis and the axial Z axis by using the width change and the edge displacement of the marker in combination with the geometric principles of sequential robust filtering and monocular imaging; and finally, outputting a three-way physical vibration waveform by using the calibration conversion factor. According to the method, the fuzzy influence is adaptively eliminated from a physical imaging mechanism, micron-level precision three-direction vibration synchronous measurement is realized only by a single camera, and the hardware cost and the deployment difficulty are reduced.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention relates to the technical field of image processing, solves the problems that in the prior art, UV flat plate printing needs multiple cameras to cover the whole printing platform, and the precision is not high, and provides a scanning image processing method and device based on a single camera, equipment and a medium. The method comprises the steps that a camera is controlled to scan a calibration graph in the main scanning direction to obtain a first area image, the first area image is analyzed to correct distortion, target images are obtained, then the camera moves in the auxiliary scanning direction and repeatedly scans to obtain a second area image, distortion correction is conducted as well, and overlapped parts of the two target images are compared to obtain a target image. And longitudinal shooting overlapping parameters are calculated, and adjacent target area images are spliced by using the two groups of longitudinal overlapping parameters through a mapping splicing technology. Scanning of the whole printing platform can be completed through a single camera, hardware configuration is simplified, image splicing is carried out after multiple times of scanning and distortion correction, and the overall printing precision is improved.

The invention provides a calibration method suitable for a single-camera high-precision scanning system. The calibration method comprises the following steps: S1, carrying out area array mode calibration on a linear array camera; by adjusting the exposure time of the line-scan digital camera, moving the base station to image at different positions of the checkerboard, and solving internal parameters and distortion parameters of the line-scan digital camera; s2, line scanning mode calibration; performing line scanning imaging on the checkerboard to obtain pixel coordinates of angular points of the checkerboard; and separating the actual rotation angle of the checkerboard from the projection error, and further deducing the rotation angle of the real camera, thereby completing calibration. According to the calibration requirement of the linear array camera in a scanning imaging scene, a two-stage algorithm combining area array mode modeling and linear scanning imaging correction is provided, and the problem that a traditional calibration method is not applicable under the condition that a single camera is fixed and moves in a single direction is effectively solved.

The invention provides a panoramic image generation method and device, equipment and a storage medium, and the method comprises the steps: carrying out the single-lens calibration of a wide-angle camera according to a checkerboard calibration method, and obtaining a distortion correction matrix; controlling the wide-angle camera and the array camera to perform time-space synchronous acquisition to obtain an initial panoramic image and a sub-image set; performing nonlinear distortion correction on the initial panoramic image according to the distortion correction matrix to generate a distortionless standardized panoramic image; acquiring sub-image feature information of each sub-image in the sub-image set, and acquiring a projection position, a perspective transformation matrix and a scale transformation matrix corresponding to each sub-image in the standardized panoramic image according to the sub-image feature information; and establishing a panoramic canvas coordinate system according to the standardized panoramic image, projecting each sub-image into the panoramic canvas coordinate system according to the corresponding perspective transformation matrix and the projection position, adjusting the size corresponding to the sub-image in the panoramic canvas coordinate system according to the scale transformation matrix, and generating a panoramic image according to the adjusted panoramic canvas coordinate system.

The invention discloses a compact stripe monocular three-dimensional measurement system and method based on a mechanical grating, and belongs to the technical field of measurement sensing. The invention aims to solve the problems that the existing three-dimensional measurement technology depends on expensive active devices, the structure is complex, and single-frame high-density fringe projection and dynamic measurement are difficult to realize. The system comprises a dense fringe projection device, a monocular acquisition module and a processing unit. The compact stripe projection device is composed of an active light source, a mechanical grating and an optical lens, and is used for projecting a single-frame compact stripe formed by the mechanical grating to the surface of an object to be measured. And the monocular acquisition module is used for acquiring a deformed stripe image which is subjected to height modulation by the surface of an object. And the processing unit is used for synchronously controlling projection and acquisition and resolving the acquired single-frame deformed stripe image based on a pre-established system calibration model so as to reconstruct a three-dimensional point cloud coordinate of the object. The passive mechanical grating is adopted to replace traditional digital projection elements such as DLP, MEMS and DOE, and monocular vision and a single-frame reconstruction algorithm are combined, so that the system has the advantages of low cost, small size, single camera and the like, and has wide application prospects in the fields of miniaturized 3D imaging modules, extremely small endoscopes, portable 3D imaging equipment, dynamic 3D imaging and the like.

The invention relates to the technical field of image shielding processing, in particular to a target graph shielding processing method and system based on variable-focus wide-view-angle cameras, and the method comprises the steps: deploying a plurality of variable-focus cameras based on seat distribution, constructing a monitoring network, and collecting an initial image; identifying desktop features according to the images, establishing a unified coordinate system and mapping each desktop; when shooting is needed, querying candidate cameras according to the coordinate system, and determining a main camera according to an imaging area and a distance score; and performing quality evaluation on the main shot image, outputting the main shot image if the main shot image reaches the standard, otherwise, triggering a dynamic re-shooting mechanism. According to the invention, through cooperation of multiple cameras and intelligent visual angle selection, the problem that a single camera is easy to block is effectively solved, and automatic and high-quality image acquisition and processing are realized.

Disclosed are embodiments for determining the position and speed of a projectile in three-dimensional world coordinates using a single camera and radar. In some embodiments, a method comprises: determining a zero radial speed time of the projectile based on radar measurements of the projectile; determining a radial speed of the projectile at a particular time based on the radar measurements; determining a radial speed slope at the particular time based on the radial speed measurements; determining a two-dimensional image point of the projectile at the particular time; determining a depth scale coefficient at the particular time based on the zero radial speed time, the radial speed, the radial speed slope, the image point, and a homographic transformation; and determining a position of the projectile in space at the particular time based on a three-dimensional position of the camera, the depth scale coefficient, the image point and the homographic transformation.

The invention relates to the technical field of industrial visual inspection, and discloses an asynchronous exposure double-view-field imaging system and method, and the system can achieve the merging of independent view field images collected by two parallel lenses into a single linear array camera through a reflector and a non-polarization beam splitting cube based on the asynchronous exposure double-view-field imaging system. Meanwhile, in cooperation with precise time sequence control, the system drives two independent stroboscopic light sources to be synchronous with line trigger signals of the camera, different view fields are alternately illuminated in odd-even line periods, the camera is made to perform asynchronous exposure, and two paths of image information are collected in a time-sharing mode; and finally, reconstructing two complete, independent and seamlessly spliced detection images from the interleaved data streams through an odd-even line separation algorithm. According to the invention, with the hardware cost of a single camera, the detection visual field is obviously expanded, the detection efficiency and the image geometric precision under the high-line scanning frequency are ensured, and the excellent balance of the performance, the cost and the efficiency is realized.

The invention relates to the technical field of deformation monitoring, and discloses a large-range deformation measurement method and device based on multi-camera space-time consistency under the complex working condition of AI + vision + IMU fusion. The method comprises the steps of multi-camera data synchronization, IMU data fusion, AI algorithm optimization, space-time consistency calculation and deformation compensation. The method comprises the following steps: enabling multiple cameras to jointly work through a space-time synchronization technology to obtain a multi-view image, recording motion information by using an IMU (Inertial Measurement Unit) to compensate visual errors, fusing and optimizing data by using an AI deep learning model, constructing a measurement model through space-time consistency analysis, and carrying out deformation compensation by combining visual and IMU information. The method avoids the measurement limitation of a single camera, effectively eliminates the measurement error of a complex environment, can achieve high-precision real-time dynamic deformation monitoring in a large range, can be widely applied to the fields of civil engineering, aerospace and the like, and has remarkable innovativeness and application value.

The invention relates to a three-dimensional projection deformation measurement and error compensation method based on a single camera, and the method comprises the steps: collecting a plurality of projection images at multiple moments through a single camera, and recording an image intensity value at each pixel coordinate in each projection image at each moment; inputting the image intensity value into a pre-trained deep learning model, and outputting three-dimensional height prediction of a corresponding pixel coordinate at a corresponding moment; performing correlation search based on the image intensity values of the pixel coordinates in each projection image at the reference moment and the deformation moment to obtain an optimal pixel displacement component; based on the three-dimensional height prediction of each pixel coordinate at the reference moment and the deformation moment, constructing an image surface displacement vector at the deformation moment by using the optimal pixel displacement component, calculating a three-dimensional displacement field at the deformation moment based on the image surface displacement vector at the deformation moment, and then performing three-dimensional strain field calculation; and finally, an efficient and robust closed-loop process is formed through error compensation, and measurement and compensation of three-dimensional deformation in a single-camera scene are realized.

Systems and methods are provided herein for creating interactive 3D content for XR devices using a single camera. This may be accomplished by receiving a first piece of content comprising a plurality of segments, wherein the first piece of content is recorded by a first camera. A system may identify a first object within a first segment of the first piece of content and compare the first object with a plurality of 3D models stored in a database. In response to determining that a first 3D model of the plurality of 3D models corresponds to the first object, the system then generates a second piece of content, by combining the first 3D model with the first piece of content. The system also generates an index associated with the second piece of content.

The invention relates to the field of furniture control, and discloses an intelligent massage armchair control method based on posture recognition, comprising a multi-modal sensing fusion module which collects vision, pressure and millimeter-wave radar data through a sensor, constructs a three-dimensional body contour value of a user, and sends the three-dimensional body contour value to the multi-modal sensing fusion module; data are transmitted to an individual feature recording module based on the collected vision, pressure and millimeter wave radar data; and the individual feature analysis module is used for receiving the information acquired by the multi-modal sensing fusion module, judging the body shape of the user by using a K-means clustering algorithm according to the information, and then selecting a preset scheme according to the body shape information of the user to carry out matching judgment on the sitting posture of the current user. According to the invention, data of the infrared camera, the pressure sensor and the millimeter wave radar are integrated through the multi-modal sensing fusion module, multi-dimensional sensing of a visual contour, pressure distribution and a skeleton structure is realized, and the problem of key feature point extraction deviation caused by body type difference of a single camera is effectively solved.

The invention belongs to the technical field of image processing, and particularly relates to a wide dynamic image fusion method and device for a fast moving scene. A single-lens double-light-path light splitting method is adopted, optical systems with the same type of sensors, the same exposure duration and different aperture coefficients are adopted for double light paths, field angles are matched, scenes are the same, and the error is smaller than 0.5 pixel; the method comprises the following steps: performing video image acquisition, performing wavelet transform on acquired two paths of image data, decomposing two paths of image matrixes to obtain a high-frequency detail image and a low-frequency-width dynamic image, and performing multi-depth-of-field fusion on the high-frequency detail image; performing multi-exposure fusion on the low-frequency wide dynamic image to generate HDR wide dynamic data, and performing tone mapping to form an LDR low-frequency image; and combining the high-frequency fusion detail image and the LDR low-frequency image. According to the method, the image without overexposure, with large depth of field and multiple details can be obtained, and the effects of video processing tasks such as target searching and tracking are effectively improved.

The invention discloses a rotating body three-dimensional vibration displacement measurement method based on monocular vision and deep learning. The method comprises the following steps: obtaining a trained semantic segmentation model and a monocular depth estimation model; the trained semantic segmentation model and the trained monocular depth estimation model take the same vibration video to be predicted shot by a high-speed camera as input; predicting the vibration video to be predicted frame by frame according to a time sequence through a trained semantic segmentation model to obtain a semantic segmentation result of the vibration image, and obtaining a center point coordinate of a rotating body according to the semantic segmentation result of the vibration image; performing frame-by-frame prediction on the vibration video to be predicted according to a time sequence through a trained monocular depth estimation model to obtain a depth prediction map of the vibration image; and integrating the center point coordinates of the rotating body and the depth prediction map to obtain the three-dimensional vibration displacement of the rotating body. Based on the trained semantic segmentation model and the monocular depth estimation model, the to-be-predicted vibration video obtained by using a single camera can be predicted, and then the three-dimensional vibration displacement of the rotating body is obtained.

The invention discloses a tunnel structure detection system and method fusing a laser radar and a rotary scanning camera, and the system employs a layered modular design, and comprises a bottom motion unit, a middle support and energy supply unit, an upper detection and synchronization unit, and a top rotary imaging unit. The synchronous control module of the upper layer detection and synchronization unit coordinates the work of each sensor, controls the self-walking servo motor of the bottom layer motion unit to drive the vehicle body to move at a constant speed, and accurately triggers the rotating motor of the top rotating imaging unit to drive the double mirror surfaces to rotate, thereby achieving the 360-degree high-definition imaging of a single camera. The laser radar scanning of the upper detection and synchronization unit is synchronously controlled, and a high-precision crystal oscillator is utilized to simulate GNSS second pulse time service, so that millisecond-level hard synchronization of multi-source data is realized; through high-precision matching fusion of three-dimensional laser point cloud data and image data, integrated high-precision detection of tunnel structure deformation and apparent diseases is completed, and full automation from data acquisition, synchronous fusion to disease identification is realized.

The invention discloses a multi-target tracking method for transformer substation cross-monitoring areas, and relates to the technical field of computer vision and video monitoring. Target detection is carried out on video streams of all cameras through an improved YOLOv8 model; a global attention mechanism and a multi-level feature pyramid are introduced to enhance the detection capability of sheltered and small targets, a detection result is input into an improved DANN model to extract appearance features with domain invariance, and a local motion track and a local identity label of a target are generated in a single camera through cross-frame association. And establishing a global feature pool by taking the main camera as a reference, and distributing global identity labels for targets in the secondary cameras by calculating appearance feature similarity to complete identity unification. And on the basis of the global identity identifier, the local motion trails of the cameras are fused to generate the global motion trail of the same target in the transformer substation area, so that the multi-target tracking precision of the transformer substation in the cross-monitoring-area scene is improved.

The invention provides a multi-mode visual touch fusion sensor of a non-coaxial light path, which realizes multi-mode cooperation of a visual mode and a visual touch mode by performing non-coaxial arrangement on the optical axis of a camera and the central normal of a touch plane on a structural level; a virtual binocular system is constructed under the condition of a single camera by means of a free-form surface light splitting biprism, the number of devices and the assembling freedom degree are reduced, and therefore the integration degree and calibration consistency are improved; a semi-transparent and semi-reflective optical element is matched with an LED light-emitting unit time sequence and a control circuit to realize automatic and rapid mode switching, so that the long-term reliability is improved, and the switching service life is prolonged; through the collaborative design of the self-assembled metal nano reflective coating and the thin polyurethane packaging layer, a high-contrast and wear-resistant deformation image is obtained on the premise of not sacrificing the tactile sensitivity, and finally continuous acquisition of a target pose and contact parameters is realized. And the comprehensive requirements of accurate grabbing and touch closed-loop control on stable imaging, rapid switching, easy integration and high reliability are met.

The invention relates to a laser communication terminal coarse tracking control system, and relates to the technical field of laser communication, the system comprises a turntable module and a data control processing module, the turntable module comprises an azimuth angle turntable and a pitch angle turntable, the pitch angle turntable is provided with a mounting rack through a connecting piece, and the upper end of the mounting rack is provided with a coarse tracking camera; the mounting frame is used for capturing the target light spot position and transmitting the target light spot position to the data processing control module, a support is fixedly arranged on one side of the upper end of the mounting frame, and an infrared camera is fixedly arranged at one end of the support. According to the application, the coarse tracking camera is arranged, the infrared camera is arranged at the upper end of the mounting frame, and the infrared camera can filter ambient visible light clutters due to high penetrability of infrared light to a target light source, so that a clear light spot image is generated, and the problem that a traditional single camera cannot be used in a complex light environment is solved. And furthermore, the success rate of target light source capturing is improved.

The invention discloses a wafer morphology and deformation measurement method, system and device, and the method comprises the steps: employing a single camera to obtain a binocular image of a wafer through different light paths, employing a template matching principle in a first image and a second image of the binocular image, obtaining a matched measurement region, and carrying out the measurement of the morphology and deformation of the wafer. A feature point matching algorithm is adopted to match feature points of a measurement area, pixel coordinates of the matched feature points are converted into three-dimensional coordinates, a reference model of the wafer measurement area is constructed, then morphology and deformation measurement is achieved, and the defect that no wafer detection substitution technology exists at present is overcome.

The invention discloses a hoisting safety detection method and system based on artificial intelligence, and belongs to the technical field of hoisting safety management, and the method comprises the steps of panoramic view perception, target extraction, spatial-temporal trajectory modeling and safety risk assessment. According to the invention, a multi-modal structured target extraction method is adopted for target extraction, RGB images and depth information are integrated, the defect that the recognition rate of a single camera is reduced under shielding and illumination changes is overcome, the three-dimensional position and the motion state of the target can be obtained, and the interaction relationship between the target and a restricted area can be judged; according to the method, spatial-temporal trajectory modeling is carried out by adopting a multi-class target trajectory modeling method, the position change trend of various types of targets is accurately predicted, and the future possible moving region range of the targets is depicted through trajectory distribution modeling, so that prospective support is provided for risk identification; safety risk assessment is carried out by adopting a risk scoring method fusing double risk factors, the comprehensive safety risk is comprehensively assessed, and the accident prevention level in the lifting operation environment is remarkably improved.

The path planning method for the binocular photographing detection machine for the substrate comprises the following steps: S1, firstly, analyzing and processing size data of a PCB (Printed Circuit Board) and two camera lens modules, including equipment parameters and detection board data; s2, carrying out region division on the detection plate based on the two camera lens modules, constraining the working range of the camera lens modules according to the sizes of the camera lens modules and the stroke range of the camera lens modules, and designing a division mode of eight working regions based on the hole site characteristics of the detection plate; s3, establishing an area task pairing scheduling integer programming model of the two camera lens modules, and pre-distributing detection task groups of the two camera lens modules according to a division result of the eight working areas; and S4, a single-region path planning algorithm is adopted, and a variable domain search algorithm is adopted to solve the shortest path of a single camera lens module for connecting all detection holes in the current region. According to the scheme, collaborative photographing is carried out on the PCB through the binocular camera, and the problem of collision interference of the two cameras is prevented.

A method trains machine learning models for filmmaking by varying a single camera parameter, capturing shots, and processing these to adjust the Al model. A system includes processors and memory to perform single-parameter variation, capture and process shots, and enhance Al model understanding. A non-transitory computer-readable medium has instructions for configuring a multi-camera environment, varying a camera parameter, and processing shots to improve an Al model.

The invention discloses a multi-modal visual language navigation method based on a dynamic environment knowledge graph and related equipment, and can be applied to the technical field of artificial intelligence. After the real-time environment image of the area to which the target robot belongs is captured through the single camera, feature extraction is carried out on the real-time environment image to obtain the image features, the to-be-processed category corresponding to the target object is coded to obtain the semantic code, and after the spatial features of the target object are extracted, the target robot is obtained. Constructing a dynamic environment knowledge graph according to the confidence, semantic coding and spatial features of the target object in combination with an external knowledge base, extracting high-dimensional graph features of the dynamic environment knowledge graph, and performing implicit fusion and display modeling in combination with image features and language features corresponding to the target instruction to obtain a navigation state vector containing long-term memory; therefore, the moving operation of the target robot can be controlled based on the navigation state vector, efficient and explainable navigation reasoning is further realized, and the language navigation accuracy is improved at relatively low cost.