93890 results about "Visual perception" patented technology

The invention discloses an ancient building risk prediction management and control method and system based on a large model, and relates to the technical field of ancient building protection. An ancient building risk prediction management and control system based on a large model comprises a multi-source data acquisition module, a multi-modal data fusion module, a large model risk prediction module, a risk assessment and early warning module, a digital twinborn dynamic management and control module and a repair decision module. According to the method, the structure state data, the visual image data and the environment data are fused, so that the state of the historic building is comprehensively perceived; the self-supervised learning pre-trained ViT-GPT model is combined with the LSTM time sequence model, so that the time sequence analysis capability and the multi-modal data processing capability of risk prediction are remarkably improved, the future damage probability and the risk trend of the historic building can be accurately predicted, and the defects of the prior art in long-time span risk prediction are overcome.

The invention provides a battery film production automation control method and system based on visual inspection, and relates to the technical field of production control, and the method comprises the steps: collecting a film image, carrying out the fusion, generating a fused film image, carrying out the noise reduction, constructing a feature point matching relation, removing wrong point pairs, carrying out the geometric correction, carrying out the sub-pixel-level alignment, and obtaining a registration image; edge enhancement is carried out based on an improved Frangian filtering algorithm, an edge region is repaired and segmented to obtain a film edge contour, three-dimensional reconstruction is carried out, three-dimensional point cloud coordinates are solved, a film three-dimensional contour model is reconstructed, a film defect detection model is constructed, and a film defect distribution diagram and defect confidence are obtained through forward propagation; performing association analysis with a production process, constructing a feature mapping model, generating a defect knowledge base and matching a process parameter optimization strategy, generating a dynamic instruction and sending the dynamic instruction to a station control unit, monitoring operation data, performing closed-loop feedback control, and obtaining a current optimal control scheme.

Using a pre-trained and fixed Vision Transformer (ViT) model as an external semantic prior, a generator is trained given only a single structure / appearance image pair as input. Given two input images, a source structure image and a target appearance image, a new image is generated by the generator in which the structure of the source image is preserved, while the visual appearance of the target image is transferred in a semantically aware manner, so that objects in the structure image are “painted” with the visual appearance of semantically related objects in the appearance image. A self-supervised, pre-trained ViT model, such as a DINO-VIT model, is leveraged as an external semantic prior, allowing for training of the generator only on a single input image pair, without any additional information (e.g., segmentation / correspondences), and without adversarial training. The method may generate high quality results in high resolution (e.g., HD).

The invention belongs to the technical field of bridge diagnosis, and discloses a bridge crack intelligent diagnosis system based on multi-modal data fusion. By fusing multi-source data such as visual images, sound wave detection and vibration signals, comprehensive perception and characterization of crack features are realized; constructing a bridge crack characteristic spectrum diagram by adopting a cross-modal feature extraction and heterogeneous feature coding technology; generating a crack evolution situation map based on space-time correlation analysis and knowledge graph construction; the robustness of the system in a complex environment is improved through environmental adaptability feature enhancement and multi-scale characterization; constructing a bridge safety risk hypergraph in combination with multi-dimensional risk analysis and multi-agent collaborative diagnosis; analyzing and revealing a crack evolution mechanism by applying a causal relationship; and finally, through dynamic fusion and uncertainty quantification, a crack intelligent diagnosis comprehensive report is generated. According to the system, the limitation of traditional single-mode diagnosis is broken through, and dynamic prediction and accurate risk assessment of fracture evolution are realized.

The invention belongs to the technical field of visual special effects, and discloses an AI visual special effect dynamic generation system fusing multi-modal perception. Through cooperative work of six core modules of multi-modal input processing, importance analysis, parameter configuration, parameterized rendering, parameter optimization and rendering and output, deep fusion of special effects and contents is realized. An image sequence, audio data and scene parameters can be processed at the same time, a multi-modal feature data set is constructed, a scene key visual area is scientifically recognized, a visual importance distribution map is generated, and the special effect parameter configuration and rendering process is guided. An improved neural radiation field technology and a physical constraint model are adopted to ensure the consistency and reality of the special effect at different visual angles; and a multi-dimensional quality evaluation and parameter automatic optimization mechanism is introduced, so that the visual expressive force and the artistic value of the special effect are guaranteed. The creation threshold is remarkably reduced, the cooperative expression ability of the special effect and the content is improved, and the special effect becomes a powerful tool for enhancing the narration and enhancing the emotion.

The invention relates to the technical field of image processing, and discloses an intelligent machine vision detection method and system based on image processing, and a storage medium. The method comprises the steps of performing candidate region generation and geometric constraint optimization processing on fusion features through a cascade bidirectional region suggestion network to obtain a target candidate region, and performing instance segmentation processing on the target candidate region through a graph convolutional network and a boundary dynamic adjustment mechanism to obtain a segmentation result; performing feature extraction processing on the segmentation result through a space-time joint attention network to obtain space-time context features; performing multi-target tracking processing on the spatio-temporal context features through depth correlation measurement and motion mode matching to obtain a trajectory prediction result; and performing abnormal mode modeling processing on the trajectory prediction result through hierarchical contrast learning and multi-modal feature fusion to obtain abnormal features, and generating a visual inspection report according to the abnormal features. According to the invention, the accuracy of intelligent machine vision detection based on image processing is improved.

The invention discloses an exhibition hall robot visual language navigation method based on a large model, belongs to the field of exhibition hall robots, and aims at solving the problems existing in an existing exhibition hall robot. According to the method, an exhibition hall robot multi-mode scene sensing system senses a current scene, and candidate target objects are obtained through fusion features of RGB images and laser point clouds; extracting image features of all candidate target objects in the scene; according to the exhibition hall robot multi-mode environment understanding system, an exhibition hall robot receives a voice instruction, a visual language multi-mode large model is used for understanding a target object and a user intention, then high-dimensional semantic features of a target object text are extracted from the system, and the target object is found out from candidate target objects; according to the exhibition hall robot intelligent navigation system based on the multi-modal large model, an exhibition hall robot carries out path planning and autonomous movement according to a pose sequence generated by an intention, and self behaviors are dynamically adjusted according to a real-time changing scene provided by the exhibition hall robot multi-modal scene sensing system.

The invention provides a circuit board welding fault identification method and system based on machine vision, and relates to the technical field of visual identification, and the method comprises the steps: obtaining an image of a circuit board welding region through employing a structured light projection system and a multi-camera imaging system; realizing accurate positioning of a defect region in the region of interest by utilizing three-dimensional shape information and through morphological operation and an edge detection algorithm; for the accurately positioned defect area, obtaining high-precision three-dimensional point cloud data of the defect area, extracting a multi-scale curvature feature descriptor, and fusing the multi-scale curvature feature descriptor with visual features to form a multi-feature descriptor; inputting the multi-feature descriptors into a defect classification and recognition module based on ensemble learning to obtain the types and severity of the welding defects; and according to a defect identification result, the three-dimensional space distribution and severity of the welding defects are visually displayed.

The invention discloses a high-precision image processing method and system based on illumination adaptive compensation, and relates to the technical field of computer vision and image processing, and the method comprises the steps: inputting an original image, and dividing the image into a high-frequency edge layer, an intermediate-frequency texture layer and a low-frequency illumination layer through a multi-scale residual network; acquiring illumination intensity, color temperature and scene categories in real time by using an ambient light sensor and a scene semantic segmentation model, and generating dynamic compensation parameters; carrying out dynamic range expansion on a low-frequency illumination layer based on a physical illumination model, and adjusting the weight of highlight suppression and dark area enhancement through a self-adaptive S-shaped exposure curve; a double-branch generative adversarial network is adopted, noise suppression and super-resolution reconstruction are carried out on the high-frequency layer, and texture detail enhancement is carried out on the intermediate-frequency layer; aligning the data of the depth camera and the infrared sensor with the visible light image through a cross-modal fusion module; and performing tone mapping on the fused image based on human visual characteristics, and outputting an enhanced image with a high dynamic range and reserved details.

The invention relates to the technical field of camera linkage alarm, and discloses a camera linkage alarm method and system for intelligent environment monitoring, and the method comprises the steps: synchronously collecting the video stream data of a plurality of cameras and the multi-parameter monitoring data of an environment sensor; executing multi-path feature analysis and cross-modal feature fusion to obtain environment-vision joint features; environment scene self-adaptive classification is executed, and environment-vision joint features are analyzed through an anomaly judgment model to obtain an abnormal event judgment result; executing multi-camera collaborative verification and vision-environment data association analysis to obtain multi-dimensional abnormal event description information; and selecting a multi-camera linkage response strategy and coordinating the plurality of cameras to carry out three-dimensional monitoring on the abnormal area to obtain a linkage response execution result, so that a false alarm mode can be identified and an abnormal judgment rule can be dynamically adjusted, continuous optimization of the performance of the plurality of cameras is realized, and the false alarm rate and the missing report rate of the plurality of cameras in long-term operation are reduced.

The invention discloses a welded pipe surface defect detection system, particularly relates to the technical field of metal pipe surface quality detection, and is used for solving the problem of defect misjudgment and missing detection caused by mixed reflection interference under single light source irradiation in the existing visual detection technology. An annular light source is adopted for time-sharing pulse triggering through a light source control module, reflected light images at different angles are synchronously collected, a feature extraction module constructs a diffuse reflection intensity ratio and mirror reflection angle distribution matrix to generate a three-dimensional reflection feature spectrum, and reflection anomaly characterization of a defect area is enhanced; the area positioning module screens candidate areas and inhibits interference by combining strength ratio circumferential deviation degree and reflection angle gradient change, and the distortion correction module compensates geometric distortion errors based on curvature radius and station movement parameters, filters pseudo defects of abnormal time sequence fluctuation, and improves the accuracy of the distortion correction. And the defect judgment module utilizes a gradient-gray scale space coupling classification model to distinguish the types of cracks, scratches and corrosion, and finally judges the defect authenticity in combination with reflection characteristic deviation vector superposition.

A multi-modal information tagging method, apparatus and device, and a storage medium and a product. An information tagging effect is effectively improved by means of determining visual feature information of visual information by means of a multi-modal tagging system, performing multi-modal fusion processing on the basis of the visual feature information, the visual information, recognized text information and descriptive text information to obtain image-text fusion features, determining image-text correlation information between the visual information and both the recognized text information and the descriptive text information, on the basis of the image-text fusion features and the image-text correlation information, determining a tagging result for information to be tagged, and performing multidimensional information tagging on the basis of multi-modal information fusion and cross-modal correlations.

In some embodiments, a computer system changes a visual prominence of a respective virtual object in response to detecting a threshold amount of overlap between a first virtual object and a second virtual object. In some embodiments, a computer system changes a visual prominence of a respective virtual object based on a change in spatial location of a first virtual object with respect to a second virtual object. In some embodiments, a computer system applies visual effects to representations of physical objects, virtual environments, and / or physical environments. In some embodiments, a computer system changes a visual prominence of a virtual object relative to a three-dimensional environment based on display of overlapping objects of different types in the three-dimensional environment. In some embodiments, a computer system changes a level of opacity of a first virtual object overlapping a second virtual object in response to movement of the first virtual object.

The invention relates to an automatic driving vehicle trajectory planning method, which comprises the following steps of: extracting comprehensive spatio-temporal information representation of a driving scene from automatic driving multi-modal data by adopting an end-to-end branch, and predicting and generating a trajectory sequence predicted by a vehicle in a future set time period; a visual language model branch is adopted to embed visual features related to the text instruction into a large language model, and a plurality of different track sequences in a future set time period are obtained through prediction; and fusing the trajectory sequences output by the two branches according to the loss value and the danger situation of the current driving scene to obtain a final planning trajectory. Compared with the prior art, the method has the advantages of high safety and reliability.

The invention discloses a multi-modal perception humanoid robot action adaptive control method and system, and belongs to the technical field of computers, and the method comprises the steps: obtaining first multi-modal data; the first multi-modal data comprises first image data, first point cloud data and first force sense data; performing data fusion on the first image data and the first point cloud data, and constructing a three-dimensional model of the scene in real time; generating an optimal path according to the real-time environment data of the three-dimensional model, converting the optimal path into a joint instruction of the humanoid robot, and generating an action control strategy; according to the action control strategy, arm movement of the robot is adjusted through visual servo control, and first operation data are obtained; the path and action of the robot are adjusted in a closed-loop control mode, and second operation data are obtained; and reinforcement learning and regular evaluation are carried out through a DDPG algorithm, and an action control strategy is continuously optimized. According to the invention, multi-modal sensing robot motion adaptive control with strong adaptive capability, high operation precision and good robustness can be realized.

A knowledge fusion multi-modal interaction method and apparatus based on an improved alignment method. The method comprises: acquiring multi-modal data (S101); constructing a query transformation model (S102); on the basis of a multi-stream feature extraction method, performing improved processing on a vision transformer to obtain a multi-scale vision transformer (S103); on the basis of a joint text method, training the multi-scale vision transformer to obtain a target vision transformer (S104); freezing the target vision transformer, and on the basis of the target vision transformer, performing vision-text alignment training processing on the query transformation model to obtain a target query transformation model (S105); connecting the target query transformation model to a language model to obtain a knowledge fusion multi-modal interaction model (S106); and inputting the multi-modal data into the knowledge fusion multi-modal interaction model to obtain an interaction result (S107). In the embodiments of the present invention, the knowledge fusion multi-modal interaction model aligns visual representation with text representation, so that the mutual information between the visual representation and the text representation can be maximized, and the present invention can be widely applied to the technical field of artificial intelligence.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving a user text sequence that characterizes a robotic task to be performed by the agent in the environment; converting the user text sequence into an input text sequence in a standardized format for a visual language understanding task; obtaining an observation image characterizing a state of the environment; processing (i) the input text sequence in the standardized format and (ii) the observation image using a policy neural network to generate a policy output that defines an action to be performed by the agent in response to the observation image; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

The invention discloses a target recognition and grabbing posture planning system for a collaborative robot, and belongs to the technical field of mechanical arms. According to material posture information provided by a sensing module, the optimal grabbing posture is intelligently matched from a preset grabbing strategy library by fusing mechanical arm movement track parameters and target material physical characteristics; the initial accuracy of the grabbing scheme is ensured, in the grabbing process, the camera continuously collects the material poses in real time, and the offset is accurately calculated by comparing the real-time poses with the expected poses. Once material deviation is detected, a visual and force closed-loop adjusting mechanism is triggered immediately, a force-position hybrid control model is utilized, position deviation is synchronously corrected, and grabbing force parameters are dynamically adjusted, so that the grabbing stability and precision are remarkably improved, the grabbing failure rate is effectively reduced, manual intervention and reset time are greatly shortened, and the grabbing efficiency is improved. Therefore, the production efficiency is comprehensively improved.

The invention provides a commodity information processing and querying method and system. Belongs to the technical field of information processing. The method comprises the steps of collecting text information of commodities, preprocessing the text information of the commodities through natural language processing information, and constructing a structured commodity text database; based on the high-definition image of the commodity, key features of the commodity are extracted through machine vision and are associated with information in the text database to form multi-modal commodity information representation; the method comprises the following steps: encoding a commodity text by adopting a BERT model based on Transform, and obtaining a deep semantic representation of the text; performing feature extraction on the commodity image by using a pre-trained convolutional neural network to obtain an advanced visual feature of the image; through natural language processing and machine vision technologies, the text information and the image information of the commodity are subjected to structured and multi-modal fusion, so that the processing efficiency of the commodity information is greatly improved.

The invention discloses a concrete bridge crack detection method based on computer vision, and relates to bridge detection, and the method comprises the steps: collecting a crack image, GPS data and a depth image of a concrete bridge; according to the GPS data, establishing a mapping relation between the pixel coordinates and the fracture physical space coordinates; based on feature matching of ORB features and a registration optimization algorithm based on mutual information, obtaining alignment of the crack image and the depth image at a pixel level, and obtaining a color crack image and a depth crack image; a MaskR-CNN model is adopted to extract a pixel-level crack segmentation mask of the crack region; according to the crack segmentation mask and the registered depth crack image, depth information of a crack region is estimated through a depth completion algorithm based on a conditional generative adversarial network (CGAN), and a crack depth distribution map is obtained; according to the crack depth distribution map, geometric parameters of the crack are extracted through a morphological algorithm and a central axis transformation algorithm. Aiming at the low concrete bridge crack detection precision in the prior art, the detection precision is improved.

The invention provides an industrial robot autonomous collaborative decision-making method and system based on multi-modal sensing and a medium, and belongs to the technical field of industrial robot intelligent control. The method comprises the steps of performing cross-modal space-time alignment to eliminate data space-time differences by collecting visual, tactile and auditory information, realizing multi-modal feature fusion in combination with a dynamic weight adjustment mechanism and an attention calculation model, and generating a joint decision strategy through a deep learning optimization model. The system dynamically allocates sensor weights according to task types, introduces a multi-objective optimization mechanism of energy consumption, precision and safety, and sets a fault-tolerant rule to automatically recover the weights or recalibrate the sensors. According to the method, the problems of rigid data fusion and single optimization dimension in traditional multi-modal decision making are solved, and the precision, the response speed and the environmental adaptability of collaborative operation of the industrial robot are remarkably improved.

The invention discloses a campus green space ownership perception assessment method and system based on multi-modal learning, and the method specifically comprises the steps: inputting the output results of a visual feature coding network and an auditory feature coding network into an audio-visual modal feature fusion module for fusion processing, and obtaining an environment joint representation; inputting the physiological response data set and the psychological assessment data set into a physiological and psychological collaborative encoder for processing to obtain a physiological and psychological joint representation; inputting the environment joint characterization and the physiological and psychological joint characterization into a global fusion module for deep fusion to obtain a final fusion feature; and inputting the final fusion feature into a multi-task prediction channel for processing, and training and optimizing in combination with a loss function to obtain an environment comfort comprehensive scoring result and an explanatory decision basis. According to the invention, comprehensive assessment of campus green space perception is realized, the accuracy and scientificity of assessment are improved, and decision support is provided for optimization of campus space design.

The invention discloses a stainless steel surface defect detection method based on machine vision, and relates to the field of material defect detection.The stainless steel surface defect detection method comprises the steps that stainless steel surface images are collected, the collected multispectral surface images are subjected to image fusion, and a multispectral composite image is generated; performing detail enhancement on the low-resolution image through a super-resolution reconstruction algorithm to obtain a high-resolution image set; positioning a defect area position from the high-resolution image set through a defect detection algorithm, and obtaining a defect position data set; extracting a historical image defect feature set and a real-time image defect feature set from the defect position data set, and constructing a defect classification model according to the historical image defect feature set; inputting the real-time image defect feature set into a defect classification model, and performing defect classification and evaluation to obtain a defect classification data set; and generating and distributing a defect detection report. By combining the multispectral imaging algorithm, the super-resolution reconstruction algorithm and the defect classification model, the automation, precision and efficiency of stainless steel surface defect detection are improved.

The invention discloses a fine-grained multi-modal prompt learning method based on a visual language pre-training model, and relates to the technical field of visual language pre-training models.The method comprises the steps that a multi-modal prompt learning vector used for learning visual features and text features is constructed, and a fine-grained multi-modal prompt learning model is constructed based on the fine-grained multi-modal prompt learning model; global differences, subtle distinguishing visual features and category difference text features among different visual categories are generated, and a multi-modal prompt learning vector is optimized based on a cosine similarity loss function. According to the invention, through combination of dual-granularity visual prompt and text prompt, the problems that in an existing prompt learning tuning method, cooperative reasoning cannot be carried out in a dual-mode prompt representation space, and fine distinguishing features are difficult to capture are effectively solved; and superior migration generalization ability and few-sample learning ability are verified on a plurality of main image identification reference data sets, and the performance is remarkably improved in downstream tasks.

The invention provides a bridge structure crack position identification method based on deep learning and computer vision, and the method comprises the steps: obtaining a multi-view visible light image and a depth image, fusing the visible light image and the depth image according to registration parameters, and generating a fusion image containing the texture and depth information of a bridge; for the fused image, extracting a crack region by adopting an adaptive threshold segmentation algorithm, and dynamically adjusting the threshold to obtain a crack binary image; according to the crack binary image, calculating three-dimensional coordinates of crack pixel points through a three-dimensional reconstruction algorithm, and performing reconstruction to obtain three-dimensional point cloud data containing crack space position and form information; on the basis of the three-dimensional point cloud data, three-dimensional local feature descriptors are adopted to describe geometric attributes of the surface of the crack, crack fragment parameters are calculated, and a three-dimensional crack model is constructed; based on a deep learning model, a transfer learning strategy is adopted, a self-adaptive transferable bridge crack three-dimensional detection method is formed, and model parameters are updated so that the model parameters can adapt to the characteristics of a target bridge.

Automatic generation of in-store product information and navigation guidance, using Augmented Reality (AR) and a Vision-and-Language Model (VLM) and multi-modal Artificial Intelligence (AI). An automated method includes: providing to the VLM images that are captured within a retailer venue by an electronic device that is a smartphone or an AR device or smart glasses; (b) automatically feeding into the VLM those images, or pre-sliced or pre-cropped image-portions of those images that were sliced or cropped using Machine Learning that performs object boundaries detection and not product recognition; (c) invoking the VLM to generate outputs of VLM analysis of content of those fed images or image-portions. The VLM outputs can be: VLM-based product recognition, VLM-based product-related information, VLM-generated virtual shopping assistance, VLM-generated in-store navigation guidance, or other VLM-generated outputs. Based on the VLM-generated outputs, the electronic device provides real-time information about products depicted in the images, VLM-generated shopping assistance, and VLM-generated in-store navigation guidance.

The embodiment of the invention discloses a visual inspection and quality evaluation method for a small structural component of an airplane, and the method comprises the steps: S1, obtaining three-dimensional point cloud data, and carrying out the preprocessing of the three-dimensional point cloud data; s2, carrying out multi-angle two-dimensional image acquisition; s3, mapping the two-dimensional image texture obtained in the S2 to the surface of a three-dimensional model, and generating the three-dimensional model comprising surface information; s4, extracting geometric features and surface texture features from the three-dimensional model; s5, fusion and comparative analysis are carried out to identify potential anomalies and defects; s6, outputting specific abnormal types and positions; and S7, performing comprehensive quality evaluation according to an anomaly detection result in the step S6, and generating a quality evaluation report. According to the technical scheme provided by the embodiment of the invention, by fusing three-dimensional scanning and high-resolution two-dimensional image technologies and adopting the classification model based on machine learning, the comprehensiveness, accuracy and efficiency of detection are remarkably improved, so that the production quality and safety are effectively improved.

A computer system facilitates participation in a communication session with one or more participants, including displaying a spatial representation of one of the participants. In response to detecting that a virtual object is in spatial conflict with a first portion of the spatial representation, the computer system displays the portion of the virtual object in spatial conflict with the spatial representation with reduced visual prominence compared to the visual prominence of a portion of the virtual object not in spatial conflict with the spatial representation.

The invention relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle autonomous obstacle avoidance decision-making method and system based on multi-source sensor fusion, and the method comprises the steps: achieving the time-space synchronization of a laser radar, a visual camera and a millimeter-wave radar through timestamp alignment and coordinate mapping, and constructing a dynamic obstacle grid map; fusing multi-source data based on a dynamic Bayesian network, and dynamically adjusting the confidence coefficient weight of the sensor in combination with the environment illumination intensity and the barrier surface material; and adopting a reinforcement learning model to generate an incremental obstacle avoidance strategy, and triggering a grading response instruction according to the risk assessment grade. The problem of fusion errors caused by spatial-temporal asynchronization of multi-source sensor data is solved, and the real-time obstacle avoidance success rate of dynamic obstacles is increased.

In some embodiments, a computer system changes a visual prominence of a respective virtual object in response to detecting a threshold amount of overlap between a first virtual object and a second virtual object. In some embodiments, a computer system changes a visual prominence of a respective virtual object based on a change in spatial location of a first virtual object with respect to a second virtual object. In some embodiments, a computer system applies visual effects to representations of physical objects, virtual environments, and / or physical environments. In some embodiments, a computer system changes a visual prominence of a virtual object relative to a three-dimensional environment based on display of overlapping objects of different types in the three-dimensional environment. In some embodiments, a computer system changes a level of opacity of a first virtual object overlapping a second virtual object in response to movement of the first virtual object.