104 results about "Ir image" patented technology

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention relates to the technical field of remote sensing monitoring, in particular to a remote sensing target detection method and system for a low-visibility image. The method comprises the following steps: acquiring multi-modal remote sensing image data; carrying out defogging enhancement processing on the low-visibility input image; normalizing the defogged RGB image and the defogged IR image, and then splicing and fusing the RGB image and the IR image; carrying out layer-by-layer coding on the multi-modal fusion image by adopting a mixed trunk structure fusing Transform, Mamba and CNN (Convolutional Neural Network); performing frequency domain decomposition on the trunk output features based on two-dimensional wavelet transform; generating an HR feature map by adaptively selecting a key region; and carrying out cross-scale aggregation on the HR feature map to obtain a detection target frame. Through the multi-modal image defogging enhancement and feature distillation mechanism, the definition and contrast of the remote sensing image in severe weather such as haze and rainy days are effectively enhanced, the shielding interference of environmental degradation on small target detection is weakened, and the stability and adaptability of the model in complex weather scenes are enhanced.

The invention discloses a multi-modal target detection method based on a dual-backbone YOLO architecture, which is applied to the technical field of multi-modal target detection, and comprises the following steps: constructing a multi-modal target detection model based on the dual-backbone YOLO architecture; wherein the detection trunk comprises a double-flow feature extraction network and three fusion Mama blocks, the detection network is composed of a neck module and a head module which are used for multi-modal target detection, and the input of the detection network is the output of the three fusion Mama blocks; the double-flow feature extraction network is used for extracting local features from the RGB image and the IR image respectively; each fusion Mama block comprises a Z scanning state space channel fusion module for performing shallow feature fusion on local features and a double Z scanning state space fusion module for performing deep feature fusion on a shallow feature fusion result; and inputting a to-be-detected image to the multi-modal target detection model to obtain a target detection result. According to the invention, the target detection precision is effectively improved.

The invention relates to a multi-modal dynamic fusion method based on graph attention network low-rank decomposition. The method comprises the following steps: respectively decomposing local reconstruction features and global reconstruction features of different modals from an obtained RGB image, point cloud data and an infrared image by adopting wavelet transform and KA transform; constructing a first graph structure based on the local reconstruction features of different modes, and aggregating nodes in the first graph structure to obtain local fusion features; constructing a second graph structure based on the global reconstruction features of different modalities, and aggregating nodes in the second graph structure to obtain global fusion features; combining the local fusion features and the global fusion features into a combined feature tensor, and constructing an approximate combined feature tensor based on the combined feature tensor through low-rank decomposition; and inputting the approximate joint feature tensor into a Transform encoder, and pooling an output result to obtain a final dynamic fusion feature, and the method improves the accuracy and efficiency of information processing.

Thermal imaging systems are provided. An example thermal imaging system includes an infrared (IR) imager that acquires IR image data of a field of view of the IR imager. The thermal imaging system further includes video analysis circuitry operably coupled to the IR imager. The video analysis circuitry receives first temperature data of a first field reference within the field of view of the IR imager, receives second temperature data of a second field reference within the field of view of the IR imager, and receives IR image data from the IR imager. The video analysis circuitry calibrates the IR imager based upon the first temperature data, the second temperature data, and the IR image data. The thermal imaging system may further include a temperature control chamber enclosing the IR imager and configured to thermally isolate the IR imager and temperature sensors thermally coupled to the IR imager.

A leak of a cold fluid (e.g., a chilled fluid, or a fluid that is initially pressurized and becomes cold on leakage) is detected using a sequence of Infrared (IR) and Visual (VI) images. Using neural nets, in each of VI and IR image-level features are extracted from images and compared with image-level features from images of different times to obtain motion-enhanced features. The motion enhanced features from VI and IR are then compared to obtain fused features from which the leak is detected. The image-level features may be extracted using a neural net with multiple stages. The motion-enhanced and fused features may be obtained in parallel using the image-level features from the multiple stages, and the leak detection based on the stage-specific fused features.

The invention discloses a reflective column scene abnormal depth detection method based on multi-modal fusion, and belongs to the technical field of abnormal depth detection. The method comprises the following steps: firstly, aligning RGB and IR images with different sizes, extracting a ground area in the RGB image through a segmentation network, and mapping the ground area into a depth map to accurately identify the ground; for the problem that the ground near the reflective column is difficult to distinguish, more reliable ground identification is realized by introducing a segmentation network. Meanwhile, for other abnormal noise points in the depth map, a method for screening by using the standard deviation relative difference of the two-frame depth difference and the single-frame standard deviation is provided, so that the robustness of random noise is enhanced. According to the method, the defects of an existing multi-mode method in a high-reflection scene are overcome, and a new solution thought is provided for quality improvement of the ToF depth data in a complex environment.

Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to a Registration Engine that identifies a region of interest from an infrared image portraying a target portion of a physical anatomy in a current physical pose. The Registration Engine generates a 3D point cloud of the region of interest. The Registration Engine identifies region of interest isosurface data by detecting matches between isosurface data and the 3D point cloud. The Registration Engine determines display location, at the target portion of the physical anatomy, for the region of interest isosurface data. The Registration Engine renders an Augmented Reality (AR) display of medical data with respect to the display location.

The invention relates to the technical field of intelligent household appliance control, in particular to an intelligent frying and baking equipment self-adaptive temperature control method and system based on a multi-mode sensor. The system comprises a multi-source sensing module, a feature calculation module, a confidence evaluation module, a model deduction module and a fusion control module. The system calculates temperature fluctuation and angle change rate by collecting infrared, image and angle data; the method is characterized in that a sensor confidence coefficient factor is generated by using a confidence coefficient attenuation model, and a deduction temperature is calculated by combining a heat balance equation; the weights of the measured value and the deduced value are dynamically adjusted based on the confidence coefficient, and fusion temperature is generated for closed-loop control; according to the invention, lampblack shielding and uncovering interference are effectively identified, the problem of sensing distortion is solved, and accurate temperature control in a complex environment is realized.

The invention relates to a test system for testing a LIDAR device (1), comprising a controller (5) for generating LIDAR information relating to a simulated LIDAR object based on a real object to be reproduced for a test process, wherein the LIDAR information includes at least one item of two-dimensional contour information and an item of depth information relating to a virtual distance of the simulated LIDAR object; comprising a LIDAR image generation device (6) for generating the simulated LIDAR object based on the LIDAR information; and comprising a LIDAR projection device (7) for projecting the simulated LIDAR object (9) onto a projection surface (8). The depth information is determined on the basis of a time component between the projection surface (8) and the LIDAR device (1) to be tested, which is defined by the controller (5). In one embodiment, a video image generation device can be provided, in order to generate a visible video image of at least one real object on the projection surface. The IR image which is not visible for people and which is projected by the LIDAR projection device is overlayed in a synchronised manner with a video image that is visible for people. In a variant, the test system is suitable for also testing a stereo camera, in addition to the LIDAR device, e.g. which is integrated in a vehicle to be checked, but can also be provided independently, yet in combination with the LIDAR device.

The invention discloses an abnormal behavior identification method based on a visual large model and a cognitive Agent, and relates to the technical field of computer vision, and the method comprises the steps: collecting an RGB image and an infrared thermal imaging IR image in a monitoring scene, and extracting a human body key point sequence; performing time sequence difference processing on the key point sequence, and constructing a skeleton velocity vector containing a motion trend; inputting the skeleton velocity vector into a Mamba model, adjusting model parameters by using a selective state space mechanism, and carrying out long-time-sequence modeling on the action sequence; predicting the speed variation of the next frame by adopting a residual prediction architecture and reconstructing a human body posture; and calculating an abnormal score according to the prediction error, triggering a cognitive Agent when the score exceeds a preset threshold value, calling a visual large model (VLM) to perform semantic analysis on the current scene environment, and generating a natural language warning containing an abnormal type and an environmental cause. According to the method, the forgetting problem of a traditional model in a long sequence and the jitter problem of a prediction action are solved.

The invention provides an image processing method and device and electronic equipment, and belongs to the technical field of image processing. The image processing method comprises the following steps: acquiring ambient brightness of a shooting scene; under the condition that the ambient brightness is greater than or equal to a brightness threshold value, acquiring a first RGB image shot by the first camera and a second RGB image shot by the second camera, and acquiring the ambient brightness according to the first RGB image and the second RGB image, calculating depth information of an overlapped shooting area of the first camera and the second camera in the shooting scene; under the condition that the ambient brightness is smaller than the brightness threshold value, a first IR image shot by the first camera and a second IR image shot by the second camera are obtained, and according to the first IR image and the second IR image, the brightness of the ambient light is obtained; and calculating depth information of an overlapped shooting area of the first camera and the second camera in the shooting scene. The depth information acquisition accuracy can be effectively improved.

Optical states in TIR-based image displays may be modulated by movement of electrophoretically mobile particles into and out of the evanescent wave region at the interface of a high refractive index convex protrusions and a low refractive index medium. The movement of particles into the evanescent wave region may frustrate TIR and form dark states at pixels. Movement of particles out of the evanescent wave region may allow for TIR of incident light to form bright states at pixels. The movement of the particles may be controlled by employing the drive methods of pulse width modulation, voltage modulation or a combination thereof.

A system includes an infrared (IR) light source(s); IR camera(s); an optical combiner, wherein a set of IR-responsive markers are located in at least one of: (i) within the optical combiner, (ii) on a semi-reflective surface of the optical combiner; and processor(s) configured to: control the IR camera(s) to capture IR image(s) of the optical combiner, whilst controlling the IR light source(s) to emit IR light towards the optical combiner; detect at least a subset of the set of IR-responsive markers in the IR image(s); for a given IR-responsive marker detected in the IR image(s), determine a deformation in a shape of the given IR-responsive marker with respect to a reference shape; and determine a curvature of the optical combiner, based on respective deformations in shapes of IR-responsive markers in at least said subset.

Systems, methods, and computer program products for thermal contrast determinations are provided. An example imaging system includes a first infrared (IR) imaging device that generates first IR image data of a field of view of the first IR imaging device and a computing device connected with the first IR imaging device. The computing device receives probe temperature data from a temperature probe indicative of an external environment of the imaging system and receives the first IR image data from the first IR imaging device. The computing device determines background temperature data based upon the first IR image data, determines gas temperature data based upon the probe temperature data, and determines a thermal contrast for each pixel based upon a comparison between the background temperature data and the gas temperature data. The computing device further determines a detection limit for each pixel as a function of thermal contrast.

An image is rendered based a neural radiance field (NeRF) volumetric representation of a scene, where the NeRF representation is based on captured frames of video data, each frame including a color image, a widefield IR image, and a plurality of depth IR images of the scene. Each depth IR image is captured when the scene is illuminated by a different pattern of points of IR light, and the illumination by the patterns occurs at different times. The NeRF representation provides a mapping between positions and viewing directions to a color and optical density at each position in the scene, where the color and optical density at each position enables a viewing of the scene from a new perspective, and the NeRF representation provides a mapping between positions and viewing directions to IR values for each of the different patterns of points of IR light from the new perspective.

Various embodiments disclosed herein describe a divided-aperture infrared spectral imaging (DAISI) system that is adapted to acquire multiple IR images of a scene with a single-shot (also referred to as a snapshot). The plurality of acquired images having different wavelength compositions that are obtained generally simultaneously. The system includes at least two optical channels that are spatially and spectrally different from one another. Each of the at least two optical channels are configured to transfer IR radiation incident on the optical system towards an optical FPA unit comprising at least two detector arrays disposed in the focal plane of two corresponding focusing lenses. The system further comprises at least one temperature reference source or surface that is used to dynamically calibrate the two detector arrays and compensate for a temperature difference between the two detector arrays.

The application discloses a die temperature control method, an electronic device and a storage medium. The die temperature control method comprises the following steps: acquiring an infrared image of a die casting die and real-time temperature data of an internally embedded thermocouple, fusing the infrared image and the thermocouple data, generating a three-dimensional corrected temperature field containing the surface and internal temperature of the die, when the three-dimensional corrected temperature field is abnormal, outputting an adjustment value of a target adjustment parameter through a parameter prediction model, wherein the parameter prediction model introduces a thermodynamic equation residual constraint in a loss function during training, sending the adjustment value to a die temperature control system for execution, collecting a new temperature field in the next production cycle, and if the deviation from the target temperature field exceeds a threshold value, incrementally adjusting the parameter through a PID algorithm. Through the die temperature control method, the die temperature of the die casting model in die casting can be controlled, and the speed and accuracy of die temperature control can be improved.

The invention discloses a method and system for heating a pole piece by using laser, and the method comprises the following steps: obtaining an image of the pole piece to be heated, processing the image, and recognizing the total width of the pole piece, the width of a coating region and the width of a blank region, so as to determine the positions of the coating region and the blank region; a continuous laser is combined with a laser shaping device to form a light spot matched with the size of the coating area to heat the coating area; an infrared image of the heated pole piece is obtained, a pole piece area lower than a temperature threshold value is recognized, and the pole piece area lower than the temperature threshold value is scanned and heated to a set temperature through a pulse laser; the rolling difficulty of the high-compaction-density pole piece can be reduced, the energy consumption is reduced, the heating temperature uniformity of the pole piece is improved, the thickness consistency of the pole piece is improved, and the phenomena of wrinkles and belt breakage are reduced.

The present invention relates to an IR imaging device including: an image capturing unit which includes a pixel array having a spatially varying exposure (SVE) pixel pattern of a size of N x N in which pixels are classified into pixel groups so that pixels having an identical sensitivity belong to an identical group and pixels having multiple sensitivities are arranged so that pixels having an identical sensitivity are not adjacent to each other, wherein the multiple pixels having the multiple sensitivities concurrently detect light with the multiple sensitivities so as to capture an image; and an image restoring unit for acquiring multiple images having different sensitivities by generating an image having a low resolution for each pixel group by using only data of multiple pixels belonging to an identical pixel group among the data of the captured image and interpolating data of pixels belonging to different pixel groups in a demosaicing method to restore the resolution thereof.

A system comprises a computer including one or more processors and memory. The memory includes instructions such that the one or more processors are programmed to receive at least one of Red-Green-Blue (RGB) image data or infrared (IR) image data from one or more cameras communicatively connected to the one or more processors, where the RGB image data and the IR image data represents an environment including at least one individual disposed along a support surface. The one or more processors receive annotated depth image data that corresponds to the RGB image data and the IR image data. The one or more processors train a neural network with at least one of the RGB image data and the IR image data corresponding to the annotated depth image data, where the neural network is trained to predict when the individual is exiting the support surface.

The invention belongs to the technical field of intelligent driving assistance systems, and particularly relates to an in-vehicle target detection method and system based on binocular multi-modal fusion and a two-stage attention mechanism. According to the method, RGB and IR video streams are synchronously collected through a binocular camera, and invalid frames are removed through time synchronization buffering and quality evaluation; in the first stage, an attention thermodynamic diagram is generated by using lightweight ResNet, ROI is extracted, and target features are weighted and enhanced; carrying out geometric registration on RGB and IR images, and adaptively selecting a parallel multi-modal fusion path according to an illumination index; in the second stage, an improved YOLO network is combined with cross-scale feature fusion, Anchor and dynamic reasoning scheduling detection targets are optimized, and a detection result is subjected to weighted fusion and time consistency optimization. According to the method, the problems of missing detection of small targets in the vehicle, poor environmental adaptability and high calculation delay are effectively solved, and high-precision and high-robustness real-time detection is realized on vehicle gauge level hardware.

Systems, methods, and computer program products for multi-model emission determinations are provided. An example imaging system includes an infrared (IR) imaging device configured to generate second IR image data of a first field of view of the IR imaging device at a second time and a computing device operably connected with the IR imaging device. The computing device receives the second IR image data of the first field of view from the IR imaging device and accesses a first detection model associated with the first field of view of the IR imaging device. The first detection model is generated based upon first IR image data of the first field of view of the IR imaging device generated at a first time. The computing device further generates first spectral absorption data based upon the second IR image data and the first detection model for detecting a fugitive emission.

The present application relates to the technical field of intelligent household appliance control, in particular to a self-adaptive temperature control method and system for intelligent frying and roasting equipment based on multi-modal sensors; comprising multi-source perception, feature calculation, confidence assessment, model deduction and fusion control module; the system collects infrared, image and angle data, calculates temperature fluctuation and angle change rate; the core is to generate sensor confidence factor by using confidence decay model, and to calculate and deduce temperature in combination with heat balance equation; based on confidence, the weight of measured value and deduced value is dynamically adjusted to generate fusion temperature for closed-loop control; the present application effectively identifies oil fume shielding and cover opening interference, solves the problem of perception distortion, and realizes accurate temperature control in complex environment.

Embodiments of systems and methods for vehicle information capture using white light are disclosed. In embodiments, the method may include capturing two or more near-infrared (NIR) or infrared (IR) images of a license plate of a vehicle. The method may include, in response to a determination that the license plate was captured, determining if images containing contiguous images of the license plate were captured. The method may include, in response to a determination that images containing contiguous images of the license plate were captured, determining a target illumination zone and a time that the vehicle will pass through the target illumination zone. The method may include, in response to a determination that the vehicle is in a target illumination zone, initiating a pulse of a white light and capturing a white light image. The method may include determining a license plate number based on the white light image.

The invention is notably directed to a computer-implemented method of detecting obstacles (40) in an area (45) adjacent a vehicle (2) based on low-beam images (32). A low-beam image (32) is an image of a given area, obtained by a camera (223) of the vehicle (2) at night or in low-light conditions while partially illuminating the given area (45) with a low-beam lighting system (236) of the vehicle (2). The method comprises repeatedly performing algorithmic cycles at a processing unit (110) of the vehicle (2). Each cycle comprises: accessing (S311) a low-beam image (32) of an area (45) adjacent the vehicle (2); running (S316) a computational model (50) based on the low-beam image (32) accessed, wherein the computational model is a computer vision model that was trained (S300) based on low-beam images (32) labelled in accordance with potential obstacles therein, for the computational model to detect one or more potential obstacles (40) in the low-beam image (32) accessed; and signalling (S318), in response to an obstacle (40) detected (S317: Yes) in the low-beam image (32) accessed, a control unit (250) of the vehicle (2) to autonomously trigger (S319) a behavioural change of the vehicle (2), such as braking of the vehicle. The model may advantageously rely on IR images obtained while fully illuminating the given area with a high-beam infrared lighting system (234) of the vehicle (2), so as to exploit complementarities between the two types of images. The invention is further directed to related computerized units and computer program products.

A sensor module includes a silicon substrate. A set of isolation walls defines, in the silicon substrate, an array of silicon-based image sensor pixels and an array of cavities. An infrared (IR)-sensitive material in the array of cavities forms an array of IR sensor pixels in a same focal plane as the array of silicon-based image sensor pixels.

The invention relates to the field of early warning and monitoring, in particular to a temperature monitoring and early warning system of a gas turbine, which performs infrared image acquisition based on a monitoring module through cooperative work of the monitoring module, a feature extraction module, an analysis module and an early warning module. The feature extraction module extracts the axial gradient temperature difference of the gas compressor area and the change features of the specific contour of the combustion chamber area. And the analysis module calculates a temperature flow anomaly tendency parameter based on the characteristics of the two dimensions, and judges potential anomaly. And the early warning module controls the second acquisition unit to perform strabismus movement acquisition after receiving the potential abnormality judgment, and analyzes the discrete difference characteristic value of each area. And when the discrete difference characteristic value exceeds a preset standard, the system carries out verification acquisition and judges whether an early warning signal is sent out or not. The system can accurately monitor the temperature change of the gas turbine in real time, early warns potential abnormity in time, and improves the operation safety and reliability of the gas turbine.

This invention provides a robot visual localization method based on target detection, comprising: an image acquisition module that records in real time IR and depth images of the forklift robot's picking direction; an edge computing module that detects the pallet's border information based on the IR image; a false detection filtering module that filters the pallet's border information based on prior knowledge and establishes internal relationships within the borders; and a distance determination module that performs visual localization of the pallet based on the internal relationships within the borders and the depth image. This invention also provides an apparatus for this method, using a more easily deployable single-stage image target detection model to detect the pallet's support legs, enabling the robot to locate the pallet. Then, by combining the depth image with the detection results, the distance between the pallet and the forklift robot is evaluated, and a route is planned to pick up the pallet along with the goods on it.

The present application relates to a kind of fruit damage class and area identification method based on multi-source image, comprising: the visible light and IR imaging system is built;Through visible light and IR imaging system, fruit image acquisition and processing are carried out, obtain multi-modal image dataset, multi-modal image dataset is divided into training set, verification set and test set;DeepLabv3 deep learning model is improved, and ADS-DeepLabv3+ network model is obtained;ADS-DeepLabv3+ network model is trained using training set, the fruit image to be identified is input into the ADS-DeepLabv3+ network model after training, and the fruit damage type and area are obtained.The present application can better obtain the "anti- -trans" characteristics of fruit, improve detection accuracy, have better adaptability and anti-interference;ADS-DeepLabv3+ network model improves the recognition ability of small target damage by integrating feature pyramid network and convolution attention module, and the prediction ability for difficult-to-distinguish samples is improved by the mixed loss function obtained by combining Dice_Loss and Focal_loss loss function.