67 results about "Ir image" patented technology

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention relates to a multi-modal dynamic fusion method based on graph attention network low-rank decomposition. The method comprises the following steps: respectively decomposing local reconstruction features and global reconstruction features of different modals from an obtained RGB image, point cloud data and an infrared image by adopting wavelet transform and KA transform; constructing a first graph structure based on the local reconstruction features of different modes, and aggregating nodes in the first graph structure to obtain local fusion features; constructing a second graph structure based on the global reconstruction features of different modalities, and aggregating nodes in the second graph structure to obtain global fusion features; combining the local fusion features and the global fusion features into a combined feature tensor, and constructing an approximate combined feature tensor based on the combined feature tensor through low-rank decomposition; and inputting the approximate joint feature tensor into a Transform encoder, and pooling an output result to obtain a final dynamic fusion feature, and the method improves the accuracy and efficiency of information processing.

A leak of a cold fluid (e.g., a chilled fluid, or a fluid that is initially pressurized and becomes cold on leakage) is detected using a sequence of Infrared (IR) and Visual (VI) images. Using neural nets, in each of VI and IR image-level features are extracted from images and compared with image-level features from images of different times to obtain motion-enhanced features. The motion enhanced features from VI and IR are then compared to obtain fused features from which the leak is detected. The image-level features may be extracted using a neural net with multiple stages. The motion-enhanced and fused features may be obtained in parallel using the image-level features from the multiple stages, and the leak detection based on the stage-specific fused features.

The invention discloses a reflective column scene abnormal depth detection method based on multi-modal fusion, and belongs to the technical field of abnormal depth detection. The method comprises the following steps: firstly, aligning RGB and IR images with different sizes, extracting a ground area in the RGB image through a segmentation network, and mapping the ground area into a depth map to accurately identify the ground; for the problem that the ground near the reflective column is difficult to distinguish, more reliable ground identification is realized by introducing a segmentation network. Meanwhile, for other abnormal noise points in the depth map, a method for screening by using the standard deviation relative difference of the two-frame depth difference and the single-frame standard deviation is provided, so that the robustness of random noise is enhanced. According to the method, the defects of an existing multi-mode method in a high-reflection scene are overcome, and a new solution thought is provided for quality improvement of the ToF depth data in a complex environment.

The invention relates to the technical field of intelligent household appliance control, in particular to an intelligent frying and baking equipment self-adaptive temperature control method and system based on a multi-mode sensor. The system comprises a multi-source sensing module, a feature calculation module, a confidence evaluation module, a model deduction module and a fusion control module. The system calculates temperature fluctuation and angle change rate by collecting infrared, image and angle data; the method is characterized in that a sensor confidence coefficient factor is generated by using a confidence coefficient attenuation model, and a deduction temperature is calculated by combining a heat balance equation; the weights of the measured value and the deduced value are dynamically adjusted based on the confidence coefficient, and fusion temperature is generated for closed-loop control; according to the invention, lampblack shielding and uncovering interference are effectively identified, the problem of sensing distortion is solved, and accurate temperature control in a complex environment is realized.

The invention relates to a test system for testing a LIDAR device (1), comprising a controller (5) for generating LIDAR information relating to a simulated LIDAR object based on a real object to be reproduced for a test process, wherein the LIDAR information includes at least one item of two-dimensional contour information and an item of depth information relating to a virtual distance of the simulated LIDAR object; comprising a LIDAR image generation device (6) for generating the simulated LIDAR object based on the LIDAR information; and comprising a LIDAR projection device (7) for projecting the simulated LIDAR object (9) onto a projection surface (8). The depth information is determined on the basis of a time component between the projection surface (8) and the LIDAR device (1) to be tested, which is defined by the controller (5). In one embodiment, a video image generation device can be provided, in order to generate a visible video image of at least one real object on the projection surface. The IR image which is not visible for people and which is projected by the LIDAR projection device is overlayed in a synchronised manner with a video image that is visible for people. In a variant, the test system is suitable for also testing a stereo camera, in addition to the LIDAR device, e.g. which is integrated in a vehicle to be checked, but can also be provided independently, yet in combination with the LIDAR device.

The invention discloses an abnormal behavior identification method based on a visual large model and a cognitive Agent, and relates to the technical field of computer vision, and the method comprises the steps: collecting an RGB image and an infrared thermal imaging IR image in a monitoring scene, and extracting a human body key point sequence; performing time sequence difference processing on the key point sequence, and constructing a skeleton velocity vector containing a motion trend; inputting the skeleton velocity vector into a Mamba model, adjusting model parameters by using a selective state space mechanism, and carrying out long-time-sequence modeling on the action sequence; predicting the speed variation of the next frame by adopting a residual prediction architecture and reconstructing a human body posture; and calculating an abnormal score according to the prediction error, triggering a cognitive Agent when the score exceeds a preset threshold value, calling a visual large model (VLM) to perform semantic analysis on the current scene environment, and generating a natural language warning containing an abnormal type and an environmental cause. According to the method, the forgetting problem of a traditional model in a long sequence and the jitter problem of a prediction action are solved.

The invention provides an image processing method and device and electronic equipment, and belongs to the technical field of image processing. The image processing method comprises the following steps: acquiring ambient brightness of a shooting scene; under the condition that the ambient brightness is greater than or equal to a brightness threshold value, acquiring a first RGB image shot by the first camera and a second RGB image shot by the second camera, and acquiring the ambient brightness according to the first RGB image and the second RGB image, calculating depth information of an overlapped shooting area of the first camera and the second camera in the shooting scene; under the condition that the ambient brightness is smaller than the brightness threshold value, a first IR image shot by the first camera and a second IR image shot by the second camera are obtained, and according to the first IR image and the second IR image, the brightness of the ambient light is obtained; and calculating depth information of an overlapped shooting area of the first camera and the second camera in the shooting scene. The depth information acquisition accuracy can be effectively improved.

Optical states in TIR-based image displays may be modulated by movement of electrophoretically mobile particles into and out of the evanescent wave region at the interface of a high refractive index convex protrusions and a low refractive index medium. The movement of particles into the evanescent wave region may frustrate TIR and form dark states at pixels. Movement of particles out of the evanescent wave region may allow for TIR of incident light to form bright states at pixels. The movement of the particles may be controlled by employing the drive methods of pulse width modulation, voltage modulation or a combination thereof.

Systems, methods, and computer program products for thermal contrast determinations are provided. An example imaging system includes a first infrared (IR) imaging device that generates first IR image data of a field of view of the first IR imaging device and a computing device connected with the first IR imaging device. The computing device receives probe temperature data from a temperature probe indicative of an external environment of the imaging system and receives the first IR image data from the first IR imaging device. The computing device determines background temperature data based upon the first IR image data, determines gas temperature data based upon the probe temperature data, and determines a thermal contrast for each pixel based upon a comparison between the background temperature data and the gas temperature data. The computing device further determines a detection limit for each pixel as a function of thermal contrast.

An image is rendered based a neural radiance field (NeRF) volumetric representation of a scene, where the NeRF representation is based on captured frames of video data, each frame including a color image, a widefield IR image, and a plurality of depth IR images of the scene. Each depth IR image is captured when the scene is illuminated by a different pattern of points of IR light, and the illumination by the patterns occurs at different times. The NeRF representation provides a mapping between positions and viewing directions to a color and optical density at each position in the scene, where the color and optical density at each position enables a viewing of the scene from a new perspective, and the NeRF representation provides a mapping between positions and viewing directions to IR values for each of the different patterns of points of IR light from the new perspective.

The invention discloses a method and system for heating a pole piece by using laser, and the method comprises the following steps: obtaining an image of the pole piece to be heated, processing the image, and recognizing the total width of the pole piece, the width of a coating region and the width of a blank region, so as to determine the positions of the coating region and the blank region; a continuous laser is combined with a laser shaping device to form a light spot matched with the size of the coating area to heat the coating area; an infrared image of the heated pole piece is obtained, a pole piece area lower than a temperature threshold value is recognized, and the pole piece area lower than the temperature threshold value is scanned and heated to a set temperature through a pulse laser; the rolling difficulty of the high-compaction-density pole piece can be reduced, the energy consumption is reduced, the heating temperature uniformity of the pole piece is improved, the thickness consistency of the pole piece is improved, and the phenomena of wrinkles and belt breakage are reduced.

A system comprises a computer including one or more processors and memory. The memory includes instructions such that the one or more processors are programmed to receive at least one of Red-Green-Blue (RGB) image data or infrared (IR) image data from one or more cameras communicatively connected to the one or more processors, where the RGB image data and the IR image data represents an environment including at least one individual disposed along a support surface. The one or more processors receive annotated depth image data that corresponds to the RGB image data and the IR image data. The one or more processors train a neural network with at least one of the RGB image data and the IR image data corresponding to the annotated depth image data, where the neural network is trained to predict when the individual is exiting the support surface.

The invention belongs to the technical field of intelligent driving assistance systems, and particularly relates to an in-vehicle target detection method and system based on binocular multi-modal fusion and a two-stage attention mechanism. According to the method, RGB and IR video streams are synchronously collected through a binocular camera, and invalid frames are removed through time synchronization buffering and quality evaluation; in the first stage, an attention thermodynamic diagram is generated by using lightweight ResNet, ROI is extracted, and target features are weighted and enhanced; carrying out geometric registration on RGB and IR images, and adaptively selecting a parallel multi-modal fusion path according to an illumination index; in the second stage, an improved YOLO network is combined with cross-scale feature fusion, Anchor and dynamic reasoning scheduling detection targets are optimized, and a detection result is subjected to weighted fusion and time consistency optimization. According to the method, the problems of missing detection of small targets in the vehicle, poor environmental adaptability and high calculation delay are effectively solved, and high-precision and high-robustness real-time detection is realized on vehicle gauge level hardware.

Systems, methods, and computer program products for multi-model emission determinations are provided. An example imaging system includes an infrared (IR) imaging device configured to generate second IR image data of a first field of view of the IR imaging device at a second time and a computing device operably connected with the IR imaging device. The computing device receives the second IR image data of the first field of view from the IR imaging device and accesses a first detection model associated with the first field of view of the IR imaging device. The first detection model is generated based upon first IR image data of the first field of view of the IR imaging device generated at a first time. The computing device further generates first spectral absorption data based upon the second IR image data and the first detection model for detecting a fugitive emission.

The present application relates to the technical field of intelligent household appliance control, in particular to a self-adaptive temperature control method and system for intelligent frying and roasting equipment based on multi-modal sensors; comprising multi-source perception, feature calculation, confidence assessment, model deduction and fusion control module; the system collects infrared, image and angle data, calculates temperature fluctuation and angle change rate; the core is to generate sensor confidence factor by using confidence decay model, and to calculate and deduce temperature in combination with heat balance equation; based on confidence, the weight of measured value and deduced value is dynamically adjusted to generate fusion temperature for closed-loop control; the present application effectively identifies oil fume shielding and cover opening interference, solves the problem of perception distortion, and realizes accurate temperature control in complex environment.

Embodiments of systems and methods for vehicle information capture using white light are disclosed. In embodiments, the method may include capturing two or more near-infrared (NIR) or infrared (IR) images of a license plate of a vehicle. The method may include, in response to a determination that the license plate was captured, determining if images containing contiguous images of the license plate were captured. The method may include, in response to a determination that images containing contiguous images of the license plate were captured, determining a target illumination zone and a time that the vehicle will pass through the target illumination zone. The method may include, in response to a determination that the vehicle is in a target illumination zone, initiating a pulse of a white light and capturing a white light image. The method may include determining a license plate number based on the white light image.

The invention is notably directed to a computer-implemented method of detecting obstacles (40) in an area (45) adjacent a vehicle (2) based on low-beam images (32). A low-beam image (32) is an image of a given area, obtained by a camera (223) of the vehicle (2) at night or in low-light conditions while partially illuminating the given area (45) with a low-beam lighting system (236) of the vehicle (2). The method comprises repeatedly performing algorithmic cycles at a processing unit (110) of the vehicle (2). Each cycle comprises: accessing (S311) a low-beam image (32) of an area (45) adjacent the vehicle (2); running (S316) a computational model (50) based on the low-beam image (32) accessed, wherein the computational model is a computer vision model that was trained (S300) based on low-beam images (32) labelled in accordance with potential obstacles therein, for the computational model to detect one or more potential obstacles (40) in the low-beam image (32) accessed; and signalling (S318), in response to an obstacle (40) detected (S317: Yes) in the low-beam image (32) accessed, a control unit (250) of the vehicle (2) to autonomously trigger (S319) a behavioural change of the vehicle (2), such as braking of the vehicle. The model may advantageously rely on IR images obtained while fully illuminating the given area with a high-beam infrared lighting system (234) of the vehicle (2), so as to exploit complementarities between the two types of images. The invention is further directed to related computerized units and computer program products.

A sensor module includes a silicon substrate. A set of isolation walls defines, in the silicon substrate, an array of silicon-based image sensor pixels and an array of cavities. An infrared (IR)-sensitive material in the array of cavities forms an array of IR sensor pixels in a same focal plane as the array of silicon-based image sensor pixels.

This invention provides a robot visual localization method based on target detection, comprising: an image acquisition module that records in real time IR and depth images of the forklift robot's picking direction; an edge computing module that detects the pallet's border information based on the IR image; a false detection filtering module that filters the pallet's border information based on prior knowledge and establishes internal relationships within the borders; and a distance determination module that performs visual localization of the pallet based on the internal relationships within the borders and the depth image. This invention also provides an apparatus for this method, using a more easily deployable single-stage image target detection model to detect the pallet's support legs, enabling the robot to locate the pallet. Then, by combining the depth image with the detection results, the distance between the pallet and the forklift robot is evaluated, and a route is planned to pick up the pallet along with the goods on it.

The present application relates to a kind of fruit damage class and area identification method based on multi-source image, comprising: the visible light and IR imaging system is built;Through visible light and IR imaging system, fruit image acquisition and processing are carried out, obtain multi-modal image dataset, multi-modal image dataset is divided into training set, verification set and test set;DeepLabv3 deep learning model is improved, and ADS-DeepLabv3+ network model is obtained;ADS-DeepLabv3+ network model is trained using training set, the fruit image to be identified is input into the ADS-DeepLabv3+ network model after training, and the fruit damage type and area are obtained.The present application can better obtain the "anti- -trans" characteristics of fruit, improve detection accuracy, have better adaptability and anti-interference;ADS-DeepLabv3+ network model improves the recognition ability of small target damage by integrating feature pyramid network and convolution attention module, and the prediction ability for difficult-to-distinguish samples is improved by the mixed loss function obtained by combining Dice_Loss and Focal_loss loss function.

The utility model discloses a dual-light module for a cooking robot, and the dual-light module comprises a dual-light assembly which comprises an RGB camera and an IR camera; the light supplementing assembly comprises a light supplementing lamp, and the light supplementing lamp is matched with the RGB camera and the IR camera; the shell comprises a shell body, the shell body is provided with a first containing cavity, the double-light assembly and the light supplementing assembly are arranged in the first containing cavity, and the RGB camera, the IR camera and the light supplementing lamp are matched with mounting holes in the shell body respectively. Due to the fact that the double-light assembly comprises the RGB camera and the IR camera, the RGB camera and the IR camera are combined, and the RGB camera is used for shooting food materials in the boiler barrel so as to obtain RGB images of the food materials in the boiler barrel; the IR camera is used for shooting food materials in the boiler barrel to obtain IR images of the food materials in the boiler barrel, so that RGB images and the IR images in the cooking process are obtained, the temperature of the food materials in the cooking process can be obtained through background processing, and a data basis is provided for accurately controlling the cooking process.

A LiDAR system includes a LiDAR sensor and an infrared (IR) camera. The LiDAR sensor includes one or more light sources configured to emit a set of light pulses, one or more detectors configured to receive a set of return light pulses, and a processor configured to determine a time of flight for each return light pulse, and obtain a point cloud based on the times of flight of the set of return light pulses. The IR camera is configured to capture IR images concurrently with operation of the LiDAR sensor, analyze the IR images to detect presence of suspicious IR pulses, and provide information about the suspicious IR pulses to the LiDAR sensor. The processor of the LiDAR sensor is further configured to identify one or more points of the point cloud as suspicious points due to interference based on the information about the suspicious IR pulses.

A method for processing RGB-Infrared (RGB-IR) sensor data is provided that includes receiving a raw RGB-IR image, determining whether to process the raw RGB-IR image in day mode or night mode, generating, when day mode is determined, an infrared (IR) subtracted raw Bayer image from the raw RGB-IR image and processing the IR subtracted raw Bayer image in an image signal processor (ISP), and generating, when night mode is determined, an IR image from the raw RGB-IR image.

Provided are an obstacle detection device having a false detection prevention function, an obstacle detection method, and a storage medium. The obstacle detection device (100) of the present invention includes: a stereo camera (110L, 110R) that images left and right IR images; a point cloud data generation section (120) that generates point cloud data having 3-dimensional position information from the IR images imaged by the stereo camera (110L, 110R); an obstacle detection section (130) that detects an obstacle based on the point cloud data; an invalid region identification section (140) that identifies the presence or absence of an invalid region of the point cloud in the point cloud data; and an object recognition section (150) that recognizes a specific object in the invalid region of the point cloud based on the IR images. The obstacle detection section (130) does not determine the invalid region of the point cloud as an obstacle in a case where the specific object is recognized by the object recognition section (150).

A thermal image-forming attachment for a sighting apparatus has a combiner for overlaying a generated thermal image from a display against a visible scene defined along an optical axis of the sighting apparatus, wherein the combiner intersects the optical axis between an objective lens and the visible scene. A first IR camera has a first field of view aligned with the sighting apparatus optical axis; a second IR camera has a second field of view narrower than the first field of view, aligned with the sighting apparatus optical axis. An image processor generates the thermal image on the display, wherein IR image content rendered from the second IR camera is centered within IR image content from the first IR camera. Collimating optics project the thermal image from the display toward the combiner as collimated light aligned with collimated light from the visible scene.

The invention discloses a face recognition method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring a face RGB image and a face IR image of a user, and performing binocular living body verification; after the verification is passed, acquiring illumination intensity; extracting target face RGB features and target face IR features; if the illumination intensity is greater than or equal to a first threshold value, performing face RGB feature matching to obtain a plurality of first matching values; if the maximum first matching value is greater than or equal to a second threshold value, the face recognition is successful; if the maximum first matching value is smaller than a second threshold value, determining that the user is a stranger; if the illumination intensity is smaller than the first threshold value, face IR feature matching is carried out, and a plurality of second matching values are obtained; if the maximum second matching value is greater than or equal to a third threshold value, the face recognition is successful; and if the maximum second matching value is smaller than a third threshold value, determining that the user is a stranger. By implementing the technical scheme provided by the invention, the accuracy of face recognition can be improved.

A camera device according to the present embodiment may comprise: a first light-emitting unit for irradiating an object with an infrared structured light pattern; a second light-emitting unit for irradiating the object with infrared light; a light-receiving unit including an RGB-IR sensor for detecting the infrared structured light pattern, the infrared light, and visible light, which are reflected from the object; an image processing unit for acquiring depth information, IR image information, and visible light information of the object by using the infrared structured light pattern, the infrared light, and the visible light received by the light-receiving unit; and a control unit for controlling a period for irradiating the object with the infrared structured light pattern by the first light-emitting unit and a period for irradiating the object with the infrared light by the second light-emitting unit.

The invention discloses an image fusion method and device, a shooting terminal and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: determining image data obtained through image collection, and the image data comprises a first RGB image, a first IR image and a first depth image; performing epipolar correction on the image data to obtain a second RGB image corresponding to the first RGB image, a second IR image corresponding to the first IR image and a second depth image corresponding to the first depth image; inputting the second RGB image and the second IR image into a pre-trained deep learning model, and outputting to obtain a disparity map between the second RGB image and the second IR image; and performing depth map conversion on the disparity map to obtain a third depth map, and performing image fusion on the second depth map and the third depth map to obtain a target depth map. The accuracy of the depth map shot by the shooting terminal is improved.

The application provides an interactive control method and system based on robot control, and relates to the technical field of robot control.The method comprises the following steps: performing differential processing on the visible light image and the infrared image in the early education environment, and extracting the suspicious target area in the differential image; designing a feature descriptor for the suspicious target area, and performing feature analysis on the suspicious target area according to the feature descriptor to obtain a feature analysis result; performing preliminary discrimination on the suspicious target area according to the feature analysis result, and processing the target set after discrimination through a clustering algorithm to obtain a target list.The application improves the accuracy of target recognition in the early education environment, and enables the early education robot to perform personalized interactive response according to the recognition result, thereby enhancing the educational interactivity and interest.