196 results about "Scene segmentation" patented technology

The invention relates to the technical field of image processing, in particular to an image fusion driven anti-collision identification method and system. The method comprises the following steps: acquiring multi-source image data, and performing semantic guidance image fusion according to the multi-source image data to obtain image fusion data; performing scene segmentation according to the image fusion data to obtain scene segmentation data, and performing target detection on the scene segmentation data to obtain scene target data; performing confidence hierarchical space positioning according to the scene target data to obtain target positioning data, and performing motion trail prediction according to the target positioning data to obtain motion trail prediction data; and performing collision area propagation risk assessment according to the motion trail prediction data to obtain collision risk data. By fusing image perception depth, semantic understanding dimension and trajectory evolution modeling, the method has higher environmental adaptability, weak target recovery capability and system interpretability.

The invention relates to the technical field of camera adjustment, in particular to a camera adaptive adjustment method and system and a storage medium. The method comprises the following steps: acquiring a multi-source environment sensing data set and an initial environment image; performing environment feature recognition on the initial environment image according to the multi-source environment sensing data set to obtain a scene environment feature map; performing semantic scene segmentation on the scene environment feature map to obtain a semantic scene macroscopic feature map; performing time sequence attention target tracking on the scene environment feature map to obtain a target vehicle motion feature map; performing feature constraint fusion on the semantic scene macroscopic feature map and the target vehicle motion feature map to obtain a target vehicle-scene motion semantic map; and performing parameter adjustment on the camera according to the target vehicle-scene motion semantic map to obtain a camera parameter adjustment strategy set. According to the invention, the performance and the image quality of the camera in a changeable environment are improved.

The invention provides a semantic information reasoning method guided by a remote sensing scene graph, which can be applied to the technical field of remote sensing image processing. The method comprises the following steps: segmenting a remote sensing image to generate a scene segmentation image; executing target detection to generate an image block set; performing fine-grained analysis on the image global features and the image block set to generate a scene description text; performing grammar analysis, extracting a triple of objects, object attributes and relation information among the objects, and generating a remote sensing scene graph related to the problem; encoding the problem text into an embedded vector, projecting the embedded vector to a visual feature space, processing the embedded vector through a frozen self-attention layer and a learnable gating layer, and outputting an image-text interaction feature; carrying out attention fusion and double gating balance on the question coding features, and outputting a text code guided by a scene graph; and fusing the interaction features and the text codes, and reasoning local semantic information of the target area. The invention further provides a semantic information reasoning device and equipment guided by the remote sensing scene graph and a medium.

The invention discloses a building construction intelligent safety monitoring method based on the Internet of Things, and the method comprises the following steps: obtaining construction environment data, structure state data and operation behavior image data, and carrying out the preprocessing; environment state modeling, local structure strain and behavior recognition and operation scene segmentation are carried out through the edge intelligent processing unit; feature fusion is carried out, and a fusion situation vector is constructed; performing high-frequency anomaly identification and emergency preliminary screening, and generating an edge preliminary early warning result and a high-risk data fragment; constructing a safety evolution trajectory crossing a time window, and fusing historical data to generate a risk semantic map; cloud semantic reasoning operation is executed, and a final risk level judgment result and a corresponding trigger source identifier are generated; and automatically triggering a safety response instruction according to a risk level judgment result. According to the method, the Internet of Things and intelligent semantic analysis are fused, multi-source safety monitoring and self-adaptive response are realized, and the method has the advantages of high real-time performance, global perception and continuous optimization.

The invention discloses an infrared image enhancement method based on scene segmentation. The method comprises the following steps: (1) inputting an original infrared image into an infrared image scene segmentation network based on edge feature guidance to obtain a scene soft segmentation result; (2) decomposing the infrared image into a background layer image and a detail layer image by adopting bilateral filtering; (3) performing adaptive enhancement on the detail layer image based on characteristics of different scenes; (4) enhancing the background layer image by adopting contrast limited adaptive histogram equalization; and (5) carrying out weighted fusion on the enhanced detail layer image and the background layer image. According to the infrared image enhancement method based on scene segmentation, the semantic segmentation algorithm is introduced, the infrared image is divided into different scene areas, and the adaptive infrared image detail enhancement parameters are set for the detail information and noise features of each scene area, so that the quality of the infrared image can be effectively improved.

The invention provides a building safety assessment method and system based on optical fiber strain detection, and relates to the technical field of force detection and data processing, and the method comprises the steps: carrying out the scene segmentation of a target building structure, configuring an optical fiber sensing array, receiving optical fiber sensing data, combining a stress-optical database, carrying out the signal preprocessing and strain calculation, and determining a four-dimensional strain field; a fuzzy hierarchy module is supervised and trained, a temperature drift compensation branch is introduced, temperature drift compensation correction and building risk assessment are performed, a risk distribution field is determined, and sub-level assessment fitting is performed, so that the technical problems of high difficulty in strain acquisition and data processing and limited analysis precision and accuracy in the prior art are solved; and the building safety assessment error is large. Data validity is improved through dynamic and static scene segmentation and optimized sampling, fuzzy analytic hierarchy process and temperature drift compensation are executed with a stress-light database as an analysis base line, and the precision and efficiency of strain analysis and building evaluation can be effectively improved.

An unmanned aerial vehicle landing method, comprising: acquiring a ground image within a visual range of an unmanned aerial vehicle (S101); performing scene seg mentation on the ground image, and on the basis of a scene segmentation result, acquiring a plurality of candidate alternate landing areas from a ground range corresponding to the ground image (S102); on the basis of the current position of the unmanned aerial vehicle and the positions of the candidate alternate landing areas, planning candidate flight paths for the unmanned aerial vehicle to land at the candidate alternate landing areas (S103); on the basis of the candidate flight paths, determining a target flight path and a target alternate landing area corresponding to the target flight path (S104); and landing at the target alternate landing area on the basis of the target flight path (S105).

The invention discloses a processing method and system for full-scene ground object segmentation based on a visual large model, and belongs to the technical field of computer vision. Comprising the following steps: acquiring video data shot by an unmanned aerial vehicle, and preprocessing image data in a video; the method comprises the following steps: detecting categories in image data by using Grounding DINO through preset information, and obtaining an instance segmentation mask of a detected frame by using an SAM segmentation model; using CLIPSeg to obtain a primary segmentation mask of the category; determining a sampling point in the primary segmentation mask, and inputting the sampling point into the SAM segmentation model to obtain a secondary segmentation mask of the category; and combining the primary segmentation mask with the secondary segmentation mask to obtain a panoramic segmentation result. Compared with the prior art, the method has the beneficial effects that the problem that a traditional SAM segmentation model does not have the ability of text recognition and semantic analysis between objects is solved, and full-scene segmentation of the view angle picture of the unmanned aerial vehicle according to the cue word is realized.

A method includes obtaining, from an image sensor, image data of a real-world scene; obtaining, from a depth sensor, sparse depth data of the real-world scene; and passing the image data to a first neural network to obtain one or more object regions of interest (ROIs) and one or more feature map ROIs. Each object ROI includes at least one detected object. The method also includes passing the image data and sparse depth data to a second neural network to obtain one or more dense depth map ROIs; aligning the one or more object ROIs, one or more feature map ROIs, and one or more dense depth map ROIs; and passing the aligned ROIs to a fully convolutional network to obtain a segmentation of the real-world scene. The segmentation contains one or more pixelwise predictions of one or more objects in the real-world scene.

A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and / or other sensor data may be stitched together, stacked, and / or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and / or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and / or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

The invention belongs to the technical field of substation inspection, and provides a substation inspection robot multi-sensor fusion obstacle avoidance method and system, and the technical scheme is as follows: obtaining multi-modal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data and visual image data; preprocessing the obtained multi-modal sensor data to obtain effective observation data, and determining initial obstacle data based on the effective observation data; time synchronization and fusion are carried out on the obtained multiple kinds of initial obstacle data, the fused data serve as obstacle data, scene segmentation is carried out in combination with multiple kinds of sensor data in the fusion process, confidence coefficient decision rules are dynamically set according to different scenes, and obstacle information is determined in combination with the set confidence coefficient decision rules; and generating an optimal obstacle avoidance route based on the determined obstacle information. The problem that a traditional single sensor cannot cope with a complex scene of substation inspection business is solved, and the overall stable operation capacity of the robot is improved.

The present invention discloses a forest unstructured scene segmentation method based on multispectral image fusion. Aiming at the situations of complex background, multiple light and shadow interference in forest unstructured scenes, the method introduces enhanced vegetation index (EVI) data, designs a parallel dual encoding structure to extract RGB and EVI features, and forms multimodal complementary features in the encoding process. In addition, the encoding stage also uses dilated convolution to increase the receptive field of the network and optimize the feature extraction process. Then, the decoded features are fused with the fused features generated by the encoding for a second time to obtain a multispectral fusion convolutional neural network, and then the network is trained and RGB and EVI images are input to realize the semantic segmentation of forest unstructured scenes. The present invention effectively solves the problems of poor adaptability and mis-segmentation of unstructured segmentation methods based on RGB images, and improves the accuracy and robustness of semantic segmentation of forest unstructured scenes.

The invention discloses a double-arm clothes folding robot control method and device, equipment and a storage medium. The method comprises the following steps: determining three-dimensional point cloud data of a target workbench according to a collected image through a visual perception assembly; determining a scene label graph and a desktop edge line set through a scene division assembly according to the three-dimensional point cloud data, determining a clothing mask of the to-be-processed clothing according to the scene label graph through a clothing classification assembly, and determining a clothing description vector according to the clothing mask; determining a low-risk action set and a protection area mask from the candidate clothes folding action set through a safety protection component according to the scene label graph, the desktop edge line set and the clothes mask; determining a target clothes folding action sequence through an action planning component according to the clothes description vector, the low-risk action set and the protection area mask; and folding the to-be-processed clothes through the robot control assembly according to the target clothes folding action sequence. According to the scheme, the clothes folding efficiency and reliability of the robot are improved.

The invention relates to the technical field of image processing, and discloses a rendering method and device based on ray tracing and rasterization, equipment and a medium, and the method comprises the steps: segmenting an initial three-dimensional scene based on a scene segmentation model and local rendering parameters of the initial three-dimensional scene, generating three-dimensional sub-scenes and determining an optimal rendering mode corresponding to each three-dimensional sub-scene; performing image rendering on each three-dimensional sub-scene through a preferred rendering mode to generate a rendered sub-image; and performing image synthesis processing on each rendering sub-image to generate a target rendering image. According to the method, the initial three-dimensional scene is divided into the plurality of sub-scenes, and the optimal rendering mode is determined for each sub-scene, so that the rendering efficiency is improved. By adopting the rendering mode most suitable for the characteristics of each sub-scene, the rendering sub-image with higher quality can be generated, the rendering image with high quality can be quickly generated, and the visual effect experience of a user is improved.

Systems and techniques are provided for performing scene segmentation and object tracking. For example, a method for processing one or more frames. The method may include determining first one or more features from a first frame. The first frame includes a target object. The method may include obtaining a first mask associated with the first frame. The first mask includes an indication of the target object. The method may further include generating, based on the first mask and the first one or more features, a representation of a foreground and a background of the first frame. The method may include determining second one or more features from a second frame and determining, based on the representation of the foreground and the background of the first frame and the second one or more features, a location of the target object in the second frame.

The invention belongs to the technical field of computer vision, embedded edge calculation and intelligent video monitoring, and discloses a yolk-based night smoke and fire identification optimization method, which can effectively detect flames at night or in a scene with disordered light. A multi-source data fusion and lightweight scene segmentation technology is adopted, an on-site perception model is built, and the model is combined with a small sample learning mechanism, so that night common interference factors such as vehicle lamps and street lamps can be identified; then, a federal learning algorithm and a time sequence attention module are used for modeling the dynamic characteristics of the smoke and fire, and an obtained characteristic model has high generalization ability; then, the system accurately screens out real smoke and fire candidate targets through self-adaptive threshold matching and pixel-level interference elimination operation; and finally, based on a multi-factor decision model and a dynamic edge transmission technology, rapid alarm of smoke and fire identification is realized. The whole system can also automatically optimize the whole process parameters of the method through rule mining and reinforcement learning.

The invention discloses a photovoltaic panel fine identification segmentation method based on photovoltaic power station scene knowledge constraint, and the method comprises the following steps: carrying out the thinning of a photovoltaic power station scene remote sensing image Iscene; the method comprises the following steps of: making a scene segmentation sample set Sscenelabel of the photovoltaic power station; performing training and knowledge reasoning on the photovoltaic power station scene model; the method comprises the following steps: manufacturing a photovoltaic panel instance segmentation sample set Spanellabel; a photovoltaic panel identification model Mpanel is trained; reasoning a photovoltaic panel result set of a single photovoltaic power station scene; and carrying out multi-scene photovoltaic panel achievement integration. The photovoltaic panel fine identification cascade model formed by the method does not depend on remote sensing image time phase change and territorial scope, the defect that a traditional method is low in false target discrimination capability is overcome, and the identification reliability of the photovoltaic panel in a complex scene is improved; the management efficiency is improved and the management cost is reduced for the management and maintenance of the photovoltaic array in the photovoltaic power industry, and a good support effect of a remote sensing technology is exerted.

The embodiments of the present disclosure provide an image scene segmentation and apparatus, and a device and a storage medium. The method includes: obtaining an intermediate scene segmentation image by performing scene initial segmentation and scene initial fusion on an obtained target image; detecting, from the intermediate scene segmentation image, segmentation blocks to be processed; and obtaining a target scene segmentation image of the target image by performing segmentation correction on the segmentation blocks to be processed.

The invention discloses a three-dimensional scene segmentation method based on 3D Gaussian Splitting, the core idea is to make full use of the relation between different views in the step-by-step segmentation process, which relates to two complementary aspects: 1, the target is to enhance the algorithm for aggregating a 2D segmentation mask into 3D segmentation, especially to improve the precision and integrity in a scene with a small number of lenses; this function of a small number of lenses is crucial to providing cues for new views because the cues depend on segmentation predictions from preceding views. Second, by retaining the reliable cues used in the previous view, the new view can avoid cueing inconsistent targets. On the basis, a simple and effective lifting algorithm based on Gaussian center perspective is provided, the completeness of three-dimensional segmentation is guaranteed, boundary distinguishing is enhanced, and accurate three-dimensional segmentation is achieved. Besides, a prompt propagation strategy is introduced, and the reliable prompt from the adjacent view is effectively utilized by restoring the 2D prompt into the scene, so that accurate segmentation is realized.

The invention relates to the technical field of information monitoring, in particular to a bidding informatization platform data security monitoring and analysis method, which comprises the following steps of: acquiring bidding platform data, dividing user behaviors into a plurality of monitoring points according to operation objects, operation quantity and operation types of the bidding platform data in a continuous time period, acquiring a user behavior sequence formed by the monitoring points; performing scene segmentation on the user behavior, and determining an abnormal behavior relative to a historical behavior baseline under each scene segmentation dimension; merging each abnormal behavior into a plurality of association sets, and determining a behavior association result of each association set and the operation object; determining association paths under different scene segmentation dimensions according to a behavior association result in a monitoring point distribution form; for the intersection of the association paths under all scene segmentation dimensions, updating a path list after combination of the association paths, and taking the updated path list as an output early warning target; the response speed and the processing efficiency of the platform to the abnormal behavior are realized.

The invention belongs to the technical field of artificial intelligence, and discloses a hidden space world model construction method for a robot grabbing operation scene and related equipment thereof. The method comprises the steps of performing semantic segmentation on a scene image through an image semantic segmentation network to generate a semantic segmentation mask of each object; processing the semantic segmentation mask through a multi-task scene understanding network to generate implicit representation; based on the implicit representation of each object at the current moment and the action information of the mechanical arm, predicting implicit representation at the next moment through a state transition network; and according to the predicted implicit representation at the next moment, generating a scene segmentation image reconstruction result, an object existence judgment result and a contact relation judgment result through a decoder in the multi-task scene understanding network architecture, and constructing an implicit space world model of the scene. Based on the method, efficient characterization and dynamic prediction of the robot grabbing operation scene are achieved, and the generalization ability and the multi-task cooperation efficiency of the model in a complex environment are improved.

The invention discloses a video generation model training data set processing method, and belongs to the field of video processing. The method comprises the following steps: S1, storing a video collected in a specific service direction in a single specified folder, wherein the video has no fixed limitation in the aspects of duration, resolution and frame rate; s2, extracting the file name, the frame rate, the duration, the resolution ratio and the aspect ratio of the video in the S1, and recording the data in a CSV file; s3, performing the following screening on each video according to the information in the S2: deleting the video when the video resolution is lower than 360P; deleting the video when the aspect ratio of the video is less than 0.7; when the video frame rate is lower than 20 frames per second, deleting; and S4, performing scene segmentation on the video screened in the step S3 by adopting a PySceneDetect algorithm. According to the method, representative and dynamically changing core frames can be uniformly extracted from a large number of original videos, so that the diversity and quality of a video data set are improved, and high-quality training data are provided for a video generation model.

The invention relates to an automatic driving vehicle-oriented street scene real-time semantic segmentation method and device. The method comprises the steps that an image acquisition module performs real-time original image acquisition through a monocular camera; the real-time semantic segmentation module constructs a real-time semantic segmentation model to realize real-time segmentation of the original image; and the vehicle-mounted camera real-time segmentation module obtains an environment picture in real time through a vehicle-mounted camera, and sends the environment picture to the trained real-time semantic segmentation model to obtain a final scene segmentation result. According to the method, real-time scene segmentation of the autonomous vehicle facing different street scenes is established, high-precision segmentation can be carried out on different street scenes at a high reasoning speed, and the method has high stability.

The invention relates to a self-adaptive perception strategy switching method and system for long-distance automatic driving, and the method comprises the steps: carrying out the analysis of a public data set through employing a multi-target joint optimization clustering method, generating a standard scene prototype, and constructing an initial scene-optimal strategy mapping database; acquiring multi-dimensional scene features of a navigation route, performing global optimal scene division and recognition on a target road section, and generating a perception strategy map between continuous cells; querying a scene-optimal strategy mapping database continuously optimized by reinforcement learning, and selecting an optimal perception strategy for each cell based on a Q value maximization principle; when the vehicle enters the new cell, the self-adaptive switching control module automatically activates the preloaded optimal sensing strategy; and evaluating a strategy execution effect, generating an instant reward, transmitting an experience tuple containing a state, an action and the reward to the cloud, and continuously optimizing the Q-value database. The method and the system can improve the perception performance and the resource utilization efficiency of the automatic driving system.

The invention discloses an orthograph generation method and device, equipment, a storage medium and a product. The method comprises the following steps: determining a corresponding scene sparse point cloud and a camera external parameter according to an acquired scene aerial image sequence of a target scene and a camera internal parameter; performing region segmentation on the target scene according to the camera internal reference and the camera external reference to obtain scene segmentation sub-regions; performing Gaussian splash processing on the scene sparse point cloud in each scene segmentation sub-region based on a preset two-dimensional Gaussian splash algorithm to generate a local two-dimensional Gaussian scene corresponding to each scene segmentation sub-region; rendering each local two-dimensional Gaussian scene based on an orthographic camera model to generate a corresponding local scene Gaussian orthographic graph; and splicing the Gaussian orthographs of the local scenes according to the external parameters of the camera to generate an orthograph corresponding to the target scene. The problems that a traditional method is limited by high-precision topographic information requirements and high-precision DSM generation costs too many computing resources are solved, and the high-precision orthograph can be quickly generated.

A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and / or other sensor data may be stitched together, stacked, and / or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and / or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and / or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

The invention provides a scene segmentation and editing method and system based on three-dimensional Gaussian sputtering, and the method comprises the steps: carrying out the two-dimensional instance segmentation of a three-dimensional scene multi-view two-dimensional image, obtaining a two-dimensional instance mask, initializing the three-dimensional reconstruction through three-dimensional Gaussian sputtering, and obtaining a three-dimensional Gaussian body set; associating cross-view consistent semantic features extracted by the pre-training visual language model for each Gaussian body, constructing and iterating a three-dimensional instance prototype library based on a two-dimensional instance mask, and performing conversion to obtain a cross-view consistent two-dimensional supervision signal; adding a learnable instance identity code for each Gaussian body, constructing a multi-dimensional loss function in combination with a two-dimensional supervision signal, a spatial neighborhood relationship and cross-view consistent semantic features, and optimizing learnable parameters containing the instance identity codes; and realizing Gaussian body instance grouping based on the optimized instance identity codes to complete instance-level scene segmentation and editing. According to the method, the technical problems of instance identity association and high-quality instance level generation under the conditions of sparse view angle and close adjacent objects are solved.

The invention belongs to the technical field of medical information processing, and relates to a laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation. Firstly, a video stream composed of multiple frames of images of an endoscope is input into the system, and then the spatial feature of each frame of image is obtained through a multi-frame spatial feature extraction module; the spatial features of all the frames are then input into a multi-frame time feature fusion module to obtain time features fused with different time lengths, and then the time features are used for outputting operation stages, operation steps and operation ternary body labels; meanwhile, only the spatial features of the current frame are extracted from the spatial features of all the frames, and an operation scene segmentation label is output through a single-frame feature restoration module. According to the laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation, computing resources can be greatly reduced, and compared with an existing single-granularity surgery scene technology, the laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation can output surgery scene information of multiple granularities at a time by means of the mutual coordination effect of multiple tasks.

A method and processing device for providing a scene segmentation map for a total field of view of an image capturing device comprises obtaining a scene segmentation map for the total field of view; receiving an indication that a new focus value has been set for the image capturing device for acquiring images of a current field of view; comparing the new focus value with a stored focus value associated with a focus region in the current field of view, wherein the stored focus value represents a previously used focus value; upon the new focus value deviating from the stored focus value more than a trigger threshold, triggering a scene segmentation map update process; and otherwise maintaining the scene segmentation map.

The invention discloses an ultrahigh-resolution immersive scene rendering method based on hexagonal picture segmentation. The method comprises the following steps: 1, determining a hexagonal segmentation style of an immersive display surface; 2, unfolding the immersive display surface into a plane; 3, acquiring 3D point information of each hexagonal surface, respectively calculating the width W and the height H of the enveloping quadrilateral surface, and calculating a local coordinate system V of the hexagonal surface S; 4, using W, H and V to construct a view cone and a viewpoint matrix of the three-dimensional scene; 5, setting a shade M of a plane expansion view for the hexagonal surface S; 6, calling a rendering scene of the 3D scene for the mask region by using information of the view cone and the viewpoint matrix, and rendering the mask region into a picture at the position S of the hexagonal surface; and 7, repeating the steps 5-6, and sequentially carrying out the same treatment on each hexagonal surface. According to the method and the device, the ultrahigh-resolution picture can be rendered in a scene segmentation form, and the method and the device are suitable for ultrahigh-resolution picture rendering under immersive display equipment.