A target two-dimensional spatial orientation relationship identification method based on a spatial orientation vector field

Abstract

The application provides a target two-dimensional spatial orientation relationship identification method based on a spatial orientation vector field, relates to the technical field of visual relationship identification, and comprises the following steps: constructing a target two-dimensional spatial orientation relationship identification network model based on a spatial orientation vector field; constructing a supervision label and a loss function of the target two-dimensional spatial orientation relationship identification network based on the spatial orientation vector field; presetting four types of two-dimensional spatial orientation basic relationships, collecting image data sets, completing labeling and data set division, iteratively optimizing network model parameters through the labeled training set and the composite loss function, adjusting hyperparameters with a verification set, and completing identification model training; inputting a to-be-identified image into the trained identification model, respectively analyzing output data of a target center point detection head and a spatial orientation detection head, calculating a vector field score between any two targets, determining a two-dimensional spatial orientation relationship between the targets through a preset threshold, and outputting a final identification result in a triple format.

Description

Technical Field

[0001] This invention relates to the field of visual relationship recognition technology, and more particularly to a method for recognizing target two-dimensional spatial orientation relationships based on a spatial orientation vector field. Background Technology

[0002] Recognizing visual relationships between objects in images is a key area of ​​computer vision, aiming to reveal semantic relationships between objects. The development of this technology relies on fundamental research in deep learning, object detection, and image segmentation, promoting the integration of object recognition and scene understanding. Initially, relationship detection mainly relied on handcrafted features and rules, but with the advancement of convolutional neural networks (CNNs), researchers began to adopt end-to-end models, constructing richly annotated visual relationship datasets to drive progress in relationship learning and modeling. Furthermore, the introduction of multimodal learning has further promoted the fusion of image and language, laying the foundation for applications such as visual relationship question answering and the generation of descriptive representations of relationships between objects in images. Despite significant progress in this field, spatial orientation relationships, as fundamental semantic features of images, still require in-depth research, playing a crucial role in helping machines understand the location, behavior, and interactions of objects. Summary of the Invention

[0003] In view of the technical problems mentioned in the background, a method for identifying two-dimensional spatial orientation relationships of a target based on a spatial orientation vector field is provided.

[0004] The technical means employed in this invention are as follows:

[0005] A method for identifying two-dimensional spatial orientation relationships of targets based on spatial orientation vector fields includes the following steps: Step 1: Construct a target two-dimensional spatial orientation relationship recognition network model based on spatial orientation vector field; Step 2: Constructing a supervised labeling system for a target two-dimensional spatial orientation relationship recognition network based on a spatial orientation vector field. and loss function ;in, The loss function represents the target center point; This represents the spatial orientation loss function between targets; and All represent coefficients; Step 3: Preset four types of basic two-dimensional spatial orientation relationships, collect image datasets and complete annotation and dataset division, iteratively optimize network model parameters through the labeled training set and the composite loss function, fine-tune hyperparameters with the validation set, and complete the recognition model training. Step 4: Input the image to be recognized into the trained recognition model, parse the output data of the target center point detection head and the spatial orientation detection head respectively, calculate the vector field score between any two targets, determine the two-dimensional spatial orientation relationship between targets through a preset threshold, and output the final recognition result in triplet format.

[0006] Furthermore, step 1 includes the following steps: Step 11: Select a deep convolutional feature extraction network as the encoder of the two-dimensional spatial orientation relationship recognition network to downsample the image: ; in, Indicates the input image; This represents a feature extraction network; These represent the parameters of the feature extraction network; This represents the output feature map. ; These represent the output feature maps respectively. Width, height, and passageway; Step 12: Construct an upsampling module as a decoder to process the output feature map. Restored to a higher resolution output space, outputting enhanced feature maps. : ; in, Indicates an upsampling network; Represents the structural parameters of the network; , and These represent enhanced feature maps. Width, height, and passageway; Step 13: Constructing a target category recognition branch will enhance the feature map. The output is the target center point detection head. : ; in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels; Step 14: Construct the target 2D spatial orientation recognition branch and output X as the spatial orientation detection head. : ; in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels.

[0007] Furthermore, step 2 includes the following steps: Step 21: Construct the target center point detection head of At each target center point Construct a Gaussian distribution at [location]. ,Right now: ; in, and These represent the target center point at... The x and y coordinates on the graph; It is not the adaptive standard deviation parameter of the target Gaussian circular distribution; Step 22: Construct the target spatial orientation detection head of , Construction between each pair of targets Vector fields and Vector field, for superior Vector fields and The pixels in the vector field construct the truth value, that is: ; in, Indicates the first in the field The value of each pixel; Represented as Pixels in the field; Represented as Pixels in the field; This represents the coordinate system along the image with the center point of the main target. x The unit vector of the axis; Represented as along the image coordinate system constructed with the center point of the main target. The unit vector of the axis; Represented as the center point of the main target and the first A vector consisting of elements; Step 23: Construct the spatial orientation loss function between targets, that is: ; in, This indicates the number of spatial orientation vector fields; and Represents the x and y coordinates of a pixel on the feature map; Channels representing feature maps; This represents the hyperparameters used in training.

[0008] Furthermore, in step 3, the parameters of the target two-dimensional spatial orientation relationship recognition model based on the spatial orientation vector field are trained using a loss function, namely: ; in, Indicates the number of times the model has been trained; Indicates the learning rate; This indicates the loss function with respect to the parameters. The gradient.

[0009] Furthermore, step 4 includes the following steps: Step 41, on the detection head The target two-dimensional location is found by nonmaximum suppression. Target category and confidence level ,Right now: ; in, express The x and y coordinates of the pixel; express The passage; Step 42: Obtain each main objective and object target Between Score of vector field and Score of vector field ; Step 43: Spatial orientation relationship between targets and confidence level ; Step 44: Output the target's two-dimensional spatial orientation relationship in the form of triplets, i.e. ;in, and These represent the subject objective and the object objective, respectively. Indicates the category of the target. Indicates the confidence level of the category; Categories representing spatial orientation relationships; This indicates the confidence level of the spatial orientation relationship category.

[0010] Furthermore, the acquisition of each subject target and object target Between Score of vector field and Score of vector field The calculation formula is: ; in, express Forecast Channels of a vector field; express Forecast Channels of a vector field; and These represent the x and y coordinates of a pixel, respectively. Indicates the width of the field.

[0011] Furthermore, the spatial orientation relationship between the targets and confidence level The calculation formula is: ; in, and Representing the main objectives and object target exist The value of the ordinate; and This represents the threshold parameter.

[0012] Compared with the prior art, the present invention has the following advantages: This invention introduces an end-to-end spatial orientation relationship recognition model. This network model has two branches: one branch identifies target categories based on keypoint detection technology, and the other branch introduces a spatial orientation vector field to identify the two-dimensional spatial orientation relationships between targets. This innovative method enhances the understanding of spatial relationships between targets in images by analyzing deep semantic features, enabling more accurate inference of the existence and location of targets. This allows it to better adapt to various detection tasks in complex scenes, such as target detection, action recognition, activity detection, human-target interaction recognition, image caption generation, and image retrieval. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2This is a diagram of the network model structure of the present invention.

[0016] Figure 3 This is a schematic diagram illustrating the effect of the present invention in PASCALL VOC. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] like Figures 1-3 As shown, this invention provides a method for identifying two-dimensional spatial orientation relationships of a target based on a spatial orientation vector field, comprising the following steps: Step 1: Construct a target two-dimensional spatial orientation relationship recognition network model based on a spatial orientation vector field. In this application, Step 1 includes the following steps: Step 11: Select a deep convolutional feature extraction network as the encoder of the two-dimensional spatial orientation relationship recognition network to downsample the image: ; in, Indicates the input image; This represents a feature extraction network; These represent the parameters of the feature extraction network; This represents the output feature map. ; These represent the output feature maps respectively. Width, height, and passageway; Step 12: Construct an upsampling module as a decoder to process the output feature map. Restored to a higher resolution output space, outputting enhanced feature maps. : ; in, Indicates an upsampling network; Represents the structural parameters of the network; , and These represent enhanced feature maps. Width, height, and passageway; Step 13: Constructing a target category recognition branch will enhance the feature map. The output is the target center point detection head. : ; in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels; Step 14: Construct the target 2D spatial orientation recognition branch and output X as the spatial orientation detection head. : ; in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels.

[0020] Step 2: Constructing a supervised labeling system for a target two-dimensional spatial orientation relationship recognition network based on a spatial orientation vector field. and loss function ;in, The loss function represents the target center point; This represents the spatial orientation loss function between targets; and All represent coefficients.

[0021] In a preferred embodiment, step 2 includes the following steps: Step 21: Construct the target center point detection head of At each target center point Construct a Gaussian distribution at [location]. ,Right now: ; in, and These represent the target center point at... The x and y coordinates on the graph; It is not the adaptive standard deviation parameter of the target Gaussian circular distribution; Step 22: Construct the target spatial orientation detection head of , Construction between each pair of targets Vector fields and Vector field, for superior Vector fields and The pixels in the vector field construct the truth value, that is: ; in, Indicates the first in the field The value of each pixel; Represented as Pixels in the field; Represented as Pixels in the field; This represents the coordinate system along the image with the center point of the main target. x The unit vector of the axis; Represented as along the image coordinate system constructed with the center point of the main target. The unit vector of the axis; Represented as the center point of the main target and the first A vector consisting of elements; Step 23: Construct the spatial orientation loss function between targets, that is: ; in, This indicates the number of spatial orientation vector fields; and Represents the x and y coordinates of a pixel on the feature map; Channels representing feature maps; This represents the hyperparameters used in training.

[0022] Step 3: Preset four types of basic two-dimensional spatial orientation relationships, collect image datasets and complete annotation and dataset division, iteratively optimize network model parameters through the annotated training set and the composite loss function, fine-tune hyperparameters with the validation set, and complete the recognition model training.

[0023] Based on the general visual perception of objects in images, four basic types of two-dimensional spatial orientation relationships are defined. (about), (up and down), (bottom left - top right) and (Top left - bottom right). Obtain publicly available image data and annotate the target images using annotation tools. Divide the data into training, validation, and test sets according to proportions. Use all training set image data for model training, the validation set for adjusting model hyperparameters, and the test set for evaluating model performance. In step 3, the parameters of the target two-dimensional spatial orientation relationship recognition model based on the spatial orientation vector field are trained using a loss function, i.e.: ; in, Indicates the number of times the model has been trained; Indicates the learning rate; This indicates the loss function with respect to the parameters. The gradient.

[0024] Step 4: Input the image to be recognized into the trained recognition model, parse the output data of the target center point detection head and the spatial orientation detection head respectively, calculate the vector field score between any two targets, determine the two-dimensional spatial orientation relationship between targets through a preset threshold, and output the final recognition result in triplet format.

[0025] In a preferred embodiment, step 4 includes the following steps: Step 41, on the detection head The target two-dimensional location is found by nonmaximum suppression. Target category and confidence level ,Right now: ; in, express The x and y coordinates of the pixel; express The passage; Step 42: Obtain each main objective and object target Between Score of vector field and Score of vector field The acquisition of each subject target and object target Between Score of vector field and Score of vector field The calculation formula is: ; in, express Forecast Channels of a vector field; express Forecast Channels of a vector field; and These represent the x and y coordinates of a pixel, respectively. Indicates the width of the field.

[0026] Step 43: Spatial orientation relationship between targets and confidence level Spatial orientation relationship between the targets and confidence level The calculation formula is: ; in, and Representing the main objectives and object target exist The value of the ordinate; and This represents the threshold parameter.

[0027] Step 44: Output the target's two-dimensional spatial orientation relationship in the form of triplets, i.e. ;in, and These represent the subject objective and the object objective, respectively. Indicates the category of the target. Indicates the confidence level of the category; Categories representing spatial orientation relationships; This indicates the confidence level of the spatial orientation relationship category.

[0028] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the descriptions of each embodiment have their own emphasis; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the two-dimensional spatial orientation relationship of a target based on a spatial orientation vector field, characterized in that, Includes the following steps: Step 1: Construct a target two-dimensional spatial orientation relationship recognition network model based on spatial orientation vector field; Step 2: Constructing a supervised labeling system for a target two-dimensional spatial orientation relationship recognition network based on a spatial orientation vector field. and loss function ;in, The loss function represents the target center point; This represents the spatial orientation loss function between targets; and All represent coefficients; Step 3: Preset four types of basic two-dimensional spatial orientation relationships, collect image datasets and complete annotation and dataset division, iteratively optimize network model parameters through the labeled training set and the composite loss function, fine-tune hyperparameters with the validation set, and complete the recognition model training. Step 4: Input the image to be recognized into the trained recognition model, parse the output data of the target center point detection head and the spatial orientation detection head respectively, calculate the vector field score between any two targets, determine the two-dimensional spatial orientation relationship between targets through a preset threshold, and output the final recognition result in triplet format.

2. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 1, characterized in that, Step 1 includes the following steps: Step 11: Select a deep convolutional feature extraction network as the encoder of the two-dimensional spatial orientation relationship recognition network to downsample the image: ； in, Indicates the input image; This represents a feature extraction network; These represent the parameters of the feature extraction network; This represents the output feature map. ; These represent the output feature maps respectively. Width, height, and passageway; Step 12: Construct an upsampling module as a decoder to process the output feature map. Restored to a higher resolution output space, outputting enhanced feature maps. : ； in, Indicates an upsampling network; Represents the structural parameters of the network; , and These represent enhanced feature maps. Width, height, and passageway; Step 13: Constructing a target category recognition branch will enhance the feature map. The output is the target center point detection head. : ； in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels; Step 14: Construct the target 2D spatial orientation recognition branch and output X as the spatial orientation detection head. : ； in, Represents a continuous convolutional structure; Indicates structural parameters; , and They represent Width, height, and number of channels.

3. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 1, characterized in that, Step 2 includes the following steps: Step 21: Construct the target center point detection head of At each target center point Construct a Gaussian distribution at [location]. ,Right now: ； in, and These represent the target center point at... The x and y coordinates on the graph; It is not the adaptive standard deviation parameter of the target Gaussian circular distribution; Step 22: Construct the target spatial orientation detection head of , Construction between each pair of targets Vector fields and Vector field, for superior Vector fields and The pixels in the vector field construct the truth value, that is: ； in, Indicates the first in the field The value of each pixel; Represented as Pixels in the field; Represented as Pixels in the field; This represents the coordinate system along the image with the center point of the main target. x The unit vector of the axis; Represented as along the image coordinate system constructed with the center point of the main target. The unit vector of the axis; Represented as the center point of the main target and the first A vector consisting of elements; Step 23: Construct the spatial orientation loss function between targets, that is: ； in, This indicates the number of spatial orientation vector fields; and Represents the x and y coordinates of a pixel on the feature map; Channels representing feature maps; This represents the hyperparameters used in training.

4. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 1, characterized in that, In step 3, the parameters of the target two-dimensional spatial orientation relationship recognition model based on the spatial orientation vector field are trained using a loss function, namely: ； in, Indicates the number of times the model has been trained; Indicates the learning rate; This indicates the loss function with respect to the parameters. The gradient.

5. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 1, characterized in that, Step 4 includes the following steps: Step 41, on the detection head The target two-dimensional location is found by nonmaximum suppression. Target category and confidence level ,Right now: ； in, express The x and y coordinates of the pixel; express The passage; Step 42: Obtain each main objective and object target Between Score of vector field and Score of vector field ; Step 43: Spatial orientation relationship between targets and confidence level ; Step 44: Output the target's two-dimensional spatial orientation relationship in the form of triplets, i.e. ;in, and These represent the subject objective and the object objective, respectively. Indicates the category of the target. Indicates the confidence level of the category; Categories representing spatial orientation relationships; This indicates the confidence level of the spatial orientation relationship category.

6. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 5, characterized in that, The goal of obtaining each subject and object target Between Score of vector field and Score of vector field The calculation formula is: ； in, express Forecast Channels of a vector field; express Forecast Channels of a vector field; and These represent the x and y coordinates of a pixel, respectively. Indicates the width of the field.

7. The target two-dimensional spatial orientation relationship recognition method based on a spatial orientation vector field according to claim 5, characterized in that, Spatial orientation relationship between the targets and confidence level The calculation formula is: ； in, and Representing the main objectives and object target exist The value of the ordinate; and This represents the threshold parameter.

Images