Optimization method, device and electronic device for saliency representation of anchor vector

Abstract

The present invention provides a method, device and electronic device for optimizing the saliency representation of an anchor vector. In the method, through multiple attention mechanisms (self-attention mechanism and cross-attention mechanism), the anchor vector can capture more fine-grained saliency information and enhance the global feature expression capability. The finally obtained saliency-optimized anchor vector has better adaptability to dynamic environmental changes, and the multiple attention mechanism also improves the accuracy of capturing saliency information.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and deep learning, and in particular to an optimization method, device and electronic device for saliency representation of an anchor vector. Background Art

[0002] In traditional scene recognition methods, a set of anchor vectors are randomly generated in the original image, and then these anchor vectors are learned through the network. The final learned set of anchor vectors may not focus on the salient information in the scene corresponding to the original image (cannot represent the salient information in the original image). In other words, the final set of anchor vectors is difficult to effectively represent the salient information in the original image, resulting in insufficient scene representation capabilities, which in turn affects the accuracy of scene recognition. In addition, the above process lacks dynamic interaction between anchor vectors, making it difficult to capture implicit associations in complex scenes, and the ability to express global features is insufficient.

[0003] In summary, traditional anchor vectors have the technical problem of being difficult to effectively represent saliency information and insufficient in expressing global features. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an optimization method, device and electronic device for saliency representation of anchor vectors, so as to alleviate the technical problems that traditional anchor vectors are difficult to effectively represent saliency information and have insufficient global feature expression capabilities.

[0005] In a first aspect, an embodiment of the present invention provides a method for optimizing saliency representation of an anchor vector, comprising:

[0006] Obtaining a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple;

[0007] A self-attention mechanism is used to establish global consistency between the anchor vectors, thereby obtaining a set of anchor vectors with implicit intrinsic relationships;

[0008] A cross-attention mechanism is used to interact the anchor vector with implicit intrinsic relationship with local features to obtain a set of anchor vectors with saliency information, wherein the local features are obtained by discretizing the feature maps corresponding to the anchor vectors;

[0009] Performing nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors;

[0010] The optimized anchor point vectors are normalized and concatenated to obtain a saliency optimized anchor point vector.

[0011] Furthermore, a self-attention mechanism is used to establish global consistency between the anchor vectors, including:

[0012] Normalizing the anchor point vector to obtain a normalized anchor point vector;

[0013] A self-attention mechanism is used to perform feature aggregation using the normalized anchor vector as query, key and value;

[0014] The vector obtained by feature aggregation is fused with the anchor vector to obtain a set of anchor vectors with implicit internal relations.

[0015] Furthermore, the local features are obtained by using a pre-trained backbone network to extract features from an original image corresponding to the anchor point vector and then discretizing the extracted feature map, wherein the number of the local features is multiple.

[0016] Furthermore, a cross attention mechanism is used to interact the anchor vector with implicit intrinsic relationship with the local features, including:

[0017] Normalizing the anchor point vector with the implicit intrinsic relationship to obtain a normalized anchor point vector with the implicit intrinsic relationship;

[0018] Performing dimension alignment and normalization processing on the local features to obtain normalized local features;

[0019] A cross attention mechanism is used to take the normalized anchor vector with implicit intrinsic relationship as query, and the normalized local features as keys and values ​​for feature aggregation;

[0020] The vector obtained by feature aggregation is fused with the normalized anchor vector with implicit internal relationship to obtain a set of anchor vectors with saliency information.

[0021] Furthermore, nonlinear optimization is performed on the anchor vector with saliency information, including:

[0022] Normalizing the anchor point vector with saliency information to obtain a normalized anchor point vector with saliency information;

[0023] Using a multi-layer perceptron to perform nonlinear optimization on the normalized anchor point vector with significant information;

[0024] The vector obtained after nonlinear optimization is fused with the anchor point vector with saliency information after normalization to obtain a set of optimized anchor point vectors.

[0025] Furthermore, the method further comprises:

[0026] Calculating the similarity between the saliency-optimized anchor vector and a preset saliency-optimized anchor vector in a scene recognition relationship, wherein the scene recognition relationship is a relationship between the preset saliency-optimized anchor vector and a target scene;

[0027] A target preset saliency optimized anchor point vector corresponding to the saliency optimized anchor point vector is determined according to the similarity, and then a target scene corresponding to the target preset saliency optimized anchor point vector is used as the recognized scene.

[0028] In a second aspect, an embodiment of the present invention provides an optimization device for saliency representation of an anchor vector, comprising:

[0029] An acquisition unit, used to acquire a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple;

[0030] A global consistency establishing unit, used for establishing global consistency between the anchor vectors by using a self-attention mechanism to obtain a set of anchor vectors with implicit internal relations;

[0031] An interaction unit, used for interacting the anchor vector with implicit intrinsic relationship with local features by using a cross attention mechanism to obtain a set of anchor vectors with saliency information, wherein the local features are obtained by discretizing feature maps corresponding to the anchor vectors;

[0032] A nonlinear optimization unit, used for performing nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors;

[0033] The normalization and splicing processing unit is used to perform normalization and splicing processing on the optimized anchor point vector to obtain an anchor point vector with optimized saliency.

[0034] Furthermore, the global consistency establishing unit is also used for:

[0035] Normalizing the anchor point vector to obtain a normalized anchor point vector;

[0036] A self-attention mechanism is used to perform feature aggregation using the normalized anchor vector as query, key and value;

[0037] The vector obtained by feature aggregation is fused with the anchor vector to obtain a set of anchor vectors with implicit internal relations.

[0038] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0040] In an embodiment of the present invention, a method for optimizing the saliency representation of an anchor vector is provided, comprising: obtaining a group of anchor vectors with global representation information, wherein the number of the group of anchor vectors is multiple; using a self-attention mechanism to establish global consistency between the anchor vectors to obtain a group of anchor vectors with implicit intrinsic relationships; using a cross-attention mechanism to interact the anchor vectors with implicit intrinsic relationships with local features to obtain a group of anchor vectors with saliency information, wherein the local features are obtained by discretizing feature maps corresponding to the anchor vectors; performing nonlinear optimization on the anchor vectors with saliency information to obtain a group of optimized anchor vectors; and performing normalization and concatenation processing on the optimized anchor vectors to obtain an anchor vector with saliency optimization. From the above description, it can be seen that in the optimization method of the saliency representation of the anchor vector of the present invention, through multiple attention mechanisms (self-attention mechanism and cross-attention mechanism), the anchor vector can capture more fine-grained saliency information and enhance the global feature expression ability. The final saliency optimized anchor vector has better adaptability to dynamic environmental changes. The multiple attention mechanism also improves the accuracy of capturing saliency information, alleviating the technical problem that traditional anchor vectors are difficult to effectively represent saliency information and have insufficient global feature expression ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flowchart of a method for optimizing the saliency representation of an anchor vector provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of a multiple attention mechanism provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of an optimization device for saliency representation of an anchor vector provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Traditional anchor vectors are difficult to effectively represent saliency information and lack the ability to express global features.

[0048] Based on this, in the optimization method of the saliency representation of the anchor vector of the present invention, through multiple attention mechanisms (self-attention mechanism and cross-attention mechanism), the anchor vector can capture more fine-grained saliency information and enhance the global feature expression capability. The final saliency-optimized anchor vector has better adaptability to dynamic environmental changes, and the multiple attention mechanism also improves the accuracy of capturing saliency information.

[0049] To facilitate understanding of this embodiment, firstly, a method for optimizing the saliency representation of an anchor point vector disclosed in an embodiment of the present invention is introduced in detail.

[0050] Embodiment 1:

[0051] According to an embodiment of the present invention, an embodiment of a method for optimizing saliency representation of an anchor vector is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0052] Figure 1 is a flow chart of a method for optimizing the saliency representation of an anchor vector according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0053] Step S102, obtaining a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple;

[0054] Specifically, the above anchor point vector may be generated randomly or by some traditional methods.

[0055] The above anchor vector is used for the first time in the task of visual scene recognition. The previous anchor vectors were all used in the image detection task. In the image detection task, anchor points are used to locate the target of interest in the image. Here, the saliency information in the image is represented by the anchor points.

[0056] Step S104, using a self-attention mechanism to establish global consistency between anchor vectors, and obtaining a set of anchor vectors with implicit internal relationships;

[0057] The above self-attention mechanism is the interaction of internal information between anchor vectors, and then learns global consistency, that is, gradually learns the intrinsic correlation between anchor vectors and captures the implicit internal relationship of salient information.

[0058] Step S106, using a cross-attention mechanism to interact the anchor vectors with implicit internal relationships with the local features to obtain a set of anchor vectors with saliency information, wherein the local features are obtained by discretizing the feature maps corresponding to the anchor vectors;

[0059] Specifically, in the process of the above-mentioned cross-attention mechanism, the salient information in the scene can be captured and dynamic focusing of the salient area can be achieved.

[0060] In the cross-attention mechanism, the anchor vectors with implicit intrinsic relationships interact with local features during the learning process, and through interaction, the salient information of specific salient areas can be determined. That is, each anchor vector with implicit intrinsic relationships is dynamically and adaptively focused on the local features (i.e. salient information) that should be focused on in the characteristics of the scene, and then the salient information is integrated into its own anchor vector to obtain a set of anchor vectors with salient information.

[0061] Step S108, performing nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors;

[0062] Specifically, nonlinear optimization can further improve the expressive power of anchor vectors.

[0063] Step S110, normalizing and concatenating the optimized anchor point vectors to obtain an anchor point vector with optimized saliency.

[0064] Specifically, the above-mentioned saliency-optimized anchor vector is used to compactly represent the scene. The above process uses a hybrid attention mechanism to identify the salient areas in the scene, and deeply associates the saliency information of the salient areas with the anchor vector, so that the anchor vector has the ability to represent the saliency information in the scene, and finally represents the image of the scene with the saliency-optimized anchor vector.

[0065] In an embodiment of the present invention, a method for optimizing the saliency representation of an anchor vector is provided, comprising: obtaining a group of anchor vectors with global representation information, wherein the number of the group of anchor vectors is multiple; using a self-attention mechanism to establish global consistency between the anchor vectors to obtain a group of anchor vectors with implicit intrinsic relationships; using a cross-attention mechanism to interact the anchor vectors with implicit intrinsic relationships with local features to obtain a group of anchor vectors with saliency information, wherein the local features are obtained by discretizing feature maps corresponding to the anchor vectors; performing nonlinear optimization on the anchor vectors with saliency information to obtain a group of optimized anchor vectors; and performing normalization and concatenation processing on the optimized anchor vectors to obtain an anchor vector with saliency optimization. From the above description, it can be seen that in the optimization method of the saliency representation of the anchor vector of the present invention, through multiple attention mechanisms (self-attention mechanism and cross-attention mechanism), the anchor vector can capture more fine-grained saliency information and enhance the global feature expression ability. The final saliency optimized anchor vector has better adaptability to dynamic environmental changes. The multiple attention mechanism also improves the accuracy of capturing saliency information, alleviating the technical problem that traditional anchor vectors are difficult to effectively represent saliency information and have insufficient global feature expression ability.

[0066] The above content briefly introduces the optimization method of the saliency representation of the anchor point vector of the present invention. The specific contents involved are described in detail below.

[0067] In an optional embodiment of the present invention, a self-attention mechanism is used to establish global consistency between anchor vectors, which specifically includes the following steps: Figure 2 :

[0068] (1) normalizing the anchor point vector to obtain a normalized anchor point vector;

[0069] (2) Using the self-attention mechanism to perform feature aggregation with the normalized anchor vector as query, key, and value;

[0070] (3) The vector obtained by feature aggregation is fused with the anchor vector to obtain a set of anchor vectors with implicit internal relationships.

[0071] In an optional embodiment of the present invention, the local features are obtained by using a pre-trained backbone network to perform feature extraction on the original image corresponding to the anchor point vector, and then discretizing the extracted feature map, wherein the number of local features is multiple, and each local feature is essentially a local feature vector.

[0072] In an optional embodiment of the present invention, a cross attention mechanism is used to interact the anchor vector with implicit intrinsic relationship with the local feature, which specifically includes the following steps: Figure 2 :

[0073] (1) normalizing the anchor point vector with implicit internal relationship to obtain the normalized anchor point vector with implicit internal relationship;

[0074] (2) Dimensionally aligning and normalizing the local features to obtain normalized local features;

[0075] (3) Using the cross-attention mechanism, the normalized anchor vector with implicit internal relationship is used as the query, and the normalized local features are used as keys and values ​​for feature aggregation;

[0076] (4) The vector obtained by feature aggregation is fused with the normalized anchor vector with implicit intrinsic relationship to obtain a set of anchor vectors with saliency information.

[0077] In an optional embodiment of the present invention, nonlinear optimization is performed on the anchor vector with saliency information, specifically comprising the following steps: Figure 2 :

[0078] (1) normalizing the anchor point vector with saliency information to obtain a normalized anchor point vector with saliency information;

[0079] (2) Using a multi-layer perceptron to perform nonlinear optimization on the normalized anchor vectors with significant information;

[0080] (3) The vector obtained after nonlinear optimization is fused with the normalized anchor vector with saliency information to obtain a set of optimized anchor vectors.

[0081] In an optional embodiment of the present invention, the method further comprises the following steps:

[0082] (1) calculating the similarity between the saliency optimized anchor vector and the preset saliency optimized anchor vector in the scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset saliency optimized anchor vector and the target scene;

[0083] (2) Determine the target preset saliency optimized anchor vector corresponding to the saliency optimized anchor vector according to the similarity, and then use the target scene corresponding to the target preset saliency optimized anchor vector as the recognized scene.

[0084] The method of the present invention has the following effects:

[0085] 1. Enhanced global expression capability: Through multiple attention mechanisms, anchor vectors can capture more fine-grained saliency information.

[0086] 2. Improved robustness: The optimized anchor vector (i.e., the saliency-optimized anchor vector) is more adaptable to dynamic environmental changes.

[0087] 3. Reduce noise interference: Multiple attention mechanisms improve the accuracy of capturing salient information.

[0088] Embodiment 2:

[0089] An embodiment of the present invention also provides an optimization device for the saliency representation of an anchor point vector. The optimization device based on the saliency representation of an anchor point vector is mainly used to execute the optimization method for the saliency representation of an anchor point vector provided in the first embodiment of the present invention. The following is a specific introduction to the optimization device for the saliency representation of an anchor point vector provided in the embodiment of the present invention.

[0090] Figure 3 is a schematic diagram of an optimization device for saliency representation of an anchor vector according to an embodiment of the present invention, such as Figure 3 As shown, the device mainly includes: an acquisition unit 10, a global consistency establishment unit 20, an interaction unit 30, a nonlinear optimization unit 40, and a normalization and splicing processing unit 50, wherein:

[0091] An acquisition unit, used to acquire a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple;

[0092] A global consistency building unit is used to establish global consistency between anchor vectors using a self-attention mechanism to obtain a set of anchor vectors with implicit intrinsic relationships;

[0093] The interaction unit is used to interact the anchor vector with implicit intrinsic relationship with the local features by using the cross attention mechanism to obtain a set of anchor vectors with saliency information, where the local features are obtained by discretizing the feature map corresponding to the anchor vector;

[0094] A nonlinear optimization unit is used to perform nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors;

[0095] The normalization and splicing processing unit is used to normalize and splice the optimized anchor point vector to obtain a saliency optimized anchor point vector.

[0096] In an embodiment of the present invention, a device for optimizing the saliency representation of an anchor vector is provided, comprising: obtaining a group of anchor vectors with global representation information, wherein the number of a group of anchor vectors is multiple; using a self-attention mechanism to establish global consistency between the anchor vectors to obtain a group of anchor vectors with implicit intrinsic relationships; using a cross-attention mechanism to interact the anchor vectors with implicit intrinsic relationships with local features to obtain a group of anchor vectors with saliency information, wherein the local features are obtained by discretizing feature maps corresponding to the anchor vectors; performing nonlinear optimization on the anchor vectors with saliency information to obtain a group of optimized anchor vectors; and performing normalization and concatenation processing on the optimized anchor vectors to obtain an anchor vector with saliency optimization. From the above description, it can be seen that in the optimization device for the saliency representation of the anchor vector of the present invention, through multiple attention mechanisms (self-attention mechanism and cross-attention mechanism), the anchor vector can capture more fine-grained saliency information and enhance the global feature expression capability. The final saliency-optimized anchor vector has better adaptability to dynamic environmental changes. The multiple attention mechanism also improves the accuracy of capturing saliency information, alleviating the technical problem that traditional anchor vectors are difficult to effectively represent saliency information and have insufficient global feature expression capabilities.

[0097] Optionally, the global consistency establishing unit is also used to: normalize the anchor vector to obtain a normalized anchor vector; use the self-attention mechanism to perform feature aggregation using the normalized anchor vector as query, key and value; and fuse the vector obtained by feature aggregation with the anchor vector to obtain a set of anchor vectors with implicit intrinsic relationships.

[0098] Optionally, the local features are obtained by extracting features from an original image corresponding to an anchor vector using a pre-trained backbone network and then discretizing the extracted feature map, wherein the number of local features is multiple.

[0099] Optionally, the interaction unit is also used to: normalize the anchor vector with implicit intrinsic relationship to obtain the normalized anchor vector with implicit intrinsic relationship; dimensionally align and normalize the local features to obtain the normalized local features; adopt the cross-attention mechanism to perform feature aggregation with the normalized anchor vector with implicit intrinsic relationship as query and the normalized local features as key and value; fuse the vector obtained by feature aggregation with the normalized anchor vector with implicit intrinsic relationship to obtain a set of anchor vectors with saliency information.

[0100] Optionally, the nonlinear optimization unit is also used to: normalize the anchor point vector with saliency information to obtain the normalized anchor point vector with saliency information; use a multilayer perceptron to nonlinearly optimize the normalized anchor point vector with saliency information; and fuse the vector obtained after the nonlinear optimization with the normalized anchor point vector with saliency information to obtain a set of optimized anchor point vectors.

[0101] Optionally, the device is also used to: calculate the similarity between the saliency optimized anchor point vector and a preset saliency optimized anchor point vector in a scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset saliency optimized anchor point vector and the target scene; determine the target preset saliency optimized anchor point vector corresponding to the saliency optimized anchor point vector based on the similarity, and then use the target scene corresponding to the target preset saliency optimized anchor point vector as the identified scene.

[0102] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0103] like Figure 4 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the optimization method for saliency representation of the anchor vector as described above.

[0104] Specifically, the memory 602 and the processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the optimization method for the saliency representation of the anchor vector can be executed.

[0105] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0106] Corresponding to the above-mentioned optimization method for the significance representation of the anchor point vector, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned optimization method for the significance representation of the anchor point vector.

[0107] The optimization device for the saliency representation of the anchor vector provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present application are the same as those of the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0108] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0109] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0112] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the optimization method for the significance representation of the anchor vector described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0113] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0114] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for optimizing the saliency representation of anchor vectors, characterized in that: include: Obtaining a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple; A self-attention mechanism is used to establish global consistency between the anchor vectors, thereby obtaining a set of anchor vectors with implicit intrinsic relationships; A cross-attention mechanism is used to interact the anchor vector with implicit intrinsic relationship with local features to obtain a set of anchor vectors with saliency information, wherein the local features are obtained by discretizing the feature maps corresponding to the anchor vectors; Performing nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors; The optimized anchor point vectors are normalized and concatenated to obtain a saliency optimized anchor point vector.

2. The method according to claim 1, characterized in that: A self-attention mechanism is used to establish global consistency between the anchor vectors, including: Normalizing the anchor point vector to obtain a normalized anchor point vector; A self-attention mechanism is used to perform feature aggregation using the normalized anchor vector as query, key and value; The vector obtained by feature aggregation is fused with the anchor vector to obtain a set of anchor vectors with implicit internal relations.

3. The method according to claim 1, characterized in that The local features are obtained by using a pre-trained backbone network to extract features from the original image corresponding to the anchor point vector and then discretizing the extracted feature map, wherein the number of the local features is multiple.

4. The method according to claim 3, characterized in that The anchor vector with implicit intrinsic relationship is interacted with local features by using a cross attention mechanism, including: Normalizing the anchor point vector with the implicit intrinsic relationship to obtain a normalized anchor point vector with the implicit intrinsic relationship; Performing dimension alignment and normalization processing on the local features to obtain normalized local features; A cross attention mechanism is used to take the normalized anchor vector with implicit intrinsic relationship as query, and the normalized local features as keys and values ​​for feature aggregation; The vector obtained by feature aggregation is fused with the normalized anchor vector with implicit internal relationship to obtain a set of anchor vectors with saliency information.

5. The method according to claim 1, characterized in that The anchor point vector with saliency information is nonlinearly optimized, including: Normalizing the anchor point vector with saliency information to obtain a normalized anchor point vector with saliency information; Using a multi-layer perceptron to perform nonlinear optimization on the normalized anchor point vector with significant information; The vector obtained after nonlinear optimization is fused with the anchor point vector with saliency information after normalization to obtain a set of optimized anchor point vectors.

6. The method according to claim 1, characterized in that The method further comprises: Calculating the similarity between the saliency-optimized anchor vector and a preset saliency-optimized anchor vector in a scene recognition relationship, wherein the scene recognition relationship is a relationship between the preset saliency-optimized anchor vector and a target scene; A target preset saliency optimized anchor point vector corresponding to the saliency optimized anchor point vector is determined according to the similarity, and then a target scene corresponding to the target preset saliency optimized anchor point vector is used as the recognized scene.

7. An optimization device for saliency representation of anchor vectors, characterized in that: include: An acquisition unit, used to acquire a set of anchor vectors having global representation information, wherein the number of the set of anchor vectors is multiple; A global consistency establishing unit, used for establishing global consistency between the anchor vectors by using a self-attention mechanism to obtain a set of anchor vectors with implicit internal relations; An interaction unit, used for interacting the anchor vector with implicit intrinsic relationship with local features by using a cross attention mechanism to obtain a set of anchor vectors with saliency information, wherein the local features are obtained by discretizing feature maps corresponding to the anchor vectors; A nonlinear optimization unit, used for performing nonlinear optimization on the anchor point vectors with saliency information to obtain a set of optimized anchor point vectors; The normalization and splicing processing unit is used to perform normalization and splicing processing on the optimized anchor point vector to obtain an anchor point vector with optimized saliency.

8. The device according to claim 7, characterized in that The global consistency establishing unit is also used for: Normalizing the anchor point vector to obtain a normalized anchor point vector; A self-attention mechanism is used to perform feature aggregation using the normalized anchor vector as query, key and value; The vector obtained by feature aggregation is fused with the anchor vector to obtain a set of anchor vectors with implicit internal relations.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to execute the method according to any one of claims 1 to 6.

Images