Multi-modal dimension expansion method and device for power field test set, computer device, readable storage medium and program product

Abstract

The application relates to a multi-modal dimension expansion method and device for a power field test set, computer equipment, a computer readable storage medium and a computer program product, relates to the field of power data application, and can generate a high-quality multi-modal test set conforming to a power real scene to improve the accuracy of model performance analysis. The method comprises the following steps: acquiring multi-modal sample data, extracting corresponding basic features and power field adaptive features to determine feature differences and field fusion features; mapping power knowledge graph entities to a unified feature space and clustering to generate power semantic anchor points; mapping the fusion features to the unified feature space, and based on the association relationship with the anchor points, generating a region identifier indicating an effective power business region; based on the feature differences, the semantic anchor points and the region identifier, performing a derivation operation on the fusion features in the unified feature space to construct an expanded multi-modal test set.

Description

Technical Field

[0001] This application relates to the field of power data application technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for multimodal dimension expansion of test sets in the power field. Background Technology

[0002] With the development of artificial intelligence technology, multimodal large models are increasingly being used in equipment inspection and on-site safety monitoring in the power industry. In order to accurately evaluate the performance of the models in real-world complex scenarios, it is necessary to build high-quality validation test sets.

[0003] Currently, the industry typically uses general-purpose visual datasets or expands test sets through basic image enhancement operations (such as adjusting brightness, rotation, and scaling). However, general-purpose datasets lack the specialized attributes and operational constraints of power equipment, making it difficult to meet the in-depth evaluation needs of vertical industries; while basic image enhancement methods involve blind pixel-level processing, easily violating the physical laws of the image and the logic of image-text correspondence. Relying on manual collection of long-tail anomalies or violations is extremely costly.

[0004] Therefore, how to overcome the shortcomings of existing test data, such as its single dimension and poor domain adaptability, and automatically generate complex test samples that conform to the logic of real power business in order to improve the accuracy of model performance analysis is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for expanding the multimodal dimensions of test sets in the power field, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for expanding the multimodal dimensions of test sets in the power sector, including:

[0007] Acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power industry.

[0008] Based on the domain adaptation features and the basic features, determine the feature differences and domain fusion features;

[0009] Multiple domain entities in the pre-constructed power domain knowledge graph are mapped to a unified feature space and clustered to obtain the power semantic anchor points corresponding to each domain entity.

[0010] The domain fusion features are mapped to the unified feature space, and the association between the mapped domain adaptation features and the power semantic anchor is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space.

[0011] Based on the feature differences, the power semantic anchor points, and the effective region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features;

[0012] Based on the derived features, the extended multimodal test set is determined.

[0013] In one embodiment, the derived features include a first derived feature; the step of performing feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor, and the effective region identifier to obtain derived features includes:

[0014] Obtain a preset scalar scaling factor, and use the scalar scaling factor to adjust the amplitude of the feature difference to obtain a state offset vector;

[0015] Within the valid power service area indicated by the valid area identifier, the state offset vector is superimposed on the domain fusion feature to generate a first derived feature characterizing the state changes of power equipment.

[0016] In one embodiment, the derived feature includes a second derived feature; the step of performing feature derivation operations on the domain fusion feature within the unified feature space based on the feature difference, the power semantic anchor point, and the effective region identifier to obtain the derived feature further includes:

[0017] Based on a pre-defined knowledge graph of the power domain, the adjacent anchor points of the power semantic anchor point are determined, and the topological evolution direction from the power semantic anchor point to the adjacent anchor point is calculated.

[0018] Along the direction of topological evolution, the local features indicated by the effective region identifier in the domain fusion features are translated to generate a second derived feature representing the change of business scenario.

[0019] In one embodiment, the derived features include a third derived feature; the step of performing feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor, and the effective region identifier to obtain derived features includes:

[0020] The first modal feature and the second modal feature contained in the multimodal sample data are obtained, and the inverse feature that has an exclusion relationship with the power semantic anchor is determined according to the power semantic anchor; the first modal feature and the second modal feature are derived from sample data of different modalities;

[0021] Within the valid power service area indicated by the valid area identifier, a third modal feature is determined to replace the second modal feature based on the reverse feature;

[0022] Based on the first modal feature and the third modal feature, a third derived feature is determined to characterize cross-modal logic conflict.

[0023] In one embodiment, before determining the extended multimodal test set based on the derived features, the method further includes:

[0024] The derived features are mapped to the unified feature space, and the feature distance between the mapped derived features and the corresponding power semantic anchor points is obtained.

[0025] If the feature distance is greater than the preset effective semantic boundary of the corresponding power semantic anchor point, then the derived feature is constrained back to the corresponding effective power service area, so as to determine the multimodal test set based on the constrained derived feature; and,

[0026] The feature location where the derived feature changes is obtained. If the feature location is outside the effective power service area, the derived feature is removed, and the multimodal test set is determined based on the remaining derived features after removal.

[0027] In one embodiment, determining the feature differences and domain fusion features based on the domain adaptation features and the basic features includes:

[0028] The domain-adaptive features are fused with the basic features to obtain the domain-fused features;

[0029] The feature differences are determined based on the feature offset direction and feature offset magnitude of the basic features relative to the domain-adaptive features.

[0030] Secondly, this application also provides a multimodal dimension expansion device for power field test sets, comprising:

[0031] The data preprocessing and feature extraction module is used to acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain-adaptive features that match the power industry.

[0032] The domain fusion and difference analysis module is used to determine feature differences and domain fusion features based on the domain adaptation features and the basic features.

[0033] The power semantic anchor construction module is used to map multiple domain entities in a pre-built power domain knowledge graph to a unified feature space and perform clustering to obtain power semantic anchors corresponding to each domain entity.

[0034] The effective business space mapping module is used to map the domain fusion features to the unified feature space, obtain the association relationship between the mapped domain adaptation features and the power semantic anchor point, and generate an effective area identifier based on the association relationship to indicate the effective power business area corresponding to the domain entity in the unified feature space.

[0035] The feature enhancement and derivation module is used to perform feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor points, and the effective region identifiers to obtain derived features.

[0036] The test set construction and expansion module is used to determine the expanded multimodal test set based on the derived features.

[0037] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory acquires multimodal sample data and extracts the basic features of the multimodal sample data and the domain adaptation features that match the power field;

[0038] Based on the domain adaptation features and the basic features, determine the feature differences and domain fusion features;

[0039] Multiple domain entities in the pre-constructed power domain knowledge graph are mapped to a unified feature space and clustered to obtain the power semantic anchor points corresponding to each domain entity.

[0040] The domain fusion features are mapped to the unified feature space, and the association between the mapped domain adaptation features and the power semantic anchor is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space.

[0041] Based on the feature differences, the power semantic anchor points, and the effective region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features;

[0042] Based on the derived features, the extended multimodal test set is determined.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0044] Acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power industry.

[0045] Based on the domain adaptation features and the basic features, determine the feature differences and domain fusion features;

[0046] Multiple domain entities in the pre-constructed power domain knowledge graph are mapped to a unified feature space and clustered to obtain the power semantic anchor points corresponding to each domain entity.

[0047] The domain fusion features are mapped to the unified feature space, and the association between the mapped domain adaptation features and the power semantic anchor is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space.

[0048] Based on the feature differences, the power semantic anchor points, and the effective region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features;

[0049] Based on the derived features, the extended multimodal test set is determined.

[0050] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0051] Acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power industry.

[0052] Based on the domain adaptation features and the basic features, determine the feature differences and domain fusion features;

[0053] Multiple domain entities in the pre-constructed power domain knowledge graph are mapped to a unified feature space and clustered to obtain the power semantic anchor points corresponding to each domain entity.

[0054] The domain fusion features are mapped to the unified feature space, and the association between the mapped domain adaptation features and the power semantic anchor is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space.

[0055] Based on the feature differences, the power semantic anchor points, and the effective region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features;

[0056] Based on the derived features, the extended multimodal test set is determined.

[0057] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for expanding the multimodal dimensions of a test set for the power sector acquire multimodal sample data and extract its basic features and domain-adaptive features matching the power sector. Based on the domain-adaptive features and basic features, feature differences and domain fusion features are determined. Multiple domain entities in a pre-constructed power sector knowledge graph are mapped to a unified feature space and clustered to obtain power semantic anchors corresponding to each domain entity. Domain fusion features are mapped to the unified feature space, and the association between the mapped domain-adaptive features and power semantic anchors is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space. Based on feature differences, power semantic anchors, and valid region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features. Based on the derived features, the expanded multimodal test set is determined. In this application, by extracting the feature differences of multimodal samples and combining them with power semantic anchors generated by knowledge graph mapping, the traditional blind data perturbation is transformed into a directional feature evolution driven by power expertise, ensuring that the derived features conform to the physical common sense and business logic of power equipment. At the same time, by using effective region identifiers to clearly define the spatial scope of feature derivation operations, the dimensional expansion only occurs within the relevant power business area, effectively eliminating the interference of irrelevant background noise. Thus, without relying on a large amount of manual on-site data collection, a high-quality multimodal test set that conforms to the real power scenario can be automatically generated to improve the accuracy of model performance analysis. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is an application environment diagram of a multimodal dimension expansion method for a power field test set in one embodiment;

[0060] Figure 2 This is a flowchart illustrating a method for expanding the multimodal dimensions of a test set in the power sector, as shown in one embodiment.

[0061] Figure 3 This is a block diagram of a multimodal dimension extension system for a power field test set in one embodiment;

[0062] Figure 4 This is a flowchart illustrating a method for expanding the multimodal dimensions of a test set for the power sector, as described in another embodiment.

[0063] Figure 5 This is a structural block diagram of a multimodal dimension expansion device for a power field test set in one embodiment;

[0064] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0067] The multimodal dimension expansion method for power field test sets provided in this application embodiment can be applied to, for example... Figure 1 The application environment is illustrated. The terminal communicates with the server via a network. The data storage system stores the data the server needs to process. The data storage system can be integrated onto the server or located in the cloud or on other network servers. The terminal device, acting as the data acquisition or interaction end for power business data, can include power mobile inspection terminals, safety monitoring cameras, substation field sensing devices, etc., responsible for acquiring raw multimodal sample data including power equipment images, on-site operation voice, or business instructions, and sending it to the server via wired or wireless networks. The server can be a standalone physical server, a server cluster, or a cloud computing platform, internally deployed with a power domain knowledge graph and related feature derivation algorithm models. It receives the multimodal sample data and performs computational operations such as feature extraction, semantic space mapping, feature difference analysis, and dimensional expansion, thereby generating an extended multimodal test set for the power domain.

[0068] In one exemplary embodiment, such as Figure 2As shown, a method for expanding the multimodal dimensions of test sets in the power industry is provided, and this method is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps S201 to S206. Wherein:

[0069] Step S201: Obtain multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power sector.

[0070] Among them, multimodal sample data can be a heterogeneous data set including power equipment inspection images, power grid infrastructure operation videos, power operation specification texts or equipment maintenance logs. It is usually used as the original material of the test set to be expanded, and is used to build cross-modal correlations in a unified feature space.

[0071] Basic features can be obtained by encoding multimodal sample data based on a pre-trained multimodal large model in a general domain. These features represent the underlying semantic information of the samples at the level of general visual contours or linguistic logic.

[0072] Domain-adaptive features can be feature vectors extracted using pre-tuned adaptation networks, expert subnetworks, or side-path adapters tailored to the power industry vertical. These vectors are used to capture professional business attributes in the samples that are closely related to power production, safety supervision violations, or equipment defects, ensuring that subsequent feature derivation is carried out under the premise of having knowledge constraints in the power industry.

[0073] Specifically, the server inputs the acquired image data, video data, and text data describing the on-site operation into a pre-defined multimodal coding framework. During processing, the server uses the encoder's multimodal large model to extract the basic features of the samples. These features reflect common object category information and basic semantic logic within the samples. Subsequently, the server synchronously processes the same input data through a parallel-deployed domain adaptation network, extracting domain adaptation features deeply coupled with the semantics of power business, such as feature descriptions specific to power equipment models, voltage levels, or power safety supervision rules.

[0074] For example, for the input image data, video data, and text data, the basic feature representations of each modality are first extracted through the corresponding basic coding networks, denoted as... Among them, the basic feature representation is used to characterize the semantic information of the original sample under the general pre-trained model.

[0075] After obtaining the basic feature representation Then, it is input into the bottleneck mapping branch, where dimensionality reduction, nonlinear transformation, and dimensionality restoration are performed sequentially to obtain the domain-adaptive feature representation, denoted as... .

[0076] Step S202: Determine the feature differences and domain fusion features based on the domain adaptation features and basic features.

[0077] Among them, the domain fusion feature can be a composite feature representation generated by processing the basic feature and the domain adaptation feature through a specific fusion operator. It is used to simultaneously carry the general semantic logic and power business attributes in a single vector space, and serves as the basic alignment representation for subsequent dimensional expansion operations.

[0078] Feature differences can be quantified by calculating the offset of basic features relative to domain-adaptive features in the feature space, which can reflect the direction and magnitude of the shift from general representation to power-specific representation.

[0079] Specifically, the server maps and aligns the general semantic information represented by the basic features with the power industry knowledge represented by the domain-adaptive features. Through a pre-defined feature overlay logic, it generates an aligned feature representation that simultaneously reflects both the general visual features of the sample and the semantics of power business. Simultaneously, the server analyzes the relative positions of the two types of features in the same high-dimensional space to determine the feature differences between them. By quantifying the changing trends during the transformation from basic features to domain-adaptive features, the server extracts offset vectors containing domain bias information, thereby clarifying the representational shift of general capabilities when processing specific power business.

[0080] Step S203: Map multiple domain entities in the pre-constructed power domain knowledge graph to a unified feature space and perform clustering to obtain the power semantic anchor points corresponding to each domain entity.

[0081] Among them, the knowledge graph in the power field can be a pre-constructed semantic network containing the topology of the power system, equipment attributes and operating rules, which is used to provide structured prior knowledge guidance for the feature derivation of the test set.

[0082] Domain entities can be the smallest business units with clear physical meaning extracted from knowledge graphs, such as transformers, insulators, workers, or specific categories of violations, which are represented as specific semantic nodes in the feature space.

[0083] A unified feature space can be a high-dimensional vector set used to align image features, text semantics, and knowledge entity features, enabling the computation and association of different modalities under the same metric.

[0084] Electricity semantic anchors can be representative feature vectors formed by clustering and centralizing domain entities in a unified feature space. They are used to establish coordinate benchmarks with business meaning within the unified feature space.

[0085] Specifically, the server maps these discrete domain entities to a predefined unified feature space, transforming them into computable entity vectors. During the mapping process, the server ensures that this unified feature space and the multimodal features extracted in the preceding steps are distributed along the same dimension. Subsequently, the server performs clustering operations on the mapped entity vectors within the unified feature space. By analyzing the spatial distribution patterns of a large number of similar entities, it identifies the cluster centers corresponding to various types of entities. The server determines these cluster centers as power semantic anchors, with each anchor representing the standard semantic location of a specific object or scenario in the power business. In this way, the server transforms the abstract industry knowledge graph into concrete, visible, and measurable geometric reference points in the feature space.

[0086] For example, core equipment entities and their attribute description texts are extracted from the knowledge graph of the power industry, and the equipment entities and attribute description texts are mapped to the same feature space; then, the obtained entity features are clustered, and each cluster center is defined as a power semantic anchor point, denoted as . Power semantic anchors are used to characterize the central position of different power equipment categories or business objects in a unified power semantic feature space.

[0087] Step S204: Map the domain fusion features to a unified feature space, obtain the association between the mapped domain adaptation features and the power semantic anchor points, and generate a valid region identifier to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space based on the association.

[0088] Among them, the association relationship can be the spatial distance, vector angle or semantic similarity measure between the mapped domain features and each power semantic anchor point, which is used to quantify the degree of matching between the high-dimensional representation of the input sample and the preset knowledge entity.

[0089] An effective power business area can be a multi-dimensional subspace boundary defined in the feature space around a specific power semantic anchor point. It represents a reasonable range of feature distribution that conforms to common sense in power physics, equipment form or business logic.

[0090] The effective region identifier can be a spatial mask or attention weight matrix that is dynamically generated based on the above-mentioned correlation. It can binarize or divide the feature space based on a set similarity threshold, so as to accurately locate the local area where feature changes are allowed in the subsequent sample derivation process, thereby shielding background noise that is irrelevant to the target power business.

[0091] Specifically, the server synchronously projects the domain fusion features generated in the aforementioned steps onto the established unified feature space, placing them in the same computable metric environment as various power semantic anchors. After mapping, the server further analyzes the relative position and distribution patterns of the features (especially the retained domain adaptation features) with various power semantic anchors in the space, thereby extracting the correlation between them. By evaluating the strength of this correlation, the server can identify which local information in the current sample features is highly correlated with specific domain entities (such as transformers, insulators, etc.). Subsequently, based on the extracted correlations, the server generates exclusive effective region identifiers for these highly correlated feature regions, enabling these identifiers to clearly define which feature dimensions and locations belong to effective power business regions that can be expanded and evolved, and which belong to background redundant information that does not require attention within the feature space.

[0092] For example, the domain fusion features generated in the aforementioned steps are output as features. Perform a nonlinear mapping to obtain the aligned feature representation, denoted as... Aligned feature representation This serves as the input for subsequent distance constraints and region filtering in this step.

[0093] Based on this, distance constraints are used to ensure that image features and corresponding text features are semantically close within a unified power semantic feature space, while maintaining category consistency between image features and their respective power semantic anchors, thereby achieving cross-modal feature alignment.

[0094] Furthermore, for the aligned feature representation The local feature vectors corresponding to each spatial location are used to calculate their relationship with each power semantic anchor point. The similarity is calculated, and the maximum similarity is taken as the response intensity of the spatial location. Then, a binary mask is generated according to a preset threshold. The binary mask is used to identify the effective area related to power business. For the area with a value of 0 in the binary mask, the corresponding background features are suppressed; for the area with a value of 1 in the binary mask, the corresponding features are retained for subsequent dimensional expansion processing.

[0095] Step S205: Based on feature differences, power semantic anchors, and effective region identifiers, perform feature derivation operations on the domain fusion features within a unified feature space to obtain derived features.

[0096] Feature derivation operations can be automated data augmentation or feature evolution processing performed within a high-dimensional unified feature space. These operations simulate different operating states of power equipment, changes in topological relationships, or logical conflicts between images and text. They can be achieved by applying controlled vector perturbations, translations, or permutations to the original fused features. In a specific embodiment, the derivation operations include generating adjacency association test dimensions based on the semantic relationship between the target power equipment and adjacent power equipment; generating derivation test dimensions under different state differences based on the difference in feature vectors; and constructing logical inconsistency samples between images and text within the effective business area.

[0097] Derived features can be high-dimensional representations with entirely new semantic dimensions generated after the above evolution process. They are used to represent various extended scenarios that conform to the common sense of power physics and business logic (such as deepening of equipment defects, changes in line connections, etc.), thus serving as the core data elements for expanding the multimodal test set.

[0098] Specifically, the server uses domain fusion features as the operational basis, refers to the direction and magnitude of the evolution from general attributes to power-specific attributes indicated by feature differences, and combines the spatial benchmark provided by power semantic anchors to carry out multi-dimensional feature derivation operations on the domain fusion features. During the derivation process, the server strictly relies on the aforementioned generated effective region identifiers to limit the scope of the operation, ensuring that all feature changes precisely fall on local dimensions related to the target power business, while maintaining feature stability in background-irrelevant regions. Through this controlled spatial operation, the server can automatically generate derived features representing various complex power scenarios without relying on actual manual physical sampling, such as feature representations of gradual changes in equipment state, scene element evolution, or cross-modal information mismatches, thereby greatly enriching the semantic diversity of the test samples.

[0099] Step S206: Based on the derived features, determine the expanded multimodal test set.

[0100] The extended multimodal test set can be an independent dataset used to evaluate the comprehensive cognitive and reasoning capabilities of artificial intelligence models in the business scenarios of the power vertical industry. It is used to verify the recognition accuracy and generalization robustness of the model under extended conditions such as gradual changes in complex equipment states, scene topology evolution, or cross-modal logic misalignment, thereby providing a standardized and multi-dimensional evaluation benchmark for the online application of various business models in the power system.

[0101] Specifically, the server systematically sorts and structures the generated large number of derived features. Considering the massive data volume of multimodal high-dimensional features, the server typically does not directly store all feature matrices in their entirety during implementation. Instead, it extracts the key control information that dominates the derivation process and encapsulates the derived features into a lightweight parametric description. The parametric description includes at least semantic anchor point numbers, feature offset directions, scaling factors, and transformation type identifiers. Subsequently, based on different power application scenarios (such as substation equipment defect detection, construction site violation identification, or image-text consistency judgment testing tasks), the server classifies, labels, and indexes the encapsulated derived instances, organically aggregating discrete feature points into a structurally complete extended multimodal test set.

[0102] In this embodiment, by extracting the feature differences of multimodal samples and combining them with power semantic anchors generated by knowledge graph mapping, the traditional blind data perturbation is transformed into a directional feature evolution driven by power expertise, ensuring that the derived features conform to the physical common sense and business logic of power equipment. At the same time, the spatial scope of feature derivation operations is clearly defined by using effective region identifiers, so that dimensional expansion only occurs within the relevant power business area, effectively eliminating the interference of irrelevant background noise. Thus, without relying on a large amount of manual on-site data collection, a high-quality multimodal test set that conforms to the real power scenario can be automatically generated to improve the accuracy of model performance analysis.

[0103] In one embodiment, the derived features include a first derived feature; based on feature differences, power semantic anchors, and effective region identifiers, feature derivation operations are performed on the domain fusion features within a unified feature space to obtain derived features, including:

[0104] Obtain a preset scalar scaling factor, and use the scalar scaling factor to adjust the amplitude of feature differences to obtain a state offset vector; within the effective power business area indicated by the effective area identifier, superimpose the state offset vector onto the domain fusion feature to generate the first derived feature characterizing the state change of power equipment.

[0105] Among them, the first derived feature can be a derived feature generated by performing a restricted evolution on the target sample based on the domain bias information in the unified feature space. It is used to characterize the nonlinear expansion of the target sample to new business dimensions that have not yet been fully covered, while retaining the original power semantic benchmark.

[0106] Scalar adjustment can be a processing logic that uses a preset step size factor or scaling factor to quantize and scale the difference in feature vectors. It is used to dynamically adjust the degree to which derived features deviate from the original features according to test requirements.

[0107] Specifically, when the server performs feature derivation operations to generate the first derived feature, it first obtains a preset scalar scaling factor. For example, the server matches the corresponding coefficient value from the configuration library according to the state level of the power equipment to be simulated (such as slight overheating, severe overheating, etc.), and uses the coefficient value to adjust the amplitude of the determined feature difference, thereby obtaining the state offset vector.

[0108] Subsequently, the server locates the valid power service area indicated by the valid area identifier and superimposes the calculated state offset vector onto the domain fusion feature within that area. Through this local feature enhancement and offset, the server is able to generate a first derived feature characterizing the state changes of power equipment.

[0109] For example, the server generates derivation directions for corresponding new state samples based on business scenarios not yet fully covered by multimodal sample data. These new state samples may include equipment state difference samples (such as the evolution from minor rust to severe corrosion on equipment surfaces), operational risk difference samples (such as critical changes in safety distances during construction operations), and scene semantic change samples (such as interference from weather and lighting on the equipment background). Subsequently, the server superimposes the scalar-adjusted feature differences onto the domain fusion features mapped from these state samples, thereby determining the first derived feature. During this process, the server ensures that the derivation result is a limited extension of the original semantics, so that the generated derived features possess novel testing dimensions without deviating from the boundaries of real power business logic.

[0110] In this embodiment, the first derived feature obtained through the above steps improves the evaluation depth of the power multimodal test set when dealing with extreme and edge risk scenarios, thereby providing a high-quality data source for improving the robustness verification of power artificial intelligence models.

[0111] In one embodiment, the derived features include a second derived feature; based on feature differences, power semantic anchors, and effective region identifiers, feature derivation operations are performed on the domain fusion features within a unified feature space to obtain the derived features, which also includes:

[0112] Based on the pre-defined knowledge graph of the power domain, the adjacent anchor points of the power semantic anchor points are determined, and the topological evolution direction from the power semantic anchor points to the adjacent anchor points is calculated. Along the topological evolution direction, the local features indicated by the effective region identifiers in the domain fusion features are translated to generate the second derived features representing the changes in business scenarios.

[0113] Among them, the second derived feature can be a derived representation generated after performing evolution processing on the domain fusion features corresponding to the sample data based on the physical connection relationship between power entities. It is used to simulate the changes or switching of configuration relationships of related objects in the power scenario within the feature space, such as simulating the coordinated changes of adjacent switch states in a substation.

[0114] Adjacent anchors can be semantic centers of other power entities that have a direct physical connection with the current target entity, identified from the power domain knowledge graph.

[0115] The topology evolution direction can be a unit direction vector connecting the power semantic anchor point with its adjacent anchor points, which is used to indicate the path of logical translation or restricted offset of features within a unified feature space.

[0116] Specifically, the first-order and second-order adjacency entities of the target power equipment in the power domain knowledge graph are obtained, and the semantic anchor points corresponding to the target power equipment are extracted respectively. semantic anchors corresponding to adjacent devices Then, based on the directional relationship between the aligned feature representation of the target device and the semantic anchor points of adjacent devices, the feature offset direction is determined, and a restricted offset is performed within a preset constraint range to obtain the derived test dimension related to the adjacency semantics, i.e., the second derived feature. The derived result is still limited to the effective semantic boundary corresponding to the target category.

[0117] For example, when the server performs feature derivation operations to generate a second derived feature, it specifically utilizes the structured information of the knowledge graph to drive the evolution of the feature space. The server first accesses a pre-defined power domain knowledge graph and, based on the topological connection rules defined in the graph, finds and determines the adjacent anchor points of the current power semantic anchor point. For instance, if the anchor point of the current target entity represents a transformer, the server can determine the directly adjacent circuit breaker or transformer as the adjacent anchor point based on the physical connection relationships in the graph. Subsequently, the server calculates the spatial vector from the power semantic anchor point to its adjacent anchor points, thereby determining the direction of topological evolution. After obtaining the evolution direction, the server locates the local feature region indicated by the effective region identifier and performs a translation operation on the local feature along the direction of topological evolution. In this way, the server can simulate the business scenario transformation with topological association with the target device while preserving its core features, generating a second derived feature representing the dynamic evolution of the scenario.

[0118] In this embodiment, the automated derivation mechanism based on topological logic effectively verifies the depth of understanding of the spatial positional relationships and physical connection attributes between objects in complex scenarios by the power multimodal model, and enhances the applicability of the test set in the assessment of systemic correlation risks.

[0119] In one embodiment, the derived features include a third derived feature; based on feature differences, power semantic anchors, and effective region identifiers, feature derivation operations are performed on the domain fusion features within a unified feature space to obtain derived features, including:

[0120] The system acquires the first and second modal features contained in the multimodal sample data, and determines the inverse features that have an exclusion relationship with the power semantic anchor based on the power semantic anchor. The first and second modal features are derived from sample data of different modalities. Within the effective power business area indicated by the effective area identifier, a third modal feature is determined based on the inverse feature to replace the second modal feature. Based on the first and third modal features, a third derived feature representing cross-modal logical conflict is determined.

[0121] Among them, the third derived feature can be derived data that represents cross-modal logical conflicts (such as inconsistencies between graphic and textual information), and can be used to simulate complex situations in the power scenario where visual features contradict business text within the feature space.

[0122] First-modal features and second-modal features can be low-level representations from different sources but with corresponding semantics in multimodal sample data, such as visual features on the image side and descriptive features on the text side.

[0123] The exclusion relationship can be a mutually exclusive association established based on the pre-set equipment anomaly rules in the power knowledge graph (such as "wearing a safety helmet" and "not wearing a safety helmet"), while the reverse feature can be a feature fragment extracted based on the exclusion relationship that has a logical violation with the current power semantic anchor.

[0124] The third modal feature can be a new modal representation generated after coverage reconstruction in a specific business area using inverse features.

[0125] Specifically, within the valid region identifier (binary mask identifier), the feature representation of one modality remains unchanged, while local feature fragments that do not match the current semantic anchor are selected from the feature representation of the other modality and replaced with the corresponding feature representation within the target region. This generates derived samples that are semantically inconsistent between the image and text, or between different visual modalities. The replacement operation is only performed within the valid business region of the binary mask identifier to avoid introducing invalid perturbations in irrelevant background regions.

[0126] For example, the server first parses the original multimodal sample data, separating first-modal features (such as image features containing the operation scene) and second-modal features (such as text features describing the operation specifications). Then, the server, combined with the currently interested power semantic anchor, queries a pre-defined power domain knowledge graph to identify inverse features that have a business logic-rejecting relationship with the anchor. Next, the server strictly limits the operation scope to the valid power business area indicated by the valid area identifier, and based on the extracted inverse features, replaces the corresponding local areas of the original second-modal features to identify the logically deceptive third-modal features. Finally, the server combines the original and correct first-modal features with the locally logically modified third-modal features to determine the third derived feature representing cross-modal logical conflict.

[0127] In this embodiment, local feature replacement based on mutual exclusion logic avoids data distortion caused by global modifications, giving the third derived feature superficial rationality and constructing cross-modal conflicts at the deep semantic level. These automatically generated logically inconsistent samples expand the difficult negative samples in the test set, providing an objective verification method for evaluating the graph-text consistency verification capability, causal reasoning capability, and robustness against false alarms of the power multimodal model in complex scenarios.

[0128] In one embodiment, before determining the extended multimodal test set based on derived features, the method further includes:

[0129] The derived features are mapped to a unified feature space, and the feature distance between the mapped derived features and the corresponding power semantic anchor is obtained. If the feature distance is greater than the preset effective semantic boundary of the corresponding power semantic anchor, the derived features are constrained back to the corresponding effective power business area, so as to determine the multimodal test set based on the constrained derived features. The feature location where the derived features change is obtained. If the feature location is outside the effective power business area, the derived features are removed, so as to determine the multimodal test set based on the remaining derived features after removal.

[0130] Among them, feature distance can be a mathematical measure used to quantify the degree of semantic deviation between derived features and the original benchmark. It can be calculated based on Euclidean distance or cosine angle in high-dimensional vector space and is used to evaluate whether semantic drift occurs in the extended derived features.

[0131] The effective semantic boundary can be the maximum deviation threshold set based on a specific power semantic anchor point, which represents the characteristic deformation limit of a power physical entity while maintaining its own business attributes.

[0132] Feature location can be the activation region where the derived feature changes relative to the basic feature in a unified feature space.

[0133] Specifically, the server remaps the previously generated derived features to a unified feature space and measures the feature distance between these mapped derived features and their corresponding power semantic anchors. The server performs semantic boundary verification by comparing this feature distance with a preset effective semantic boundary. If the server finds that the feature distance of a certain derived feature is too large, exceeding the effective semantic boundary allowed for that category, it will take corrective measures to constrain it back to the boundary of the corresponding effective power business area along the original deviation path, thereby obtaining the constrained derived feature.

[0134] Building upon this, the server also performs a regional consistency check, which identifies the specific locations where the derived feature has changed compared to the original feature. If the server detects that most or key parts of these feature locations are outside the valid power service area indicated by the binary mask (e.g., the change occurs on an irrelevant background sky or grass), it determines that the derived feature is a poor-quality sample that introduces invalid perturbations and discards it directly. Finally, the server only performs structured encapsulation of the subsequent multimodal test set based on those derived features that have passed the boundary constraints and regional checks and are retained.

[0135] In this embodiment, based on the constraints of feature distance and semantic boundaries, the semantic collapse phenomenon during feature extrapolation can be suppressed, ensuring that derived samples maintain the physical contours of the original device. Simultaneously, region verification based on feature locations can filter out invalid derived features caused by background noise. This dual verification mechanism improves the data quality and engineering usability of the multimodal test set, avoiding model evaluation distortion due to abnormal negative samples.

[0136] In one embodiment, feature differences and domain fusion features are determined based on domain adaptation features and basic features, including:

[0137] The domain-adaptive features and the basic features are fused to obtain the domain-fused features; the feature differences are determined based on the feature offset direction and feature offset magnitude of the basic features relative to the domain-adaptive features.

[0138] Among them, feature fusion operation can be a computational logic used to structurally integrate semantic information from different representation levels in a unified high-dimensional space, including but not limited to weighted concatenation, attention mechanisms or cross mapping, so that the final generated single feature vector can simultaneously carry general visual cognition and power industry rules.

[0139] The feature offset direction and feature offset magnitude can be used as parameters to quantify the spatial changes of multimodal data during the migration process from the general semantic domain to the power professional semantic domain. The feature offset direction indicates a specific dimensional path of business logic evolution (e.g., offset towards "severe defects"), and the feature offset magnitude indicates the spatial span of semantic bias.

[0140] Specifically, the server acquires the basic features and domain-adaptive features extracted by the front end and performs feature fusion on them. During this process, the server aligns the parameters and reorganizes the general, large-scale model-based knowledge (such as common object shapes and colors) represented by the basic features with the specialized knowledge of the power vertical domain (such as specific insulator models and standard circuit breaker opening distances) represented by the domain-adaptive features, thereby generating a comprehensive domain-fusion feature, which serves as the foundation for the entire dimensional expansion process. Simultaneously, to extract the deep semantic gap between these two types of features, the server performs in-depth analysis of the relative geometric relationship between the basic features and the domain-adaptive features within the feature space. Using the basic features as the starting point and the domain-adaptive features as the ending point, the server accurately assesses the feature offset direction when the basic features are transformed relative to the domain-adaptive features in space, and calculates the high-dimensional spatial distance traversed by this transformation to determine the feature offset magnitude.

[0141] For example, the domain adaptation feature is represented Representation of basic features Weighted fusion is performed to obtain the output features. Furthermore, computational domain-adaptive feature representations Representation of basic features The difference between them yields the feature differences. (Vector values). Eigenvector differences Used to characterize the direction and magnitude of the shift from general characterization to characterization in the power field.

[0142] In this embodiment, by decoupling and recombining general features with prior knowledge of electricity, and quantifying the domain bias into feature offset direction and magnitude, the generated fused features are ensured to conform to the physical constraints of electricity. Furthermore, the separated feature differences provide physically meaningful guidance for subsequent sample extrapolation and state evolution, avoiding disordered expansion of the feature space and improving the domain accuracy of the multimodal test set while reducing computational overhead.

[0143] To enable those skilled in the art to better understand the above steps, the following example illustrates the embodiments of this application, but it should be understood that the embodiments of this application are not limited thereto.

[0144] With the deepening application of artificial intelligence technology in the power industry, multimodal large-scale models are playing an increasingly important role in safety supervision, infrastructure construction, and production. During the model development and deployment cycle, the test set, as an evaluation benchmark independent of the training data, directly determines the accuracy of the assessment of the model's real-world operational capabilities. In complex power business scenarios, model evaluation often relies not only on single-modal perception but also on changing on-site visual factors (such as personnel status, equipment occlusion, and perspective changes) and cross-modal logical consistency between image content and safety regulations and work order descriptions. Therefore, constructing a high-quality multimodal test set that can realistically and completely reflect the complexity and safety constraints of power operations is of great significance for objectively verifying the robustness and comprehensive discriminative ability of the model in actual operating environments.

[0145] In existing technologies, test set construction for multimodal tasks typically follows general domain processing schemes. These methods primarily expand the number of test samples through manual screening, random sampling, or conventional data augmentation techniques (such as adjusting image brightness, cropping, rotating, blurring, or performing simple synonym replacement on text). When processing multimodal data, existing construction logic usually involves separately processing the collected on-site images, video clips, and text descriptions before performing a simple physical combination, heavily relying on human experience to maintain the basic quality of the samples. This traditional construction approach remains fundamentally focused on applying general pixel-level or literal-level perturbations to the original samples, without establishing a dedicated test dimension generation mechanism based on prior knowledge for specific vertical industries.

[0146] However, the aforementioned existing technologies have significant limitations when applied to practical applications in the power sector. On the one hand, existing sample generation processes do not embed deep business constraints such as equipment attributes and operational standards in the power industry, and it is difficult to automatically generate test data in batches that covers complex correlation changes (such as key component obstruction and personnel location violations), resulting in a disconnect between the sample set and real business scenarios and a narrow scenario coverage. On the other hand, existing methods lack specific test designs for consistency between text and graphics, lack systematic cross-modal logical conflict samples, and the organization and management of test samples are relatively crude, making it impossible to finely locate the model's shortcomings in specific evaluation dimensions. These shortcomings together make it difficult for existing test sets to comprehensively and accurately evaluate the comprehensive cognitive ability and robustness of power-specific multimodal models in complex field situations.

[0147] To address the aforementioned issues, this application also provides, for example: Figure 3 The multimodal dimension extension system shown is designed for power sector test sets. Figure 4 The multimodal dimension expansion method for power domain test sets shown can be applied to the system, and the method is executed by the system, wherein:

[0148] The sample parsing module receives and processes image, video, and text data from power business scenarios, extracting domain-adaptive feature vectors through cross-modal pre-trained models and side-path adapters. The knowledge constraint module accesses the power domain knowledge graph, constructs a unified power semantic feature space for feature mapping, generates background filtering masks using gating networks, and outputs a feature subset that conforms to professional representation constraints. The dimension expansion module performs constrained feature derivation operations within the feature space, including feature offset units based on knowledge graph adjacency relationships and state extrapolation units based on feature vector differences, outputting derived test dimensions. The consistency verification module performs semantic boundary verification and image-text consistency verification to determine whether the derived dimensions conform to established business rules and cross-modal correspondences. The test set encapsulation module receives compliant feature representations, performs anchor indexing and parameter description encapsulation, and finally outputs a structured evaluation dataset.

[0149] In this embodiment, the alignment and constraint expression of image and text modalities in a unified power semantic feature space are realized; by introducing a feature derivation mechanism based on knowledge graphs, test dimensions oriented towards power business scenarios can be automatically generated; and through image-text consistency verification and effective region filtering mechanisms, the business relevance and evaluation effectiveness of the generated test samples are improved.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0151] Based on the same inventive concept, this application also provides a device for expanding the multimodal dimensions of a power field test set, which implements the above-described method for expanding the multimodal dimensions of a power field test set. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for expanding the multimodal dimensions of a power field test set provided below can be found in the limitations of the method for expanding the multimodal dimensions of a power field test set described above, and will not be repeated here.

[0152] In one exemplary embodiment, such as Figure 5 As shown, a multimodal dimension expansion device for power industry test sets is provided, including: a data preprocessing and feature extraction module 510, a domain fusion and difference analysis module 520, a power semantic anchor construction module 530, an effective business space mapping module 540, a feature enhancement and derivation module 550, and a test set construction and expansion module 560, wherein:

[0153] The data preprocessing and feature extraction module 510 is used to acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power field.

[0154] Domain fusion and difference analysis module 520 is used to determine feature differences and domain fusion features based on the domain adaptation features and the basic features;

[0155] The power semantic anchor construction module 530 is used to map multiple domain entities in the pre-built power domain knowledge graph to a unified feature space and perform clustering to obtain the power semantic anchors corresponding to each domain entity.

[0156] The effective business space mapping module 540 is used to map the domain fusion features to the unified feature space, obtain the association relationship between the mapped domain adaptation features and the power semantic anchor point, and generate an effective area identifier based on the association relationship to indicate the effective power business area corresponding to the domain entity in the unified feature space.

[0157] The feature enhancement and derivation module 550 is used to perform feature derivation operations on the domain fusion features in the unified feature space based on the feature differences, the power semantic anchor points, and the effective region identifiers to obtain derived features.

[0158] The test set construction and expansion module 560 is used to determine the expanded multimodal test set based on the derived features.

[0159] In one embodiment, the domain fusion and difference analysis module 520 is further configured to:

[0160] The domain-adaptive features are fused with the basic features to obtain the domain-fused features;

[0161] The feature differences are determined based on the feature offset direction and feature offset magnitude of the basic features relative to the domain-adaptive features.

[0162] In one embodiment, the derived feature includes a first derived feature; the feature enhancement and derivation module 550 is further configured to:

[0163] Obtain a preset scalar scaling factor, and use the scalar scaling factor to adjust the amplitude of the feature difference to obtain a state offset vector;

[0164] Within the valid power service area indicated by the valid area identifier, the state offset vector is superimposed on the domain fusion feature to generate a first derived feature characterizing the state changes of power equipment.

[0165] In one embodiment, the derived feature includes a second derived feature; the feature enhancement and derivation module 550 is further configured to:

[0166] Based on a pre-defined knowledge graph of the power domain, the adjacent anchor points of the power semantic anchor point are determined, and the topological evolution direction from the power semantic anchor point to the adjacent anchor point is calculated.

[0167] Along the direction of topological evolution, the local features indicated by the effective region identifier in the domain fusion features are translated to generate a second derived feature representing the change of business scenario.

[0168] In one embodiment, the derived feature includes a third derived feature; the feature enhancement and derivation module 550 is further configured to:

[0169] The first modal feature and the second modal feature contained in the multimodal sample data are obtained, and the inverse feature that has an exclusion relationship with the power semantic anchor is determined according to the power semantic anchor; the first modal feature and the second modal feature are derived from sample data of different modalities;

[0170] Within the valid power service area indicated by the valid area identifier, a third modal feature is determined to replace the second modal feature based on the reverse feature;

[0171] Based on the first modal feature and the third modal feature, a third derived feature is determined to characterize cross-modal logic conflict.

[0172] In one embodiment, the test set construction and expansion module 560 is further configured to:

[0173] The derived features are mapped to the unified feature space, and the feature distance between the mapped derived features and the corresponding power semantic anchor points is obtained.

[0174] If the feature distance is greater than the preset effective semantic boundary of the corresponding power semantic anchor point, then the derived feature is constrained back to the corresponding effective power service area, so as to determine the multimodal test set based on the constrained derived feature; and,

[0175] The feature location where the derived feature changes is obtained. If the feature location is outside the effective power service area, the derived feature is removed, and the multimodal test set is determined based on the remaining derived features after removal.

[0176] Each module in the aforementioned multimodal dimension expansion device for power sector test sets can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0177] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a multimodal dimension expansion method for a test set in the power industry.

[0178] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0179] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0180] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0181] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0183] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0184] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0185] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for expanding the multimodal dimensions of test sets in the power sector, characterized in that, The method includes: Acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain adaptation features that match the power industry. Based on the domain adaptation features and the basic features, determine the feature differences and domain fusion features; Multiple domain entities in the pre-constructed power domain knowledge graph are mapped to a unified feature space and clustered to obtain the power semantic anchor points corresponding to each domain entity. The domain fusion features are mapped to the unified feature space, and the association between the mapped domain adaptation features and the power semantic anchor is obtained. Based on the association, a valid region identifier is generated to indicate the corresponding valid power business area of ​​the domain entity in the unified feature space. Based on the feature differences, the power semantic anchor points, and the effective region identifiers, feature derivation operations are performed on the domain fusion features within the unified feature space to obtain derived features; Based on the derived features, the extended multimodal test set is determined.

2. The method according to claim 1, characterized in that, The derived features include a first derived feature; the process of performing feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor point, and the effective region identifier to obtain derived features includes: Obtain a preset scalar scaling factor, and use the scalar scaling factor to adjust the amplitude of the feature difference to obtain a state offset vector; Within the valid power service area indicated by the valid area identifier, the state offset vector is superimposed on the domain fusion feature to generate a first derived feature characterizing the state changes of power equipment.

3. The method according to claim 2, characterized in that, The derived features include second derived features; the step of performing feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor point, and the effective region identifier to obtain derived features further includes: Based on a pre-defined knowledge graph of the power domain, the adjacent anchor points of the power semantic anchor point are determined, and the topological evolution direction from the power semantic anchor point to the adjacent anchor point is calculated. Along the direction of topological evolution, the local features indicated by the effective region identifier in the domain fusion features are translated to generate a second derived feature representing the change of business scenario.

4. The method according to claim 1, characterized in that, The derived features include a third derived feature; the feature derivation operation on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor point, and the effective region identifier to obtain derived features includes: The first modal feature and the second modal feature contained in the multimodal sample data are obtained, and the inverse feature that has an exclusion relationship with the power semantic anchor is determined according to the power semantic anchor; the first modal feature and the second modal feature are derived from sample data of different modalities; Within the valid power service area indicated by the valid area identifier, a third modal feature is determined to replace the second modal feature based on the reverse feature; Based on the first modal feature and the third modal feature, a third derived feature is determined to characterize cross-modal logic conflict.

5. The method according to claim 1, characterized in that, Before determining the extended multimodal test set based on the derived features, the process further includes: The derived features are mapped to the unified feature space, and the feature distance between the mapped derived features and the corresponding power semantic anchor points is obtained. If the feature distance is greater than the preset effective semantic boundary of the corresponding power semantic anchor point, then the derived feature is constrained back to the corresponding effective power service area, so as to determine the multimodal test set based on the constrained derived feature; and, The feature location where the derived feature changes is obtained. If the feature location is outside the effective power service area, the derived feature is removed, and the multimodal test set is determined based on the remaining derived features after removal.

6. The method according to claim 1, characterized in that, The step of determining feature differences and domain fusion features based on the domain adaptation features and the basic features includes: The domain-adaptive features are fused with the basic features to obtain the domain-fused features; The feature differences are determined based on the feature offset direction and feature offset magnitude of the basic features relative to the domain-adaptive features.

7. A multimodal dimension expansion device for test sets in the power industry, characterized in that, The device includes: The data preprocessing and feature extraction module is used to acquire multimodal sample data and extract the basic features of the multimodal sample data as well as the domain-adaptive features that match the power industry. The domain fusion and difference analysis module is used to determine feature differences and domain fusion features based on the domain adaptation features and the basic features. The power semantic anchor construction module is used to map multiple domain entities in a pre-built power domain knowledge graph to a unified feature space and perform clustering to obtain power semantic anchors corresponding to each domain entity. The effective business space mapping module is used to map the domain fusion features to the unified feature space, obtain the association relationship between the mapped domain adaptation features and the power semantic anchor point, and generate an effective area identifier based on the association relationship to indicate the effective power business area corresponding to the domain entity in the unified feature space. The feature enhancement and derivation module is used to perform feature derivation operations on the domain fusion features within the unified feature space based on the feature differences, the power semantic anchor points, and the effective region identifiers to obtain derived features. The test set construction and expansion module is used to determine the expanded multimodal test set based on the derived features.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images