Gateway quality determination method and device, equipment and medium

Abstract

The invention provides a gateway quality determination method and device, equipment and a medium, and relates to the technical field of computers. The method comprises the following steps: firstly, acquiring target vibration data for performing a vibration test on an internal component of a target gateway and a target component video formed by monitoring the process of performing the vibration test on the internal component; secondly, through a semantic mining unit in a target quality determination network, respectively performing semantic mining on the target vibration data and the target component video to form vibration semantic features and component semantic features; then, semantic enhancement is carried out on the component semantic features based on the vibration semantic features through a semantic enhancement unit, and enhanced component semantic features are formed; and finally, through a quality determination unit, based on the semantic features of the enhanced component, determining target quality data. Based on the content, the problem that the reliability of gateway quality determination is relatively low in the prior art can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology. Specifically, it relates to a method and device, equipment, and medium for determining gateway quality. Background Art

[0002] As a high-performance network device, the FTTR (Fiber to the Room) gateway undertakes the core function of data transmission in scenarios such as homes and enterprises. With the continuous development of network technology, the performance requirements of the FTTR gateway are constantly increasing, especially in terms of reliability and stability. To ensure the long-term stable operation of the FTTR gateway in actual use, it is necessary to comprehensively evaluate and verify its quality. For example, in the application of the FTTR gateway, it may face a complex environment, such as being affected by external forces, and the connection stability of internal components may be reduced during vibration. Therefore, it is necessary to test its connection stability. However, in the prior art, generally, after vibration, testers observe to determine the corresponding connection quality. In this way, there will be a problem that the reliability of quality determination is relatively low. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method and device, equipment, and medium for determining gateway quality to improve the problem of relatively low reliability in determining gateway quality existing in the prior art.

[0004] To achieve the above purpose, this application adopts the following technical solutions: A method for determining gateway quality includes: Obtaining target vibration data from a vibration test on internal components of a target gateway and a target component video formed by monitoring the process of the vibration test on the internal components; Through a semantic mining unit in a target quality determination network, respectively performing semantic mining on the target vibration data and the target component video to form vibration semantic features and component semantic features, where the target quality determination network further includes a semantic enhancement unit and a quality determination unit; Through the semantic enhancement unit, based on the vibration semantic features, performing semantic enhancement on the component semantic features to form enhanced component semantic features; Through the quality determination unit, based on the enhanced component semantic features, determining target quality data corresponding to the target gateway, where the target quality data is used to characterize the connection stability of the internal components of the target gateway.

[0005] In a preferred option of the present application, in the above gateway quality determination method, the step of respectively performing semantic mining on the target vibration data and the target component video by the semantic mining unit in the target quality determination network to form vibration semantic features and component semantic features includes: Segment the target vibration data according to the target duration carried by the semantic mining unit in the target quality determination network to form a plurality of local vibration data. Wherein, the target vibration data belongs to time series data and at least includes vibration frequency, vibration amplitude, vibration type, and vibration direction, and the duration corresponding to each local vibration data is equal to the target duration; Based on the target duration, perform frame extraction on the target component video to form a plurality of target component video frames. Wherein, the time interval between the timestamps corresponding to every two adjacent target component video frames is equal to the target duration, and there is a one-to-one correspondence between the plurality of target component video frames and the plurality of local vibration data; Perform semantic mining on the plurality of local vibration data through the first semantic mining branch in the semantic mining unit to form vibration semantic features, where the vibration semantic features include a plurality of local vibration semantic features corresponding one-to-one to the plurality of local vibration data; Perform semantic mining on the plurality of target component video frames through the second semantic mining branch in the semantic mining unit to form component semantic features, where the component semantic features include a plurality of local component semantic features corresponding one-to-one to the plurality of target component video frames.

[0006] In a preferred option of the present application, in the above gateway quality determination method, the step of performing semantic mining on the plurality of local vibration data through the first semantic mining branch in the semantic mining unit to form vibration semantic features includes: For each local vibration data in the plurality of local vibration data, load the local vibration data into the first semantic mining branch in the semantic mining unit; Perform word embedding processing on the parameter sequences of each dimension in the local vibration data to form word embedding features corresponding to each parameter of each dimension. Wherein, the feature size of each word embedding feature is 1*n; Concatenate the word embedding features corresponding to each parameter in the parameter sequences of the same dimension to form a concatenated embedding feature, where the size of the concatenated embedding feature is 1*a, a is equal to m*n, and m is equal to the number of parameters in the parameter sequence of the same dimension; Combine the concatenated embedding features corresponding to each dimension of the local vibration data to form a combined embedding feature, where the size of the combined embedding feature is b*a, and b is equal to the number of dimensions corresponding to the local vibration data; Perform self-attention processing on the combined embedding feature to form the local vibration semantic feature corresponding to the local vibration data, and determine the vibration semantic feature based on the local vibration semantic feature corresponding to each local vibration data.

[0007] In a preferred selection of the present application, in the above gateway quality determination method, the step of performing semantic mining on the multiple target component video frames through the second semantic mining branch in the semantic mining unit to form component semantic features includes: Load each of the multiple target component video frames into the second semantic mining branch in the semantic mining unit in sequence; Perform convolution processing on the currently loaded target component video frame to obtain a component convolution feature; Fuse the local component semantic feature corresponding to the previously loaded target component video frame into the component convolution feature to form the local component semantic feature corresponding to the currently loaded target component video frame, where the local component semantic feature corresponding to the previously loaded target component video frame of the first loaded target component video frame is the semantic feature carried by the second semantic mining branch, and this semantic feature is formed during the training process of the second semantic mining branch; Determine the component semantic feature based on the local component semantic feature corresponding to each target component video frame.

[0008] In a preferred selection of the present application, in the above gateway quality determination method, the step of fusing the local component semantic feature corresponding to the previously loaded target component video frame into the component convolution feature to form the local component semantic feature corresponding to the currently loaded target component video frame includes: Obtain the power module component contour map corresponding to the currently loaded target component video frame, and perform convolution processing on the power module component contour map to obtain a contour map convolution feature; Calculate the feature correlation parameters between the contour map convolution feature and the component convolution feature and the local component semantic feature corresponding to the previously loaded target component video frame respectively, and update the component convolution feature and the local component semantic feature corresponding to the previously loaded target component video frame respectively based on the corresponding feature correlation parameters to form a first component update feature and a second component update feature; Calculate the feature correlation parameter between the first component update feature and the second component update feature, and update the first component update feature based on the feature correlation parameter to form the local component semantic feature corresponding to the currently loaded target component video frame.

[0009] In a preferred option of the present application, in the above gateway quality determination method, the step of forming the enhanced component semantic feature by enhancing the component semantic feature based on the vibration semantic feature through the semantic enhancement unit includes: Load the local vibration semantic feature in the vibration semantic feature and the local component semantic feature corresponding to the local vibration semantic feature in the component semantic feature into the semantic enhancement unit; Perform forward mining on the local component semantic feature, and apply noise features during the forward mining process to form a component forward mining feature; Perform backward reduction on the component forward mining feature, and fuse the local vibration semantic feature during the backward reduction process to form an enhanced component semantic feature. Among them, both forward mining and backward reduction include multiple stages. During the forward mining process, as the stage progresses, the size of the mined feature gradually decreases. During the backward reduction process, as the stage progresses, the size of the reduced feature gradually increases.

[0010] In a preferred option of the present application, in the above gateway quality determination method, the step of performing backward reduction on the component forward mining feature, and fusing the local vibration semantic feature during the backward reduction process to form an enhanced component semantic feature includes: For the first backward reduction stage, perform multiple feature reductions on the component forward mining feature and the local vibration semantic feature respectively to form corresponding multiple first reduction features and multiple second reduction features. Calculate the feature correlation parameter between each first reduction feature and the corresponding second reduction feature respectively to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. Based on the target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the updated component semantic feature of the current stage. Among them, feature reduction includes transposed convolution and / or interpolation; For each subsequent backward reduction stage starting from the second stage, perform multiple feature reductions on the semantic features of the update components in the previous stage and the second reduction features corresponding to the last feature reduction in the previous stage, respectively, to form corresponding multiple first reduction features and multiple second reduction features. Further, calculate the feature correlation parameters between each first reduction feature and the corresponding second reduction feature, respectively, to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. Based on this target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the semantic features of the update components in the current stage; Determine the semantic features of the reinforcement components based at least on the semantic features of the update components in the last stage.

[0011] This application also provides a gateway quality determination device, including: A test data acquisition module, configured to acquire target vibration data obtained by performing a vibration test on the internal components of a target gateway and a target component video formed by monitoring the process of performing the vibration test on the internal components; A data semantic mining module, configured to respectively perform semantic mining on the target vibration data and the target component video through a semantic mining unit in a target quality determination network to form vibration semantic features and component semantic features, where the target quality determination network further includes a semantic reinforcement unit and a quality determination unit; A feature semantic reinforcement module, configured to perform semantic reinforcement on the component semantic features based on the vibration semantic features through the semantic reinforcement unit to form reinforced component semantic features; A component quality analysis module, configured to determine target quality data corresponding to the target gateway based on the reinforced component semantic features through the quality determination unit, where the target quality data is used to characterize the connection stability of the internal components of the target gateway.

[0012] On the basis of the above, this application also provides an electronic device, including: A memory, configured to store a computer program; A processor connected to the memory, configured to execute the computer program stored in the memory to implement the above gateway quality determination method.

[0013] On the basis of the above, this application also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs, it executes each step of the above gateway quality determination method.

[0014] The gateway quality determination method, device, equipment, and medium provided by this application obtain target vibration data from vibration testing of the internal components of the target gateway and a target component video formed by monitoring the process of vibration testing the internal components. Secondly, through the semantic mining unit in the target quality determination network, semantic mining is respectively performed on the target vibration data and the target component video to form vibration semantic features and component semantic features. Then, through the semantic enhancement unit, the component semantic features are semantically enhanced based on the vibration semantic features to form enhanced component semantic features. Finally, through the quality determination unit, target quality data is determined based on the enhanced component semantic features. Based on the above, on the one hand, potential semantic information in the target component video is mined based on a neural network, so that the obtained component semantic features have a high semantic representation ability. On the other hand, the component semantic features are enhanced based on the vibration semantic features corresponding to the target vibration data, so that the semantic representation accuracy of the formed enhanced component semantic features is further improved. Therefore, the reliability of the target quality data determined based on the enhanced component semantic features can be improved, thereby improving the relatively low reliability problem of gateway quality determination existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To make the above objects, features, and advantages of this application more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows.

[0016] Figure 1 It is a structural block diagram of an electronic device provided by an embodiment of this application.

[0017] Figure 2 It is a schematic flowchart of a gateway quality determination method provided by an embodiment of this application.

[0018] Figure 3 It is a schematic diagram of segmentation processing and frame extraction processing provided by an embodiment of this application.

[0019] Figure 4 It is a schematic diagram of backward restoration provided by an embodiment of this application.

[0020] Figure 5 It is a schematic block diagram of a gateway quality determination device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objects, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Usually, the components of the embodiments of this application described and shown in the drawings here can be arranged and designed in various different configurations.

[0022] Accordingly, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0023] As Figure 1 shown, an embodiment of the present application provides an electronic device. Among them, the electronic device may include a memory, a processor, and a gateway quality determination device.

[0024] Specifically, the memory and the processor are directly or indirectly electrically connected to achieve data transmission or interaction. For example, the memory and the processor may be electrically connected through one or more communication buses or signal lines. The gateway quality determination device includes at least one software function module stored in the memory in the form of software or firmware. The processor is configured to execute the executable computer programs stored in the memory, such as the software function modules and computer programs included in the gateway quality determination device, to implement the gateway quality determination method provided by the embodiment of the present application.

[0025] Optionally, the memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0026] Moreover, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0027] It can be understood that Figure 1The structure shown is only schematic, and the electronic device may further include more or fewer components than those shown in Figure 1 or have a different configuration from that shown in Figure 1 . For example, it may further include a communication unit for information interaction with other devices.

[0028] In combination with Figure 2 , an embodiment of the present application further provides a gateway quality determination method applicable to the above-mentioned electronic device. Among them, the method steps defined by the processes related to the gateway quality determination method can be implemented by the electronic device. The following will elaborate in detail on Figure 2 the specific process shown.

[0029] Step S110, obtain target vibration data obtained by performing a vibration test on the internal components of a target gateway and a target component video formed by monitoring the process of performing the vibration test on the internal components.

[0030] In an embodiment of the present application, the electronic device can obtain target vibration data obtained by performing a vibration test on the internal components of the target gateway (such as the operating parameters of the corresponding vibration device, and the target gateway can be fixed on the vibration device to perform the corresponding vibration test) and a target component video formed by monitoring the process of performing the vibration test on the internal components (which can be obtained by monitoring through a corresponding video monitoring device). In addition, the specific application scenario of the target gateway can be FTTR-B (Business FTTR), etc., that is, a business enterprise all-optical networking solution, which is an all-optical networking solution for business enterprise scenarios and is a Wi-Fi solution specifically designed for enterprises. Using the all-optical networking solution of fiber access + fiber-optic composite cable + Wi-Fi6, Wi-Fi is covered in every corner of the enterprise, providing network services for the networking work of enterprise employees, such as live broadcasts, online meetings, and other activities. In addition, the target gateway can also be used to provide IPTV (Internet Protocol Television) services, etc.

[0031] Step S120, respectively perform semantic mining on the target vibration data and the target component video through a semantic mining unit in a target quality determination network to form vibration semantic features and component semantic features.

[0032] In an embodiment of the present application, after obtaining the target vibration data and the target component video, the electronic device can respectively perform semantic mining on the target vibration data and the target component video through a semantic mining unit in the target quality determination network to form vibration semantic features and component semantic features, that is, respectively extract the potential semantic information in the target vibration data and the target component video. Additionally, in an embodiment of the present application, the manifestation form of each feature can be a vector (or matrix). The target quality determination network (which can be a trained neural network model. During the training process, it can learn the mapping relationship between the sample vibration data and the sample component video and the corresponding quality labels. The quality labels can be formed by expert annotation or identified and determined by other network models) further includes a semantic enhancement unit and a quality determination unit.

[0033] Step S130: Through the semantic enhancement unit, perform semantic enhancement on the component semantic features based on the vibration semantic features to form enhanced component semantic features.

[0034] In an embodiment of the present application, after extracting the vibration semantic features and the component semantic features, the electronic device can perform semantic enhancement on the component semantic features based on the vibration semantic features through the semantic enhancement unit to form enhanced component semantic features. That is to say, the vibration semantic features can be fused into the component semantic features, so that the semantic information in the component semantic features is enhanced, thereby forming enhanced component semantic features with better representation ability.

[0035] Step S140: Through the quality determination unit, determine the target quality data corresponding to the target gateway based on the enhanced component semantic features.

[0036] In an embodiment of the present application, after forming the semantic features of the enhanced component, the electronic device may, through the quality determination unit, determine the target quality data corresponding to the target gateway based on the semantic features of the enhanced component. The target quality data is used to characterize the connection stability of the internal components of the target gateway. Exemplarily, the quality determination unit may include a fully connected network layer and an output layer, and the output layer may include a linear regression function or a classification function (such as softmax, etc.). Specifically, in one implementation, the target quality data may be a value within the continuous interval of 0-1. The larger the value, the higher the connection stability. Thus, after the fully connected network layer processes the semantic features of the enhanced component, a parameter may be output, and then, this parameter may be mapped to the interval 0-1 through a linear regression function (such as an identity mapping, etc.), that is, the target quality data is obtained. In another implementation, the target quality data may be values such as 0, 1, 2, 3,......, 10 (or types such as high quality, low quality, etc.). Thus, the fully connected network layer may convert the semantic features of the enhanced component into a vector, such as a vector with a size of 1*10, and then, through a classification function, map this vector into a probability distribution with a size of 1*10. Among them, one parameter represents the probability of one type, such as the probability of "0", the probability of "1", the probability of "2", the probability of "10", etc. Thus, the type with the highest probability may be determined as the target quality data.

[0037] Based on the above content, on the one hand, potential semantic information in the target component video is mined based on a neural network, so that the obtained component semantic features have a high semantic representation ability. On the other hand, based on the vibration semantic features corresponding to the target vibration data, the component semantic features are enhanced, so that the semantic representation accuracy of the formed enhanced component semantic features is further improved. Therefore, the reliability of the target quality data determined based on the enhanced component semantic features can be improved, thereby improving the relatively low reliability problem of gateway quality determination existing in the prior art.

[0038] First, regarding step S120, it should be noted that the specific ways of performing semantic mining on the target vibration data and the target component video are not limited and can be selected according to actual needs.

[0039] For example, in an alternative implementation, in order to improve the accuracy of semantic mining and make the semantic information represented by the mined vibration semantic features and component semantic features more reliable, step S120 described above may further include the following steps S121, S122, S123, and S124, and the specific content of each step is described as follows.

[0040] Step S121: Determine the target duration carried by the semantic mining unit in the network according to the target quality, and segment the target vibration data to form multiple local vibration data.

[0041] In the embodiment of the present application, the target duration carried by the semantic mining unit in the network can be determined according to the target quality (which can be used as a network parameter of the semantic mining unit, formed during the training process, and can be a randomly generated value in the initial stage), and the target vibration data is segmented to form multiple local vibration data. Among them, the target vibration data belongs to time series data and at least includes vibration frequency (for example, the frequency range is 5 Hz to 2000 Hz to cover various possible vibrations in the application environment), vibration amplitude, vibration type (such as sine wave vibration, random vibration, etc.), and vibration direction. The duration corresponding to each local vibration data is equal to the target duration.

[0042] Step S122: Based on the target duration, perform frame extraction on the target component video to form multiple target component video frames.

[0043] In the embodiment of the present application, frame extraction can also be performed on the target component video based on the target duration to form multiple target component video frames. For example, the first target component video frame is a video frame corresponding to the target duration starting from the initial timestamp. Among them, the interval duration between the timestamps corresponding to every two adjacent target component video frames is equal to the target duration. There is a one-to-one correspondence between the multiple target component video frames and the multiple local vibration data. For example, the first target component video frame corresponds to the first local vibration data, the second target component video frame corresponds to the second local vibration data, as shown in Figure 3 shown.

[0044] Step S123: Through the first semantic mining branch in the semantic mining unit, perform semantic mining on the multiple local vibration data to form vibration semantic features.

[0045] In the embodiment of the present application, after segmenting to form the multiple local vibration data, semantic mining can be performed on the multiple local vibration data through the first semantic mining branch in the semantic mining unit to form vibration semantic features. Among them, the vibration semantic features include multiple local vibration semantic features corresponding one-to-one to the multiple local vibration data. For example, the first local vibration data corresponds to the first local vibration semantic feature, the second local vibration data corresponds to the second local vibration semantic feature, and the third local vibration data corresponds to the third local vibration semantic feature.

[0046] Step S124: Through the second semantic mining branch in the semantic mining unit, perform semantic mining on the multiple target component video frames to form component semantic features.

[0047] In the embodiment of the present application, after the multiple target component video frames are formed by truth extraction, semantic mining can be performed on the multiple target component video frames through the second semantic mining branch in the semantic mining unit to form component semantic features. Among them, the component semantic features include multiple local component semantic features corresponding one-to-one to the multiple target component video frames. For example, the first target component video frame corresponds to the first local component semantic feature, the second target component video frame corresponds to the second local component semantic feature, and the third target component video frame corresponds to the third local component semantic feature.

[0048] It can be understood that in the above step S123, the specific method of performing semantic mining on the multiple local vibration data is not limited. For example, in an alternative embodiment, in order to ensure the accuracy of semantic mining and make the semantic representation accuracy of the obtained vibration semantic features higher, the above step S123 may further include the following content: First, for each local vibration data in the multiple local vibration data, load the local vibration data into the first semantic mining branch in the semantic mining unit; Second, word embedding processing can be performed on the parameter sequences of each dimension in the local vibration data (which can be implemented through a corresponding word embedding model) to form word embedding features corresponding to each parameter (one time point corresponds to one parameter) of each dimension (such as four dimensions of vibration frequency, vibration amplitude, vibration type, and vibration direction), where the feature size of each word embedding feature is 1*n; Then, the word embedding features corresponding to each parameter in the parameter sequence belonging to the same dimension can be concatenated to form a concatenated embedding feature, where the size of the concatenated embedding feature is 1*a, a is equal to m*n, and m is equal to the number of parameters in the parameter sequence of the same dimension. For example, the word embedding features of the parameters at each time point corresponding to the vibration frequency can be concatenated to form a concatenated embedding feature corresponding to the vibration frequency; the word embedding features of the parameters at each time point corresponding to the vibration amplitude can be concatenated to form a concatenated embedding feature corresponding to the vibration amplitude; the word embedding features of the parameters at each time point corresponding to the vibration type can be concatenated to form a concatenated embedding feature corresponding to the vibration type; the word embedding features of the parameters at each time point corresponding to the vibration direction can be concatenated to form a concatenated embedding feature corresponding to the vibration direction; Further, the concatenated embedding features corresponding to each dimension in the local vibration data can be combined to form a combined embedding feature, where the size of the combined embedding feature is b*a, where b is equal to the number of dimensions corresponding to the local vibration data, such as the above-mentioned 4 dimensions; Finally, self-attention processing can be performed on the combined embedding feature to form the local vibration semantic feature corresponding to the local vibration data, and based on the local vibration semantic feature corresponding to each local vibration data, the vibration semantic feature can be determined; based on this, since the combined embedding feature includes the semantic information of each dimension, therefore, through self-attention processing, the semantic information of each dimension can be associated and mined, thereby improving the reliability of the mined semantic information.

[0049] It can be understood that in the above step S124, the specific method for semantic mining of the multiple target component video frames is not limited. For example, in an alternative embodiment, in order to improve the accuracy of semantic mining and make the reliability of the obtained component semantic features higher, the above step S124 can further include step S124a, step S124b, step S124c, and step S124d, and the specific content of each step is as follows.

[0050] Step S124a: Load each of the multiple target component video frames into the second semantic mining branch in the semantic mining unit in sequence.

[0051] In the embodiment of the present application, each of the multiple target component video frames can be loaded into the second semantic mining branch in the semantic mining unit in sequence. In this way, subsequent processing can be performed in the second semantic mining branch, where the second semantic mining branch is different from the first semantic mining branch to achieve semantic mining of data in different modalities.

[0052] Step S124b: Perform convolution processing on the currently loaded target component video frame to obtain a component convolution feature.

[0053] In the embodiment of the present application, convolution processing can be performed on the currently loaded target component video frame to obtain a component convolution feature. Exemplarily, there is a convolution network layer in the second semantic mining branch. In this way, through this convolution network layer, convolution processing can be performed on the target component video frame to obtain the corresponding component convolution feature.

[0054] Step S124c: Fuse the local component semantic feature corresponding to the previously loaded target component video frame into the component convolution feature to form the local component semantic feature corresponding to the currently loaded target component video frame.

[0055] In an embodiment of the present application, after obtaining the component convolution features, the local component semantic features corresponding to the previously loaded target component video frame can be fused into the component convolution features to form the local component semantic features corresponding to the currently loaded target component video frame. Among them, the local component semantic features corresponding to the previously loaded target component video frame of the first loaded target component video frame are the semantic features carried by the second semantic mining branch, and this semantic feature is formed during the training process of the second semantic mining branch (initially, it can be a randomly generated vector). Based on this, since the semantic information of the previous target component video frame has a certain auxiliary effect on the semantic information of the subsequent target component video frame, the local component semantic features corresponding to the previously loaded target component video frame can be fused into the component convolution features to form the local component semantic features corresponding to the currently loaded target component video frame, so that the semantics of the local component semantic features can be richer.

[0056] Step S124d: Determine the component semantic features based on the local component semantic features corresponding to each of the target component video frames.

[0057] In an embodiment of the present application, after obtaining the local component semantic features corresponding to each of the target component video frames, the component semantic features can be determined based on the local component semantic features corresponding to each of the target component video frames.

[0058] It can be understood that in the above step S124c, the specific manner of fusing the local component semantic features corresponding to the previously loaded target component video frame into the component convolution features is not limited. For example, in an alternative embodiment, in order to further improve the semantic representation ability of the component semantic features, the above step S124c can further include the following content: First, the power module component contour map corresponding to the currently loaded target component video frame can be obtained (existing contour extraction techniques can be used, and no specific limitations and descriptions are made here), and the power module component contour map is subjected to convolution processing to obtain the contour map convolution features; Secondly, the feature correlation parameters between the contour map convolution features and the target component convolution features and the local component semantic features corresponding to the previously loaded target component video frame can be calculated respectively. And, based on the corresponding feature correlation parameters respectively, the target component convolution features and the local component semantic features corresponding to the previously loaded target component video frame are updated to form the first component update feature and the second component update feature. Based on this, the capture of semantic information related to the power module component contour in the target component convolution features and the local component semantic features corresponding to the previously loaded target component video frame can be realized. The power module component is responsible for providing stable power supply for all other components of the device in the target gateway. It is the basis for the operation of the device. If the power module component fails, the entire device cannot work properly. Therefore, the power module is crucial in vibration and shock tests. Any interruption or instability of the power supply will cause the device to be unusable and may even cause system damage. Thus, the semantic representation ability of the first component update feature and the second component update feature in the connection stability test can be made better. In addition, the calculation of the feature correlation parameter can refer to calculating the dot product between two features, and the update of the feature can refer to performing a weighted sum calculation on the feature based on the feature correlation parameter. Then, the feature correlation parameter between the first component update feature and the second component update feature can be calculated, and based on this feature correlation parameter, the first component update feature is updated to form the local component semantic feature corresponding to the currently loaded target component video frame. Based on this, the semantic information in the second component update feature can be fused into the first component update feature to realize the capture of relevant semantic features and improve the semantic representation ability.

[0059] In the second aspect, it should be noted that for step S130, the specific manner of semantic enhancement of the component semantic features based on the vibration semantic features is not limited and can be selected according to actual needs.

[0060] For example, in an alternative embodiment, in order to avoid the problem of overfitting while fully integrating the vibration semantic features into the component semantic features, the above step S130 may further include step S131, step S132, and step S133, and the specific content is as follows.

[0061] Step S131, load the local vibration semantic feature in the vibration semantic feature and the local component semantic feature corresponding to the local vibration semantic feature in the component semantic feature into the semantic enhancement unit.

[0062] In the embodiments of the present application, the local vibration semantic features in the vibration semantic features and the local component semantic features corresponding to the local vibration semantic features in the component semantic features can be loaded into the semantic enhancement unit for subsequent fusion processing.

[0063] Step S132: Perform forward mining on the local component semantic features, and apply noise features during the forward mining process to form component forward mining features.

[0064] In the embodiments of the present application, the local component semantic features can be subjected to forward mining, and noise features can be applied during the forward mining process to form component forward mining features. Among them, through forward mining, high-level and abstract features in the local component semantic features can be captured. In addition, by applying noise features, the problem of overfitting can be avoided. And the noise feature can be a randomly generated vector and can follow a normal distribution.

[0065] Step S133: Perform backward reduction on the component forward mining features, and fuse the local vibration semantic features during the backward reduction process to form enhanced component semantic features.

[0066] In the embodiments of the present application, after the component forward mining features are formed, the component forward mining features can be subjected to backward reduction, and the local vibration semantic features can be fused during the backward reduction process to form enhanced component semantic features. Among them, both forward mining and backward reduction include multiple stages, and during the forward mining process (such as implemented through cascaded convolutional network layers), as the stages progress, the size of the mined features gradually decreases, and during the backward mining process, as the stages progress, the size of the reduced features gradually increases.

[0067] It can be understood that in the above step S133, the specific manner of forming the enhanced component semantic features is not limited. For example, in an alternative embodiment, in order to be able to fully fuse the local vibration semantic features and to a certain extent reduce the computational amount and improve the efficiency, the above step S133 can further include the following content (combined with Figure 4 ) First, for the first backward reduction stage, perform multiple feature reductions on the component forward mining features and the local vibration semantic features respectively to form corresponding multiple first reduction features and multiple second reduction features. Also, calculate the feature correlation parameters between each first reduction feature and the corresponding second reduction feature respectively to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. And, based on this target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the updated component semantic feature of the current stage. Among them, the feature reduction includes transposed convolution and / or interpolation, so that the feature size can gradually increase. In addition, since the sizes of the multiple first reduction features are different and the sizes of the multiple second reduction features are also different, therefore, the sizes of the corresponding multiple feature correlation parameters are also different. Thus, the feature correlation parameters with small sizes can be interpolated so that the sizes of the processed feature correlation parameters are the same. Then, the mean value can be calculated to obtain the corresponding target correlation parameter. Or, the weighted mean value can also be calculated. The closer the corresponding feature reduction is to the front, the smaller the weight coefficient, and the closer the corresponding feature reduction is to the back, the larger the weight coefficient; Second, for each subsequent backward reduction stage starting from the second one, perform multiple feature reductions on the updated component semantic feature of the previous stage and the second reduction feature corresponding to the last feature reduction of the previous stage respectively to form corresponding multiple first reduction features and multiple second reduction features. Also, calculate the feature correlation parameters between each first reduction feature and the corresponding second reduction feature respectively to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. And, based on this target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the updated component semantic feature of the current stage; Then, determine the enhanced component semantic feature based at least on the updated component semantic feature of the last stage; Exemplarily, the updated component semantic feature of the last stage can be determined as the enhanced component semantic feature, or the updated component semantic features of each stage can be concatenated to form the corresponding enhanced component semantic feature.

[0068] Combined with Figure 5 , the embodiment of the present application further provides a gateway quality determination device applicable to the above electronic device. Among them, the gateway quality determination device includes a test data acquisition module, a data semantic mining module, a feature semantic enhancement module, and a component quality analysis module.

[0069] The test data acquisition module is used to acquire target vibration data for vibration testing of internal components of a target gateway and a target component video formed by monitoring the process of vibration testing of the internal components. In an embodiment of the present application, the test data acquisition module can be used to execute Figure 2 the steps S110 shown. For the related content of the test data acquisition module, reference can be made to the description of step S110 above.

[0070] The data semantic mining module is used to, through semantic mining units in a target quality determination network, perform semantic mining on the target vibration data and the target component video respectively to form vibration semantic features and component semantic features. The target quality determination network further includes a semantic enhancement unit and a quality determination unit. In an embodiment of the present application, the data semantic mining module can be used to execute Figure 2 the steps S120 shown. For the related content of the data semantic mining module, reference can be made to the description of step S120 above.

[0071] The feature semantic enhancement module is used to, through the semantic enhancement unit, perform semantic enhancement on the component semantic features based on the vibration semantic features to form enhanced component semantic features. In an embodiment of the present application, the feature semantic enhancement module can be used to execute Figure 2 the steps S130 shown. For the related content of the feature semantic enhancement module, reference can be made to the description of step S130 above.

[0072] The component quality analysis module is used to, through the quality determination unit, determine target quality data corresponding to the target gateway based on the enhanced component semantic features, where the target quality data is used to characterize the connection stability of the internal components of the target gateway. In an embodiment of the present application, the component quality analysis module can be used to execute Figure 2 the steps S110 shown. For the related content of the component quality analysis module, reference can be made to the description of step S110 above.

[0073] In an embodiment of the present application, corresponding to the above gateway quality determination method applied to the electronic device, a computer-readable storage medium is further provided. A computer program is stored in the computer-readable storage medium, and when the computer program runs, it executes each step of the gateway quality determination method.

[0074] Among them, each step executed when the foregoing computer program runs will not be elaborated one by one here, and reference can be made to the foregoing explanation of the gateway quality determination method.

[0075] In summary, the gateway quality determination method, apparatus, device, and medium provided in this application obtain target vibration data obtained by performing a vibration test on internal components of a target gateway and a target component video formed by monitoring the process of performing a vibration test on the internal components. Secondly, through a semantic mining unit in a target quality determination network, semantic mining is respectively performed on the target vibration data and the target component video to form a vibration semantic feature and a component semantic feature. Then, through a semantic enhancement unit, the component semantic feature is semantically enhanced based on the vibration semantic feature to form an enhanced component semantic feature. Finally, through a quality determination unit, based on the enhanced component semantic feature, target quality data is determined. Based on the above, on the one hand, potential semantic information in the target component video is mined based on a neural network, so that the obtained component semantic feature has a high semantic representation ability. On the other hand, based on the vibration semantic feature corresponding to the target vibration data, the component semantic feature is enhanced, so that the semantic representation accuracy of the formed enhanced component semantic feature is further improved. Therefore, the reliability of the target quality data determined based on the enhanced component semantic feature can be improved, thereby improving the relatively low reliability problem of gateway quality determination existing in the prior art.

[0076] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of apparatuses, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0077] In addition, each functional module in each embodiment of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0078] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs. It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or device including the said element.

[0079] The foregoing are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A method for determining gateway quality, characterized in that Including: Obtaining target vibration data for vibration testing of internal components of a target gateway and a target component video formed by monitoring the process of vibration testing of the internal components; Determining a semantic mining unit in a network through target quality, and respectively performing semantic mining on the target vibration data and the target component video to form vibration semantic features and component semantic features, wherein the target quality determination network further includes a semantic enhancement unit and a quality determination unit; Through the semantic enhancement unit, based on the vibration semantic features, performing semantic enhancement on the component semantic features to form enhanced component semantic features; Through the quality determination unit, based on the enhanced component semantic features, determining target quality data corresponding to the target gateway, wherein the target quality data is used to characterize the connection stability of the internal components of the target gateway.

2. The gateway quality determination method according to claim 1, wherein The step of respectively performing semantic mining on the target vibration data and the target component video through the semantic mining unit in the target quality determination network to form vibration semantic features and component semantic features includes: According to the target duration carried by the semantic mining unit in the target quality determination network, performing segmentation processing on the target vibration data to form a plurality of local vibration data, wherein the target vibration data belongs to time series data and at least includes vibration frequency, vibration amplitude, vibration type, and vibration direction, and the duration corresponding to each local vibration data is equal to the target duration; Based on the target duration, performing frame extraction processing on the target component video to form a plurality of target component video frames, wherein the time interval between the timestamps corresponding to every two adjacent target component video frames is equal to the target duration, and there is a one-to-one correspondence between the plurality of target component video frames and the plurality of local vibration data; Through the first semantic mining branch in the semantic mining unit, performing semantic mining on the plurality of local vibration data to form vibration semantic features, wherein the vibration semantic features include a plurality of local vibration semantic features corresponding one-to-one to the plurality of local vibration data; Through the second semantic mining branch in the semantic mining unit, performing semantic mining on the plurality of target component video frames to form component semantic features, wherein the component semantic features include a plurality of local component semantic features corresponding one-to-one to the plurality of target component video frames.

3. The gateway quality determination method according to claim 2, wherein The step of performing semantic mining on the plurality of local vibration data through the first semantic mining branch in the semantic mining unit to form vibration semantic features includes: For each local vibration data in the plurality of local vibration data, loading the local vibration data into the first semantic mining branch in the semantic mining unit; Performing word embedding processing on the parameter sequences of each dimension in the local vibration data respectively to form word embedding features corresponding to each parameter of each dimension, wherein the feature size of each word embedding feature is 1*n; Concatenate the word embedding features corresponding to each parameter in the parameter sequence belonging to the same dimension to form a concatenated embedding feature, where the size of the concatenated embedding feature is 1*a, a is equal to m*n, and m is equal to the number of parameters in the parameter sequence of the same dimension; Combine the concatenated embedding features corresponding to each dimension in the local vibration data to form a combined embedding feature, where the size of the combined embedding feature is b*a, and b is equal to the number of dimensions corresponding to the local vibration data; Perform self-attention processing on the combined embedding feature to form the local vibration semantic feature corresponding to the local vibration data, and determine the vibration semantic feature based on the local vibration semantic feature corresponding to each local vibration data.

4. The gateway quality determination method according to claim 2, wherein The step of forming the component semantic feature by performing semantic mining on the multiple target component video frames through the second semantic mining branch in the semantic mining unit includes: Sequentially load each target component video frame in the multiple target component video frames into the second semantic mining branch in the semantic mining unit; Perform convolution processing on the currently loaded target component video frame to obtain a component convolution feature; Fuse the local component semantic feature corresponding to the previously loaded target component video frame into the component convolution feature to form the local component semantic feature corresponding to the currently loaded target component video frame, where the local component semantic feature corresponding to the previously loaded target component video frame of the first loaded target component video frame is the semantic feature carried by the second semantic mining branch, and this semantic feature is formed during the training process of the second semantic mining branch; Determine the component semantic feature based on the local component semantic feature corresponding to each target component video frame.

5. The gateway quality determination method according to claim 4, wherein The step of fusing the local component semantic feature corresponding to the previously loaded target component video frame into the component convolution feature to form the local component semantic feature corresponding to the currently loaded target component video frame includes: Obtain the power module component contour map corresponding to the currently loaded target component video frame, and perform convolution processing on the power module component contour map to obtain a contour map convolution feature; Calculate the feature correlation parameters between the contour map convolution feature and the component convolution feature and the local component semantic feature corresponding to the previously loaded target component video frame respectively, and update the component convolution feature and the local component semantic feature corresponding to the previously loaded target component video frame respectively based on the corresponding feature correlation parameters to form a first component update feature and a second component update feature; Calculate the feature correlation parameter between the first component update feature and the second component update feature, and update the first component update feature based on this feature correlation parameter to form the local component semantic feature corresponding to the currently loaded target component video frame.

6. The gateway quality determination method according to claim 1, wherein The step of forming the enhanced component semantic feature by performing semantic enhancement on the component semantic feature based on the vibration semantic feature through the semantic enhancement unit includes: Load the local vibration semantic feature in the vibration semantic feature and the local component semantic feature corresponding to the local vibration semantic feature in the component semantic feature into the semantic enhancement unit; Perform forward mining on the local component semantic feature, and apply noise features during the forward mining process to form a component forward mining feature; Perform backward reduction on the component forward mining feature, and fuse the local vibration semantic feature during the backward reduction process to form an enhanced component semantic feature. Here, both forward mining and backward reduction include multiple stages. During the forward mining process, as the stage progresses, the size of the mined feature gradually decreases. During the backward mining process, as the stage progresses, the size of the reduced feature gradually increases.

7. The gateway quality determination method according to claim 6, wherein The step of performing backward reduction on the component forward mining feature and fusing the local vibration semantic feature during the backward reduction process to form an enhanced component semantic feature includes: For the first backward reduction stage, perform multiple feature reductions on the component forward mining feature and the local vibration semantic feature respectively to form corresponding multiple first reduction features and multiple second reduction features. And calculate the feature correlation parameters between each first reduction feature and the corresponding second reduction feature respectively to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. And based on the target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the updated component semantic feature of the current stage. Here, feature reduction includes transposed convolution and / or interpolation; For each subsequent backward reduction stage starting from the second stage, perform multiple feature reductions on the updated component semantic feature of the previous stage and the second reduction feature corresponding to the last feature reduction of the previous stage respectively to form corresponding multiple first reduction features and multiple second reduction features. And calculate the feature correlation parameters between each first reduction feature and the corresponding second reduction feature respectively to form multiple feature correlation parameters, and fuse the multiple feature correlation parameters to form a target correlation parameter. And based on the target correlation parameter, update the first reduction feature corresponding to the last feature reduction to form the updated component semantic feature of the current stage; Determine the enhanced component semantic feature based at least on the updated component semantic feature of the last stage.

8. A gateway quality determination device, characterized in that, Includes: A test data acquisition module for acquiring target vibration data obtained by performing vibration tests on the internal components of a target gateway and a target component video formed by monitoring the process of performing vibration tests on the internal components; A data semantic mining module for determining a semantic mining unit in a network through target quality, and performing semantic mining on the target vibration data and the target component video respectively to form a vibration semantic feature and a component semantic feature. Here, the target quality determination network further includes a semantic enhancement unit and a quality determination unit; A feature semantics enhancement module, configured to, through the semantics enhancement unit, perform semantics enhancement on the component semantics features based on the vibration semantics features to form enhanced component semantics features; A component quality analysis module, configured to, through the quality determination unit, determine the target quality data corresponding to the target gateway based on the enhanced component semantics features, where the target quality data is used to characterize the connection stability of the internal components of the target gateway.

9. An electronic device, characterized in that, Comprising: A memory, configured to store a computer program; A processor connected to the memory, configured to execute the computer program stored in the memory to implement the gateway quality determination method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when the computer program runs, it executes the gateway quality determination method according to any one of claims 1-7.

Images