Anomaly detection method and device, electronic equipment, storage medium and program product

Abstract

This disclosure relates to an anomaly detection method, apparatus, electronic device, storage medium, and program product. The anomaly detection method includes: acquiring the Mth frame and Nth frame of a recorded video, wherein the recorded video includes video recorded for the electronic device to be detected in a set display mode, where N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N; determining the category of the Mth frame and the category of the Nth frame based on the correspondence between frames and categories, wherein different categories correspond to different display states; determining the frame difference between the Nth frame and the Mth frame based on the Nth frame, the category of the Nth frame, the Mth frame, and the category of the Mth frame; and determining the display state of the electronic device based on the relationship between the frame difference and frame difference requirements. Through the embodiments of this disclosure, the accuracy of anomaly detection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to anomaly detection methods, devices, electronic devices, storage media, and program products. Background Technology

[0002] In related technologies, it is necessary for testers to identify whether there are abnormalities in the screen display of electronic devices. When only one frame in a long video of an electronic device has a display abnormality, it is difficult for testers to find it, resulting in inaccurate detection results of the display status of the electronic device's screen. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an anomaly detection method, apparatus, electronic device, storage medium, and program product.

[0004] According to a first aspect of the present disclosure, an anomaly detection method is provided, comprising: acquiring an M-th frame and an N-th frame of a recorded video, wherein the recorded video includes a video recorded for an electronic device to be detected in a set display mode, wherein N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N; determining the category of the M-th frame and the category of the N-th frame based on the correspondence between frames and categories, wherein different categories correspond to different display states; determining the frame difference between the N-th frame and the M-th frame based on the N-th frame, the category of the N-th frame, the M-th frame, and the category of the M-th frame; and determining the display state of the electronic device based on the relationship between the frame difference and the frame difference requirement.

[0005] In one implementation, determining the image difference between the Nth frame and the Mth frame includes: determining the sequence order between the Nth frame target image and the Mth frame target image based on the order in which the Mth frame and the Nth frame were acquired, and using this sequence as a first order; constructing input data based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order; and determining the image difference between the Nth frame and the Mth frame based on the input data and the first target model.

[0006] In one implementation, constructing input data based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order includes: generating a first data sequence according to the first order based on the category corresponding to the Nth frame and the category corresponding to the Mth frame; transposing the first data sequence to obtain a second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other; and using the first data sequence and the second data sequence as the input data.

[0007] In one implementation, determining the category of the Mth frame and the Nth frame based on the correspondence between the image and the category includes: using a second target model to extract features from the Mth frame to obtain first image features, and / or extracting features from the Nth frame to obtain second image features; performing feature enhancement on the first image features to obtain enhanced first image features, and / or performing feature enhancement on the second image features to obtain enhanced second image features; classifying the images based on the enhanced first image features and an attention mechanism, and / or the second image features and an attention mechanism, to obtain classification scores for each category corresponding to the Mth frame and / or for each category corresponding to the Nth frame; wherein the classification score represents the probability of the category corresponding to the image; and determining the category corresponding to the Mth frame and / or the category corresponding to the Nth frame based on the classification score.

[0008] In one implementation, M includes N+P and NQ, wherein P is an integer greater than or equal to 1, and Q is an integer greater than or equal to 1.

[0009] According to a second aspect of the present disclosure, an anomaly detection device is provided, comprising: an acquisition unit, configured to acquire an M-th frame and an N-th frame of a recorded video, wherein the recorded video includes a video recorded for an electronic device to be detected in a set display mode, wherein N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N; and a processing unit, configured to determine the category of the M-th frame and the category of the N-th frame based on the correspondence between frames and categories, determine the frame difference between the N-th frame and the M-th frame based on the N-th frame, the category of the N-th frame, the M-th frame, and the category of the M-th frame, and to determine the display state of the electronic device based on the relationship between the frame difference and the frame difference requirement, wherein different categories correspond to different display states.

[0010] In one embodiment, the processing unit determines the image difference between the Nth frame and the Mth frame in the following manner: based on the order in which the Mth frame and the Nth frame were acquired, the sequence order between the Nth frame target image and the Mth frame target image is determined and used as a first order; input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order; and the image difference between the Nth frame and the Mth frame is determined based on the input data and the first target model.

[0011] In one embodiment, the processing unit constructs input data based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order as follows: a first data sequence is generated according to the first order based on the category corresponding to the Nth frame and the category corresponding to the Mth frame; the first data sequence is transposed to obtain a second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other; the first data sequence and the second data sequence are used as the input data.

[0012] In one implementation, the processing unit determines the category of the Mth frame and the category of the Nth frame based on the correspondence between the frame and the category as follows: It calls a second target model to extract features from the Mth frame to obtain first image features, and / or extracts features from the Nth frame to obtain second image features; it enhances the first image features to obtain enhanced first image features, and / or enhances the second image features to obtain enhanced second image features; based on the enhanced first image features and an attention mechanism, and / or the second image features and an attention mechanism, it classifies the frames to obtain classification scores for each category corresponding to the Mth frame, and / or classification scores for each category corresponding to the Nth frame; wherein the classification score represents the probability of the frame corresponding to a category; and it determines the category corresponding to the Mth frame and / or the category corresponding to the Nth frame based on the classification score.

[0013] In one implementation, M includes N+P and NQ, wherein P is an integer greater than or equal to 1, and Q is an integer greater than or equal to 1.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the anomaly detection method described in the first aspect or any embodiment of the first aspect.

[0015] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the anomaly detection method described in the first aspect or any embodiment of the first aspect.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, implements the anomaly detection method described in the first aspect or any embodiment of the first aspect.

[0017] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: Since there is a correspondence between images and categories, and different categories correspond to different display states, the display state (e.g., normal display state or abnormal display state) of the electronic device to be detected in the recorded video under a set display mode can be initially determined by acquiring the Mth and Nth frames of the recorded video. Furthermore, by further judging the images and their corresponding classification results, it can be determined whether there is a deviation in the results of the previous judgment process, thereby improving the accuracy of the detection results.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] Figure 1 This is a schematic diagram of an anomaly detection method according to an exemplary embodiment.

[0021] Figure 2 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment.

[0022] Figure 3A This is a flowchart illustrating a method for determining the corresponding category of a screen according to an exemplary embodiment.

[0023] Figure 3B This is a schematic diagram of an architecture for determining a second target model, according to an exemplary embodiment.

[0024] Figure 4 This is a flowchart illustrating a method for determining the result of the image difference between the Nth frame and the Mth frame according to an exemplary embodiment.

[0025] Figure 5This is a flowchart illustrating a method for constructing input data according to an exemplary embodiment.

[0026] Figure 6 This is a block diagram of an anomaly detection device according to an exemplary embodiment. Figure 1 .

[0027] Figure 7 This is a block diagram of an anomaly detection device according to an exemplary embodiment. Figure 2 .

[0028] Figure 8 This is block diagram three illustrating an anomaly monitoring device according to an exemplary embodiment. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0030] With the development of artificial intelligence (AI) technology, various AI technologies and models are widely used in a variety of scenarios. Among these technologies, users will use corresponding computer vision techniques to detect acquired images and perform corresponding processing based on the detection results, such as image classification.

[0031] In some scenarios, such as inspection scenarios, image classification technology is often used for anomaly detection. For example, in the scenario of electronic device inspection, image classification technology is used to detect whether the display screen of the electronic device is in a normal or abnormal state.

[0032] In this scenario, the user or detection device controls the electronic device under test to play a specified video (e.g., a video used to detect the screen display status, hereinafter referred to as the specified video). During the playback of the specified video, the user or detection device records the video using a corresponding image acquisition device. Based on the acquired video (e.g., the video recorded by the electronic device under test while playing the specified video, hereinafter referred to as the recorded video), the user or detection device determines whether the display screen of the electronic device under test (hereinafter referred to as the display screen of the electronic device under test) can display the specified video correctly. If, based on the recorded video, it is determined that the display screen of the electronic device under test can display the specified video correctly, the display status of the screen is considered normal. If, based on the recorded video, it is determined that the display screen of the electronic device under test cannot display the specified video correctly (e.g., there is color distortion or missing image frames), the display status of the screen is considered abnormal.

[0033] In this scenario, CV-related AI algorithms or AI models are typically used to detect the display status of the screen. For example, a visual model based on a Convolutional Neural Network (CNN) architecture can detect each frame of a recorded video (e.g., detect each frame of the screen of the electronic device to be detected in the recorded video) and classify them one by one to achieve the detection of the display status of the screen of the electronic device to be detected.

[0034] However, in some detection scenarios, the display abnormalities on the screen of the electronic device under test during the playback of a specified video may be minor and difficult for the AI ​​model to detect. For example, the display abnormality may only occur in one or a few frames during the playback of the specified video. Or, the display abnormality may be very minor (e.g., a slight color difference in the displayed image).

[0035] In this case, detecting the display status of the electronic device's screen based on the above method will increase the difficulty for the AI ​​model to recognize screen features, leading to inaccurate AI model output results and thus distorted detection results.

[0036] Based on this, the present disclosure proposes an anomaly detection method, with application scenarios such as... Figure 1 As shown, Figure 1This is a schematic diagram illustrating an anomaly detection method according to an exemplary embodiment. The method involves acquiring images from the electronic device detection process (e.g., images from a recorded video after frame-by-frame processing). A model is invoked to perform an initial detection based on the acquired images, obtaining the initial detection result for each frame (e.g., the result matrix in the diagram). A second detection is then performed on the acquired multiple frames and their corresponding detection results, yielding the second detection result. Based on this second detection result, the display state of the screen is determined (e.g., "√" in the diagram corresponds to a normal display state, and "×" corresponds to an abnormal display state). On one hand, by using multiple rounds of detection, the initial detection result is detected, improving the accuracy of the detection. On the other hand, since the second detection process is based on multiple frames, it is possible to determine the correlation between multiple frames, thereby identifying whether there are images in multiple frames that differ from other frames. This improves the ability to identify minor display anomalies (e.g., those with low frequency of occurrence and / or minor characteristic phenomena), thus enhancing the accuracy of anomaly detection.

[0037] It should be noted that the anomaly detection method provided in this disclosure can be applied to devices. Devices may include, for example, terminals or servers. Terminals include, but are not limited to, at least one of the following: mobile phone, wearable device, IoT device, car with communication capabilities, smart car, tablet computer, computer with wireless transceiver capabilities, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home. Servers may include, but are not limited to, independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0038] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0039] The various data involved in the embodiments of this application (such as recorded video footage, various models, etc.) can be stored in the blockchain so that the data demander can obtain them, and the immutability mechanism of the blockchain can be used to ensure the trustworthiness of the data.

[0040] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0041] In the embodiments of this disclosure, terms such as “in response to…”, “in response to determining…”, “in the case of…”, “when…”, “if…”, “if…”, etc., can be used interchangeably.

[0042] In the embodiments disclosed herein, terms such as “greater than”, “greater than or equal to”, “not less than”, “more than”, “more than or equal to”, “not less than”, “higher than”, “higher than or equal to”, “not lower than”, and “above” can be substituted for each other, and terms such as “less than”, “less than or equal to”, “not greater than”, “less than”, “less than or equal to”, “not more than”, “lower than”, “lower than or equal to”, “not higher than”, and “below” can be substituted for each other.

[0043] For ease of understanding, some technical terms involved in the embodiments of this disclosure will be explained by way of example below:

[0044] Cross-Stage Partial Networks with Feature Fusion (C2f) are network modules used for feature extraction. They process input data by dividing it into two branches: one branch directly passes the data to the output, while the other branch is processed through multiple bottleneck modules. This branching design understandably increases the network's nonlinearity and representational capabilities, thereby enhancing its ability to model complex data.

[0045] Bottleneck module: A widely used neural network structure in deep learning, primarily for feature extraction and enhancement. Its core idea is to extract features through a series of convolutional operations while controlling the number of channels to achieve compression and expansion, thereby improving the model's expressive power while keeping the number of model parameters and computational cost relatively low. In this embodiment, the second target model can be, for example, a model with both C2f and Bottleneck modules.

[0046] Spatial Pyramid Pooling Fast (SPPF): This strategy is used in convolutional neural networks to process input images of different sizes. It is particularly useful in tasks requiring multi-scale features, such as object detection and image segmentation. By using SPPF, the model can better adapt to targets of different scales, improving detection accuracy and robustness.

[0047] Temporal Convolutional Network (TCN) is a deep learning model used to process sequential data, specifically designed for time series prediction problems. TCN is a variant of Convolutional Neural Network (CNN) that captures temporal dependencies in time series data by applying convolutions along the temporal dimension. In embodiments of this disclosure, the first target model may be, for example, a model based on a TCN architecture, which, in capturing temporal dependencies in time series data, can be understood as capturing the differences between frames N and M (the smaller the dependency, the larger the difference).

[0048] Figure 2 This is a flowchart illustrating an anomaly detection method according to an exemplary embodiment, such as... Figure 2 As shown, the anomaly detection method is used in a device (e.g., a terminal) and includes the following steps.

[0049] In step S11, the Mth frame and the Nth frame of the recorded video are acquired.

[0050] In step S12, based on the correspondence between the images and categories, the category of the Mth frame and the category of the Nth frame are determined respectively.

[0051] In step S13, the differences between the Nth frame and the Mth frame are determined based on the Nth frame, the category of the Nth frame, the Mth frame, and the category of the Mth frame.

[0052] In step S14, the display state of the electronic device is determined based on the relationship between the screen difference and the screen difference requirement.

[0053] Where N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, M is different from N, and different categories correspond to different display states.

[0054] In this embodiment, since there is a correspondence between images and categories, and different categories correspond to different display states, the display state (e.g., normal display state or abnormal display state) of the electronic device to be detected in the recorded video under a set display mode can be initially determined by acquiring the Mth and Nth frames of the recorded video. Furthermore, by further judging the images and their corresponding classification results, it can be determined whether there is a deviation between the results of the previous judgment process and the actual situation, thereby improving the accuracy of the detection results. Moreover, since the judgment is based on multiple frames (e.g., the Mth and Nth frames) and the corresponding display states of these frames, the correlation between the specified image and other images can be determined by combining the correlations between multiple frames during the judgment process. This improves the identification of differences between the specified image and other images, enhances the ability to identify minor anomalies (e.g., those occurring at a low frequency and / or with a relatively mild phenomenon characterization), and thus improves the accuracy of the detection results.

[0055] In some embodiments, the terms "image," "image frame," "picture," "picture frame," and "frame" can be used interchangeably.

[0056] It should be noted that the anomaly detection method proposed in this embodiment can be applied to scenarios beyond the detection of the display status of electronic devices, and can also be extended to other scenarios for image detection and classification.

[0057] It should be noted that the term "frame difference" is used to characterize the correlation between the M-th frame and the N-th frame. The less correlation there is between the M-th and N-th frames, the greater the difference between them. Therefore, the requirement for frame difference can be understood as a requirement for the correlation between the M-th and N-th frames (e.g., a degree requirement or a threshold requirement).

[0058] It should be noted that, for Figure 2 The electronic devices mentioned in the related embodiments can be understood as electronic devices to be detected, such as electronic devices capable of displaying images.

[0059] For ease of understanding, the following will use an electronic device (such as a smartphone) as an example to illustrate the relevant embodiments of this disclosure.

[0060] In some embodiments, setting a display mode can be understood as a mode used to detect whether the display state of an electronic device is abnormal.

[0061] For example, an electronic device may play a specified video when a certain display mode is set. This specified video can be understood as a video used to detect whether there are any abnormalities in the display status of the electronic device. In some scenarios, these videos used to detect the display status of the electronic device may also be referred to as test cases.

[0062] For recorded video, this could include video recorded for the electronic device under test in a set display mode. In some scenarios, the video can be acquired based on something different from the electronic device under test (e.g., acquired based on an image acquisition device different from the electronic device under test, or acquired based on a device with storage capabilities, etc.).

[0063] In some scenarios, if the recorded video is acquired using an image acquisition device that is different from the electronic device being detected, it can be acquired in real time using an image acquisition device that is different from the electronic device being detected, or it can be pre-recorded using an image acquisition device that is different from the electronic device being detected.

[0064] In other scenarios, if the recorded video is obtained from a device with storage capabilities, the recorded video can be retrieved from the storage area of ​​the device with storage capabilities (such as a terminal or server).

[0065] In some embodiments, acquiring the images displayed in the recorded video (e.g., the Mth frame and the Nth frame) can be implemented based on the scenario described above. It should be noted that, as mentioned in the embodiments of this disclosure, the acquired Mth frame and Nth frame can be understood, for example, as acquiring multiple frames displayed by the electronic device under test in a set display mode within the recorded video.

[0066] For example, in some embodiments, M can be multiple values, such as N+P and / or NQ, where P is an integer greater than or equal to 1, Q is an integer greater than or equal to 1, and P and Q may or may not be equal. That is, the acquired M-th frame and N-th frame can be understood as the following cases A1) to C1):

[0067] A1) Obtain the NQth frame and the Nth frame.

[0068] B1) Obtain the Nth frame and the N+Pth frame.

[0069] C1) acquires the NQ-th frame, the N-th frame, and the N+P-th frame.

[0070] It should be noted that if N is 1 (that is, the Nth frame is the first frame), then in some scenarios it can only correspond to the case of B1).

[0071] For ease of understanding, the following will use case C1) as an example to illustrate the relevant embodiments of this disclosure. It should be noted that the relevant embodiments of this disclosure are also applicable to cases A1) and B1) (the Nth frame is not the first frame).

[0072] In case C1), based on the correspondence between frames and categories, the categories of the Mth frame and the Nth frame are determined separately. For example, this can be understood as: based on the correspondence between frames and categories, the categories of the NQth frame, the Nth frame, and the N+Qth frame are determined separately. For ease of understanding, in the following related embodiments, M can be understood as "N+P" and "NQ", which will not be elaborated further.

[0073] To determine the category of the Mth frame and the Nth frame based on the correspondence between the images and categories, for example, it can be achieved based on a relevant classification model.

[0074] For example, based on the trained second target model, the Mth frame and the Nth frame are classified respectively to obtain the category corresponding to the Mth frame and the category corresponding to the Nth frame.

[0075] To facilitate understanding, the following will be explained through... Figure 3A An exemplary description is provided of the process for determining the category corresponding to the image. Figure 3A This is a flowchart illustrating a method for determining the corresponding category of a screen according to an exemplary embodiment. For example... Figure 3A As shown, the method for determining the category of an image includes the following steps.

[0076] In step S21, the second target model is called to extract features from the Mth frame to obtain the first image features, and / or to extract features from the Nth frame to obtain the second image features.

[0077] In step S22, feature enhancement is performed on the first image features to obtain enhanced first image features, and / or feature enhancement is performed on the second image features to obtain enhanced second image features.

[0078] In step S23, the images are classified based on the enhanced first image features and the attention mechanism, and / or the second image features and the attention mechanism, to obtain the classification scores for each category of the Mth frame and / or the classification scores for each category of the Nth frame.

[0079] In step S24, the category corresponding to the Mth frame and / or the category corresponding to the Nth frame are determined based on the classification score.

[0080] Among them, the classification score represents the probability of the category corresponding to the image.

[0081] In this embodiment of the disclosure, since the first image feature is associated with the Mth frame and the second image feature is associated with the Nth frame, the first image feature and / or the second image feature are respectively enhanced by calling the second target model, and classification is performed based on the enhanced features to improve the accuracy of the model classification results (e.g., the category corresponding to the image).

[0082] It should be noted that, for example, calling the second target model to extract features from the Mth frame and / or the Nth frame can be understood as the second target being able to call the second target model to extract features from the Mth frame and the second target model to extract features from the Nth frame, which can be processed in parallel or sequentially. It is understood that, in this embodiment of the disclosure, other "and / or" related behaviors can also be understood as behaviors that can be processed in parallel or sequentially.

[0083] The second objective model can be understood, for example, as a model that can be used for image classification. For ease of understanding, the following will illustrate this... Figure 3B The second objective model is described with relevant examples.

[0084] Figure 3B This is a schematic diagram illustrating an architecture for determining a second target model according to an exemplary embodiment. Figure 3B The second target model architecture shown may include, for example, multiple convolutional layers (such as convolutional layer 1, convolutional layer 2, etc. shown in the figure), a feature enhancement module, an optimization module, an attention module, and other modules or network layers. It should be noted that... Figure 3B This paper only provides illustrative representations of some modules or network layers of the second target model; other potentially involved modules are not included. Figure 3B These are drawn out (e.g., fully connected layers), but are still considered as modules or network layers that the second target model may have.

[0085] The convolutional layers in the second target model architecture can be used for feature extraction. For example, feature extraction can be performed on the Mth frame to obtain the feature map corresponding to the Mth frame, and / or feature extraction can be performed on the Nth frame to obtain the feature map corresponding to the Nth frame. Figure 3B In this model, multiple convolutional layers (such as convolutional layer 1 and convolutional layer 2 shown in the figure) can be set to enable the model to learn feature representations at different levels. This allows the model to recognize more features in images with a lot of detail, thereby improving the accuracy of the model's output.

[0086] In some scenarios, convolutional layers can, for example, split the feature maps obtained during the feature extraction process into multiple sub-feature maps, such as multiple sub-feature maps corresponding to the Mth frame and / or multiple sub-feature maps corresponding to the Nth frame. A portion of these sub-feature maps is then input into the target module.

[0087] For the target module, the acquired features (e.g., sub-feature maps input to convolutional layers) can be enhanced to enable the model to recognize more refined features in the M-th frame and / or the N-th frame. In some scenarios, the target module may include modules such as the C2f module and the Bottleneck module.

[0088] The optimization module is used to pool the acquired features. In some scenarios, the optimization module may be a pooling module that incorporates the SPPF strategy.

[0089] The attention module could be, for example, a module with an attention mechanism that enhances the ability to capture channel features. The attention module is used to increase the attention paid to features of different channels by the second target model, so that the network can utilize the input features more effectively. For example, the example of the attention module increasing the second model's attention to features of different channels can be implemented as follows (A2).

[0090] A2) Assume the input of the attention module is X, the height of the feature map corresponding to X is H', the width of the feature map is W', and the number of channels of the feature map is C', for example, denoted as X∈H′×W′×C′. Performing a convolution transformation on X yields the convolution-transformed feature U, for example, denoted as U∈H×W×C. Performing global pooling (squeeze) on each channel of feature U (e.g., global average pooling, or global max pooling, etc.), compressing (or aggregating) the height (e.g., H) and width (e.g., W) of each channel, yields feature Z, for example, denoted as Z∈1×1×C. After obtaining feature Z, performing an excitation operation based on feature Z yields a weight vector. Assuming the weight vector is denoted as s, the weight vector can be determined based on the following mathematical expression (1).

[0091] s = sigmoid(W2ReLU(W1Z)) (1)

[0092] For the mathematical expression (1), W1 and W2 represent the weights corresponding to the fully connected layer, respectively.

[0093] After determining the weight vector s, a scaling operation can be performed, multiplying the feature U and the weight vector s to obtain a new feature map. Since each feature channel usually represents a feature learned by the network, an independent weight vector is assigned to each channel through multiplication. This allows the attention mechanism to adjust the weights of each channel, which is more precise than adjusting the weights of the entire feature map. Assuming the new feature map is denoted as x, the new feature map can be determined based on the following mathematical expression (2).

[0094] x=U·s (2)

[0095] After obtaining the new feature x, the input M-th frame and / or N-th frame can be classified based on the corresponding classification module to obtain the corresponding classification results, which can also be referred to as the category corresponding to the M-th frame and / or the category corresponding to the N-th frame.

[0096] For the classification result, assuming the classification result is denoted as r, r can be represented by the following mathematical expression (3).

[0097] r = [rate1, rate2, rate3] (3)

[0098] Here, rate1 represents the probability (e.g., probability) of the classification result corresponding to the first category, rate2 represents the probability of the classification result corresponding to the second category, and rate3 represents the probability of the classification result corresponding to the third category. It can be understood that if there are N preset classification categories, the result of r can be denoted as rateN, where r, rate1, rate2, or one or more of rate3 can be understood as classification scores.

[0099] Understandably, different classification results can correspond to different display states. For example, if rate1 corresponds to the category "Normal", then the display state of rate1 is normal. If rate2 corresponds to the category "Irregular Color", then the display state of rate2 is abnormal. If rate3 corresponds to the category "Distorted Screen", then the display state of rate3 is abnormal.

[0100] Therefore, through the above examples, the classification results corresponding to the Mth frame and / or the Nth frame can be obtained based on the second target model, thereby determining the display state that has been initially judged by the second target model.

[0101] Understandably, the classification result for the Mth frame determined by the second objective model is based on the image features corresponding to the Mth frame (and the same applies to the classification result for the Nth frame). Therefore, if there are even slight abnormalities (such as frequency of occurrence) in the display state, it will increase the difficulty for the second objective model to identify the image features corresponding to the abnormal phenomena, potentially leading to incorrect output results. For example, incorrect output results could be achieved by misrepresenting the actual normal display state of the electronic device as the result corresponding to the abnormal display state, or by misrepresenting the abnormal display state.

[0102] It is understandable that the acquired multi-frame images are usually continuous, and there may be differences between the frames (e.g., semantic differences, image information differences, content differences, etc.). Therefore, the differences between the frames can be identified. For example, the differences between different display categories, the differences between different display categories, or the differences between different display categories, or the differences between different display categories.

[0103] Therefore, by obtaining the category corresponding to the Mth frame and the category corresponding to the Nth frame output by the second target model, the image differences between the Mth and Nth frames can be determined. Furthermore, based on these differences, it can be determined whether the Nth frame corresponds to an abnormal display state. For example, the relationship between the image differences between the Mth and Nth frames and the image difference requirements can be used to determine whether the Nth frame corresponds to an abnormal display state.

[0104] As for the result of determining the difference between the Nth frame and the Mth frame, for example, it can be achieved through... Figure 4 This is achieved in the following way. Figure 4 This is a flowchart illustrating a method for determining the image difference between the Nth frame and the Mth frame, according to an exemplary embodiment. Figure 4 As shown, the method for determining the image difference between the Nth frame and the Mth frame includes the following steps.

[0105] In step S41, based on the order in which the Mth frame and the Nth frame are acquired, the sequence order between the Nth frame target image and the Mth frame target image is determined and used as the first order.

[0106] In step S42, input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order.

[0107] In step S43, based on the input data and the first target model, the image difference between the Nth frame and the Mth frame is determined.

[0108] In this embodiment, input data is constructed based on the sequential order of the Mth and Nth frames, giving the constructed input data a sequence characteristic. Therefore, during the processing of the input data by the first target model, the first target model can systematically determine the differences between multiple frames based on the constructed sequence characteristics, making the determined frame differences more accurate.

[0109] In some embodiments, the order in which the Mth frame and the Nth frame are acquired can be determined by acquiring the frame number corresponding to the Mth frame and the frame number of the Nth frame. Thus, the corresponding first order can be determined based on the frame numbers corresponding to multiple frames.

[0110] It is understood that, in some embodiments, the order of the Mth frame and the Nth frame can be determined based on the timestamps corresponding to the Mth frame and the Nth frame, thereby determining the first order.

[0111] For the first target model, since the input data is sequential, in order for the first target model to determine the image differences between the category corresponding to the Nth frame and the category corresponding to the Mth frame based on the input data, the first target model needs to be a model capable of capturing the data dependencies in the data sequence, such as a model related to the TCN architecture.

[0112] Understandably, in some scenarios, to accurately identify the differences between multiple frames, it's necessary to consider not only the differences between a specified frame and the previous frame, but also the differences between the specified frame and the subsequent frame. For example, if the specified frame is the Nth frame, and we need to consider the differences between the NPth and Nth frames, we must simultaneously consider the differences between the N+Pth frames (where P can be an integer greater than or equal to 1).

[0113] Therefore, it is necessary to construct input data that can be bidirectionally processed by the first target model, so that the first target model can capture the bidirectional dependencies of data in the data sequence.

[0114] For input data that enables bidirectional processing of the target model, for example, it can be processed as follows: Figure 5 The implementation method shown is adopted. Figure 5 This is a flowchart illustrating a method for constructing input data according to an exemplary embodiment. Figure 5As shown, the method for constructing input data includes the following steps.

[0115] In step S51, a first data sequence is generated based on the classification results corresponding to the Nth frame and the classification results corresponding to the Mth frame, in the first order.

[0116] In step S52, the first data sequence is transposed to obtain the second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other;

[0117] In step S53, the first data sequence and the second data sequence are used as the input data.

[0118] In this embodiment of the disclosure, since the first data sequence is the inverse of the second data sequence, the second data sequence includes the same data elements as the first data sequence, and the data order of the second data sequence is the reverse of the data order of the first data sequence. Therefore, by inputting both the first and second data sequences as input data into the first target model, the model can determine the image differences between consecutive frames at a specified frame based on the input data.

[0119] For ease of understanding, the first data sequence and the second data sequence will be described in detail below.

[0120] Assume the category corresponding to the Nth frame is r. now The category corresponding to the (N-1)th frame is r. pre The category corresponding to the N+1th frame is r. nex The first data sequence x can be represented by the following mathematical expression (4):

[0121] x = [r pre r now r nex ,] (4)

[0122] The first data sequence is transposed to obtain the second data sequence x. -1 For example, it can be represented by the following mathematical expression (5):

[0123] x -1 =[r now ,,r nex ,,r pr e (5)

[0124] It should be noted that the classification categories in the above mathematical expressions (4) and (5) can be described as described in the relevant content of the above mathematical expression (3), and will not be repeated here.

[0125] Having obtained the first data sequence and the second data sequence, the first data sequence and the second data sequence can be used as input data and input into the first target model.

[0126] It is understandable that, since the first data sequence and the second data sequence are inverse sequences of each other, the input data constructed based on the first and second data sequences exhibits a bidirectional arrangement characteristic in the dimension of data arrangement order. Consequently, the first target model can obtain the bidirectional dependencies captured in the data sequence based on the input data. Therefore, the first target model can extract more feature information in both directions of the data sequence based on the input data, enabling it to output prediction results more accurately. Furthermore, the bidirectional dependencies in the input data allow the first target model to fully consider the integrity of information between the preceding context (e.g., between frame N-1 and frame N) and / or the following context (e.g., between frame N and frame N+1) during the processing stage, resulting in more accurate output from the first target model.

[0127] The first model is determined based on the input data. The following section will determine the first target model based on the input data.

[0128] For example, assuming the obtained input data is the data sequence corresponding to mathematical expression (4) and mathematical expression (5), continuing from the above... Figure 4 In related embodiments, the first data sequence x and the second data sequence x -1 The input to the first target model can be represented, for example, based on the following mathematical expressions (6) and (7):

[0129] tx=TCN(x) (6)

[0130] tx -1 =TCN(x -1 (7)

[0131] Mathematical expression (6) represents the processing scheme for inputting the first data sequence into the first target model. Mathematical expression (7) represents the processing scheme for inputting the second data sequence into the first target model. t represents the relevant parameters in the time dimension. In some scenarios, t can be determined based on the frame numbers and / or timestamps of the acquired multi-frame images.

[0132] Therefore, based on mathematical expressions (6) and (7), the result x of the first target model output can be obtained, for example, through the following mathematical expression (8):

[0133] x = tx + tx -1 (8)

[0134] For example, mathematical expression (8) can be understood as adding the output of the first target model corresponding to mathematical expression (6) to the output of the first target model corresponding to mathematical expression (7) to obtain the final output x of the first target model. This output x represents the image difference between the Nth frame, the (N-1)th frame, and the (N+1)th frame. In some scenarios, image differences can also be referred to as correlations. The greater the correlation, the smaller the image difference; the smaller the correlation, the greater the image difference.

[0135] In some scenarios, based on the obtained output result x, the image difference corresponding to the Nth frame can be evaluated using the sigmoid function to determine whether the current output result meets the image difference requirements. For example, if the output value of the sigmoid function meets the threshold requirement, the image difference corresponding to the Nth frame is considered to meet the image difference requirements.

[0136] In some embodiments, to meet the image difference requirement, which can be understood as the image difference between the Nth frame and the frames before and after it, the second target model correctly classifies the Nth frame.

[0137] In some embodiments, if the image difference requirement is not met, for example, the image difference between the Nth frame and the frames before and after it is not met, the classification result of the second target model for the Nth frame is incorrect. Incorrect classification results could include, for example, classifying a frame that is actually in an abnormal display state as a frame that is in a normal display state, or classifying a frame that is actually in a normal display state as a frame that is in an abnormal display state, or classifying a frame that is actually in the first category as a frame that is in the second category, etc.

[0138] Therefore, the classification results of the second target model can be corrected by the first target model, reducing the probability of incorrect results in the classification results of the first target model, thereby improving the accuracy of detecting abnormal display status of electronic devices.

[0139] Based on the same concept, embodiments of this disclosure also provide an anomaly detection device.

[0140] It is understood that the anomaly detection device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0141] Figure 6 This is a block diagram of an anomaly detection device according to an exemplary embodiment. Figure 1 . Reference Figure 6 The device 100 includes an acquisition unit 101 and a processing unit 102.

[0142] The acquisition unit 101 is used to acquire the Mth frame and the Nth frame of the recorded video, wherein the recorded video includes video recorded for the electronic device to be detected in a set display mode, where N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N.

[0143] The processing unit 102 is used to determine the category of the Mth frame and the category of the Nth frame based on the correspondence between the frame and the category, determine the frame difference between the Nth frame and the Mth frame based on the Nth frame, the category of the Nth frame, the Mth frame, and the category of the Mth frame, and determine the display state of the electronic device based on the relationship between the frame difference and the frame difference requirements, wherein different categories correspond to different display states.

[0144] In some embodiments, the processing unit 102 determines the image difference between the Nth frame and the Mth frame in the following manner: based on the order in which the Mth frame and the Nth frame are acquired, the sequence order between the Nth frame target image and the Mth frame target image is determined and used as the first order; input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order; and the image difference between the Nth frame and the Mth frame is determined based on the input data and the first target model.

[0145] In some embodiments, the processing unit 102 constructs input data based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and a first order as follows: a first data sequence is generated according to the first order based on the category corresponding to the Nth frame and the category corresponding to the Mth frame; the first data sequence is transposed to obtain a second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other; and the first data sequence and the second data sequence are used as input data.

[0146] In some embodiments, the processing unit 102 determines the category of the Mth frame and the category of the Nth frame based on the correspondence between the frame and the category in the following manner: It calls a second target model to extract features from the Mth frame to obtain first image features, and / or extracts features from the Nth frame to obtain second image features; it enhances the first image features to obtain enhanced first image features, and / or enhances the second image features to obtain enhanced second image features; it classifies the frames based on the enhanced first image features and an attention mechanism, and / or the second image features and an attention mechanism, to obtain classification scores for each category corresponding to the Mth frame, and / or classification scores for each category corresponding to the Nth frame; wherein the classification score represents the probability of the category corresponding to the frame; and it determines the category corresponding to the Mth frame and / or the category corresponding to the Nth frame based on the classification score.

[0147] In some embodiments, M includes N+P and NQ, where P is an integer greater than or equal to 1 and Q is an integer greater than or equal to 1.

[0148] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0149] Figure 7 This is a block diagram of an anomaly detection device according to an exemplary embodiment. Figure 2 The device 200 can be provided as a terminal for performing anomaly detection methods. For example, the device 200 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0150] Reference Figure 7 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0151] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0152] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0153] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0154] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0155] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0156] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0157] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0158] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0159] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0160] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0161] Figure 8 This is block diagram three illustrating an anomaly monitoring device according to an exemplary embodiment. For example, device 300 may be provided as a server. (Refer to...) Figure 8 The device 300 includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in the memory 332 may include one or more modules, each corresponding to a set of instructions.

[0162] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0163] Based on the same concept, this disclosure also provides a computer program product, wherein the computer program product includes a computer program. This computer program can be executed by a processor, and when executed by the processor, it can perform any of the anomaly detection methods described above.

[0164] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0165] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0166] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0167] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0168] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0169] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0170] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An anomaly detection method characterized by, include: Acquire the Mth frame and Nth frame of the recorded video, wherein the recorded video includes a video recorded for the electronic device to be detected in a set display mode, where N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N; Based on the correspondence between images and categories, the categories of the Mth frame and the Nth frame are determined respectively, wherein different categories correspond to different display states; Based on the Nth frame, the category of the Nth frame, the Mth frame, and the category of the Mth frame, determine the frame differences between the Nth frame and the Mth frame; The display state of the electronic device is determined based on the relationship between the image differences and the image difference requirements.

2. The method according to claim 1, characterized in that, Determining the image difference between the Nth frame and the Mth frame includes: Based on the order in which the Mth frame and the Nth frame are acquired, the sequence order between the Nth frame target image and the Mth frame target image is determined and used as the first order; Input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order. Based on the input data and the first target model, the differences between the Nth frame and the Mth frame are determined.

3. The method according to claim 2, characterized in that, The input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order, including: Based on the category corresponding to the Nth frame and the category corresponding to the Mth frame, a first data sequence is generated in the first order; The first data sequence is transposed to obtain the second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other; The first data sequence and the second data sequence are used as the input data.

4. The method according to claim 1, characterized in that, The process of determining the category of the Mth frame and the Nth frame based on the correspondence between frames and categories includes: The second target model is invoked to extract features from the Mth frame to obtain the first image features, and / or the Nth frame is extracted to obtain the second image features; The first image features are enhanced to obtain enhanced first image features, and / or the second image features are enhanced to obtain enhanced second image features; Based on the enhanced first image features and attention mechanism, and / or the second image features and attention mechanism, the images are classified to obtain the classification scores for each category of the Mth frame and / or the classification scores for each category of the Nth frame. The classification score represents the probability of the category corresponding to the image; The category corresponding to the Mth frame and / or the category corresponding to the Nth frame are determined based on the classification score.

5. The method according to any one of claims 1 to 4, characterized in that, The M includes: N+P and NQ, where P is an integer greater than or equal to 1, and Q is an integer greater than or equal to 1.

6. An anomaly detection device, characterized in that, include: The acquisition unit is used to acquire the Mth frame and the Nth frame of the recorded video, wherein the recorded video includes a video recorded for the electronic device to be detected in a set display mode, where N is an integer greater than or equal to 1, M is an integer greater than or equal to 1, and M is different from N; The processing unit is configured to determine the category of the Mth frame and the category of the Nth frame based on the correspondence between the frame and the category, determine the frame difference between the Nth frame and the Mth frame based on the Nth frame, the category of the Nth frame, the Mth frame, and the category of the Mth frame, and determine the display state of the electronic device based on the relationship between the frame difference and the frame difference requirement, wherein different categories correspond to different display states.

7. The apparatus according to claim 6, characterized in that, The processing unit determines the image difference between the Nth frame and the Mth frame in the following manner: Based on the order in which the Mth frame and the Nth frame are acquired, the sequence order between the Nth frame target image and the Mth frame target image is determined and used as the first order; Input data is constructed based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order. Based on the input data and the first target model, the differences between the Nth frame and the Mth frame are determined.

8. The apparatus according to claim 7, characterized in that, The processing unit constructs input data based on the category corresponding to the Nth frame, the category corresponding to the Mth frame, and the first order in the following manner: Based on the category corresponding to the Nth frame and the category corresponding to the Mth frame, a first data sequence is generated in the first order; The first data sequence is transposed to obtain the second data sequence, wherein the first data sequence and the second data sequence are inverse sequences of each other; The first data sequence and the second data sequence are used as the input data.

9. The apparatus according to claim 6, characterized in that, The processing unit determines the category of the Mth frame and the category of the Nth frame based on the correspondence between the frame and the category, respectively: The second target model is invoked to extract features from the Mth frame to obtain the first image features, and / or the Nth frame is extracted to obtain the second image features; The first image features are enhanced to obtain enhanced first image features, and / or the second image features are enhanced to obtain enhanced second image features; Based on the enhanced first image features and attention mechanism, and / or the second image features and attention mechanism, the images are classified to obtain the classification scores for each category of the Mth frame and / or the classification scores for each category of the Nth frame. The classification score represents the probability of the category corresponding to the image; The category corresponding to the Mth frame and / or the category corresponding to the Nth frame are determined based on the classification score.

10. The apparatus according to any one of claims 6 to 9, characterized in that, The M includes: N+P and NQ, where P is an integer greater than or equal to 1, and Q is an integer greater than or equal to 1.

11. An electronic device, characterized in that, include: processor: Memory used to store processor-executable instructions; The processor is configured to execute the anomaly detection method according to any one of claims 1 to 5.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the processor to perform the anomaly detection method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the anomaly detection method as described in any one of claims 1 to 5.

Images