Computer vision based electromagnetic susceptibility test method

Abstract

The present application provides a computer vision-based electromagnetic sensitivity testing method, which is suitable for electronic devices with screens. The method includes a training phase and a testing phase. In the training phase, the electronic device receives test data and displays a first image on the screen, a camera captures the first image to generate a template video, and a processor generates a plurality of template images based on the template video. The testing phase includes: after generating the plurality of template images, the antenna sends an interference signal to the electronic device, the electronic device receives test data and displays a second image on the screen, a camera captures the second image to generate a test video, a processor generates a test image based on the test video, and calculates the difference ratio between the test image and each template image, and sends a warning signal when the difference ratio is greater than a threshold value. The present application can automatically detect any abnormalities in the image displayed on the screen.

Description

TECHNICAL FIELD

[0001] The present application relates to electromagnetic susceptibility testing and image comparison, and in particular to a computer vision-based electromagnetic susceptibility testing method. BACKGROUND

[0002] Electromagnetic susceptibility (EMS) refers to the tolerance or immunity to electromagnetic interference (EMI) or radio frequency interference (RFI). If an electronic device does not generate EMI or RFI to other devices and has good EMS, the electronic device has good electromagnetic compatibility (EMC). In other words, the goal of EMC is to ensure that different devices operate correctly in a common electromagnetic environment.

[0003] Radiated susceptibility testing for EMS usually uses a high-power radio frequency or electromagnetic device and a radiating antenna to direct energy to the electronic device under test, such as a notebook computer. The electronic device under test is placed in an RF anechoic chamber, and a variety of test data are displayed on the screen one by one. While the antenna emits electromagnetic waves, a camera captures the screen of the electronic device under test, and a test personnel outside the RF anechoic chamber watches the captured test image to see if there are any abnormalities and records the electromagnetic wave frequency and setting parameters when the abnormalities occur. The so-called abnormalities include, for example, a completely black screen, flickering screen, distortion of lines in the test pattern, etc.

[0004] However, the labor-intensive test can easily cause eye fatigue, and the test personnel can miss some abnormal phenomena. SUMMARY

[0005] Therefore, the present application proposes a computer vision-based electromagnetic susceptibility testing method, which uses a computer vision-based EMS testing system to continuously receive test images captured by a camera inside an RF anechoic chamber, simulates the mechanism of a test personnel watching the images and making judgments, and then judges whether there are any abnormalities in the test images and automatically records various setting parameters when the abnormalities occur.

[0006] A computer vision-based electromagnetic sensitivity testing method according to an embodiment of the present application is suitable for an electronic device having a screen. The method includes a training phase, which includes: the electronic device receiving a test data and displaying a first picture corresponding to the test data on the screen; a camera capturing the first picture to generate a template video; and a processor generating a plurality of template images according to the template video; and a testing phase, which includes: after the plurality of template images are generated, an antenna sending an interference signal to the electronic device; the electronic device receiving the test data and displaying a second picture corresponding to the test data on the screen upon receiving the interference signal; the camera capturing the second picture to generate a test video; the processor generating a test image according to the test video; the processor calculating a difference ratio between the test image and each of the plurality of template images; and the processor sending an alert signal when the difference ratio is greater than a threshold value.

[0007] In summary, the present application provides a computer vision-based electromagnetic sensitivity testing method. In the training phase, the template images belonging to normal pictures are dynamically updated from the video captured by the camera. In the testing phase, the test image is obtained from the video captured by the camera and compared with the template images, so as to automatically detect any abnormality in the picture displayed on the screen. In addition, the threshold value for abnormality judgment can be dynamically adjusted according to the past abnormality detection records.

[0008] The above description and the following description of the embodiments are intended to demonstrate and explain the spirit and principle of the present application, and provide further explanation of the scope of the patent application of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a flowchart of a computer vision-based electromagnetic sensitivity testing method according to an embodiment of the present application;

[0010] Figure 2 is Figure 1 is a detailed flowchart of step S3 in

[0011] Figure 3 is Figure 2 is a detailed flowchart of step S33 in

[0012] Figure 4 is Figure 1 is a detailed flowchart of an embodiment of step T5 in

[0013] Figure 5 is Figure 1 is a detailed flowchart of another embodiment of step T5 in

[0014] SYMBOL DESCRIPTION

[0015] S: training phase

[0016] T: testing phase

[0017] Steps: S1-S3, S31-S32, T1-T7, T51-T54, T5', T51', T52' DETAILED DESCRIPTION

[0018] The detailed features and characteristics of the present application are described in detail in the embodiments below, which are sufficient for any person skilled in the relevant art to understand the technical content of the present application and implement it, and any person skilled in the relevant art can easily understand the related ideas and characteristics of the present application according to the content disclosed in the specification, the scope of the application and the drawings. The following examples further illustrate the ideas of the present application, but do not limit the scope of the present application in any way.

[0019] The present application proposes a computer vision-based electromagnetic susceptibility (EMS) test method, which is suitable for testing electronic devices (hereinafter referred to as electronic devices) with screens.

[0020] Figure 1 is a flowchart of a computer vision-based electromagnetic susceptibility test method according to an embodiment of the present application, which includes a training phase S and a testing phase T. The training phase S includes steps S1-S3, and the testing phase T includes steps T1-T6. The implementation details of each step are introduced in order below.

[0021] Step S1 is "the electronic device receives test data, and the screen displays a first picture". The first picture corresponds to the test data, and the types of test data include static test pictures, semi-static test pictures and dynamic test pictures. Figure 2 is a schematic diagram corresponding to the above three types of test data. The static test picture is a picture, and the color of each pixel in the picture always remains unchanged. The dynamic test picture is a video, and the color of each pixel in the picture changes over time. The semi-static test picture is equivalent to the integration of static test pictures and dynamic test pictures, in other words, all the pixels in the picture are divided into two parts, one part of the pixels always remains unchanged, and the color of the other part of the pixels changes over time.

[0022] Step S2 is "the camera shoots the first picture to generate a template video". The template video is a video obtained by shooting the screen.

[0023] Step S3 is "the processor generates at least a plurality of template images according to the template video", please refer to Figure 2which is a detailed flowchart of step S3. Step S31 is "setting a region of interest in the template video", step S32 is "performing a correction operation according to the region of interest", and step S33 is "performing an algorithm to generate a plurality of template images".

[0024] In step S31, the processor sets a region of interest (ROI) in the template video. In practice, the image captured by the camera can contain the environment around the screen, such as the desktop on which the electronic device is placed and the wall behind the screen, but only the part of the template image that belongs to the first picture is needed for subsequent processes, so step S31 needs to be performed to filter out the part that is not related to the first picture. Step S31 can be implemented in two ways: automatically and manually.

[0025] The automatic setting of ROI is to separate the part that belongs to the first picture from each frame of the template video according to the color (or brightness) of the environment where the electronic device is located and an adaptive thresholding algorithm. The adaptive thresholding algorithm can be, for example, OTSU, K-means, or the adaptiveThreshold function in OpenCV. In addition, if there are no other light-emitting components in the anechoic chamber except the screen, the color of the environment where the electronic device is located is black, so it is easy to separate the light-emitting area from the template video as the template image.

[0026] The manual setting of ROI is to use another screen to display the template video, and the user decides the range of the first picture in the screen.

[0027] In practice, the electronic device is placed on a rotating table, and the electronic device is uniformly affected by the interference signal during the test phase T by rotating the electronic device. The camera is usually set in a fixed position, so the shape and position of the screen in the template video captured by the camera will change with the rotation angle. Therefore, if the camera can rotate synchronously with the electronic device so that the lens always points to the screen, step S31 can be omitted in other embodiments. For example, the processor first determines whether the template video contains parts other than the first picture. If the result is "no", step S31 can be skipped, and if the result is "yes", step S31 is performed.

[0028] In step S32, the correction operation is, for example, a perspective transformation. Assuming that the template video is displayed on another screen, the picture seen by the user on the other screen can have visual differences compared to the first picture, due to the internal parameters of the camera itself and the optical parameters of the lens. For example, if the camera uses a fisheye lens, the edges of the picture displayed on the other screen can have a deformation or curvature phenomenon compared to the edges of the first picture, so that the deformed ROIs in the template video need to be returned to the shape of the first picture (typically a rectangle, but the present application is not limited thereto) by the correction operation.

[0029] In step S33, the processor can acquire a plurality of frames from the template video according to a specified sampling frequency, and then execute an algorithm according to the frames to determine the foreground property and the background property of each pixel in each frame. The pixels with the background property are the standard for subsequent comparison, and the pixels with the foreground property will be regarded as abnormal pixels in most cases.

[0030] In a first embodiment of step S33, the algorithm is a Gaussian mixture model or ViBE.

[0031] In a second embodiment of step S33, the details of the algorithm are as follows:

[0032] First, the processor executes a corresponding program according to the type of test data to generate the template image.

[0033] If the test data is a dynamic test picture, the processor acquires at least one frame of the template video as the template image. The present application does not limit the number of acquired frames, nor the acquisition rule of the frames. For example, if the template video contains 30 frames F1-F30, the processor can set all the frames F1-F30 as the template image, or can only select F1-F10 as the template image. The more template images are selected, the higher the comparison accuracy in the test phase T.

[0034] If the test data is a static test picture, the processor will execute a background initialization program according to the first frame of the template image, and execute a background update program according to the second frame. The second frame is one of the frames, and the time sequence of these frames is after the time sequence of the first frame. Please refer to Figure 3 which is the flowchart of step S33 when the test data is a static test picture.

[0035] The background initialization procedure is shown in step S331, "obtaining a plurality of reference pixels adjacent to a first center pixel". The first center pixel is each pixel in the first frame. Assuming the frame size is 3x3, the frame has 9 pixels arranged in a 3x3 grid. The first center pixel is in the center of the 3x3 grid, and N reference pixels can be selected from the remaining 8 pixels, N≤8. N represents the sampling depth of the reference pixels. Unless the position is at the edge of the frame, each first center pixel can basically select N reference pixels from the surrounding pixels as the standard for pixel comparison. The distance adjacent to the first center pixel is adjusted according to the sampling depth N. For example, when the sampling depth N is 20, the selection range of the reference pixels must be expanded to a 5x5 area, which is the range formed by extending two pixels in all directions from the first center pixel as the center.

[0036] The background update procedure is shown in steps S332, S333, and S334. Step S332 is "judging the similarity of the second center pixel to each reference pixel", step S333 is "determining the attribute of the second center pixel according to the similarity", and step S334 is "replacing the first center pixel or the reference pixel with the second center pixel according to the specified probability, attribute, and consecutive frame number".

[0037] In steps S332 and S333, the second center pixel is each pixel in the second frame. When the number of pixels similar to the second center pixel in the N reference pixels of the first frame is less than the threshold value m, the processor sets the foreground attribute of the second center pixel. Otherwise, when the number of pixels similar to the second center pixel in the N reference pixels of the first frame is not less than the threshold value, the processor sets the background attribute of the second center pixel. In one embodiment, the condition for judging whether two pixels are similar is that the difference in Euclidean distance between the two pixels is less than a default value. In another embodiment, the condition for judging whether two pixels are similar is that the difference in pixel intensity between the two pixels is less than a default value.

[0038] In step S334, when the second center pixel has a background attribute, the first center pixel or the reference pixel is replaced with the second center pixel according to the specified probability r.

[0039] In step S334, if the second center pixel at the same position in the consecutive C frames of the template video has the foreground attribute, the processor replaces the first center pixel in the first frame with the second center pixel. For example, during the training stage S, the user inadvertently moves the mouse cursor from the upper left corner of the screen to the lower right corner, and then stays at the lower right corner. In this case, the pixels that make up the cursor are set to have the foreground attribute at the beginning of the training stage S because the mouse cursor is not part of the test data, but these pixels are identified as part of the template image because they persist in the consecutive frames.

[0040] In an embodiment, the following parameters are used: the reference pixel sampling depth N = 5, the threshold m = 2, the specified probability r = 1 / 16, and the number of consecutive frames C = 15, based on the consideration of the instruction cycle and the accuracy.

[0041] If the test data is a semi-static test image, the test data can be divided into a dynamic part and a static part. The above-mentioned methods for dynamic test images and static test images are respectively adjusted to generate the template image for the dynamic part and the static part.

[0042] During the training stage S, the processor continuously updates the background model based on the new second frame of the template video and Figure 3 The background model is updated by the flow shown in FIG. 6. The background model is composed of a base template image and a reference pixel dictionary. The base template image is composed of a plurality of first center pixels (equivalent to the updated first frame). The reference pixel dictionary records a plurality of reference pixels corresponding to each pixel of the base template image. In another aspect, the background model can be regarded as being composed of a plurality of template images. Each template image is equivalent to a combination of a first center pixel and a plurality of reference pixels selected from each pixel position of the base template image.

[0043] After the training stage S is completed, the processor has generated one or more template images and recorded them in the storage device, so that the template images can be read as templates during the test stage T.

[0044] In the above-described embodiment, the processor directly uses the first frame of the template video as the initial template image, and then updates the template image based on the second frame of the template video. In another embodiment, the processor can first use the test data as the initial template image, and then update the template image based on the template image. In this way, the number of frames used during the training stage S can be reduced while maintaining the comparison accuracy, thereby reducing the calculation cost during the training.

[0045] The test stage T includes steps T1-T6, wherein step T1 is "the antenna sends an interference signal to the electronic device".

[0046] Step T2 is "the electronic device receives the test data, and the screen displays a second image", the second image corresponds to the test data. In steps T2 and S1, the types of test data used are the same.

[0047] Step T3 is "the camera captures the second image to generate test video", steps T2-T3 are similar to steps S1-S2, but the difference is that in the training phase T, the electronic device is affected by the interference signal, so abnormal phenomena may appear in the second image, such as a black screen, flickering, distortion of lines in the test pattern, etc. In the training phase S, the electronic device is not affected by the interference signal, so the template of the normal image can be induced.

[0048] Step T4 is "the processor generates a test image according to the test video", step T4 is similar to step S3, but the difference is that in step T4, the processor generates a test image according to each frame of the test video. In an embodiment, if the process of setting the ROI and correcting as shown in step S3 is included, the process as shown in step T4 is adapted to be applicable to step T4. In addition, if you want to speed up the execution speed of the test phase T, in step T4, the processor selects part of the frames from all the frames of the test video to generate multiple test images. Figure 2 Figure 2 Step T5 is "the processor calculates the difference ratio of the test image and each template image", according to the type of test data, the processor executes the corresponding program to calculate the difference ratio. When the test data is a dynamic test pattern, the way to calculate the difference ratio is as shown in step T5 of

[0049] When the test data is a static test pattern, the way to calculate the difference ratio is as shown in step T5' of Figure 4 Figure 5

[0050] When the test data is a dynamic test pattern, multiple template images are generated after the training phase S ends, and the processor compares the feature similarity of each template image and the input image in the neural network based on the perceptual similarity metric.

[0051] Please refer to Figure 4 Step T51 is "input the test image and each template image into the neural network model", the neural network has multiple layers of convolution operations, and the neural network model is, for example, AlexNet, which is used to calculate the perceptual difference of the two images. In step T51, the operation of resizing the test image to the size of the template image can also be included.

[0052] ​​​Step T52 is "calculating a plurality of difference values according to the feature map of the test image and the feature map of each template image". In each layer of convolution operation, the feature map of the test image and the feature map of each template image are obtained. In an embodiment, in the first three layers of convolution operation, the processor upsamples the feature map of the template image and the feature map of the test image, enlarges the two feature maps to the same size, and then calculates the difference value between the two feature maps. The calculation method is as follows: for two pixels at the same position L in the two feature maps, the processor calculates the Euclidean distance D between the two pixels as the difference value of the difference map at position L. The smaller the Euclidean distance D, the more similar the two pixels are; the larger the Euclidean distance D, the more different the two pixels are. In an embodiment, the Euclidean distance D between the two pixels is calculated according to the RGB values of the two pixels respectively, so the difference value is a three-dimensional data.

[0053] Step T53 is "generating a difference map according to the plurality of difference values and calculating a difference ratio". In the difference map, if the Euclidean distance D of the pixel at position L is greater than a default value, the processor marks the pixel at position L in the difference map as a difference pixel. The difference ratio is related to the number of all difference pixels in the difference map and the total number of pixels in the difference map, wherein the difference value of each difference pixel is greater than the default value. In an embodiment, the processor calculates the area of the block composed of all difference pixels, and the proportion of the area in the total area of the difference map is the difference ratio.

[0054] After the processes of steps T51-T5, each template image and the test image will generate a difference map and a difference ratio. In an embodiment, the processor can select the maximum difference ratio from the difference ratios as the standard for judging whether the test image passes the test, or can calculate the weighted average of the difference ratios as the standard for judging whether the test image passes the test, and the present application does not limit this.

[0055] When the test data is a static test image, the method for calculating the difference ratio is as shown in step T5' of Figure 5 , which includes step T51' and step T52'. In the test stage T, the background model has stopped updating.

[0056] Step T51' is "calculating the number of difference pixels according to the background model and the test image". In detail, for the pixel P T located at position L in the test image and the pixel P M corresponding to position L in the background model, the processor calculates the number of pixels similar to pixel P M in P T . The processor can calculate the number of difference pixels (difference number) according to the number of similar pixels (similar number) and the number of pixels P M .M The first center pixel and the corresponding plurality of reference pixels.

[0057] Step T52' is "determining the difference pixels according to the number of differences and generating a difference map and a difference ratio".

[0058] In an embodiment, for each pixel of the test image, if the number of differences of the pixel exceeds a tolerance value, the pixel is marked as a difference pixel in the difference map, otherwise the pixel is not marked as a difference pixel; and the difference ratio is the ratio of the difference pixels in the difference map to all pixels in the difference map.

[0059] Please refer to Figure 1 Step T6 is "judging whether the difference ratio is greater than a threshold value".

[0060] If the result of the judgment is "yes", go to step T7, if the result of the judgment is "no", return to step S1, and the processor selects another type of test data and returns to step S1 to perform the next round of EMS test.

[0061] In step S7, the processor issues an alarm signal. In addition, the processor records the type of anomaly, the frequency of the interference signal, the type of test data, the configuration information of the electronic device, and the currently used threshold value, etc. in the database of the storage device.

[0062] In an embodiment, a step of dynamically adjusting the threshold value is further included before step T6. In detail, the processor reads the configuration information of the electronic device, and searches for historical test records similar to the configuration information in the database. If a historical test record and a historical threshold value used therein are found, the processor adjusts the threshold value used in the next step T6 according to the historical threshold value.

[0063] In summary, the present application proposes a computer vision-based electromagnetic sensitivity test method. In the training phase, the template image belonging to the normal picture is dynamically updated from the video captured by the camera. In the test phase, the test image is obtained from the video captured by the camera and compared with the template image, so as to automatically detect any anomaly in the picture displayed on the screen. In addition, the present application can dynamically adjust the threshold value used for anomaly judgment according to the past anomaly detection records.

[0064] Although the present application is disclosed with the above-mentioned embodiments, it is not intended to limit the present application. Any modification and improvement made without departing from the spirit and scope of the present application shall fall within the patent protection scope of the present application. For the protection scope of the present application, please refer to the appended patent claim scope.

Claims

1. A computer vision-based electromagnetic sensitivity test method, characterized by, A method for an electronic device with a screen, the method comprising: a training phase comprising: the electronic device receiving a test data and displaying a first frame corresponding to the test data on the screen; a camera capturing the first frame to generate a template video; and a processor generating a plurality of template images based on at least the template video; and a testing phase comprising: after the plurality of template images are generated, an antenna emitting an interference signal to the electronic device; the electronic device receiving the test data and displaying a second frame corresponding to the test data on the screen upon receiving the interference signal; the camera capturing the second frame to generate a test video; the processor generating a test image based on the test video; the processor calculating a difference ratio of the test image and each of the plurality of template images; and the processor emitting an alert signal when the difference ratio is greater than a threshold value. the processor generating the plurality of template images based on at least the template video comprises: setting a region of interest in the template video; performing a correction operation based on the region of interest; and performing an algorithm to generate the plurality of template images.

2. The computer vision-based electromagnetic sensitivity test method of claim 1, wherein, the algorithm is Gaussian Mixture Model or ViBE.

3. The computer vision-based electromagnetic sensitivity test method of claim 1, wherein, the test data is a dynamic test image, and the algorithm comprises: the processor obtaining at least one frame of the template image as the plurality of template images.

4. The computer vision-based electromagnetic susceptibility test method of claim 1, wherein, the test data is a static test image, and the algorithm comprises: a background initialization procedure comprising: obtaining a plurality of reference pixels adjacent to a first center pixel, the first center pixel being each pixel of a first frame in the template image; and the background update procedure comprising: judging a similarity of a second center pixel and each of the plurality of reference pixels, wherein the second center pixel is each pixel of a second frame in the template image, and the second frame is subsequent to the first frame in time sequence; setting a foreground attribute for the second center pixel when a number of pixels in the plurality of reference pixels similar to the second center pixel is less than a threshold value; otherwise, setting a background attribute for the second center pixel; when the second center pixel has the background attribute, replacing the first center pixel or the reference pixel with the second center pixel based on a specified probability; and when the second center pixel has the foreground attribute, replacing the first center pixel with the second center pixel in a plurality of consecutive frames of the template video.

5. The computer vision-based electromagnetic susceptibility test method of claim 1, wherein, the test image is a dynamic test image, and the processor calculating the difference ratio of the test image and each of the plurality of template images comprises: inputting the test image and each of the plurality of template images into a neural network model, the neural network having a plurality of convolutional operations; calculating a feature map of the test image and a feature map of each of the plurality of template images in each of the plurality of convolutional operations; upsampling the feature map of the test image and the feature map of each of the plurality of template images in each of the plurality of convolutional operations, and Based on a pixel scale, a Euclidean distance is calculated as a difference value in a difference map for the feature map of the test image and the feature map of each of the plurality of template images. The difference ratio is associated with a number of difference pixels in the difference map and a total number of pixels in the difference map, wherein the difference value of each of the plurality of difference pixels is greater than a default value.

6. The computer vision-based electromagnetic susceptibility test method of claim 1, wherein, The test data is a static test image, and the processor calculates the difference ratio of the test image and each of the plurality of template images includes: calculating a difference number of each pixel according to a background model and the test image, wherein the background model is associated with the plurality of template images; and determining a difference pixel according to the difference number, and generating a difference map and the difference ratio.

7. The computer vision-based electromagnetic susceptibility test method of claim 1, wherein, The processor generates the plurality of template images according to at least the template video includes: The processor generates the plurality of template images according to the test data and the template video.

8. The computer vision-based electromagnetic susceptibility test method of claim 1, wherein, The test stage further includes: The processor obtains a configuration information of the electronic device; The processor searches a database to obtain a historical threshold in a historical test record, wherein a historical configuration information of the historical test record is similar to the configuration information; and The processor adjusts the threshold according to the historical threshold.

9. The computer vision based electromagnetic sensitivity test method of claim 1, wherein, Before the processor calculates the difference ratio of the test image and each of the plurality of template images, it further includes adjusting the size of the test image according to the size of each of the plurality of template images.

Images