A method, device, system and computer equipment for determining a light flicker frequency

Abstract

The application relates to the technical field of computer equipment, and discloses a light flicker frequency determination method, device and system and computer equipment, wherein the light flicker frequency determination method comprises the following steps: acquiring a plurality of first pictures of light in a preset time period; selecting a picture combination in the plurality of first pictures, wherein the picture combination at least comprises two characteristic pictures arranged according to shooting time, one of the two characteristic pictures is a light-on picture, and the other is a light-off picture; and determining the flicker frequency of the light by using the picture combination. The light flicker frequency determination method can solve the problem that the flicker frequency of a hard disk light needs to be detected manually by acquiring a plurality of first pictures of light in a preset time period, selecting a picture combination in the plurality of first pictures, and then determining the flicker frequency of the light by using the picture combination.

Description

Technical Field

[0001] This invention relates to the field of computer equipment technology, and specifically to a method, apparatus, system, and computer equipment for determining the flashing frequency of a light. Background Technology

[0002] Current general-purpose servers have high demand and a high degree of customization, so automated equipment is needed to test them. One of the goals of server hard drive quality testing is to check whether the flashing frequency of the hard drive light meets the requirements.

[0003] Currently, the flashing frequency of the hard drive light needs to be detected manually, which makes it difficult to make an accurate and objective judgment and requires a lot of manpower. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, system, computer equipment and storage medium for determining the flicker frequency of a light, so as to solve the problem of needing to manually detect the flicker frequency of a hard drive light.

[0005] In a first aspect, the present invention provides a method for determining the flicker frequency of a light, comprising the following steps: acquiring multiple first images of a light within a preset time period; selecting an image combination from the multiple first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of the two feature images being a lit image and the other being a dark image; and determining the flicker frequency of the light using the image combination.

[0006] The above-mentioned method for determining the flashing frequency of the light involves acquiring multiple first images of the light within a preset time period, selecting a combination of images from these first images, and then using this image combination to determine the flashing frequency of the light. This solves the problem of needing to manually detect the flashing frequency of the hard drive light.

[0007] In one optional implementation, acquiring multiple first images of the light within a preset time period includes: acquiring a second image of the light; determining whether the second image contains at least one flickering state, wherein the flickering state is the state of the light changing from bright to dim and then bright again or the state of the light changing from dim to bright and then dim again; when the second image contains at least one flickering state, determining the shooting time period of the second image; and acquiring multiple first images of the light within the shooting time period.

[0008] The method for determining the light flicker frequency is to eliminate cases where the hard drive light does not flicker by judging whether the second image contains at least one flicker state, thereby making the calculated light flicker frequency more accurate.

[0009] In one optional implementation, determining whether the second image contains at least one flickering state includes: inputting the second image into a trained second target detection model, and determining whether the second image contains at least one flickering state based on the output of the second target detection model.

[0010] This is because if there is light flickering during the long exposure shooting period, the resulting image is A; if there is no light flickering during the long exposure shooting period, the resulting image is B. Image A will be darker than image B. Therefore, by processing the images obtained from long exposure shooting using the second object detection model, it is possible to determine whether there is light flickering during the long exposure shooting period.

[0011] In one optional implementation, selecting an image combination from multiple first images includes: inputting each first image into a trained first object detection model, determining whether each first image is a lit image or a dark image based on the output of the first object detection model; and selecting an image combination from multiple first images.

[0012] This is because the brightness of images with lights on is different from that of images with lights off. Therefore, by using the first object detection model to process multiple first images, it is possible to identify whether each first image is an image with lights on or an image with lights off.

[0013] In one alternative implementation, determining the flashing frequency of a light using image combinations includes: determining a first time interval between two adjacent image combinations; and determining the flashing frequency of the light based on the first time interval.

[0014] This allows for a relatively accurate determination of the light's flicker frequency.

[0015] In one optional implementation, determining the first time interval between two adjacent image combinations includes: acquiring feature images from the two image combinations respectively; and determining the first time interval based on the feature images.

[0016] This allows for a more accurate determination of the first time interval, i.e., the period of the light flashing.

[0017] In one optional implementation, determining the first time interval based on the feature image includes: acquiring the first shooting time of the lit image in the preceding image combination of two adjacent image combinations; acquiring the second shooting time of the lit image in the following image combination of two adjacent image combinations; subtracting the first shooting time from the second shooting time to obtain the first time interval; or, acquiring the third shooting time of the dark image in the preceding image combination of two adjacent image combinations; acquiring the fourth shooting time of the dark image in the following image combination of two adjacent image combinations; subtracting the third shooting time from the fourth shooting time to obtain the first time interval.

[0018] This allows for a more accurate determination of the first time interval, i.e., the period of the light flashing.

[0019] In one optional implementation, determining the flashing frequency of the light based on the first time interval includes: acquiring all first time intervals within the time period; calculating the flashing frequency of the light using the remaining first time intervals within the time period and the least squares method; or, acquiring all first time intervals within the time period; calculating the average value of all first time intervals to obtain the flashing frequency of the light.

[0020] In one optional implementation, inputting the second image into a trained second object detection model and determining whether the second image contains at least one flickering state based on the output of the second object detection model includes: downsampling and feature extraction of the second image to obtain multiple second downsampling features of different sizes; upsampling, feature extraction, and fusion processing of the multiple second downsampling features to obtain multiple processed second downsampling features; and determining whether the second image contains at least one flickering state based on the processed multiple second downsampling features.

[0021] Alternatively, each first image can be input into a trained first object detection model. Determining whether each first image is a lit or dark image based on the output of the first object detection model includes: downsampling and feature extraction of each first image to obtain multiple first downsampling features of different sizes; upsampling, feature extraction, and fusion processing of the multiple first downsampling features to obtain multiple processed first downsampling features; and determining whether each first image is a lit or dark image based on the processed multiple first downsampling features.

[0022] This allows for accurate identification of whether the second image contains at least one flashing state or whether the first image is a bright or dark image.

[0023] In one optional implementation, downsampling and feature extraction of each first image to obtain multiple first downsampled features of different sizes includes: performing convolution and normalization on each first image to obtain a first feature; downsampling the first feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current first downsampled feature; downsampling the current first downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next first downsampled feature; using the next first downsampled feature as the current first downsampled feature, and returning to the steps of downsampling the current first downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next first downsampled feature, until the number of downsampling times reaches a predetermined number. The first threshold is set; or, the second image is downsampled and feature extracted to obtain multiple second downsampled features of different sizes, including: performing convolution and normalization on the second image to obtain a second feature; downsampling the second feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current second downsampled feature; downsampling the current second downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next second downsampled feature; using the next second downsampled feature as the current second downsampled feature, and returning to the steps of downsampling the current second downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next second downsampled feature, until the number of downsampling times reaches the preset second threshold.

[0024] This allows for the generation of output images in sizes of 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 from the headers of the first and second object detection models.

[0025] In one optional implementation, multiple first downsampled features are upsampled, feature extracted, and fused to obtain multiple processed first downsampled features; the smallest first downsampled feature is subjected to convolution and normalization, max pooling, and upsampling; the other first downsampled features besides the smallest first downsampled feature are subjected to convolution and normalization, and upsampling; and at least two processed first downsampled features are fused.

[0026] Alternatively, multiple second-downsampled features can be upsampled, feature extracted, and fused to obtain processed second-downsampled features. This involves: performing convolution and normalization, max pooling, and upsampling on the smallest second-downsampled feature; performing convolution and normalization, and upsampling on the other second-downsampled features (excluding the smallest one); and fusing at least two of the processed second-downsampled features. This removes computational bottlenecks, reduces GPU memory usage, and accelerates network inference.

[0027] In some optional implementations, before performing convolution and normalization, max pooling, and upsampling on the smallest first downsampled feature, the method further includes performing min pooling on the smallest first downsampled feature; or, before performing convolution and normalization, max pooling, and upsampling on the smallest second downsampled feature, the method further includes performing min pooling on the smallest second downsampled feature.

[0028] Therefore, it can be concluded that the first target detection model and the second target detection model have better recognition performance.

[0029] In some optional implementations, the training method for the first object detection model or the second object detection model includes: acquiring a training set; setting the size of the prior boxes in the K-means clustering method; clustering the training set using the K-means clustering method, selecting the intersection and union of the predicted boxes and the ground truth boxes in the K-means clustering method, and using the prior box with the largest intersection and union as the positive sample; training the model using a preset loss function set based on the positive samples to obtain the first object detection model or the second object detection model.

[0030] In some alternative implementations, the loss functions in the loss function set include the location regression loss function, the object confidence loss, and the classification loss function.

[0031] In some alternative implementations, the size of the prior box corresponds to that of the small object.

[0032] This is because the objects that the first and second object detection models need to detect are relatively small, accounting for less than 0.01 pixels in the image. Therefore, the size of the prior box needs to correspond to the small object, which can make the first and second object detection modules perform better in detecting small objects.

[0033] In one optional implementation, after determining the flashing frequency of the light using image combination, the method further includes: obtaining a preset standard flashing frequency; determining whether the flashing frequency of the light meets the preset requirements based on the standard flashing frequency; and issuing a prompt message when it does not meet the requirements.

[0034] This not only allows for accurate determination of the light's flashing frequency, but also enables timely alerts when the hard drive malfunctions, facilitating quick handling of abnormal situations by staff.

[0035] According to a second aspect, embodiments of the present invention also provide a device for determining the flicker frequency of a light, comprising an acquisition module, an image combination selection module, and a flicker frequency determination module. The acquisition module is used to acquire multiple first images of a light within a preset time period. The image combination selection module is used to select an image combination from the multiple first images, wherein the image combination includes at least two feature images arranged chronologically according to the shooting time, one of the two feature images being a lit image and the other being a dark image. The flicker frequency determination module is used to determine the flicker frequency of the light using the image combination.

[0036] According to a third aspect, embodiments of the present invention also provide a computer device, including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for determining the light flicker frequency described in the first aspect or any embodiment of the first aspect.

[0037] According to a fourth aspect, embodiments of the present invention also provide a system for determining the flicker frequency of a light, comprising: a long exposure camera, a high frame rate camera, and a computer device according to a third aspect, wherein the long exposure camera and the high frame rate camera are both communicatively connected to the computer device.

[0038] In one alternative implementation, the system for determining the light flicker frequency further includes a light source controller, which is communicatively connected to a computer device.

[0039] In one alternative implementation, the frame rate of the high frame rate camera is more than twice the flashing frequency of the light.

[0040] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the method for determining the light flashing frequency of the first aspect or any embodiment of the first aspect. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1This is a flowchart illustrating a method for determining the flicker frequency of a light according to an embodiment of the present invention.

[0043] Figure 2 This is a flowchart illustrating another method for determining the flashing frequency of a light according to an embodiment of the present invention;

[0044] Figure 3 This is a flowchart illustrating another method for determining the flashing frequency of a light according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the overall structure of the second target detection model according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of the ECA unit according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram showing the variation of the head structure according to an embodiment of the present invention;

[0048] Figure 7 This is a flowchart illustrating another method for determining the flashing frequency of a light according to an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the structure of a light flicker frequency determination device according to an embodiment of the present invention;

[0050] Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention;

[0051] Figure 10 This is a schematic diagram of a processing method for a light flicker frequency determination system according to an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] During server use, the flashing frequency of the hard drive light may be normal in some cases, but too high or too low in others. Therefore, it is necessary to test the flashing frequency of the hard drive light.

[0054] According to an embodiment of the present invention, a method for determining the flashing frequency of a light is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0055] This embodiment provides a method for determining the flashing frequency of a light, which can be used in computer devices. In this embodiment, the light refers to the hard drive backlight. Figure 1 This is a flowchart illustrating a method for determining the flicker frequency of a light according to an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps:

[0056] Step S101: Obtain multiple first images of the lights within a preset time period.

[0057] Specifically, the first image was taken using a high frame rate camera. High frame rate cameras can achieve high frame rates and are generally used to photograph fast-moving objects.

[0058] Multi-frame-rate camera image capture requires setting up to capture multiple images within 1 second. This embodiment uses a 120FPS camera, meaning the maximum number of images captured within 1 second is 120 (normally there will be some error, within three images). The multiple images are stored in a queue in chronological order. Each image is set as a class containing time, name, and sequence ID, corresponding to the relevant time, ensuring that all parameters in this class can be retrieved when each item is dequeued.

[0059] A camera's frame rate is the number of images it can capture per second; a light's flashing frequency is the number of times the light flashes per second. To stably capture the flashing frequency of a light, the camera's frame rate must be greater than the light's flashing frequency. For example, a high frame rate camera's frame rate is more than twice the light's flashing frequency.

[0060] Step S102: Select an image combination from multiple first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of which is a lit image and the other is a dark image.

[0061] The "Lights On" image is a picture taken with a high frame rate camera when the lights are on; the "Lights Off" image is a picture taken with a high frame rate camera when the lights are off.

[0062] Two feature images arranged chronologically by shooting time are two feature images taken at adjacent times. The image can be either bright (on light) or dark (off light). It's important to note that while both can be arranged sequentially, only one sorting method can be used in a single selection process: bright light before dark, or dark light before bright. It's not possible to select both combinations of images. Of course, different sorting methods can be used in different selection processes.

[0063] For example, the first images for a preset time period are: bright 1, bright 2, dark 3, dark 4, dark 5, dark 6, bright 7, bright 8, dark 9, dark 10, dark 11, dark 12, bright 13, bright 14, dark 15, dark 16, dark 17, dark 18, bright 19, bright 20...

[0064] When selecting images from multiple first images in the order of bright images first and dark images last, the resulting two feature images are (bright 2, dark 3), (bright 8, dark 9), (bright 14, dark 15)...

[0065] When selecting images from multiple first images according to the order of dark images first and bright images last, the resulting two feature images are (dark 6, bright 7), (dark 12, bright 13), (dark 18, bright 19)...

[0066] An image set can include only two feature images, or it can include other images besides the two feature images. For example, multiple first images for a preset time period are: Bright 1, Bright 2, Dark 3, Dark 4, Dark 5, Dark 6, Bright 7, Bright 8, Dark 9, Dark 10, Dark 11, Dark 12, Bright 13, Bright 14, Dark 15, Dark 16, Dark 17, Dark 18… Here, “Bright 1” indicates that the first image is a lit image, and “Dark 2” indicates that the second image is a dark image. In this example, Bright 2 and Dark 3 are two feature images, Dark 6 and Bright 7 are two feature images, Bright 8 and Dark 9 are two feature images, Dark 12 and Bright 13 are two feature images, and so on.

[0067] For Bright 2 and Dark 3, the combined image can be Bright 2 and Dark 3, or Bright 1, Bright 2, Dark 3 and Dark 4, or Bright 1, Bright 2, Dark 3, Dark 4 and Dark 5.

[0068] Step S103: Determine the flashing frequency of the light using image combination.

[0069] The method for determining the flashing frequency of a light provided in this embodiment obtains multiple first images of the light within a preset time period, selects a combination of images from the multiple first images, and then uses the image combination to determine the flashing frequency of the light. This solves the problem of needing to manually detect the flashing frequency of the hard drive light.

[0070] This embodiment provides a method for determining the flashing frequency of a light, which can be used in computer devices. In this embodiment, the light refers to the hard drive backlight. Figure 2 This is a flowchart illustrating another method for determining the flicker frequency of a light according to an embodiment of the present invention, as shown below. Figure 2 As shown, the process includes the following steps:

[0071] Step S201: Obtain the second image of the light.

[0072] Specifically, the second image is a picture taken using a long exposure camera or a picture taken using a regular camera with a long exposure.

[0073] Step S202: Determine whether the second image contains at least one flashing state, wherein the flashing state is the state in which the light changes from bright to dim and then bright again or the light changes from dim to bright and then dim again.

[0074] If the first images for a preset time period are: bright 1, bright 2, dark 3, dark 4, dark 5, dark 6, bright 7, bright 8, dark 9, dark 10, dark 11, dark 12, bright 13, bright 14, dark 15, dark 16, dark 17, dark 18..., then one flashing state corresponds to the above first images as "bright 1, bright 2, dark 3, dark 4, dark 5, dark 6, bright 7, bright 8", that is, from bright to dark and then bright again, or "dark 3, dark 4, dark 5, dark 6, bright 7, bright 8, dark 9, dark 10, dark 11, dark 12", that is, from dark to bright and then dark again.

[0075] To more accurately determine the light flicker frequency, step S202 can be modified to determine whether the second image contains at least two flicker states.

[0076] Step S203: When the second image contains at least one flashing state, determine the shooting time period of the second image.

[0077] Specifically, when using a long-exposure camera to photograph the hard drive light, not only can a second image be obtained, but also the time period in which the second image was taken.

[0078] Step S204: Obtain multiple first images of the lights within the shooting time period.

[0079] Step S205: Select an image combination from the multiple first images. The image combination must include at least two feature images arranged chronologically by shooting time, one of which is a lit image and the other is a dark image. For details, please refer to [link to details]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0080] Step S206: Determine the flashing frequency of the light using image combination.

[0081] The method for determining the light flicker frequency provided in this embodiment obtains a second image of the light and determines whether the second image contains at least one flicker state. It can obtain multiple first images of the light within the shooting time period of the second image only when the second image contains at least one flicker state. In other words, by determining whether the second image contains at least one flicker state, the case where the hard drive light does not flicker can be eliminated, thereby making the calculated light flicker frequency more accurate.

[0082] This embodiment provides a method for determining the flashing frequency of a light, which can be used in computer devices. In this embodiment, the light refers to the hard drive backlight. Figure 3 This is a flowchart illustrating another method for determining the flicker frequency of a light according to an embodiment of the present invention, as shown below. Figure 3 As shown, the process includes the following steps:

[0083] Step S301: Obtain the second image of the light.

[0084] Step S302: Input the second image into the trained second target detection model, and determine whether the second image contains at least one flickering state based on the output of the second target detection model.

[0085] This is because if there is light flickering during the long exposure shooting period, the resulting image is A; if there is no light flickering during the long exposure shooting period, the resulting image is B. Image A will be darker than image B. Therefore, by processing the images obtained from long exposure shooting using the second object detection model, it is possible to determine whether there is light flickering during the long exposure shooting period.

[0086] Similarly, the images obtained from shooting long exposures will differ depending on whether there are two light flashes during the shooting period or only one light flash. Therefore, by using the second object detection model to process the images obtained from long exposures, it is possible to determine whether there are at least two light flashes during the long exposure shooting period.

[0087] As an optional implementation, inputting the second image into a trained second target detection model and determining whether the second image contains at least one flickering state based on the output of the second target detection model includes: downsampling and feature extraction of the second image to obtain multiple second downsampling features of different sizes; upsampling, feature extraction, and fusion processing of the multiple second downsampling features to obtain multiple processed second downsampling features; and determining whether the second image contains at least one flickering state based on the multiple processed second downsampling features.

[0088] In other words, such as Figure 4 As shown, the second object detection model includes a second backbone module, a second neck module, and a second prediction module. The second backbone module is used to downsample and extract features from the image to be processed, obtaining multiple downsampled features of different sizes. The image to be processed includes a first image and a second image. The second neck module is used to upsample, extract features, and fuse the multiple downsampled features to obtain multiple processed downsampled features. The second prediction module is used to obtain the prediction result of the image to be processed based on the multiple processed downsampled features. When the second object detection model adopts the above structure, it can accurately identify whether the second image contains at least one flickering state.

[0089] As an optional implementation, downsampling and feature extraction of the second image to obtain multiple second downsampled features of different sizes includes: performing convolution and normalization on the second image to obtain second features; downsampling the second features, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current second downsampled feature; downsampling the current second downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next second downsampled feature; using the next second downsampled feature as the current second downsampled feature, and returning to the steps of downsampling the current second downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next second downsampled feature, until the number of downsampling reaches a preset second threshold. For example, the second threshold can be greater than or equal to 5 times, that is, at least 5 downsampling operations.

[0090] In other words, such as Figure 4As shown, the second main module includes a CBM unit and a CSP unit; the CBM unit is used to perform convolution and normalization on the image to be processed to obtain the first feature; the CSP unit is connected to the CBM unit and includes at least 5 CSP sub-units connected in sequence. Each CSP sub-unit is used to downsample the first feature or the previous downsampled feature, divide the downsampled feature map into two parts of features, and then merge the two parts of features through a cross-stage hierarchical structure to obtain the current downsampled feature.

[0091] Specifically, the CBM unit includes a convolutional layer (Conv), batch normalization (Bn), and the Mish activation function, f(x) = x∗tanh(softplus(x)). The CSP unit stands for cross-stage partial network unit.

[0092] This allows for the generation of output images in sizes of 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 from the head of the second object detection model.

[0093] As an optional implementation, multiple second downsampled features are upsampled, feature extracted, and fused to obtain processed second downsampled features, including: performing convolution and normalization, max pooling, and upsampling on the smallest second downsampled feature; performing convolution and normalization, and upsampling on the other second downsampled features besides the smallest one; and fusing at least two of the processed second downsampled features. In one optional implementation, this means, as... Figure 4 As shown, the second neck module includes multiple CBL units, SPP units, multiple upsampling units, and multiple connection units; the multiple CBL units are used to perform convolution and normalization processing on each downsampled feature; the SPP units are used to perform max pooling processing on the downsampled feature with the smallest size; the multiple upsampling units are used to perform upsampling processing on the other downsampled features except for the downsampled feature with the largest size; and the multiple connection units are used to perform fusion processing on at least two downsampled features.

[0094] Specifically, the CBL unit includes a convolutional layer (Conv), batch normalization (Bn), and the Leaky_relu activation function, f(x)=max[αx,x]).

[0095] Specifically, the SPP unit is a spatial pyramid module that can fuse features of more than ten sizes. It mainly consists of three max-pooling modules and a concat module. Compared with simply using K*K max-pooling, the SPP module approach can greatly increase the receptive field and significantly separate the most important contextual features.

[0096] like Figure 4 As shown, after inputting a 768*768 image, the following processing is performed in the second main module:

[0097] (1) The image is convolved and normalized in the CBM cell to obtain the first feature.

[0098] (2) In the CSP (cross-stage partial) unit, the first feature is downsampled multiple times. The first feature after each downsampling is mapped into two parts, and then the two parts are merged through the cross-stage hierarchical structure to obtain multiple downsampled features. For example Figure 4 As shown, a CSP unit includes at least 5 CSP sub-units. Figure 4 The process passes through each CSP sub-unit in sequence to obtain the first downsampled feature (192*192), the second downsampled feature (96*96), the third downsampled feature (48*48), the fourth downsampled feature (24*24), and the fifth downsampled feature (12*12).

[0099] The following processing is performed in the second neck module and the second prediction module:

[0100] (1) For the fifth downsampled feature (12*12), convolution and normalization are performed in the CBL unit, max pooling is performed in the SPP unit, and upsampling is performed in the upsampling unit (i.e. upsampling 1); For the fourth downsampled feature (24*24), convolution and normalization are performed in the CBL unit.

[0101] The processed fifth downsampled feature and the processed fourth downsampled feature are fused in the connection unit (i.e. connection 1) to obtain the first fused feature;

[0102] (2) For the third downsampled feature (48*48), convolution and normalization are performed in the CBL unit; for the first fusion feature, convolution and normalization are performed in the CBL unit, and upsampling is performed in the upsampling unit (i.e. upsampling 2).

[0103] The processed third downsampled feature and the processed first fused feature are fused in the connection unit (i.e. connection 2) to obtain the second fused feature;

[0104] (3) For the second downsampled feature (96*96), convolution and normalization are performed in the CBL unit; for the second fused feature, convolution and normalization are performed in the CBL unit, and upsampling is performed in the upsampling unit (i.e., upsampling 3).

[0105] The processed second downsampled feature and the processed second fused feature are fused in the connection unit (i.e. connection 3) to obtain the third fused feature;

[0106] (4) For the first downsampled feature (192*192), convolution and normalization are performed in the CBL unit; for the third fused feature, convolution and normalization are performed in the CBL unit, and upsampling is performed in the upsampling unit (i.e., upsampling 4).

[0107] The processed first downsampled feature and the processed third fused feature are fused in the connection unit (i.e. connection 4) to obtain the fourth fused feature;

[0108] (5) For the fourth fusion feature, convolution and normalization are performed in the CBL unit, and convolution is performed in the convolution unit (i.e., convolution 1) to obtain the feature prediction result of 192*192.

[0109] (6) For the fourth fusion feature, convolution and normalization are performed in the CBL unit; for the third fusion feature, convolution and normalization are performed in the CBL unit.

[0110] The processed fourth fusion feature and the processed third fusion feature are fused together in the connection unit (i.e., connection 5) to obtain the fifth fusion feature;

[0111] For the fifth fusion feature, convolution and normalization are performed in the CBL unit, and convolution is performed in the convolution unit (i.e., convolution 2) to obtain a 96*96 feature prediction result;

[0112] (7) For the fifth fusion feature, convolution and normalization are performed in the CBL unit; for the second fusion feature, convolution and normalization are performed in the CBL unit.

[0113] The processed fifth fusion feature and the processed second fusion feature are fused together in the connection unit (i.e., connection 6) to obtain the sixth fusion feature;

[0114] For the sixth fusion feature, convolution and normalization are performed in the CBL unit, and convolution is performed in the convolution unit (i.e., convolution 3) to obtain a 48*48 feature prediction result;

[0115] (8) For the sixth fusion feature, convolution and normalization are performed in the CBL unit; for the first fusion feature, convolution and normalization are performed in the CBL unit.

[0116] The processed sixth fusion feature and the processed first fusion feature are fused together in the connection unit (i.e., connection 7) to obtain the seventh fusion feature;

[0117] For the seventh fusion feature, convolution and normalization are performed in the CBL unit, and convolution is performed in the convolution unit (i.e., convolution 4) to obtain a 24*24 feature prediction result;

[0118] (9) For the seventh fusion feature, convolution and normalization are performed in the CBL unit;

[0119] The processed seventh fusion feature and the fifth downsampled feature after convolution and normalization are fused in the connection unit (i.e. connection 8) to obtain the eighth fusion feature;

[0120] For the eighth fused feature, convolution and normalization are performed in the CBL unit, and convolution is performed in the convolution unit (i.e., convolution 5) to obtain a 12*12 feature prediction result.

[0121] As an optional implementation, before performing convolution and normalization, max pooling, and upsampling on the smallest second downsampled feature, the method further includes performing min pooling on the smallest second downsampled feature.

[0122] As one implementation method, the second target detection model also includes an ECA unit; such as Figure 4 The ECA unit shown is connected to the last CSP sub-unit in the CSP unit and is used to perform min-pooling on the smallest downsampled feature.

[0123] This is because, compared to other attention mechanisms, the ECA attention mechanism avoids dimensionality reduction and efficiently achieves local cross-channel interaction using one-dimensional convolution, extracting the dependencies between channels.

[0124] like Figure 5 As shown, the operation steps are as follows:

[0125] Perform global average pooling on the input features;

[0126] Perform convolution and one-dimensional convolution operations of size k, and obtain the weights W of each channel by passing through the sigmoid activation function, w=σ(C1D_k (y)).

[0127] Where k represents the kernel size, C represents the number of channels (1×1×c means one channel per pixel), G represents Global Average Pooling, and σ represents sigmoid.

[0128] As an optional implementation, the training method for the second object detection model includes: obtaining a training set; wherein the second object detection model uses a first-class image including flashing lights and a second-class image excluding flashing lights as the training set; setting the size of the prior boxes in the K-means clustering method; clustering the training set using the K-means clustering method, selecting the intersection and union of the predicted boxes and the ground truth boxes in the K-means clustering method, and using the prior box with the largest intersection and union as the positive sample; training the model using a preset loss function set based on the positive samples to obtain the second object detection model.

[0129] The second category of images, excluding flickering lights, can include photos taken when the lights are off or when the lights are on.

[0130] Specifically, a robotic arm is used to move the light source to the shooting position for taking pictures. The light source used is a white strip light source, and the long exposure time is set to 200ms. The pictures taken with the long exposure camera, P0, with the flickering light and P2 without the flickering light, will be significantly different. There are two types of pictures without the flickering light: one is P1, which is taken when the light is off, and the other is P2, which is taken when the light is on.

[0131] It should be noted that setting different parameters when training the YOLOv4 model can enable the second object detection model to perform different functions. For example, with different training parameters, the second object detection model in this embodiment of the invention can determine whether the second image contains at least one flickering state, or it can determine whether the second image contains at least two flickering states.

[0132] As an optional implementation, the loss functions in the loss function set include the location regression loss function. Loss of confidence in objects and classification loss function .

[0133] As an optional implementation, the size of the prior bounding box corresponds to the size of the small object. This is because the objects that the second object detection model needs to detect are relatively small, accounting for less than 0.01 pixels in the image. Therefore, the size of the prior bounding box needs to correspond to the size of the small object, which can make the second object detection module perform better in detecting small objects.

[0134] Specifically, if the width and height of an object are less than 1 / 10 of the original image's width and height, it can be considered a small object.

[0135] For example, the second object detection model can be the YOLOv4 model. The YOLOv4 model is divided into one-stage and two-stage object detection depending on the detection method. One-stage object detection is characterized by its speed, ability to avoid false positives due to background errors, and learning generalized object features. Two-stage object detection is characterized by high accuracy, but it is relatively slower, has a longer training time, and a relatively higher false positive rate.

[0136] As an optional implementation, the YOLOv4 model is a one-stage object detection model. In the earliest days, one-stage object detection directly extracted features through convolution to predict object classification and localization. However, YOLOv4 later adopted the Acer idea, first performing clustering to generate multiple prior boxes. Prior boxes of various sizes are used for coarse localization of the object, and then precise filtering is performed later.

[0137] In order to achieve better results, the YOLOv4 model was improved in the following ways in this embodiment of the invention:

[0138] (1) Because the objects to be detected by the second object detection model are relatively small, accounting for less than 0.01 pixels in the image, and the second object detection model only needs to detect the object, the acer value of the YOLOV4 model part was optimized. Specifically, the feature value of the prior box of the YOLOV4 model corresponds to the small object.

[0139] For example, the values ​​[12,16;19,36;40,28;36,75;76,55;72,146;142,110;192,243;459,401] can be adjusted to [5,6;12, 16;29,17;19, 36;40, 28;28,62;36, 75;76, 55;72, 146;142,110;149,165;175,159;192, 243;459, 401;359,506], which improves the detection performance for small targets.

[0140] (2) In order to better optimize the recognition performance of the network, an ECA attention module was added to the YOLOv4 model. The ECA attention module is a channel attention module that uses a fast adaptive method to select the size of the one-dimensional convolution kernel.

[0141] (3) In related technologies, the head part of the YOLOv4 model uses the head from YOLOv3, where the head of 1 / 8, 1 / 16, and 1 / 32 of the original input size is used to detect large, medium, and small targets respectively, and the output of each scale has three prior boxes of different sizes. However, since this embodiment has high requirements for small object detection, the structure of the head part needs to be modified, such as... Figure 6 As shown, the head portion is 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64. This adjustment produces five output images of different sizes, which greatly optimizes the detection accuracy for small target objects. Here, 255 represents num_achor*(5+num_class).

[0142] Furthermore, to improve detection accuracy, the size of the input image can be modified. Previously, the size of the input image was 608*608, but now it can be changed to 768*768.

[0143] (4) The deep learning algorithm is encapsulated and optimized using C++. Because the speed will be slower when calling and displaying using Python, in order to further reduce the speed cost, this embodiment uses C++ to reconstruct the Python algorithm and uses the SSE module and the thread parallel module to accelerate and optimize the algorithm.

[0144] Step S303: When the second image contains at least one flashing state, determine the shooting time period of the second image.

[0145] Step S304: Obtain multiple first images of the lights within the shooting time period.

[0146] Step S305: Select an image combination from multiple first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of which is a lit image and the other is a dark image.

[0147] As an optional implementation, step S305 includes:

[0148] Step S3051: Input multiple first images into the trained first object detection model, and determine whether each first image is a lit image or a dark image based on the output of the first object detection model.

[0149] As an optional implementation, each first image is input into a trained first object detection model. Determining whether each first image is a lit image or a dark image based on the output of the first object detection model includes: downsampling and feature extraction of each first image to obtain multiple first downsampling features of different sizes; upsampling, feature extraction, and fusion processing of the multiple first downsampling features to obtain multiple processed first downsampling features; and determining whether each first image is a lit image or a dark image based on the processed multiple first downsampling features.

[0150] As an optional implementation, downsampling and feature extraction are performed on each first image to obtain multiple first downsampled features of different sizes. This includes: performing convolution and normalization on each first image to obtain a first feature; downsampling the first feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current first downsampled feature; downsampling the current first downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next first downsampled feature; using the next first downsampled feature as the current first downsampled feature, and returning to the steps of downsampling the current first downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next first downsampled feature, until the number of downsampling operations reaches a preset first threshold. For example, the first threshold can be greater than or equal to 5 times, that is, at least 5 downsampling operations are performed.

[0151] As an optional implementation, multiple first downsampled features are upsampled, feature extracted, and fused to obtain multiple processed first downsampled features; the smallest first downsampled feature is subjected to convolution and normalization, max pooling, and upsampling; the other first downsampled features besides the smallest first downsampled feature are subjected to convolution and normalization, and upsampling; and at least two processed first downsampled features are fused.

[0152] As an optional implementation, before performing convolution and normalization, max pooling, and upsampling on the smallest downsampled feature, the method further includes performing min pooling on the smallest downsampled feature. Specifically, the first target detection model includes a first backbone module, a first neck module, and a first prediction module. It should be noted that the first backbone module, the first neck module, and the first prediction module have the same structure as the second backbone module, the second neck module, and the second prediction module, respectively, and will not be described again here.

[0153] The training method for the first object detection model is similar to that for the second object detection model, the only difference being the training sets. In the training of the first object detection model, images of lights on and lights off are used as the training set. The training set includes multiple images of lights on and multiple images of lights off.

[0154] Step S3052: Select an image combination from multiple first images.

[0155] Step S306: Determine the flashing frequency of the light using image combination.

[0156] In this embodiment, by inputting the second image into a trained second object detection model, it is possible to determine whether the second image contains at least one flickering state based on the output of the second object detection model; furthermore, cases where the hard drive light does not flicker can be eliminated, thereby making the calculated light flicker frequency more accurate; by inputting multiple first images into a trained first object detection model, it is possible to determine whether each first image is a lit or dark image based on the output of the first object detection model; furthermore, image combinations can be selected from multiple first images to determine the light flicker frequency.

[0157] This embodiment provides a method for determining the flashing frequency of a light, which can be used in computer devices. In this embodiment, the light refers to the hard drive backlight. Figure 7 This is a flowchart illustrating another method for determining the flicker frequency of a light according to an embodiment of the present invention, as shown below. Figure 7 As shown, the process includes the following steps:

[0158] Step S701: Obtain the second image of the light.

[0159] Specifically, the second image is a picture taken using a long exposure camera or a picture taken using a regular camera with a long exposure.

[0160] Step S702: Determine whether the second image contains at least two flashing states, wherein the flashing state is the state in which the light changes from bright to dark and then bright again or the light changes from dark to bright and then dark again.

[0161] Step S703: When the second image contains at least one flashing state, determine the shooting time period of the second image.

[0162] Step S704: Obtain multiple first images of the lights within the shooting time period.

[0163] Step S705: Select an image combination from multiple first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of which is a lit image and the other is a dark image.

[0164] As an optional implementation, step S705 includes:

[0165] Step S7051: Input multiple first images into the trained first object detection model, and determine whether each first image is a lit image or a dark image based on the output of the second object detection model.

[0166] Step S7052: Select an image combination from multiple first images.

[0167] Suppose that the first images for a preset time period are: bright 1, bright 2, dark 3, dark 4, dark 5, dark 6, bright 7, bright 8, dark 9, dark 10, dark 11, dark 12, bright 13, bright 14, dark 15, dark 16, dark 17, dark 18, bright 19, bright 20...

[0168] When selecting image combinations from multiple first images according to the order of lit images first and dark images last, the resulting image combinations are (lit 1, lit 2, dark 3, dark 4), (lit 7, lit 8, dark 9, dark 10), (lit 13, lit 14, dark 15, dark 16)...

[0169] Step S706: Determine the flashing frequency of the light using image combination.

[0170] As an optional implementation, step S706 includes:

[0171] Step S7061: Determine the first time interval between two adjacent image combinations.

[0172] As an optional implementation, step S7061 includes: acquiring feature images from the two image combinations respectively; and determining a first time interval based on the feature images.

[0173] Specifically, the following four methods can be used to determine the first time interval based on the feature image.

[0174] The first method is as follows: obtain the first shooting time of the lit image in the first image combination of two adjacent image combinations; obtain the second shooting time of the lit image in the second image combination of two adjacent image combinations; and subtract the first shooting time from the second shooting time to obtain the first time interval.

[0175] As shown above, if multiple first images are selected, the resulting image combinations are (Bright 1, Bright 2, Dark 3, Dark 4), (Bright 7, Bright 8, Dark 9, Dark 10), (Bright 13, Bright 14, Dark 15, Dark 16)... The first method can be understood as follows: obtain the shooting time of Bright 2 in (Bright 1, Bright 2, Dark 3, Dark 4), obtain the shooting time of Bright 8 in (Bright 7, Bright 8, Dark 9, Dark 10), and subtract the shooting time of Bright 2 from the shooting time of Bright 8 to obtain a first time interval; obtain the shooting time of Bright 8 in (Bright 7, Bright 8, Dark 9, Dark 10), obtain the shooting time of Bright 14 in (Bright 13, Bright 14, Dark 15, Dark 16), and subtract the shooting time of Bright 8 from the shooting time of Bright 14 to obtain another first time interval... and so on.

[0176] The second method is as follows: obtain the third shooting time of the dark image in the first image combination of two adjacent image combinations; obtain the fourth shooting time of the dark image in the second image combination of two adjacent image combinations; and use the fourth shooting time to subtract the third shooting time to obtain the first time interval.

[0177] As shown above, if multiple first images are selected, the resulting image combinations are (Bright 1, Bright 2, Dark 3, Dark 4), (Bright 7, Bright 8, Dark 9, Dark 10), (Bright 13, Bright 14, Dark 15, Dark 16)... The first method can be understood as follows: obtain the shooting time of Dark 3 in (Bright 1, Bright 2, Dark 3, Dark 4), obtain the shooting time of Dark 9 in (Bright 7, Bright 8, Dark 9, Dark 10), and subtract the shooting time of Dark 3 from the shooting time of Dark 9 to obtain a first time interval; obtain the shooting time of Dark 9 in (Bright 7, Bright 8, Dark 9, Dark 10), obtain the shooting time of Dark 15 in (Bright 13, Bright 14, Dark 15, Dark 16), and subtract the shooting time of Dark 9 from the shooting time of Dark 15 to obtain another first time interval... and so on.

[0178] Step S7062: Determine the flashing frequency of the light based on the first time interval.

[0179] As an optional implementation, step S7062 includes:

[0180] Step a1: Obtain all first time intervals within the time period.

[0181] Step a2: Calculate the flicker frequency of the light using the remaining first time interval within the time period and the least squares method.

[0182] This allows for a relatively accurate determination of the light's flicker frequency.

[0183] Typically, 8 to 9 first time intervals can be obtained within 1 second. As an optional implementation, before step a2, the method further includes deleting the largest and smallest first time intervals within the time period. This can further improve the accuracy of the light flashing frequency.

[0184] As an optional implementation, step S7062 includes:

[0185] Step b1: Obtain all first time intervals within the time period.

[0186] Step b2: Calculate the average value of all first time intervals to obtain the flashing frequency of the light.

[0187] Step S707: Obtain the preset standard flashing frequency.

[0188] Specifically, the standard blink frequency can be understood as the blink frequency of the light when the hard drive is running normally.

[0189] Step S708: Determine whether the flashing frequency of the light meets the preset requirements based on the standard flashing frequency.

[0190] Specifically, the flashing frequency range can be set according to the standard flashing frequency. When the flashing frequency of the light exceeds the flashing frequency range, it is considered that the flashing frequency of the light does not meet the preset requirements; or, the calculated flashing frequency can be directly compared with the standard flashing frequency. For example, if the difference is no more than 10ms, it is considered to meet the requirements.

[0191] Step S709: If the condition is not met, issue a prompt message.

[0192] This is because the flashing frequency of the hard drive indicator light can reflect the operating status of the hard drive. When the flashing frequency of the hard drive indicator light does not meet the preset requirements, it indicates that the hard drive is not operating normally. Therefore, a prompt message needs to be issued to facilitate the staff to quickly handle the abnormal situation.

[0193] The method for determining the flashing frequency of a light provided in this embodiment of the invention can not only accurately determine the flashing frequency of the light, but also promptly issue a warning message when the hard drive is not running normally, making it convenient for staff to quickly handle abnormal situations.

[0194] This embodiment also provides a device for determining the flashing frequency of a light. This device is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0195] This embodiment provides a device for determining the flicker frequency of a light, such as... Figure 8 As shown, it includes:

[0196] The acquisition module 801 is used to acquire multiple first images of lights within a preset time period.

[0197] Image combination selection module 802 is used to select an image combination from multiple first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of which is a lit image and the other is a dark image.

[0198] The blinking frequency determination module 803 is used to determine the blinking frequency of the light by combining images.

[0199] In some optional implementations, the acquisition module 801 of the light flicker frequency determination device includes a first acquisition unit, a judgment unit, a shooting time period determination unit, and a second acquisition unit.

[0200] The first acquisition unit is used to acquire a second image of the light.

[0201] The judgment unit is used to determine whether the second image contains at least one flashing state, wherein the flashing state is the state of the light changing from bright to dim or from dim to bright.

[0202] The shooting time period determination unit is used to determine the shooting time period of the second image when the second image contains at least one flashing state.

[0203] The second acquisition unit is used to acquire multiple first images of the lights within the shooting time period.

[0204] In some optional implementations, the determination unit is specifically used to: input the second image into a trained second target detection model, and determine whether the second image contains at least one flickering state based on the output of the second target detection model.

[0205] In some optional implementations, the image combination selection module 802 is specifically used to: input multiple first images into a trained first object detection model, determine whether each first image is a lit image or a dark image based on the output of the first object detection model, and select an image combination from the multiple first images.

[0206] In some optional implementations, the flicker frequency determination module 803 includes a time interval determination unit and a flicker frequency determination unit.

[0207] The time interval determination unit is used to determine the first time interval between two adjacent image combinations.

[0208] The blinking frequency determination unit is used to determine the blinking frequency of the light based on a first time interval.

[0209] In some optional implementations, the time interval determination unit is specifically used to: acquire feature images from the two image combinations respectively; and determine a first time interval based on the feature images.

[0210] In some optional implementations, the time interval determination unit is specifically used to: obtain the first shooting time of the lit image in the previous image combination of two adjacent image combinations; obtain the second shooting time of the lit image in the next image combination of two adjacent image combinations; and obtain a first time interval by subtracting the first shooting time from the second shooting time; or, obtain the third shooting time of the dark image in the previous image combination of two adjacent image combinations; obtain the fourth shooting time of the dark image in the next image combination of two adjacent image combinations; and obtain the first time interval by subtracting the third shooting time from the fourth shooting time; or, obtain the first shooting time of the lit image in the previous image combination of two adjacent image combinations; obtain the fourth shooting time of the dark image in the next image combination of two adjacent image combinations; and obtain the first time interval based on the fourth shooting time and the first shooting time; or, obtain the third shooting time of the dark image in the previous image combination of two adjacent image combinations; obtain the second shooting time of the lit image in the next image combination of two adjacent image combinations; and obtain the first time interval based on the second shooting time and the third shooting time.

[0211] In one optional implementation, the determination unit includes a second backbone module, a second neck module, and a second prediction module. The second backbone module is used to downsample and extract features from the second image to obtain multiple downsampled features of different sizes; the second neck module is used to upsample, extract features, and fuse the multiple downsampled features to obtain multiple processed downsampled features; the second prediction module is used to determine whether the second image contains at least one flicker state based on the multiple processed downsampled features.

[0212] In one optional implementation, the image combination selection module 802 includes a first backbone module, a first neck module, and a first prediction module. The first backbone module is used to downsample and extract features from each first image to obtain multiple downsampled features of different sizes; the first neck module is used to upsample, extract features, and fuse the multiple downsampled features to obtain multiple processed downsampled features; the first prediction module is used to determine whether each first image is a lit image or a dark image based on the processed multiple downsampled features.

[0213] In one optional implementation, the first backbone module is specifically used for: performing convolution and normalization on the first image to obtain a first feature; downsampling the first feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current downsampled feature; downsampling the current downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next downsampled feature; using the next downsampled feature as the current sampled feature, and returning to the steps of downsampling the current downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next downsampled feature, until the number of downsampling times reaches a preset first threshold.

[0214] In one optional implementation, the second backbone module is specifically used for: performing convolution and normalization on the second image to obtain a second feature; downsampling the second feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the current downsampled feature; downsampling the current downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next downsampled feature; using the next downsampled feature as the current sampled feature, and returning to the steps of downsampling the current downsampled feature, dividing the downsampled feature map into two parts, and then merging the two parts through a cross-stage hierarchical structure to obtain the next downsampled feature, until the number of downsampling times reaches a preset second threshold.

[0215] In one optional implementation, the first neck module is specifically used to: perform convolution and normalization, max pooling and upsampling on the smallest first downsampled feature; perform convolution and normalization on other first downsampled features besides the smallest first downsampled feature, and upsampling; and fuse at least two processed first downsampled features.

[0216] In one optional implementation, the second neck module is specifically used to: perform convolution and normalization, max pooling and upsampling on the smallest second downsampled feature; perform convolution and normalization on other second downsampled features besides the smallest second downsampled feature, and upsampling; and fuse at least two processed second downsampled features.

[0217] In one alternative implementation, before performing convolution and normalization, max pooling, and upsampling on the smallest first downsampled feature, the first neck module is further configured to: perform min pooling on the smallest first downsampled feature.

[0218] In one alternative implementation, before performing convolution and normalization, max pooling, and upsampling on the smallest second downsampled feature, the second neck module is further configured to: perform min pooling on the smallest second downsampled feature.

[0219] In some optional implementations, the device for determining the light flicker frequency further includes a target detection module training module. Specifically, the target detection module training module is used to: acquire a training set; set the size of the prior boxes in the K-means clustering method; cluster the training set using the K-means clustering method, select the intersection and union of the predicted boxes and the ground truth boxes in the K-means clustering method, and use the prior box with the largest intersection and union as the positive sample; train the model using a preset loss function set based on the positive samples to obtain a first target detection model or a second target detection model.

[0220] In some alternative implementations, the loss functions in the loss function set include the location regression loss function, the object confidence loss, and the classification loss function.

[0221] In some alternative implementations, the size of the prior box corresponds to that of the small object.

[0222] In some optional implementations, the device for determining the light flashing frequency also includes a reminder module. After determining the light flashing frequency using image combinations, the reminder module is used to: obtain a preset standard flashing frequency; determine whether the light flashing frequency meets the preset requirements based on the standard flashing frequency; and issue a reminder message when it does not meet the preset requirements.

[0223] In this embodiment, the device for determining the light flashing frequency is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0224] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0225] This invention also provides a computer device having the above-described features. Figure 8 The device shown is for determining the flashing frequency of the light.

[0226] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 9As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0227] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0228] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0229] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0230] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0231] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 20 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0232] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0233] This invention also provides a system for determining the flicker frequency of lights, comprising a long exposure camera, a high frame rate camera, and the aforementioned... Figure 8 The computer equipment shown. Both the long-exposure camera and the high-frame-rate camera are communicatively connected to the computer equipment.

[0234] Specifically, the light flicker frequency determination system of this invention includes a long exposure camera, a high frame rate camera, and a host computer (i.e., the aforementioned computer device). The long exposure camera is used to take a picture of the hard disk light to obtain a second image; the high frame rate camera is used to take a picture of the hard disk light to obtain a first image; the host computer performs long exposure image capture analysis on the second image to determine whether the second image contains at least one flicker state; when the second image contains at least one flicker state, the first image is performed multi-frame rate image capture analysis to determine the light flicker frequency based on multiple first images.

[0235] As an optional implementation, the light flicker frequency determination system also includes a light source controller, which is communicatively connected to a computer device.

[0236] The light source controller can control the light source when taking pictures with long-exposure cameras and high-frame-rate cameras. Based on lighting effects and testing, tests were conducted on white, red, and blue ring lights, bar lights, and area lights. Because the flashing frequencies of the lights are different, to more clearly show the state of the hard drive light in the camera, less intense light sources were selected. Multiple exposure values ​​could be set, for example, a random value from 0 to 255. Multiple sets of photos were taken and image processed. After multiple tests, a white bar light source with a light source controller setting of 175-255 produced the best results. In this embodiment, a white bar light source was selected.

[0237] Specifically, the light flashing frequency determination system uses two cameras: one is a high frame rate industrial camera, and the other is a regular industrial camera. A signal generator is used to trigger both cameras, and images are captured from each camera when they are triggered.

[0238] The operation method of the light flashing frequency determination system is as follows: After manual activation, a robotic arm moves the long-exposure camera and the high-frame-rate camera to the shooting position. The brightness is set on the light source controller, and the brightness of the long-exposure camera and the high-frame-rate camera is adjusted simultaneously. Both cameras are used to take pictures at the same time. To ensure that the long-exposure camera and the high-frame-rate camera are triggered at the same time, a signal generator is used for triggering. After triggering, the system can acquire a second image taken by the long-exposure camera within a certain period and multiple first images taken by the high-frame-rate camera within the same period.

[0239] like Figure 10 As shown, the processing method in the computer device includes: first, analyzing and judging the long-exposure images within this period using an optimized neural network algorithm; if flickering exists within this period after long-exposure image recognition, then detecting the flickering frequency of the lights in the images acquired by the multi-frame-rate camera within this period, and finally outputting the flickering frequency for judgment.

[0240] Specifically, the light flicker frequency recognition system of this invention also performs the following operations: taking pictures with a camera driven by a robotic arm to achieve the purpose of detection; a software framework for a vision system is written in C#, which can realize image scaling and display of running information, including communication with the robotic arm; the algorithm implementation is encapsulated and accelerated using C++; a graphics processing unit (GPU) is used to accelerate the speed of image recognition; and the system software is developed using C#. The entire system includes communication between the host computer and the module, human interaction of the interface, display of some work logs, and image display interface after taking pictures.

[0241] The light flicker frequency recognition system of this invention is implemented by rationally utilizing light source schemes, robotic arms, visual algorithms, and industrial cameras. This process generally includes a motion module moving the camera and light source to a designated position for taking pictures, where the camera's frame rate is particularly important. First, the frame rate must be at least twice the light flicker frequency to ensure that the light is presented in both bright and dark states in the camera within the effective shooting time. Second, a long exposure setting is also required to ensure that two alternating states of brightness and darkness will occur within the long exposure time, thus displaying different light states in the camera. Using the images from the above two points, the second point confirms that the light has definitely undergone alternating states of brightness and darkness during this period. The first point ensures that multiple photos are taken by the camera within this time, and the brightness and darkness states of the light in each image at the same time interval are analyzed. After analyzing multiple sets of time results, a relatively accurate light flicker time can be obtained.

[0242] Therefore, this invention provides a hard drive light flickering recognition system based on a visual algorithm, including software system development, deep learning algorithm optimization, and a process for judging light flickering. It can bring the following benefits: 1. It can adapt to various server models and types by only configuring different parameters; 2. It reduces personnel costs and is fully automated, requiring no manual intervention; 3. It shortens the detection time and greatly improves the detection efficiency.

[0243] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0244] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for determining the flicker frequency of a light, characterized in that, include: Acquire multiple first images of lights within a preset time period, wherein the first images are captured using a high frame rate camera; A combination of images is selected from the plurality of first images, wherein the combination of images includes at least two feature images arranged in chronological order of shooting time, one of the two feature images being a picture of a light on and the other being a picture of a light off; The flashing frequency of the light is determined using the combination of the images; The acquisition of multiple first images of lights within a preset time period includes: A second image of the light is acquired, wherein the second image is captured using a long exposure camera; Determine whether the second image contains at least one flickering state, wherein the flickering state is the state in which the light changes from bright to dim and then bright again or the state in which the light changes from dim to bright and then dim again; When the second image contains at least one of the flashing states, the time period for capturing the second image is determined; Acquire multiple first images of the lights within the specified shooting time period; The step of determining whether the second image contains at least one of the flashing states includes: The second image is input into a trained second target detection model, and the output of the second target detection model is used to determine whether the second image contains at least one flickering state. The step of selecting a combination of images from the plurality of first images includes: Each of the first images is input into the trained first object detection model, and the output of the first object detection model is used to determine whether each of the first images is a bright light image or a dark light image. Select the image combination from the plurality of first images; The first target detection model includes a first backbone module, a first neck module, and a first prediction module; the second target detection model includes a second backbone module, a second neck module, and a second prediction module; the first backbone module and the second backbone module are used to downsample and extract features from the image to be processed, obtaining multiple downsampled features of different sizes, wherein the image to be processed includes a first image and a second image; the first neck module and the second neck module are used to upsample, extract features, and fuse the multiple downsampled features to obtain multiple processed downsampled features; the first prediction module and the second prediction module are used to obtain the prediction result of the image to be processed based on the multiple processed downsampled features; The training method for the first object detection model or the second object detection model includes: Obtain the training set; Set the size of the prior bounding boxes in the K-means clustering method; The training set is clustered using the K-means clustering method. The intersection and union of the predicted boxes and the ground truth boxes in the K-means clustering method are selected, and the prior box with the largest intersection and union is taken as the positive sample. The first target detection model or the second target detection model is obtained by training the positive samples using a preset set of loss functions.

2. The method according to claim 1, characterized in that, Determining the flashing frequency of the light using the image combination includes: Determine the first time interval between each of two adjacent image combinations; The flashing frequency of the light is determined based on the first time interval.

3. The method according to claim 2, characterized in that, The determination of the first time interval between two adjacent image combinations includes: Obtain the feature images from each of the two image combinations; The first time interval is determined based on the feature image.

4. The method according to claim 3, characterized in that, Determining the first time interval based on the feature image includes: Obtain the first capture time of the lit image in the preceding image combination among two adjacent image combinations; Obtain the second shooting moment of the lit image in the latter of two adjacent image combinations; The first time interval is obtained by subtracting the first shooting time from the second shooting time. or, Obtain the third shooting moment of the dark image in the previous image combination among two adjacent image combinations; Obtain the fourth shooting moment of the dark image in the latter of two adjacent image combinations; The first time interval is obtained by subtracting the third shooting time from the fourth shooting time.

5. The method according to claim 2, characterized in that, Determining the flashing frequency of the light based on the first time interval includes: Obtain all first time intervals within the stated time period; The flicker frequency of the light is calculated using the remaining first time interval within the time period and the least squares method; or, Obtain all first time intervals within the stated time period; The flashing frequency of the light is obtained by calculating the average value of all the first time intervals.

6. The method according to claim 1, characterized in that, The step of inputting the second image into the trained second object detection model and determining whether the second image contains at least one flickering state based on the output of the second object detection model includes: The second image is downsampled and its features are extracted to obtain multiple second downsampled features of different sizes; Multiple second downsampled features are upsampled, feature extracted, and fused to obtain multiple processed second downsampled features. Based on the processed multiple second downsampling features, determine whether the second image contains at least one of the flickering states; Alternatively, the step of inputting each of the first images into a trained first object detection model, and determining whether each of the first images is a lit image or a dark image based on the output of the first object detection model, includes: For each of the first images, downsampling and feature extraction are performed to obtain multiple first downsampled features of different sizes; Multiple first downsampled features are upsampled, feature extracted, and fused to obtain multiple processed first downsampled features. Based on the processed first downsampling features, we can determine whether each of the first images is a lit image or a dark image.

7. The method according to claim 6, characterized in that, The step of downsampling and feature extraction for each of the first images to obtain multiple first downsampled features of different sizes includes: Each of the first images is convolved and normalized to obtain the first feature; The first feature is downsampled, and the downsampled feature map is divided into two parts. Then, the two parts are merged through a cross-stage hierarchical structure to obtain the current first downsampled feature. The current first downsampled feature is downsampled, and the downsampled feature map is divided into two parts. Then, the two parts are merged through a cross-stage hierarchical structure to obtain the next first downsampled feature. The next first downsampled feature is used as the current first downsampled feature, and the current first downsampled feature is downsampled. The downsampled feature map is divided into two parts, and then the two parts are merged through a cross-stage hierarchical structure to obtain the next first downsampled feature. This process continues until the number of downsampling times reaches a preset first threshold. Alternatively, the step of downsampling and feature extraction of the second image to obtain multiple second downsampled features of different sizes includes: The second image is convolved and normalized to obtain the second feature; The second feature is downsampled, and the downsampled feature map is divided into two parts. Then, the two parts are merged through a cross-stage hierarchical structure to obtain the current second downsampled feature. The current second downsampled feature is downsampled, and the downsampled feature map is divided into two parts. Then, the two parts are merged through a cross-stage hierarchical structure to obtain the next second downsampled feature. The next second downsampled feature is used as the current second downsampled feature, and the current second downsampled feature is downsampled. The downsampled feature map is divided into two parts, and then the two parts are merged through a cross-stage hierarchical structure to obtain the next second downsampled feature. This process continues until the number of downsampling times reaches a preset second threshold.

8. The method according to claim 6, characterized in that, The process involves upsampling, feature extraction, and fusion of multiple first downsampled features to obtain multiple processed first downsampled features. The first downsampled feature with the smallest size is subjected to convolution and normalization, max pooling and upsampling. Convolution, normalization, and upsampling are performed on all first downsampled features except for the smallest first downsampled feature. At least two of the processed first downsampled features are fused together. Alternatively, upsampling, feature extraction, and fusion processing are performed on multiple second downsampled features to obtain multiple processed second downsampled features, including: The second downsampled feature with the smallest size is subjected to convolution and normalization, max pooling and upsampling. Convolution, normalization, and upsampling are performed on all second downsampled features except for the smallest second downsampled feature. At least two of the processed second downsampled features are fused together.

9. The method according to claim 8, characterized in that, Before performing convolution and normalization, max pooling, and upsampling on the smallest downsampled feature, the process also includes: Mini-pooling is performed on the first downsampled feature with the smallest size; Alternatively, before performing convolution and normalization, max pooling, and upsampling on the smallest second downsampled feature, the following steps are also included: Mini-pooling is performed on the second downsampled feature with the smallest size.

10. The method according to claim 1, characterized in that, The loss functions in the set of loss functions include the location regression loss function, the object confidence loss function, and the classification loss function.

11. The method according to claim 1, characterized in that, The size of the prior box corresponds to that of the small object.

12. The method according to any one of claims 1 to 11, characterized in that, After determining the flashing frequency of the light using the image combination, the method further includes: Obtain the preset standard flashing frequency; Determine whether the flashing frequency of the light meets the preset requirements based on the standard flashing frequency; If it does not meet the requirements, a prompt message will be issued.

13. A device for determining the flashing frequency of a light, characterized in that, include: The acquisition module is used to acquire multiple first images of lights within a preset time period, wherein the first images are captured by a high frame rate camera. The image combination selection module is used to select an image combination from the plurality of first images, wherein the image combination includes at least two feature images arranged in chronological order of shooting time, one of the two feature images being a lit image and the other being a dark image; A blinking frequency determination module is used to determine the blinking frequency of the light using the image combination; The acquisition module includes a first acquisition unit, a judgment unit, a shooting time period determination unit, and a second acquisition unit; The first acquisition unit is used to acquire a second image of the light, wherein the second image is captured using a long exposure camera; The judging unit is used to judge whether the second image contains at least one flashing state, wherein the flashing state is the state in which the light changes from bright to dim and then bright again or the light changes from dim to bright and then dim again; The shooting time period determination unit is used to determine the shooting time period of the second image when the second image contains at least one flashing state. The second acquisition unit is used to acquire multiple first images of lights within the shooting time period; The judgment unit is specifically used to: input the second image into a trained second target detection model, and determine whether the second image contains at least one flashing state based on the output of the second target detection model; The image combination selection module is specifically used for: inputting each of the first images into a trained first target detection model, determining whether each of the first images is a lit image or a dark image based on the output of the first target detection model; and selecting the image combination from the plurality of first images. The first target detection model includes a first backbone module, a first neck module, and a first prediction module; the second target detection model includes a second backbone module, a second neck module, and a second prediction module; the first backbone module and the second backbone module are used to downsample and extract features from the image to be processed, obtaining multiple downsampled features of different sizes, wherein the image to be processed includes a first image and a second image; the first neck module and the second neck module are used to upsample, extract features, and fuse the multiple downsampled features to obtain multiple processed downsampled features; the first prediction module and the second prediction module are used to obtain the prediction result of the image to be processed based on the multiple processed downsampled features; The device for determining the flashing frequency of the light also includes a target detection module training module, which is specifically used for: Obtain the training set; Set the size of the prior bounding boxes in the K-means clustering method; The training set is clustered using the K-means clustering method. The intersection and union of the predicted boxes and the ground truth boxes in the K-means clustering method are selected, and the prior box with the largest intersection and union is taken as the positive sample. The first target detection model or the second target detection model is obtained by training the positive samples using a preset set of loss functions.

14. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the method for determining the flashing frequency of a light as described in any one of claims 1 to 12.

15. A system for determining the flicker frequency of a light, characterized in that, include: The long exposure camera, the high frame rate camera, and the computer device of claim 14, wherein the long exposure camera and the high frame rate camera are both communicatively connected to the computer device.

16. The system for determining the flicker frequency of a light according to claim 15, characterized in that, Also includes: A light source controller, which is communicatively connected to the computer device.

17. The system for determining the flicker frequency of a light according to claim 15, characterized in that, The frame rate of the high frame rate camera is more than twice the flashing frequency of the light.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method for determining the flashing frequency of a light as described in any one of claims 1 to 12.

Images