Camera imaging defect detection method, display cabinet and storage medium

Abstract

The present disclosure provides a camera imaging defect detection method, a display cabinet and a storage medium, wherein the method comprises: obtaining a target video shot by a target camera toward the inside of the display cabinet during the period when the cabinet door is open; extracting a first image frame from the target video, and calculating the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the period when the cabinet door is open; based on a quality detection strategy corresponding to the image blur, performing quality detection on the image frame in the target video to obtain a quality detection result; and determining a defect detection result of the imaging defect of the target camera based on the quality detection result.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to a camera imaging defect detection method, a display cabinet, and a storage medium. Background Art

[0002] In recent years, display cabinets have been widely used in convenience stores and large supermarkets, such as unmanned self-service display cabinets, refrigerated display cabinets for displaying refrigerated items, etc. A single camera is usually installed for existing display cabinets to monitor the items inside the display cabinet through the single camera. At this time, if a single camera shoots abnormally, it will affect the imaging quality of the camera, thereby affecting the monitoring accuracy of the items inside the display cabinet. In the prior art, the imaging quality of each camera is usually detected by manual inspection. However, when the number of display cabinets is large, the inspection efficiency of the existing manual inspection method is low, and the existing manual inspection method will waste a lot of human resources. Summary of the invention

[0003] The disclosed embodiments at least provide a camera imaging defect detection method, a display cabinet, and a storage medium. In the disclosed embodiments, by determining a quality detection strategy based on image blur, and performing quality detection on the image in the target video according to the quality detection strategy, the problems existing in the target camera can be fully detected.

[0004] In a first aspect, an embodiment of the present disclosure provides a camera imaging defect detection method, comprising: obtaining a target video shot by a target camera toward the interior of the display cabinet during the period when the door of the display cabinet is open; extracting a first image frame from the target video, and calculating the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the door meets the requirements during the period when the door is open; based on a quality detection strategy corresponding to the image blur, performing quality detection on the image frames in the target video to obtain a quality detection result; and determining a defect detection result of the imaging defect of the target camera based on the quality detection result.

[0005] In a second aspect, an embodiment of the present disclosure provides a camera imaging defect detection device, comprising: an acquisition unit, used to acquire a target video shot toward the inside of the display cabinet by a target camera during the period when the door of the display cabinet is open; an extraction unit, used to extract a first image frame from the target video; a calculation unit, used to calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the door meets the requirement during the period when the door is open; a quality detection unit, used to perform quality detection on the image frame in the target video based on a quality detection strategy corresponding to the image blur to obtain a quality detection result; and a determination unit, used to determine the defect detection result of the imaging defect of the target camera based on the quality detection result.

[0006] In a third aspect, an embodiment of the present disclosure provides a display cabinet, comprising: a display cabinet body, a target camera and a processor, wherein the target camera is mounted on the display cabinet body, and the lens of the target camera is facing the interior of the display cabinet body; the target camera is configured to capture a target video inside the display cabinet while the door of the display cabinet is open; the processor is configured to extract a first image frame from the target video and calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the door meets the requirement during the period when the door is open; based on the quality detection strategy corresponding to the image blur, quality detection is performed on the image frame in the target video to obtain a quality detection result; and a defect detection result of the imaging defect of the target camera is determined based on the quality detection result.

[0007] In a fourth aspect, an embodiment of the present disclosure further provides a display cabinet, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0008] In a fifth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0009] In the disclosed embodiment, by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera based on the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources.

[0010] By calculating the image blurriness of the first image frame in the target video and determining different quality detection strategies based on the image blurriness, the problem with the target camera can be inferred based on the image blurriness. For example, when the image blurriness is large, the problem with the target camera may be a problem with the lens of the target camera itself, such as stains, exposure, untorn film, etc. When the image blurriness is small, the problem with the target camera may be a problem with the installation angle of the lens of the target camera on the display cabinet, such as the camera is installed upside down, cannot be fully photographed, etc. Therefore, in the disclosed embodiment, by determining the quality detection strategy based on the image blurriness and performing quality detection on the image in the target video according to the quality detection strategy, the problems with the target camera can be fully detected.

[0011] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0013] Figure 1 A flow chart of a camera imaging defect detection method provided by an embodiment of the present disclosure is shown;

[0014] Figure 2 A flowchart showing a specific method for extracting a first image frame from a target video in a camera imaging defect detection method provided by an embodiment of the present disclosure;

[0015] Figure 3 A flowchart showing a specific method for calculating the image blurriness of the first image frame in the camera imaging defect detection method provided by an embodiment of the present disclosure;

[0016] Figure 4 A flowchart showing a specific method for performing quality detection on the image frames in the target video to obtain quality detection results based on the quality detection strategy corresponding to the image blur in the camera imaging defect detection method provided by the embodiment of the present disclosure;

[0017] Figure 5A flowchart showing a specific method for performing quality detection on the image frames in the target video to obtain quality detection results based on the quality detection strategy corresponding to the image blur in the camera imaging defect detection method provided by the embodiment of the present disclosure;

[0018] Figure 6 A flow chart of another camera imaging defect detection method provided by an embodiment of the present disclosure is shown;

[0019] Figure 7 A schematic diagram of a camera imaging defect detection device provided by an embodiment of the present disclosure is shown;

[0020] Figure 8 A schematic diagram of the structure of a display cabinet provided in an embodiment of the present disclosure is shown;

[0021] Fig. 9 A schematic structural diagram of another display cabinet provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0023] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0024] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0025] After research, it is found that a single camera is usually installed in existing display cabinets to monitor the items inside the display cabinet. The single camera in the existing display cabinet may have the following defects: the camera is installed upside down, the camera does not work properly, the camera film is not torn off, the camera is deliberately blocked, the camera shooting is incomplete, the camera is exposed, the camera is damaged, the camera has color problems, and the camera has stains.

[0026] If a single camera has the above-described problems, it will affect the camera's imaging quality, thereby reducing the image processing accuracy of the images taken by the camera, and further affecting the monitoring accuracy of the items inside the display cabinet. In the prior art, the imaging quality of each camera is usually detected by manual inspection. However, when the number of display cabinets is large, the existing manual inspection method has low inspection efficiency and will waste a lot of human resources.

[0027] Based on the above research, the present disclosure provides a camera imaging defect detection method, a display cabinet, and a storage medium. In the embodiment of the present disclosure, first, a target video is obtained by shooting a target camera toward the inside of the display cabinet while the door of the display cabinet is open; then, a first image frame can be extracted from the target video, and the image blurriness of the first image frame can be calculated; then, a corresponding quality detection strategy can be determined based on the image blurriness, and a quality detection result can be obtained by performing a quality detection on the image frame in the target video according to the quality detection strategy, and a defect detection result of the target camera can be determined according to the quality detection result.

[0028] From the above description, it can be seen that by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera according to the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources.

[0029] By calculating the image blurriness of the first image frame in the target video and determining different quality detection strategies based on the image blurriness, the problem with the target camera can be inferred based on the image blurriness. For example, when the image blurriness is large, the problem with the target camera may be a problem with the lens of the target camera itself, such as stains, exposure, untorn film, etc. When the image blurriness is small, the problem with the target camera may be a problem with the installation angle of the lens of the target camera on the display cabinet, such as the camera is installed upside down, cannot be fully photographed, etc. Therefore, in the disclosed embodiment, by determining the quality detection strategy based on the image blurriness and performing quality detection on the image in the target video according to the quality detection strategy, the problems with the target camera can be fully detected.

[0030] In the embodiments of the present disclosure, the display cabinet can be any cabinet whose doors can be opened and closed. For example, the display cabinet can be a display cabinet that supports unmanned self-service functions. The display cabinet can also be a refrigerated cabinet for placing refrigerated items in a supermarket. In addition, the display cabinet can also be a warming cabinet, a freezer for placing frozen items, and any other cabinet whose doors can be opened and closed. The present disclosure does not specifically limit the types of display cabinets.

[0031] To facilitate understanding of this embodiment, a camera imaging defect detection method disclosed in the embodiment of the present disclosure is first introduced in detail.

[0032] See also Figure 1 FIG. 1 is a flowchart of a camera imaging defect detection method provided by an embodiment of the present disclosure, the method comprising steps S101 to S107, wherein:

[0033] S101: Acquire a target video obtained by shooting the interior of the display cabinet with a target camera while the door of the display cabinet is open.

[0034] In the disclosed embodiment, the target camera can be installed on the door handle of the cabinet door, and the lens of the target camera is arranged toward the inside of the showcase. When the cabinet door of the showcase is detected to be opened, the target camera starts shooting to obtain the target video.

[0035] Here, the shooting duration can be preset. When the cabinet door of the display cabinet is detected to be opened, the target camera is triggered to shoot a video, and the shooting ends after the shooting duration is satisfied to obtain the target video. The preset shooting duration can be a duration preset by the user. For example, the shooting duration can be selected as 10 seconds. In addition, it can also be set to other durations. The present disclosure does not specifically limit this, and it is subject to what can be implemented.

[0036] When the target video is not captured during the opening of the door of the display cabinet, defect reminder information of the target camera is generated, wherein the defect reminder information is used to indicate a hardware defect of the target camera.

[0037] Here, hardware defects may include camera damage and / or camera wiring issues.

[0038] S103: Extract a first image frame from the target video and calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the opening of the cabinet door.

[0039] In the embodiment of the present disclosure, after the target video is captured, the image blur (or clarity) of the image frames in the target video may be determined.

[0040] In a specific implementation, a first image frame may be selected in the target video, and the image blurriness of the first image frame may be calculated, and then the image blurriness may be determined as the image blurriness of the image frame in the target video.

[0041] Here, the number of first image frames extracted from the target video may be one or more. When the number of first image frames is more than one, the image blurriness of each first image frame may be determined, and then the multiple image blurrinesses may be averaged to obtain the image blurriness of the image frame in the target video. When the number of first image frames is one, the blurriness of the first image frame may be determined as the image blurriness of the image frame in the target video.

[0042] The process of a user taking an item from a display cabinet includes the following stages: a stage in which the door of the display cabinet is opened, a stage in which the user takes an item from the display cabinet, and a stage in which the door of the display cabinet is closed. In the "stage in which the user takes an item from the display cabinet", the door of the display cabinet is in a hold state.

[0043] Here, the moment when the opening angle of the cabinet door meets the requirement can be understood as the time period corresponding to the above-mentioned “stage when the user takes items from the display cabinet”.

[0044] S105: Based on the quality detection strategy corresponding to the image blur, perform quality detection on the image frames in the target video to obtain a quality detection result.

[0045] When the image is blurry, it indicates that the problem with the target camera may be a problem with the target camera lens itself, such as stains, exposure, or untorn film.

[0046] When the image blur is small, it indicates that the problem with the target camera may be the shooting angle of the target camera, for example, the camera is installed upside down or cannot be fully captured.

[0047] Based on this, different quality inspection strategies can be determined for different image blurs. For example, for a situation where the image blur is large, a first quality inspection strategy can be determined, wherein the first quality inspection strategy is used to perform defect detection on the lens of the target camera. For a situation where the image blur is small, a second quality inspection strategy can be determined, wherein the second quality inspection strategy is used to perform defect detection on the shooting angle of the target camera. Here, the shooting angle can be understood as the installation angle of the lens of the target camera on the display cabinet.

[0048] S107: Determine a defect detection result of the imaging defect of the target camera based on the quality detection result.

[0049] Here, the camera installed in the showcase may be a network camera. After the camera captures the target video, the target video may be sent to the server via the Internet so that the server executes the above-described steps S101 to S107.

[0050] In addition, a processor may be installed in the display cabinet, wherein the processor can be connected to the camera for communication. After the camera captures the target video, the target video may be sent to the processor so that the processor executes the above-described steps S101 to S107.

[0051] From the above description, it can be seen that by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera according to the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources. By determining the quality inspection strategy based on the image blur and performing quality inspection on the image in the target video according to the quality inspection strategy, the problems existing in the target camera can be fully detected.

[0052] The above steps will be introduced below in conjunction with specific embodiments.

[0053] In an optional embodiment, if Figure 2 As shown, the above step S103: extracting the first image frame from the target video specifically includes the following steps:

[0054] Step S201: while the door of the showcase is open, obtaining the door opening angle collected by a sensor pre-installed in the showcase;

[0055] Step S202: analyzing the angle variation trend of the cabinet door opening angle;

[0056] Step S203: determining a collection period during which the angle change trend is less than a preset trend threshold value during the period when the cabinet door is opened;

[0057] Step S204: determining the first image frame from a plurality of image frames corresponding to the acquisition period in the target video.

[0058] In the disclosed embodiment, the door opening angle can be detected in real time by a sensor. After the door opening angle is detected, the time when the door opening angle meets the requirement can be determined in the target video according to the door opening angle, that is, the time period corresponding to the "stage when the user takes the item from the display cabinet".

[0059] In specific implementation, the angle change trend of the cabinet door opening angle can be analyzed. For example, the angle change trend can be: the cabinet door opening angle increases, the cabinet door opening angle remains unchanged, and the cabinet door opening angle decreases. Here, the cabinet door opening angle remains unchanged can be understood as the angle change trend of the cabinet door opening angle is less than the preset trend threshold.

[0060] Based on this, in an embodiment of the present disclosure, it is possible to determine a collection period during which the angle change trend of the cabinet door opening angle is less than a preset trend threshold value during the cabinet door opening period. Afterwards, it is possible to determine the multiple image frames collected during the collection period in the target video, and then determine the first image frame among the multiple image frames. For example, multiple image frames can be determined as the first image frame, and the middle image frame (or the first image frame, or the last image frame) can also be determined as the first image frame among the multiple image frames. The present disclosure does not specifically limit the number and position of the first image frame in the target video, subject to what can be implemented.

[0061] In the embodiment of the present disclosure, the sensor may be a gyroscope. In addition to the gyroscope, the sensor may also be any sensor capable of detecting the opening angle of the cabinet door, and the present disclosure does not specifically limit this. The following takes the sensor as an example to introduce the working principle of the gyroscope in the display cabinet:

[0062] When the door opening angle of the display cabinet increases, the detection result of the gyroscope changes with the change of the door opening angle; when the door opening angle of the display cabinet remains unchanged, the gyroscope is reset for the first time; when the door opening angle of the display cabinet decreases, the detection result of the gyroscope changes with the change of the door opening angle; after the door of the display cabinet is closed, the gyroscope is reset again.

[0063] Based on this, it can be determined that the first reset period of the gyroscope is a collection period in which the angle change trend of the cabinet door opening angle is less than a preset trend threshold.

[0064] In the above-mentioned implementation, by determining the first image frame within the acquisition period when the angle change trend of the door opening angle is less than the preset trend threshold, an image of the interior of the display cabinet when the door is stable can be obtained. Since the target camera is also in motion when the door of the display cabinet is opened or closed, the image collected during the period when the door is opened or closed cannot accurately reflect the imaging quality of the target camera. At this time, by determining the first image frame from the multiple image frames corresponding to the acquisition period in the target video, a more accurate image blur can be obtained to more accurately reflect the imaging quality of the target camera, thereby improving the accuracy of the defect detection results of the target camera.

[0065] In an optional implementation, the above step S103: extracting the first image frame from the target video specifically includes the following steps:

[0066] Calculate the pixel difference between any adjacent video frames in the target video; determine the continuous video frames whose pixel difference is less than a preset difference threshold in the target video, and determine the first image frame based on the continuous video frames.

[0067] In a specific implementation, each video frame in the continuous video frames may be determined as the first image frame, and the video frame containing the largest number of objects in the continuous video frames may also be determined as the first image frame.

[0068] In an optional embodiment, if Figure 3 As shown, the above step S103: calculating the image blur of the first image frame specifically includes the following steps:

[0069] Step S301: performing fast Fourier transform on the first image frame to obtain a corresponding frequency domain image;

[0070] Step S302: extracting feature data related to blurriness from the frequency domain image;

[0071] Step S303: Calculate the image blur of the first image frame based on the feature data.

[0072] In the disclosed embodiment, the image blurriness of the first image frame may be calculated using a fast Fourier transform (FFT), wherein the blurrier the image, the higher the image blurriness. The specific method is as follows:

[0073] First, the first image frame is processed using the fast Fourier transform FFT to obtain the corresponding frequency domain image. After the first image frame is subjected to the fast Fourier transform, the zero-frequency component located in the upper left corner of the first image frame will be moved to the middle of the image for easy analysis. After that, the low-frequency component can be filtered in the corresponding frequency domain image to obtain the filtered frequency domain image. Next, the filtered frequency domain image can be processed by the reverse displacement algorithm to return the zero-frequency component to the upper left corner. For the filtered frequency domain image, a two-dimensional inverse Fourier transform can be performed on the filtered frequency domain image, and the logarithm of the result of the two-dimensional inverse Fourier transform is taken, and the logarithm is compressed to 0 to 255 to obtain the relevant feature data.

[0074] For the extracted relevant feature data, its mean value can be calculated, and then the image blurriness can be determined according to the mean value. The lower the mean value, the blurrier the image, that is, the higher the image blurriness.

[0075] In the embodiment of the present disclosure, in addition to processing the first image frame by fast Fourier transform FFT, other algorithms may be used to calculate the image blur of the first image frame, such as Laplace transform, sobel operator and other algorithms.

[0076] In the above implementation manner, the image blurriness of the first image frame is determined in the above-described manner, so that the accuracy of the image blurriness can be improved.

[0077] In an optional embodiment, if Figure 4 As shown, for the above step S105: based on the quality detection strategy corresponding to the image blur, the image frame in the target video is subjected to quality detection to obtain a quality detection result, which specifically includes the following steps:

[0078] Step S401: When it is detected that the image blur is greater than the blur threshold, image quality detection is performed on the first image frame to obtain the quality detection result; the image quality detection includes at least one of the following: stain detection, occlusion detection, exposure detection, and hue detection.

[0079] In the embodiment of the present disclosure, a blur threshold may be preset, and then the image blur may be compared with the blur threshold. If the image blur is greater than the blur threshold, it may be determined that the lens of the target camera has a defect. At this time, an image quality test may be performed on the first image frame to obtain a quality test result. Here, the quality test result is used to determine the defect type of the lens of the target camera.

[0080] In a specific implementation, when it is detected that the image blur is greater than the blur threshold, the first image frame can be input into the quality detection model for processing to obtain a quality detection result. For example, the quality detection result is used to indicate defects such as the lens of the target camera is not torn, or the lens of the target camera has stains, or the lens of the target camera is blocked.

[0081] Before the first image frame is processed by the quality detection model, the quality detection model may be trained. The training process is described as follows:

[0082] First, sample images captured by the target camera under various target defects are collected, including stain defects, occlusion defects, exposure defects, and hue defects. For each sample image, a sample label is added to the sample image, where the sample label is used to indicate the defect type corresponding to the corresponding sample image.

[0083] Secondly, a training sample is constructed according to the sample image and the sample label of the sample image, and then the quality detection model to be trained is trained according to the training sample, so as to obtain a quality detection model that meets the training requirements.

[0084] It can be known from the above description that the number of first image frames can be multiple or one. In the case where the number of first image frames is multiple, a sub-quality detection result can be obtained for each first image frame. At this time, the multiple sub-quality detection results can be aggregated to obtain the quality detection result described in the above steps. The specific aggregation process is described as follows:

[0085] Determine the result type of each sub-quality detection result, thereby obtaining at least one result type; and determine the result type that appears most frequently in the at least one result type as the quality detection result described in the above step.

[0086] In the above embodiment, when it is detected that the image blur is greater than the blur threshold, more detailed defect detection can be performed on the target camera by performing at least one of stain detection, occlusion detection, exposure detection, and hue detection on the first image frame to improve the detection accuracy of target camera defect detection.

[0087] In the embodiment of the present disclosure, in addition to performing quality detection on the image frames in the target video in the manner described above, quality detection on the image frames in the target video may also be performed by manual review.

[0088] In a specific implementation, the processor in the display cabinet may send the first image frame to the auditor, so that the auditor can perform a quality audit on the first image frame. Afterwards, the processor in the display cabinet may obtain the quality audit result sent by the auditor, wherein the quality audit result may be: quality audit passed, quality audit failed, and the reason for quality audit failure, such as stains, occlusion, exposure, color tone problems, etc.

[0089] In an optional embodiment, the method provided by the present disclosure further includes the following steps:

[0090] (1) when it is detected that the image blur is greater than the blur threshold, obtaining at least one historical image blur of a historical image frame, wherein the historical image frame is a video frame corresponding to a case where the opening angle of the cabinet door meets the requirement in a historical video collected during the historical opening period of the cabinet door;

[0091] (2) When it is determined that the blurriness of at least one historical image is greater than the blurriness threshold, image quality detection is performed on the image frames in the target video to obtain the quality detection result.

[0092] In the embodiment of the present disclosure, when it is detected that the image blurriness is greater than the blurriness threshold, at least one historical image blurriness of the historical image frame may also be acquired.

[0093] Here, the historical image frame is an image frame determined in the historical video collected during the historical opening of the cabinet door. The process of determining the historical image frame in the historical video is the same as the process of determining the first image frame, which will not be described in detail here.

[0094] After obtaining at least one historical image blur, if it is detected that at least one historical image blur is greater than the blur threshold, it can be determined that there is a quality problem with the first image frame, that is, there is a defect in the lens of the target camera. At this time, the first image frame can be subjected to image quality detection to obtain the quality detection result.

[0095] If it is detected that the blurriness of at least one historical image is less than the blurriness threshold, then it can be determined that the lens of the target camera may not have quality problems. At this time, a video frame that meets the requirements can be determined in the video captured during the next opening of the cabinet door, and whether the target camera has defects can be determined based on the image blurriness of the video frame.

[0096] For example, if a consumer holds a steaming food in his hand, when the food is close to the target camera, a layer of white fog may form at the lens of the target camera, resulting in a large image blur in the image frame of the target video captured by the target camera. However, the formation of white fog is not a defect of the target camera lens itself. Therefore, when the food is far away from the target camera, the target camera can shoot normally.

[0097] In the above implementation, by combining the historical image blur of the historical image frame to determine whether to perform image quality detection on the image frame in the target video, the problem of incorrectly identifying defects of the target camera due to a jump in the quality detection result can be prevented.

[0098] For the above step S105, if Figure 5 As shown, step S105: based on the quality detection strategy corresponding to the image blur, the image frame in the target video is subjected to quality detection to obtain a quality detection result, which specifically includes the following steps:

[0099] Step S501: when it is detected that the image blur is less than the blur threshold, object detection is performed on each image frame of the target video to obtain a first object detection result for each image frame;

[0100] Step S502: determining a second image frame in which the number of objects in the target video meets the requirement based on the first object detection result;

[0101] Step S503: performing image processing on the second image frame to obtain an image processing result, and determining the quality detection result based on the image processing result, wherein the image processing result is used to indicate a shooting angle of the target camera.

[0102] In the embodiment of the present disclosure, when it is detected that the image blur is less than the blur threshold, it can be determined that the lens of the target camera itself does not have defects. At this time, it can be further detected whether there are defects in the shooting angle of the target camera. Here, the shooting angle can be understood as the installation angle of the lens of the target camera on the display cabinet.

[0103] In a specific implementation, the target detection model can be used to perform object detection on each image frame in the target video, thereby obtaining a first object detection result for each image frame, wherein the first object detection result includes the detection results of each object in the corresponding image frame, for example, the bounding box of each object.

[0104] After the first object detection result is obtained, an image frame containing the largest number of objects in the target video may be determined as the second image frame based on the first object detection result.

[0105] After the second image frame is determined, image processing may be performed on the second image frame to obtain an image processing result, and the quality detection result is determined based on the image processing result.

[0106] If there are multiple second image frames, the second image frame with the highest confidence can be selected from the multiple second image frames, and image processing can be performed on the second image frame with the highest confidence to obtain an image processing result, and the quality detection result can be determined based on the image processing result.

[0107] When the target detection model performs object detection on each image frame in the target video, the obtained first object detection result includes the confidence of each object. At this time, the confidence of each object in each second image frame can be summed up to obtain the confidence of the second image frame.

[0108] In the embodiment of the present disclosure, before performing object detection on each image frame in the target video through the target detection model, the target detection model needs to be trained. The specific training process is described as follows:

[0109] First, a sample image is acquired.

[0110] In a specific implementation, sample images can be obtained by shooting with a camera in an offline display cabinet; then, the objects in each sample image can be labeled, for example, a bounding box can be labeled for each object in the sample image.

[0111] Second, data enhancement processing.

[0112] Next, the labeled sample images can be subjected to data enhancement processing, wherein the data enhancement processing may include at least one of the following: randomly modifying the parameters of the HSV (Hue, Saturation, Value) channel, using a mosaic method or other data enhancement processing method, etc.

[0113] Finally, the detection model to be trained is trained through the sample images processed by data enhancement to obtain the target detection model.

[0114] In the above embodiment, when it is detected that the image blur is less than the blur threshold, by selecting the second image frame containing the largest number of objects in the target video and determining the quality detection result based on the image processing result of the second image frame, it is possible to automatically detect whether there are defects in the shooting angle of the target camera, thereby improving the detection accuracy of the target camera defect detection.

[0115] In an optional implementation, the above step S503: performing image processing on the second image frame to obtain an image processing result specifically includes the following steps:

[0116] Step S11: determining the shelf position of the item shelf in the second image frame;

[0117] Step S12: calculating the distance between the shelf edge of the item shelf and the edge of the target image based on the shelf position; wherein the edge of the target image is the edge in the second image frame that is closest to the shelf edge;

[0118] Step S13: when the distance is less than a first preset distance threshold, obtaining a first image processing result; wherein the first image processing result is used to indicate that the shooting angle of the target camera does not cover all the items in the display cabinet.

[0119] Assume that the target camera is installed on the door handle of the display cabinet, that is, the target camera is installed on the left side of the display cabinet. At this time, if the boundary distance between the right boundary of the bounding box of the item shelf in the second image frame and the right boundary of the second image frame is detected to be less than the first preset distance threshold, or if the boundary distance between the right boundary of the bounding box of the item shelf in the second image frame and the left boundary of the second image frame is detected to be greater than the second preset distance threshold, it is considered that the target camera shooting is incomplete, that is, the target camera cannot completely shoot the interior of the display cabinet. If the shooting is incomplete at one time, it is considered that the shooting cannot be complete each time until the shooting mode of the target camera is adjusted.

[0120] Based on this, in the embodiment of the present disclosure, the shelf position of each item shelf in the second image frame can be determined based on the first item detection result of the second image frame. Then, the distance between the shelf edge of the item shelf and the edge of the target image can be calculated based on the shelf position. For example, the distance between the right shelf edge of the item shelf and the right image edge of the second image frame can be calculated.

[0121] Here, the determination of the shelf edge and the target image edge is associated with the installation position of the target camera in the showcase. When the target camera is installed at the left side of the showcase, the shelf edge is the right shelf edge, and the target image edge is the right image edge; when the target camera is installed at the right side of the showcase, the shelf edge is the left shelf edge, and the target image edge is the left image edge.

[0122] When it is determined that the distance between the edge of the shelf and the edge of the target image is less than the first preset distance threshold, a first image processing result is obtained. Here, the first image processing result is used to indicate that the shooting angle of the target camera does not cover the items in the display cabinet.

[0123] In the above implementation, by comparing the distance between the edge of the shelf and the edge of the target image with the first preset distance threshold, it is possible to accurately detect whether the target camera can completely capture each item in the display cabinet.

[0124] In the embodiment of the present disclosure, in addition to detecting that the shooting angle of the target camera does not cover the items in the display case in the manner described above, the following manner may be used to detect that the shooting angle of the target camera does not cover the items in the display case.

[0125] Based on this, in another optional implementation, step S503: performing image processing on the second image frame to obtain an image processing result specifically includes the following steps:

[0126] The length of the shelf boundary box of the item shelf in the second image frame is determined; and the length is processed based on the target mapping relationship to obtain a real distance corresponding to the length.

[0127] Here, the target mapping relationship is used to represent the relationship between the bounding box of each item shelf in the second image frame and the real distance indicated by the bounding box in the real scene.

[0128] After obtaining the real distance, the actual length of the display shelf can be obtained, and the real distance can be compared with the actual length. If the real distance is less than the actual length, it is determined that the shooting angle of the detection target camera does not cover the items in the display cabinet. Otherwise, it can be determined that the shooting angle of the detection target camera can cover the items in the display cabinet.

[0129] In an optional implementation, the above step S503: performing image processing on the second image frame to obtain an image processing result specifically includes the following steps:

[0130] Step S21: performing inversion processing on the second image frame, and performing object detection on the inverted second image frame to obtain a second object detection result;

[0131] Step S22: determining the accuracy of the first object detection result corresponding to the second image frame to obtain a first accuracy, and determining the accuracy of the second object detection result to obtain a second accuracy;

[0132] Step S23: when it is determined that the first accuracy is less than the second accuracy, a second image processing result is obtained; the second image processing result is used to indicate that the target camera is in an inverted state.

[0133] In the disclosed embodiment, when collecting sample images for training the target detection model, the sample images are collected when the camera is placed normally, and the sample images contain multiple objects that are not placed normally. The inventor found that if the image is inverted, the objects in the image will also be inverted. At this time, when the target detection model recognizes the inverted image, the target detection model will not be able to accurately recognize the inverted objects.

[0134] Based on this, in the embodiment of the present disclosure, whether the target camera is in an inverted state can be detected by inverting the second image frame.

[0135] During specific implementation, the second image frame may be inverted, and object detection may be performed on the inverted second image frame through a target detection model to obtain a second object detection result.

[0136] After obtaining the second object detection result, the accuracy of the first object detection result corresponding to the second image frame can be determined to obtain a first accuracy, and the accuracy of the second object detection result can be determined to obtain a second accuracy.

[0137] The calculation formula of the first accuracy is: first accuracy=known number of displayed objects A1 / total number of objects in the display cabinet; wherein the known number of displayed objects A1 is the number of objects detected in the first object detection result.

[0138] The calculation formula of the second accuracy is: second accuracy=known number of displayed objects A2 / total number of objects in the display cabinet; wherein the known number of displayed objects A2 is the number of objects detected in the second object detection result.

[0139] From the above description, it can be seen that since the target detection model is trained with sample images that do not contain inverted objects during the training process, the target detection model has poor detection results for inverted images, and some of the inverted objects will be identified as unknown objects. If the first accuracy is lower than the second accuracy, it means that the target camera is inverted; if the first accuracy is greater than the second accuracy, it means that the target camera is in an upright state.

[0140] Here, the total number of items in the display cabinet may be set as the known displayed commodity A1.

[0141] Assuming that the target camera is not in an inverted state, the value of the first accuracy can be determined to be 1. The second accuracy = the known number of displayed items A2 / the known number of displayed items A1. Since the known number of displayed items A2 is less than the known number of displayed items A1, the second accuracy is less than 1. At this time, it can be determined that the first accuracy is greater than the second accuracy, indicating that the target camera is not in an inverted state.

[0142] Assuming that the target camera is in an inverted state, the value of the first accuracy can be determined to be 1. Second accuracy = known number of displayed items A2 / known number of displayed items A1. Since the known number of displayed items A2 is greater than the known number of displayed items A1, the second accuracy is greater than 1. At this time, it can be determined that the first accuracy is less than the second accuracy, indicating that the target camera is in an inverted state.

[0143] In the above implementation, by inverting the second image frame and detecting the inverted state of the target camera based on the inverted second image frame, it is possible to quickly and accurately detect whether the target camera is in an inverted state, thereby more comprehensively detecting the defects of the target camera.

[0144] In the embodiment of the present disclosure, in addition to detecting that the target camera is in an inverted state in the manner described above, the target camera may also be detected in the following manner.

[0145] Based on this, in another optional implementation, step S503: performing image processing on the second image frame to obtain an image processing result specifically includes the following methods:

[0146] Method 1:

[0147] The second image frame is segmented to obtain image segmentation results of each object in the second image frame; based on the segmented shapes of each object in the image segmentation results, it is determined whether the object in the second image frame is in an inverted state. If it is determined to be yes, it is determined that the target camera is in an inverted state.

[0148] Method 2:

[0149] The display shelf in the second image frame is detected to obtain the shelf position of the item shelf in the second image frame. If the display shelf is determined to be located at the top of the second image frame based on the shelf position and there is no item above the display shelf, it is determined that the target camera is in an inverted state.

[0150] In the embodiment of the present disclosure, after obtaining the quality detection result of the image frame in the target video in the manner described above, the defect detection result of the imaging defect of the target camera can be determined based on the quality detection result, which specifically includes the following steps:

[0151] First, when it is determined based on the quality detection result that the image frame in the target video does not meet the image quality requirement, the image defect type corresponding to the image frame in the target video is determined.

[0152] Here, the image defect type may be: stain defect, occlusion defect, exposure defect, hue defect, image inversion defect, image incomplete defect, etc.

[0153] Step S602: determining a defect detection result of the imaging defect based on the image defect type.

[0154] After determining the above image defect type, the defect detection result of the imaging defect can be determined based on the image defect type. For example, the following defect detection results can be obtained: the camera is installed upside down, the camera does not work properly, the camera film is not torn off, the camera is deliberately blocked, the camera is not fully photographed, the camera is exposed, the camera is damaged, the camera has a color tone problem, and the camera has stains.

[0155] From the above description, it can be seen that by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera according to the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources. By determining the quality inspection strategy based on the image blur and performing quality inspection on the image in the target video according to the quality inspection strategy, the problems existing in the target camera can be fully detected.

[0156] See also Figure 6 As shown, it is a flow chart of another camera imaging defect detection method provided by an embodiment of the present disclosure, such as Figure 6 As shown, the method specifically includes the following processes:

[0157] When detecting that the cabinet door is opened, the target camera is triggered to collect video and the following steps are performed:

[0158] (1) Capture the target video.

[0159] In the disclosed embodiment, the target camera can be installed on the door handle of the cabinet door, and the lens of the target camera is arranged toward the inside of the showcase. When the cabinet door of the showcase is detected to be opened, the target camera starts shooting to obtain the target video.

[0160] Here, the shooting duration can be preset. When the cabinet door of the display cabinet is detected to be opened, the target camera is triggered to shoot a video, and the shooting ends after the shooting duration is satisfied to obtain the target video. The preset shooting duration can be a duration preset by the user. For example, the shooting duration can be selected as 10 seconds. In addition, it can also be set to other durations. The present disclosure does not specifically limit this, and it is subject to what can be implemented.

[0161] (2) Calculate the image blur of the image frames in the target video.

[0162] During specific implementation, a first image frame may be extracted from the target video, and the image blurriness of the first image frame may be calculated.

[0163] The process of extracting the first image frame from the target video is the same as the process described above, and will not be described in detail here. The process of calculating the image blur of the first image frame is the same as the process described above, and will not be described in detail here.

[0164] (3) Compare the image blur with the blur threshold. If the image blur is greater than the blur threshold, execute the following step (4). If the image blur is less than the blur threshold, execute the following steps (5)-(12).

[0165] (4) Image quality detection.

[0166] In specific implementation, image quality detection may be performed on the first image frame to obtain a quality detection result; the image quality detection includes at least one of the following: stain detection, occlusion detection, exposure detection, and hue detection.

[0167] In the disclosed embodiment, if the image blurriness is compared to be greater than the blurriness threshold, an audit reminder may also be sent to the user to remind the user to detect whether the target camera has the following defects: stain defects, occlusion defects, exposure defects, and hue defects.

[0168] When the above defects are detected in the target camera, the maintenance personnel can be notified to perform maintenance on the target camera.

[0169] (5) Performing target detection on the collected target video to obtain a first object detection result.

[0170] During specific implementation, object detection may be performed on each image frame of the target video to obtain a first object detection result for each image frame.

[0171] (6) Determine the image frame containing the largest number of objects based on the first object detection result, i.e., the second image frame.

[0172] In a specific implementation, an image frame containing the largest number of objects may be determined as the second image frame based on the first object detection result.

[0173] (7) Determine a shelf position of the item shelf based on the first item detection result.

[0174] (8) Based on the shelf position, detect whether the item shelf in the second image frame is located at the edge of the image. If it is detected that it is, execute step (9).

[0175] In a specific implementation, the distance between the shelf edge of the item shelf and the edge of the target image can be calculated based on the shelf position; wherein the target image edge is the edge in the second image frame that is closest to the shelf edge; and then, when the distance is less than a first preset distance threshold, it is determined that the item shelf in the second image frame is located at the edge of the image.

[0176] (9) The camera cannot capture the entire target.

[0177] At this time, it is determined that the shooting angle of the target camera does not cover the items in the display cabinet.

[0178] When the above defects are detected in the target camera, the maintenance personnel can be notified to perform maintenance on the target camera.

[0179] (10) Invert the second image frame.

[0180] (11) Perform target detection on the inverted second image frame to obtain a second object detection result.

[0181] During specific implementation, the object detection model may be used to perform object detection on the inverted second image frame to obtain a second object detection result.

[0182] (12) Determine whether the target camera is installed upside down based on the second object detection result.

[0183] During specific implementation, the accuracy of the first object detection result corresponding to the second image frame can be determined to obtain a first accuracy, and the accuracy of the second object detection result can be determined to obtain a second accuracy; when it is determined that the first accuracy is less than the second accuracy, a second image processing result is obtained; the second image processing result is used to indicate that the target camera is in an inverted state.

[0184] When it is detected that the target camera is in an inverted state, the maintenance personnel can be notified to perform maintenance on the target camera.

[0185] (13) The target video was not captured.

[0186] (14) Determine the hardware defect of the target camera, including: damage to the target camera and / or wiring problem of the target camera. At this time, the maintenance personnel can be notified to perform maintenance on the target camera.

[0187] From the above description, it can be seen that by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera according to the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources. By determining the quality inspection strategy based on the image blur and performing quality inspection on the image in the target video according to the quality inspection strategy, the problems existing in the target camera can be fully detected.

[0188] The following will introduce the camera imaging defect detection method in the above-mentioned display cabinet in detail in combination with specific application scenarios. Assume that the display cabinet is an unmanned self-service display cabinet, and the display cabinet can be placed in any environment, for example, a hospital, an office building, a convenience store, a tourist attraction, a school, etc. In specific applications, the user can open the door of the display cabinet in a predetermined manner (for example, scanning a code) and take the items in the display cabinet, so that the display cabinet automatically settles the items taken by the user.

[0189] Assume that the display cabinet is an unmanned self-service display cabinet, and a network camera is pre-installed in the display cabinet, wherein the network camera can be connected to the server network communication. Assume that the camera imaging defect detection method provided by the present disclosure mainly detects the following defects of the target camera: stain detection, occlusion detection, exposure detection, color tone detection, camera inversion detection, detection of whether the shooting angle of the target camera can completely cover the display cabinet, and camera hardware failure detection.

[0190] First, the target video is captured by the target camera, and the image blur of the image frame in the target video is calculated. If the target video is not captured, it can be determined that the camera hardware is faulty.

[0191] If it is detected that the image blur is greater than the blur threshold, the image quality detection is performed on the image frame in the target video. For example, the image frame in the target video (for example, the first image frame) can be input into the quality detection model for processing to obtain a quality detection result. For example, the quality detection result is used to indicate defects such as the lens of the target camera is not torn, or the lens of the target camera is stained, or the lens of the target camera is blocked.

[0192] If it is detected that the image blur is less than the blur threshold, target detection can be performed on the target video to obtain a first object detection result, and the image frame containing the largest number of objects (for example, the second image frame) can be determined based on the first object detection result. After that, it can be detected based on the second image frame whether the target camera can completely cover the detection in the display cabinet, and whether the target camera is in an inverted state.

[0193] After performing camera imaging defect detection on the target camera in the manner described above, a defect detection result can be obtained. At this point, the defect detection result can be sent to the operation and maintenance personnel, and the location of the target camera with defects can also be sent to the operation and maintenance personnel, so that the operation and maintenance personnel can repair and adjust the target camera according to the location. In the embodiment of the present disclosure, by determining the quality detection strategy based on the image blur, and performing quality detection on the image in the target video according to the quality detection strategy, the problems existing in the target camera can be fully detected.

[0194] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0195] Based on the same inventive concept, a camera imaging defect detection device corresponding to the camera imaging defect detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned camera imaging defect detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0196] Reference Figure 7 , which is a schematic diagram of a camera imaging defect detection device provided by an embodiment of the present disclosure, the device comprises: an acquisition unit 10, an extraction unit 20, a calculation unit 30, a quality detection unit 40 and a determination unit 50; wherein,

[0197] An acquisition unit 10 is used to acquire a target video captured by a target camera toward the interior of the showcase while the showcase door is open;

[0198] An extraction unit 20, configured to extract a first image frame from the target video;

[0199] A calculation unit 30 is used to calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the opening of the cabinet door;

[0200] A quality detection unit 40, configured to perform quality detection on the image frames in the target video based on the quality detection strategy corresponding to the image blur to obtain a quality detection result;

[0201] The determination unit 50 is used to determine a defect detection result of the imaging defect of the target camera based on the quality detection result.

[0202] From the above description, it can be seen that by performing quality inspection on the image in the target video and determining the defect detection result of the imaging defect of the target camera according to the quality inspection result, the imaging defect of the camera can be automatically detected, thereby improving the quality inspection efficiency of the camera in the display cabinet and saving a lot of human resources. By determining the quality inspection strategy based on the image blur and performing quality inspection on the image in the target video according to the quality inspection strategy, the problems existing in the target camera can be fully detected.

[0203] In a possible implementation, the quality detection unit is further used to: when it is detected that the image blur is greater than a blur threshold, perform image quality detection on the first image frame to obtain the quality detection result; the image quality detection includes at least one of the following: stain detection, occlusion detection, exposure detection, and hue detection.

[0204] In a possible implementation, the device is also used to: when it is detected that the image blur is greater than the blur threshold, obtain at least one historical image blur of the historical image frame, wherein the historical image frame is a video frame corresponding to the time when the opening angle of the cabinet door meets the requirement in the historical video collected during the historical opening period of the cabinet door; when it is determined that the at least one historical image blur is greater than the blur threshold, perform image quality detection on the image frame in the target video to obtain the quality detection result.

[0205] In a possible implementation, the quality detection unit is further used to: when it is detected that the image blur is less than a blur threshold, perform object detection on each image frame of the target video to obtain a first object detection result for each image frame; determine a second image frame in which the number of objects in the target video meets the requirement based on the first object detection result; perform image processing on the second image frame to obtain an image processing result, and determine the quality detection result based on the image processing result, wherein the image processing result is used to indicate the shooting angle of the target camera.

[0206] In a possible implementation, the quality detection unit is further used to: determine the shelf position of the item shelf in the second image frame; calculate the distance between the shelf edge of the item shelf and the target image edge based on the shelf position; wherein the target image edge is the edge in the second image frame that is closest to the shelf edge; and obtain a first image processing result when the distance is less than a first preset distance threshold; wherein the first image processing result is used to indicate that the shooting angle of the target camera does not cover each item in the display cabinet.

[0207] In a possible implementation, the quality detection unit is further used to: invert the second image frame, and perform object detection on the inverted second image frame to obtain a second object detection result; determine the accuracy of the first object detection result corresponding to the second image frame to obtain a first accuracy, and determine the accuracy of the second object detection result to obtain a second accuracy; when it is determined that the first accuracy is less than the second accuracy, obtain a second image processing result; the second image processing result is used to indicate that the target camera is in an inverted state.

[0208] In a possible implementation, the determination unit is further used to: determine the image defect type corresponding to the image frame in the target video when it is determined based on the quality detection result that the image frame in the target video does not meet the image quality requirements; and determine the defect detection result of the imaging defect based on the image defect type.

[0209] In a possible implementation, the device is further used to: generate defect reminder information of the target camera when the target video is not captured while the cabinet door is open, wherein the defect reminder information is used to indicate a hardware defect of the target camera.

[0210] In a possible implementation, the extraction unit is further used to: obtain the door opening angle collected by a sensor pre-installed in the display cabinet during the period when the door of the display cabinet is open; analyze the angle change trend of the door opening angle; determine a collection time period when the angle change trend is less than a preset trend threshold during the period when the door is open; and determine the first image frame from a plurality of image frames corresponding to the collection time period in the target video.

[0211] In a possible implementation, the computing unit is further used to: perform a fast Fourier transform on the first image frame to obtain a corresponding frequency domain image; extract blur-related feature data from the frequency domain image; and calculate the image blur of the first image frame based on the feature data.

[0212] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0213] Reference Figure 8 , which is a schematic diagram of the structure of a display cabinet provided by an embodiment of the present disclosure, the display cabinet includes: a display cabinet body 111 , a target camera 112 and a processor 113 .

[0214] The target camera 112 is installed on the showcase body 111 , and the lens of the target camera 112 faces the interior of the showcase body.

[0215] The target camera 112 is configured to capture a target video inside the display cabinet when the cabinet door is open;

[0216] The processor 113 is configured to extract a first image frame from the target video and calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the opening of the cabinet door; based on a quality detection strategy corresponding to the image blur, perform quality detection on the image frame in the target video to obtain a quality detection result; and determine a defect detection result of the imaging defect of the target camera based on the quality detection result.

[0217] Corresponds to Figure 1 The camera imaging defect detection method in the present disclosure also provides a display cabinet 900, such as Fig. 9 FIG. 1 is a schematic diagram of the structure of a display cabinet 900 provided in an embodiment of the present disclosure, including:

[0218] Processor 91, memory 92, and bus 93; memory 92 is used to store execution instructions, including internal memory 921 and external memory 922; the internal memory 921 is also called internal memory, which is used to temporarily store the operation data in the processor 91 and the data exchanged with the external memory 922 such as a hard disk. The processor 91 exchanges data with the external memory 922 through the internal memory 921. When the display cabinet 900 is running, the processor 91 communicates with the memory 92 through the bus 93, so that the processor 91 executes the following instructions:

[0219] Acquire a target video obtained by shooting the interior of the showcase with a target camera while the showcase door is open;

[0220] Extracting a first image frame from the target video, and calculating the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the opening of the cabinet door;

[0221] Based on the quality detection strategy corresponding to the image blur, performing quality detection on the image frames in the target video to obtain a quality detection result;

[0222] A defect detection result of the imaging defect of the target camera is determined based on the quality detection result.

[0223] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the camera imaging defect detection method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0224] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the camera imaging defect detection method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0225] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0226] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0229] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0230] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A camera imaging defect detection method, It is characterized in that include: Acquire a target video obtained by shooting the interior of the showcase with a target camera while the showcase door is open; Extracting a first image frame from the target video, and calculating the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the opening of the cabinet door; Based on the quality detection strategy corresponding to the image blur, performing quality detection on the image frames in the target video to obtain a quality detection result; Determine a defect detection result of the imaging defect of the target camera based on the quality detection result; The quality detection strategy based on the image blur corresponding to the image blur is used to perform quality detection on the image frame in the target video to obtain a quality detection result, including: In the case where it is detected that the image blur is greater than the blur threshold, obtaining at least one historical image blur of a historical image frame, wherein the historical image frame is a video frame corresponding to when the opening angle of the cabinet door meets the requirement in a historical video collected during the historical opening period of the cabinet door; When it is determined that the blurriness of the at least one historical image is greater than the blurriness threshold, image quality detection is performed on the image frames in the target video to obtain the quality detection result.

2. The method according to claim 1, It is characterized in that The quality detection strategy based on the image blur corresponding to the image blur is used to perform quality detection on the image frame in the target video to obtain a quality detection result, including: When it is detected that the image blur is greater than the blur threshold, image quality detection is performed on the first image frame to obtain the quality detection result; the image quality detection includes at least one of the following: stain detection, occlusion detection, exposure detection, and hue detection.

3. The method according to claim 1, It is characterized in that The quality detection strategy based on the image blur corresponding to the image blur is used to perform quality detection on the image frame in the target video to obtain a quality detection result, including: When it is detected that the image blur is less than the blur threshold, performing object detection on each image frame of the target video to obtain a first object detection result for each image frame; Determine, based on the first object detection result, a second image frame in which the number of objects in the target video meets the requirement; Performing image processing on the second image frame to obtain an image processing result, and determining the quality detection result based on the image processing result, wherein the image processing result is used to indicate a shooting angle of the target camera; The performing image processing on the second image frame to obtain an image processing result includes: determining a shelf position of an item shelf in the second image frame; Calculating the distance between the shelf edge of the item shelf and the edge of the target image based on the shelf position; wherein the edge of the target image is the edge in the second image frame that is closest to the shelf edge; When the distance is less than a first preset distance threshold, a first image processing result is obtained; wherein the first image processing result is used to indicate that the shooting angle of the target camera does not cover each item in the display cabinet; The performing image processing on the second image frame to obtain an image processing result includes: Inverting the second image frame, and performing object detection on the inverted second image frame to obtain a second object detection result; Determining the accuracy of the first object detection result corresponding to the second image frame to obtain a first accuracy, and determining the accuracy of the second object detection result to obtain a second accuracy; When it is determined that the first accuracy is less than the second accuracy, a second image processing result is obtained; the second image processing result is used to indicate that the target camera is in an inverted state.

4. The method according to claim 1, It is characterized in that The step of determining the defect detection result of the imaging defect of the target camera based on the quality detection result includes: In a case where it is determined based on the quality detection result that the image frame in the target video does not meet the image quality requirement, determining an image defect type corresponding to the image frame in the target video; A defect detection result of the imaging defect is determined based on the image defect type.

5. The method according to claim 1, It is characterized in that The method further comprises: When the target video is not captured during the opening of the door of the display cabinet, defect reminder information of the target camera is generated, wherein the defect reminder information is used to indicate a hardware defect of the target camera.

6. The method according to claim 1, It is characterized in that Extracting the first image frame from the target video includes: During the opening of the door of the display cabinet, obtaining the door opening angle collected by a sensor pre-installed in the display cabinet; Analyze the angle change trend of the cabinet door opening angle; Determine a collection period during which the angle change trend is less than a preset trend threshold value during the cabinet door opening period; The first image frame is determined among a plurality of image frames corresponding to the acquisition period in the target video.

7. The method according to claim 1, It is characterized in that The calculating the image blur of the first image frame comprises: Performing a fast Fourier transform on the first image frame to obtain a corresponding frequency domain image; Extracting feature data related to blurriness in the frequency domain image; An image blurriness of the first image frame is calculated based on the feature data.

8. A display cabinet, It is characterized in that include: A display cabinet body, a target camera and a processor, wherein the target camera is mounted on the display cabinet body, and a lens of the target camera faces the interior of the display cabinet body; The target camera is configured to capture a target video inside the display cabinet when the cabinet door is open; The processor is configured to extract a first image frame from the target video and calculate the image blur of the first image frame; wherein the first image frame is a video frame corresponding to the target video at the moment when the opening angle of the cabinet door meets the requirement during the cabinet door opening period; based on the quality detection strategy corresponding to the image blur, perform quality detection on the image frame in the target video to obtain a quality detection result; and determine a defect detection result of the imaging defect of the target camera based on the quality detection result; The processor is used to perform quality detection on the image frame in the target video based on the quality detection strategy corresponding to the image blur to obtain a quality detection result, and the processor is used to: In the case where it is detected that the image blur is greater than the blur threshold, obtaining at least one historical image blur of a historical image frame, wherein the historical image frame is a video frame corresponding to when the opening angle of the cabinet door meets the requirement in a historical video collected during the historical opening period of the cabinet door; When it is determined that the blurriness of the at least one historical image is greater than the blurriness threshold, image quality detection is performed on the image frames in the target video to obtain the quality detection result.

9. A display cabinet, It is characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the camera imaging defect detection method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the camera imaging defect detection method as described in any one of claims 1 to 7 are executed.

Images