Camera anomaly detection method, system and device and medium

Abstract

The invention relates to the technical field of equipment recovery evaluation, and discloses a camera anomaly detection method, system and equipment and a medium, the motion change between video frames of a to-be-detected camera is accurately detected through video jitter detection methods such as an optical flow method, a feature matching method and an energy method, and accurate video shooting function anomaly detection can be carried out on the to-be-detected camera. When a mobile phone, a notebook computer and other devices are recycled, whether the camera shakes or not can be directly detected to adjust the recycling price, so that the detection accuracy of the loss degree of the to-be-recycled device can be improved, and the loss caused by recycling the to-be-recycled device without detecting the camera function shaking abnormity can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment recycling assessment, and in particular to a camera anomaly detection method, system, equipment and medium. Background Art

[0002] The country's economic growth and technological progress have accelerated the popularization and replacement of smart devices. Recycling devices for mobile phones, notebooks, etc. will recycle discarded mobile phones, notebooks, etc. for secondary use to reduce chemical pollution to the environment and protect the environment. If the camera abnormality cannot be detected when recycling mobile phones, notebooks, etc., it will have a great impact on gross profit.

[0003] Existing methods for detecting abnormalities in camera shooting functions include: triggering shooting through commands inside smart devices such as mobile phones and notebooks while capturing the device's operating log, and then analyzing the log features. However, this method may fail to capture logs or have no relevant log output when the camera is shooting, resulting in misjudgment; using the API publicly available from the smart device system manufacturer to compile and generate an APP, and then installing it on the device, and judging whether the camera's video shooting function is abnormal based on the API return value. However, this method requires installing an APP on the device, which is too dependent on the API provided by the device system manufacturer, and is not suitable for application scenarios where APP installation is not allowed, and there is a situation where the camera shooting function returns normal even though it is abnormal; manually shooting and then subjectively judging whether the camera shooting function is normal through the human eye, but this method has high labor costs, low efficiency, and the accuracy cannot be effectively guaranteed.

[0004] Therefore, a new camera anomaly detection method is urgently needed. Summary of the invention

[0005] The present invention provides a camera abnormality detection method, system, device and medium, which are used to solve the defect that the existing camera shooting function abnormality detection method has poor accuracy, which affects the recycling evaluation of the device carrying the camera.

[0006] The present invention provides a camera abnormality detection method, comprising:

[0007] Obtain video data of static shooting taken by the camera under test;

[0008] According to the video data of static shooting taken by the camera to be tested, using a video jitter detection method, determine whether the camera to be tested has video jitter, wherein the video jitter detection method includes any one of the following or any combination thereof: optical flow method, feature matching method, energy method;

[0009] When the camera under test has video jitter, it is determined that the video shooting function of the camera under test is abnormal.

[0010] According to a camera abnormality detection method provided by the present invention, the method uses a video jitter detection method to determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test, including:

[0011] According to the video data of static shooting taken by the camera to be tested, the dense optical flow between two consecutive frames in the video data is obtained by using the optical flow algorithm, the optical flow matrix is ​​obtained, and the matrix norm is obtained according to the optical flow matrix;

[0012] When the matrix norm is less than the first preset threshold, it is determined that the camera to be tested has video jitter.

[0013] According to a camera anomaly detection method provided by the present invention, the optical flow algorithm is a Gunnar Farneback algorithm (dense optical flow algorithm).

[0014] According to a camera abnormality detection method provided by the present invention, the method uses a video jitter detection method to determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test, including:

[0015] According to the feature matching method, the following operations are performed on two consecutive frames of video data taken by the camera under test:

[0016] Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively;

[0017] Based on the two grayscale images obtained by the conversion, the image center position contours of the two grayscale images are obtained respectively, and the image features of the two grayscale images are constructed;

[0018] The image features of the two grayscale images are matched, and based on the matching results, it is determined whether the camera under test has video jitter.

[0019] According to a camera anomaly detection method provided by the present invention, the image features include any one of the following items or any combination thereof: the number of edges of the contour at the center position of the image, the contour area of ​​the contour at the center position of the image, and the absolute value obtained by the center coordinates of the contour at the center position of the image.

[0020] According to a camera anomaly detection method provided by the present invention, the image features of two grayscale images are matched, and according to the matching situation, it is determined whether there is video jitter in the camera to be tested, including:

[0021] The image features of the two grayscale images are matched. When the matching situation indicates that the preset normal situation is not met, it is determined that the camera under test has video jitter. The preset normal situation includes any one of the following or any combination thereof:

[0022] The number of edges of the contours at the center of the image of the two grayscale images is equal;

[0023] The difference in contour area between the contours at the center positions of the two grayscale images is within a preset second range;

[0024] The interpolation of the absolute values ​​of the two grayscale images obtained by the center coordinates of the contours at the center positions of the images is within a preset third range.

[0025] According to a camera abnormality detection method provided by the present invention, the method uses a video jitter detection method to determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test, including:

[0026] According to the energy method, the following operations are performed on two consecutive frames of video data taken by the camera under test:

[0027] Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively;

[0028] According to the grayscale image obtained by conversion, the image gradients of the two grayscale images are obtained by gradient algorithm respectively;

[0029] According to the image gradients of the two grayscale images, the difference between the image gradients of the two grayscale images is obtained;

[0030] When the difference between the image gradients of the two grayscale images is greater than a fourth preset threshold, it is determined that the camera to be tested has video jitter.

[0031] According to a camera abnormality detection method provided by the present invention, the method uses a video jitter detection method to determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test, including:

[0032] According to the video data of static shooting taken by the camera to be tested, the optical flow method, the feature matching method, and the energy method are used to obtain a first determination result, a second determination result, and a third determination result for determining whether the camera to be tested has video jitter;

[0033] The first determination result, the second determination result, and the third determination result are combined to obtain a final determination result of whether the camera to be tested has video jitter.

[0034] The present invention also provides an equipment recycling method, comprising:

[0035] Obtain video data of static shots of the equipment to be recycled;

[0036] According to the video data of static shooting of the equipment to be recycled, a camera abnormality detection result is obtained through any of the camera abnormality detection methods described above to determine whether the video shooting function of the equipment to be recycled is abnormal;

[0037] According to the camera anomaly detection result indicating whether the device to be recycled has an abnormality in the video shooting function, a recycling estimate and / or inspection status of the device to be recycled is obtained.

[0038] The present invention also provides a camera anomaly detection system, comprising:

[0039] A data acquisition module is used to: acquire video data of static shooting taken by the camera to be tested;

[0040] The video jitter detection module is used to: determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test by using a video jitter detection method, wherein the video jitter detection method includes any one of the following or any combination thereof: optical flow method, feature matching method, energy method;

[0041] The determination module is used to determine that the video shooting function of the camera under test is abnormal when the camera under test has video jitter.

[0042] The present invention also provides an electronic device, comprising a processor and a memory storing a computer program, wherein the processor implements any of the above-mentioned camera abnormality detection methods and / or equipment recovery methods when executing the computer program.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the camera anomaly detection method and / or device recovery method described above is implemented.

[0044] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute any one of the above-mentioned camera abnormality detection methods and / or equipment recovery methods.

[0045] The present invention provides a camera anomaly detection method, system, device and medium, which can accurately detect the motion changes between video frames of the camera to be tested through video jitter detection methods such as optical flow method, feature matching method, energy method, etc., and can accurately detect the abnormality of the video shooting function of the camera to be tested. When recycling mobile phones, notebooks and other devices, it is possible to directly detect whether the camera is shaking to adjust the recycling price, which can not only improve the detection accuracy of the loss degree of the equipment to be recycled, but also avoid the loss caused by recycling it without detecting the abnormal jitter of the camera function. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a process of a camera abnormality detection method provided by the present invention;

[0047] Figure 2A schematic diagram of the structure of a camera anomaly detection system provided by the present invention;

[0048] Figure 3 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0050] Figure 1 The flowchart of a camera anomaly detection method provided by the present invention is shown in FIG. The execution subject of the camera anomaly detection method provided by the present invention can be any applicable terminal side device or network side device, such as a camera anomaly detection device.

[0051] See also Figure 1 The present invention provides a camera abnormality detection method, which may include:

[0052] S110, obtaining video data of static shooting taken by the camera to be tested.

[0053] Specifically, the video data of static shooting taken by the camera to be tested may be video data shot by a mobile phone, notebook or other device using a camera lens. For example, the device may be fixed first so that the camera lens faces a static focus test card, and then the device camera is turned on to shoot a video for a period of x seconds (5 seconds in this embodiment), and the video is saved as a file such as video.mov (video formats include avi, mp4, mov, etc.), to form the video data of static shooting taken by the camera to be tested.

[0054] This embodiment uses a camera in a stationary state to shoot a stationary object (focus test card), so the captured video screen should be stationary and unchanged. When using the camera to be tested to shoot static video data, the video file video.mov can be opened by the OpenCV software. Because the opening and closing of the camera may cause slight vibrations, there may be systematic jitters within 2 seconds after the video starts shooting and before the video ends. Therefore, this embodiment only reads the video frames from the 2nd to the 3rd second of the video file of the video.mov as the video data of the static shooting shot by the camera to be tested, so as to improve the detection accuracy.

[0055] S120. Determine whether the camera under test has video jitter based on the static video data shot by the camera under test by using a video jitter detection method, wherein the video jitter detection method includes any one of the following or any combination thereof: an optical flow method, a feature matching method, and an energy method.

[0056] By calculating the optical flow between adjacent frames, the motion changes in the video can be detected. If the change in optical flow exceeds a certain threshold, it can be considered that the video is jittery. The specific operations of the optical flow method include:

[0057] According to the video data of static shooting taken by the camera to be tested, a dense optical flow between two consecutive frames in the video data is obtained by an optical flow algorithm (the present embodiment is a GunnarFarneback algorithm), an optical flow matrix is ​​obtained, and a matrix norm is obtained according to the optical flow matrix;

[0058] When the matrix norm is less than a first preset threshold (100 in this embodiment), it is determined that the camera to be tested has video jitter.

[0059] In order to increase the detection speed and shorten the detection time, two consecutive frames can be randomly selected without traversing all video frames in the video data of static shooting taken by the camera to be tested. When the matrix norm between any two consecutive frames is less than the first preset threshold, it is determined that the camera to be tested has video jitter.

[0060] Based on the image features of the focus test chart, some feature points can be extracted in each frame, and then points matching these feature points can be found in the next frame. If the number or position of the matching points changes significantly, it can be considered that the video is jittery.

[0061] Among them, the specific operations of the feature matching method include:

[0062] According to the feature matching method, the following operations are performed on two consecutive frames of video data taken by the camera under test:

[0063] Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively;

[0064] Based on the two grayscale images obtained by the conversion, the image center position contours of the two grayscale images are obtained respectively, and the image features of the two grayscale images are constructed, wherein the image features include any one of the following items or any combination thereof: the number of edges of the image center position contour, the contour area of ​​the image center position contour, and the absolute value obtained by the center coordinates of the image center position contour;

[0065] The image features of the two grayscale images are matched. When the matching situation indicates that the preset normal situation is not met, it is determined that the camera under test has video jitter. The preset normal situation includes any one of the following or any combination thereof:

[0066] The number of edges of the contours at the center of the image of the two grayscale images is equal;

[0067] The difference in contour area between the contours at the center positions of the two grayscale images is within a preset second range;

[0068] The interpolation of the absolute values ​​of the two grayscale images obtained by the center coordinates of the contours at the center positions of the images is within a preset third range.

[0069] By comparing the gradient differences between consecutive frames, the motion changes in the video can be detected. If the gradient difference exceeds a certain threshold, it can be considered that the video is shaking. The specific operations of the energy method include:

[0070] According to the energy method, the following operations are performed on two consecutive frames of video data taken by the camera under test:

[0071] Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively;

[0072] According to the grayscale image obtained by conversion, the image gradients of the two grayscale images are obtained by gradient algorithms (the present embodiment uses the Sobel operator), wherein the Sobel operator can be used to calculate the gradients of the grayscale image in the horizontal and vertical directions respectively, and then the image gradient of the grayscale image is obtained: gradient = sqrt(grad_x*grad_x+grad_y*grad_y), grad_x is the gradient in the horizontal direction, and grad_y is the gradient in the vertical direction;

[0073] According to the image gradients of the two grayscale images, the difference between the image gradients of the two grayscale images is obtained;

[0074] When the difference between the image gradients of the two grayscale images is greater than a fourth preset threshold, it is determined that the camera to be tested has video jitter.

[0075] When the camera abnormality detection method provided by the present invention is actually applied, any one of the above three methods can be used to detect the abnormality of the camera shooting function, or the detection results of the three methods can be combined to obtain the final abnormality of the camera shooting function. For example, based on the video data of static shooting taken by the camera to be tested, the optical flow method, the feature matching method, and the energy method can be used to obtain the first determination result, the second determination result, and the third determination result of determining whether the camera to be tested has video jitter; based on the weight ratio of the first determination result, the second determination result, and the third determination result (the weight ratio can be set according to the actual situation), the final determination result of whether the camera to be tested has video jitter is obtained.

[0076] S130: When the camera to be tested has video jitter, it is determined that the video shooting function of the camera to be tested is abnormal.

[0077] The camera anomaly detection method provided by the present invention can be applied to equipment recycling to form an equipment recycling method, which may include:

[0078] Obtain video data of static shots of the equipment to be recycled;

[0079] According to the video data of static shooting of the equipment to be recycled, a camera abnormality detection result is obtained through any of the camera abnormality detection methods described above to determine whether the video shooting function of the equipment to be recycled is abnormal;

[0080] According to the camera anomaly detection result indicating whether the device to be recycled has an abnormality in the video shooting function, a recycling estimate and / or inspection status of the device to be recycled is obtained.

[0081] After recycling mobile phones, laptops and other devices, professional quality inspection colleagues can also provide verification feedback on the camera anomaly detection method provided by the present invention when conducting manual quality inspection, and then modify and adjust it to improve the accuracy.

[0082] The present invention provides a camera anomaly detection method, system, device and medium, which can accurately detect the motion changes between video frames of the camera to be tested through video jitter detection methods such as optical flow method, feature matching method, energy method, etc., and can accurately detect the abnormality of the video shooting function of the camera to be tested. When recycling mobile phones, notebooks and other devices, it is possible to directly detect whether the camera is shaking to adjust the recycling price, which can not only improve the detection accuracy of the loss degree of the equipment to be recycled, but also avoid the loss caused by recycling it without detecting the abnormal jitter of the camera function.

[0083] The camera anomaly detection system provided by the present invention is described below. The camera anomaly detection system described below and the camera anomaly detection method described above can be referenced to each other.

[0084] See also Figure 2 , a camera anomaly detection system may include:

[0085] A data acquisition module is used to: acquire video data of static shooting taken by the camera to be tested;

[0086] The video jitter detection module is used to: determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test by using a video jitter detection method, wherein the video jitter detection method includes any one of the following or any combination thereof: optical flow method, feature matching method, energy method;

[0087] The determination module is used to determine that the video shooting function of the camera under test is abnormal when the camera under test has video jitter.

[0088] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute any of the above-mentioned camera abnormality detection methods and / or device recovery methods.

[0089] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0090] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute any of the camera abnormality detection methods and / or equipment recovery methods described above.

[0091] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned camera anomaly detection methods and / or equipment recovery methods.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A camera anomaly detection method, characterized in that: include: Obtain video data of static shooting taken by the camera under test; According to the video data of static shooting taken by the camera to be tested, using a video jitter detection method, determine whether the camera to be tested has video jitter, wherein the video jitter detection method includes any one of the following or any combination thereof: optical flow method, feature matching method, energy method; When the camera under test has video jitter, it is determined that the video shooting function of the camera under test is abnormal.

2. The camera anomaly detection method according to claim 1, characterized in that: The method of determining whether the camera to be tested has video jitter by using a video jitter detection method based on the video data of static shooting taken by the camera to be tested comprises: According to the video data of static shooting taken by the camera to be tested, the dense optical flow between two consecutive frames in the video data is obtained by using the optical flow algorithm, the optical flow matrix is ​​obtained, and the matrix norm is obtained according to the optical flow matrix; When the matrix norm is less than the first preset threshold, it is determined that the camera to be tested has video jitter.

3. The camera anomaly detection method according to claim 1, characterized in that: The method of determining whether the camera to be tested has video jitter by using a video jitter detection method based on the video data of static shooting taken by the camera to be tested comprises: According to the feature matching method, the following operations are performed on two consecutive frames of video data taken by the camera under test: Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively; Based on the two grayscale images obtained by the conversion, the image center position contours of the two grayscale images are obtained respectively, and the image features of the two grayscale images are constructed; The image features of the two grayscale images are matched, and based on the matching results, it is determined whether the camera under test has video jitter.

4. The camera anomaly detection method according to claim 3, characterized in that: The image features include any one of the following or any combination thereof: the number of edges of the contour at the center of the image, the contour area of ​​the contour at the center of the image, and the absolute value obtained by the center coordinates of the contour at the center of the image; and The matching of the image features of the two grayscale images and determining whether the camera to be tested has video jitter according to the matching situation include: The image features of the two grayscale images are matched. When the matching situation indicates that the preset normal situation is not met, it is determined that the camera under test has video jitter. The preset normal situation includes any one of the following or any combination thereof: The number of edges of the contours at the center of the image of the two grayscale images is equal; The difference in contour area between the contours at the center positions of the two grayscale images is within a preset second range; The interpolation of the absolute values ​​of the two grayscale images obtained by the center coordinates of the contours at the center positions of the images is within a preset third range.

5. The camera anomaly detection method according to claim 1, characterized in that: The method of determining whether the camera to be tested has video jitter by using a video jitter detection method based on the video data of static shooting taken by the camera to be tested comprises: According to the energy method, the following operations are performed on two consecutive frames of video data taken by the camera under test: Convert two consecutive frames of video data captured by the camera under test into grayscale images respectively; According to the grayscale image obtained by conversion, the image gradients of the two grayscale images are obtained by gradient algorithm respectively; According to the image gradients of the two grayscale images, a difference between the image gradients of the two grayscale images is obtained; when the difference between the image gradients of the two grayscale images is greater than a fourth preset threshold, it is determined that the camera to be tested has video jitter.

6. The camera anomaly detection method according to any one of claims 1 to 5, characterized in that: The method of determining whether the camera to be tested has video jitter by using a video jitter detection method based on the video data of static shooting taken by the camera to be tested comprises: According to the video data of static shooting taken by the camera to be tested, the optical flow method, the feature matching method, and the energy method are used to obtain a first determination result, a second determination result, and a third determination result for determining whether the camera to be tested has video jitter; The first determination result, the second determination result, and the third determination result are combined to obtain a final determination result of whether the camera to be tested has video jitter.

7. A method for recycling equipment, characterized in that: include: Obtain video data of static shots of the equipment to be recycled; According to the video data of static shooting of the equipment to be recycled, a camera abnormality detection result of whether the video shooting function of the equipment to be recycled is abnormal is obtained by the camera abnormality detection method according to any one of claims 1 to 6; According to the camera anomaly detection result indicating whether the device to be recycled has an abnormality in the video shooting function, a recycling estimate and / or inspection status of the device to be recycled is obtained.

8. A camera anomaly detection system, characterized in that: include: A data acquisition module is used to: acquire video data of static shooting taken by the camera to be tested; The video jitter detection module is used to: determine whether the camera under test has video jitter according to the video data of static shooting taken by the camera under test by using a video jitter detection method, wherein the video jitter detection method includes any one of the following or any combination thereof: optical flow method, feature matching method, energy method; The determination module is used to determine that the video shooting function of the camera under test is abnormal when the camera under test has video jitter.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the camera abnormality detection method according to any one of claims 1 to 6 and / or the equipment recovery method according to claim 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the camera abnormality detection method according to any one of claims 1 to 6 and / or the equipment recovery method according to claim 7 are implemented.

Images