Picture freezing judgment method and device, electronic equipment and storage medium

Abstract

The invention discloses a picture freezing judgment method and device, electronic equipment and a storage medium. The method comprises the steps that a light supplementing lamp is controlled to carry out light supplementing according to a preset light supplementing change rule; determining an image processing parameter when a light supplement change return-to-zero condition is satisfied, and controlling a shooting device to carry out image processing according to the image processing parameter; wherein the shooting equipment is fixed-point shooting equipment or moving-point shooting equipment; and determining a to-be-analyzed image output by the shooting equipment after image processing in the light supplementing process of the light supplementing lamp, and judging whether the image of the shooting equipment is frozen according to the to-be-analyzed image. According to the invention, whether the picture of the shooting equipment is frozen or not can be accurately judged.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic equipment and storage medium for determining whether a picture is frozen. Background Art

[0002] Screen freezing means that the image output by the shooting device is paused on a certain frame and the display content is no longer updated in real time. The main reason for screen freezing is that the main control of the shooting device is overloaded or crashed. However, in certain specific scenarios, under the premise that the shooting device is working normally, the scene of the image screen may not change, or only the display time in the image may change. For example, in the scenario where the instrument is monitored by a card camera, if the instrument has not changed for a long time, the image output by the card camera will also remain unchanged for a long time. For another example, in the scenario of forest fire prevention at night using a pan-tilt camera, when there is no fire point, the image captured when the pan-tilt camera stays at a preset position may also remain unchanged.

[0003] The current prior art lacks a technical solution for determining whether the image freeze is caused by an overload or crash of the main control of the shooting device or by an unchanged shooting scene of the shooting device when the image remains unchanged for a long time. Summary of the invention

[0004] The present invention provides a method, device, electronic equipment and storage medium for judging whether a picture is frozen, so as to accurately judge whether a picture of a shooting device is frozen.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining whether a picture is frozen, the method comprising:

[0006] Control the fill light to fill in light according to the preset fill light change rule;

[0007] Determine image processing parameters when the fill light change zeroing condition is met, and control the shooting device to perform image processing according to the image processing parameters;

[0008] Wherein, the shooting device is a fixed-point shooting device or a dynamic-point shooting device;

[0009] An image to be analyzed output by the shooting device after image processing during the fill light filling process is determined, and whether the shooting device is frozen is determined according to the image to be analyzed.

[0010] In a second aspect, an embodiment of the present invention further provides another method for determining whether a picture is frozen, the method comprising:

[0011] Controlling the moving point shooting device to rotate in the horizontal direction and / or vertical direction;

[0012] An image to be analyzed output by the moving-point shooting device is determined, and whether the moving-point shooting device is frozen is determined according to the image to be analyzed.

[0013] In a third aspect, an embodiment of the present invention further provides a device for determining whether a picture is frozen, the device comprising:

[0014] A fill light control module is used to control the fill light to perform fill light according to a preset fill light change rule;

[0015] An image processing parameter control module, used to determine the image processing parameters when the fill light change zeroing condition is met, and control the shooting device to perform image processing according to the image processing parameters;

[0016] Wherein, the shooting device is a fixed-point shooting device or a dynamic-point shooting device;

[0017] The first screen freeze judgment module is used to determine the image to be analyzed output by the shooting device after image processing during the fill light filling process, and judge whether the shooting device has a screen freeze according to the image to be analyzed.

[0018] In a fourth aspect, an embodiment of the present invention further provides another device for determining whether a picture is frozen, the device comprising:

[0019] A pan / tilt control module, used to control the moving point shooting device to rotate in the horizontal direction and / or vertical direction;

[0020] The second screen freeze judgment module is used to determine the image to be analyzed output by the moving point shooting device, and judge whether the screen of the moving point shooting device is frozen according to the image to be analyzed.

[0021] In a fifth aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for determining whether a screen is frozen as described in any one of the embodiments of the present invention is implemented.

[0022] In a sixth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the method for determining whether the screen is frozen as described in any one of the embodiments of the present invention.

[0023] The technical solution of the embodiment of the present invention controls the fill light to perform fill light according to the fill light change rule. During the fill light process, the camera is controlled to perform image processing according to the image processing parameters when the fill light change returns to zero, and the camera is judged whether the image is frozen according to the image to be analyzed output by the camera. The problem in the prior art that when the image does not change for a long time, it is impossible to accurately judge whether it is caused by the freezing of the camera or the unchanged shooting scene, and the accurate judgment of whether the image of the camera is frozen is achieved.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a method for determining whether a picture is frozen provided by the first embodiment of the present invention;

[0027] Figure 2 is a flow chart of a method for determining whether a picture is frozen provided by Embodiment 2 of the present invention;

[0028] Figure 3 is a flow chart of another method for determining whether a picture is frozen provided by Embodiment 3 of the present invention;

[0029] Figure 4 It is a structural schematic diagram of a device for determining whether a picture is frozen provided by a fourth embodiment of the present invention;

[0030] Figure 5 It is a structural schematic diagram of another device for determining whether a picture is frozen provided by a fifth embodiment of the present invention;

[0031] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the embodiments of the present application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned, and they should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must use the solution.

[0034] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0035] Embodiment 1

[0036] Figure 1 A flowchart of a method for determining whether a screen is frozen is provided for the first embodiment of the present invention. This embodiment can be applied to determining whether the screen of a shooting device is frozen. The method can be executed by a device for determining whether a screen is frozen. The device for determining whether a screen is frozen can be implemented in the form of hardware and / or software. The device for determining whether a screen is frozen can be configured in a server or an image management device and used in conjunction with a fixed-point shooting device or a moving-point shooting device.

[0037] like Figure 1 As shown, the method includes:

[0038] S110, controlling the fill light to perform fill light according to a preset fill light variation rule.

[0039] The fill light can be deployed in the shooting device or can be set independently from the shooting device. The fill light is controlled by the fill light control module to ensure that the fill light can perform fill light according to the preset fill light change rule. Specifically, the server can send a fill light instruction to the fill light control module, and the fill light control module controls the fill light to perform fill light.

[0040] The fill light needs to be able to illuminate the field of view of the camera so that the camera can perceive it when it is working normally. However, this embodiment does not limit the specific position of the position where the light is illuminated in the field of view of the camera when filling the light, and the proportion of the light illumination range to the field of view of the camera.

[0041] The fill light change law refers to the changes in the color, brightness, etc. of the light within a fill light change cycle. The light of the fill light changes according to the fill light change law.

[0042] Furthermore, the change rule of the fill light includes a change rule of light color and / or a change rule of light brightness.

[0043] The light color change rule refers to the change of the RGB (Red Green Blue) color mode of the light within a fill light change cycle, which can be the change of one color channel or the change of multiple color channels.

[0044] In a specific example, the fill light change rule may be that one fill light change cycle is 1 minute, and within one fill light change cycle, the fill light brightness remains unchanged, the G value and the B value remain fixed, for example, both remain 0, the R value of the fill light increases from 128 to 255 in the first 30 seconds, and the R value of the fill light decreases from 255 to 128 in the last 30 seconds. This is reflected in the fill light light, which changes from dark red to bright red in the first 30 seconds, and returns from bright red to dark red in the last 30 seconds.

[0045] In another specific example, the fill light change rule may be that one fill light change cycle is 1 minute, and in one fill light change cycle, the brightness of the fill light remains unchanged, the G value and B value are 0 in the first 20 seconds, and the R value increases from 128 to 255, the R value and B value are 0 in the middle 20 seconds, and the G value increases from 128 to 255, and the R value and G value are 0 in the last 20 seconds, and the B value increases from 128 to 255. In the fill light change cycle, the light of the fill light gradually changes from dark red to bright red in the first 20 seconds, from dark green to bright green in the middle 20 seconds, and from dark blue to bright blue in the last 20 seconds.

[0046] The light brightness variation rule refers to the variation of light brightness in a fill light variation cycle. In a specific example, the fill light variation rule may be that one fill light variation cycle is 1 minute, and in one fill light variation cycle, the brightness of the fill light gradually changes from 0 to 100 lm (lumens) in the first 30 seconds, and then decreases from 100 lm to 0 in the last 30 seconds. Furthermore, the brightness may be controlled to change according to the sine wave rule to achieve smooth brightness changes.

[0047] Furthermore, when the fill light is used to fill in light according to the fill light change rule, it must ensure that it does not affect the normal shooting of the shooting equipment. For example, in the scenario of monitoring the instrument through a card camera, the brightness and color changes of the fill light during fill light filling cannot affect the camera's monitoring effect on the instrument, that is, after introducing the fill light change, the instrument area in the monitoring image must be clear and easy to identify.

[0048] It is understandable that, in this embodiment, the regular changes of light are introduced into the field of view of the shooting device, and then it is determined whether the regular changes of light are fed back into the image output by the shooting device, so as to determine whether the shooting device is frozen. Therefore, in this embodiment, it is first necessary to ensure that the fill light is in a normal working state. Specifically, the server can determine whether the fill light control module is working properly through a heartbeat mechanism. Alternatively, a current transformer with a communication function can be connected to the negative pole of the fill light. When the current transformer has current, the fill light can be controlled normally, and the server determines whether the fill light is in a normal working state by communicating with the current transformer.

[0049] Furthermore, in this embodiment, the fill light can be controlled to perform fill light setting trigger conditions according to a preset fill light change rule: determine the similarity between two consecutive images output by the shooting device, for example, the similarity can be calculated by picture blocks and calculating the motion vectors of the corresponding blocks. This embodiment does not limit the calculation method of the similarity. If it is determined that a preset number of similarities are greater than or equal to the similarity threshold, the fill light is controlled to perform fill light according to the preset fill light change rule. After detecting that the image screen does not change, the judgment of the screen freezing is performed to reduce the consumption of computing resources.

[0050] In this embodiment, by introducing regularly changing fill light and irradiating the fill light into the field of view of the shooting device, when the shooting device is working normally, the output image can reflect the changes in light, thereby being able to distinguish whether the image does not change due to the freezing of the image caused by an abnormality in the shooting device, or because the actual shooting scene has not changed.

[0051] S120, determining image processing parameters when a fill light change zeroing condition is met, and controlling a photographing device to perform image processing according to the image processing parameters.

[0052] The image processing parameters refer to the parameters adjusted by the image signal processor (ISP) when processing the original image captured by the image sensor of the shooting device. Exemplarily, the image processing parameters may include AE ​​(Auto Exposure), AWB (Auto White Balance) or gain, etc. This embodiment does not limit the type and number of image processing parameters.

[0053] In an optional embodiment, determining the image processing parameters when the fill light change returns to zero condition may include: before controlling the fill light to perform fill light according to a preset fill light change rule, saving the latest image processing parameters in the image signal processor, and using the image processing parameters as the image processing parameters when the fill light change returns to zero condition is met.

[0054] In another optional embodiment, satisfying the fill light change zeroing condition may further include: if it is determined according to the fill light change law that the color of the fill light is within the color change range, and / or the brightness of the fill light is within the brightness change range, then it is determined that the fill light change zeroing condition is satisfied.

[0055] Among them, the color change range may refer to the color change range corresponding to white light, for example, it may mean that the R value, G value and B value are all 255; or, the R value is 255, the G value is 210-230 and the B value is 180-200; or, the R value, G value and B value are all 250-255, and this embodiment does not limit this.

[0056] The brightness variation range may be that the fill light brightness is 0, or the fill light brightness is 0-10 lm, which is not limited in this embodiment.

[0057] The color change range and brightness change range match the fill light change law. Specifically, since the fill light change law is the change process of the fill light within a fill light change cycle, if the fill light change law is the light color change law, then the color change range is the RGB value range when the fill light color is relatively stable and "normal" during the change process. Take the fill light change law as an example of converting red light, green light and blue light in sequence within the fill light change cycle. Since the transformation of light color is a gradual process, that is, within a fill light change cycle, it is actually a process of white light-red light-white light-green light-white light-blue light. Therefore, at this time, the RGB value range corresponding to white light is used as the color change range. If the fill light change law is the light brightness change law, then the brightness change range is the range in which the fill light brightness is 0 or smaller during the change process.

[0058] The shooting device is a fixed-point shooting device or a moving-point shooting device. A fixed-point shooting device refers to a shooting device with a relatively fixed position, which is installed in a specific position and usually does not move. A moving-point shooting device refers to a shooting device that can change its position or angle, and is usually equipped with a gimbal (for controlling the horizontal and vertical rotation of the shooting device) or a mobile device (such as a track, a drone, etc.), which can flexibly adjust the shooting position and angle as needed. It can be understood that since this embodiment affects the image effect by introducing fill light changes, there is no restriction on whether the shooting device is a moving point or a fixed point.

[0059] Controlling the shooting device to perform image processing according to the image processing parameters, specifically, refers to controlling the image signal processor to use fixed image processing parameters to perform image processing when processing the original image collected by the image sensor of the shooting device.

[0060] It is understandable that when the image signal processor processes the original image captured by the image sensor of the shooting device, it will perform automatic exposure adjustment, automatic white balance adjustment, color correction and noise reduction processing, etc., in order to obtain a clearer, balanced and natural image. When the fill light is used for filling light, the image signal processor will continuously adjust the various image processing parameters in the above processing process as the fill light color or brightness changes. For example, if the fill light is red light, the image signal processor will reduce the gain of the red channel, while appropriately increasing the gain of the blue and green channels. It may also adjust the parameters of the color correction matrix, or adjust the noise reduction parameters of the red channel, etc., so that the adjusted image is more balanced and natural. The image effect after adjustment will weaken the effect of the change of the fill light color and / or fill light brightness in the original image, which may cause the change of the fill light color and / or fill light brightness to be unclear in the image output by the shooting device, thereby causing the false judgment of the picture freezing.

[0061] Therefore, in this embodiment, the image processing parameters when the fill light change returns to zero are obtained, and the image processing parameters are fixed to process the original image. This can avoid the adjustment of the image processing parameters weakening the impact of the change in fill light color and / or fill light brightness on the image effect, thereby improving the accuracy of subsequent picture freezing judgments.

[0062] S130, determining an image to be analyzed output by the photographing device after image processing during the fill light filling process, and judging whether the picture of the photographing device is frozen according to the image to be analyzed.

[0063] In this embodiment, the fill light performs fill light according to a preset fill light rule, the image processor of the shooting device collects the original image, the image signal processor determines the image processing parameters when the fill light change returns to zero, and fixes the image processing parameters to process the collected original image, and finally outputs the image as the image to be analyzed.

[0064] Furthermore, determining the images to be analyzed output by the shooting device after image processing during the fill light filling process includes: determining each image to be analyzed output by the shooting device after image processing during the fill light filling process within a picture freeze analysis period; wherein the picture freeze analysis period is greater than or equal to the fill light change period of the fill light.

[0065] The image freeze analysis period is the shortest time period for continuously acquiring the images to be analyzed and performing image freeze judgment based on each image to be analyzed acquired during the period. The image freeze analysis period must be greater than or equal to the fill light change period of the fill light.

[0066] It can be understood that since the present embodiment determines whether a corresponding image effect is produced in the image output by the shooting device by introducing regular fill light, the duration of monitoring the output image to be analyzed must be at least greater than or equal to one fill light change cycle, so as to avoid the misjudgment of picture freezing caused by the monitored image to be analyzed happening to be an image captured during the fill light change zeroing period in the fill light change cycle, thereby improving the accuracy of picture freezing judgment.

[0067] In an optional embodiment, whether the image of the shooting device is frozen is determined based on the image to be analyzed. Taking the fill light variation rule as the light brightness variation rule as an example, the brightness mean of each continuously acquired image to be analyzed is determined, and the maximum and minimum values ​​of the brightness mean of each image to be analyzed are determined. If the difference between the maximum and minimum values ​​of the brightness mean is determined to be greater than or equal to a preset brightness mean difference threshold, and / or the time interval between the images to be analyzed corresponding to the maximum and minimum values ​​of the brightness mean matches the fill light variation cycle (for example, the difference between the time interval and the fill light variation cycle may be less than the time difference threshold, or the fill light variation cycle is an integer multiple of the time interval), it is determined that the shooting device is not frozen.

[0068] Correspondingly, taking the change rule of fill light as the change of light color as an example, the color mean of each continuously acquired image to be analyzed is determined, and the maximum and minimum values ​​of the color mean of each image to be analyzed are determined. If the difference between the maximum and minimum values ​​of the color mean is determined to be greater than or equal to a preset color mean difference threshold, and / or the time interval between the images to be analyzed corresponding to the maximum and minimum values ​​of the color mean matches the fill light change period (for example, the difference between the time interval and the fill light change period may be less than the time difference threshold, or the fill light change period is an integer multiple of the time interval), it is determined that the shooting device is not frozen.

[0069] It should be noted that the brightness mean difference threshold and the color mean difference threshold can be set accordingly according to the brightness change amplitude or the color change amplitude in the fill light change rule, and this embodiment does not limit this.

[0070] It is understandable that if the camera is not frozen and can work normally, then for each image to be analyzed that is continuously acquired during the image freezing analysis period, compared with before the fill light, the change in the fill light brightness during the fill light change period will affect the brightness of the image to be analyzed output by the camera, and the change in the fill light color will affect the color of the image to be analyzed output by the camera. Since the image freezing analysis period is greater than or equal to the fill light change period, each image to be analyzed can at least cover the impact of the fill light change on the image effect during one fill light change period. Therefore, this embodiment determines whether the camera is working normally by determining whether the difference between the maximum and minimum values ​​of the brightness mean or the color mean during the image freezing analysis period is greater than or equal to a preset threshold. This setting can avoid the misjudgment of image freezing caused by the change in the brightness of the ambient light, thereby improving the accuracy of the image freezing judgment.

[0071] When the shooting device is working normally, the image effect will be affected by the change of fill light brightness and / or color, and at the same time, this influence is consistent with the fill light change law. Since the fill light change law is diverse in form and is not limited to linear form or sine wave form, etc., and the influence of the fill light change law on the image effect does not necessarily present a linear relationship, therefore, this embodiment does not limit the specific form in which the image effect changes with the fill light change law. However, the change law of the image effect should be consistent with the fill light change law. Therefore, this embodiment determines the time interval between the images to be analyzed corresponding to the maximum and minimum values ​​of the brightness mean or the color mean, and determines whether the time interval is consistent with the fill light change period, or the fill light change period is an integer multiple of the time interval, thereby determining whether the change period of the image effect is the same as the fill light change period.

[0072] In another optional embodiment, judging whether the image of the shooting device is frozen is performed based on the image to be analyzed, and can also be achieved based on the brightness variance and / or color variance of each image to be analyzed. The specific process is similar to the above embodiment and will not be repeated in this embodiment.

[0073] It can be understood that the brightness mean can represent the overall brightness of the image, and the brightness mean of the image will increase to varying degrees when fill light is applied. The brightness variance is used to measure the degree of discreteness of the pixel brightness values ​​in the image relative to the brightness mean, and can represent the distribution of the image brightness. Since the fill light range of the fill light is usually only a local area in the field of view of the shooting device, the brightness variance of the image will increase when fill light is applied. Similarly, the color mean can represent the overall color tendency of the image. When fill light is applied, for example, when fill light is applied in red, the pixel color value of the red light channel increases, resulting in an increase in the color mean. The color variance measures the degree of discreteness of the pixel color value in each color channel relative to the color mean of the channel, and can reflect the uniformity of the distribution of the image color. Therefore, the color variance of the image will increase when fill light is applied.

[0074] In another optional embodiment, the image to be analyzed is used to determine whether the camera is frozen, and the fill light area in the image to be analyzed can also be determined. The position and size of the fill light area can be determined by pre-calibration. Taking the fill light variation law as the light brightness variation law as an example, the camera is judged to be frozen based on the brightness mean and / or brightness variance of the fill light area in each image to be analyzed. Correspondingly, when the fill light variation law is the light color variation law, the camera is judged to be frozen based on the color mean and / or color variance of the fill light area in each image to be analyzed. The specific process is similar to the above embodiment, and will not be repeated in this embodiment.

[0075] In this embodiment, through the parameters such as brightness mean, brightness variance, color mean and color variance of the image to be analyzed, it is determined whether the brightness of each image to be analyzed changes with the change of fill light brightness and / or whether the color of each image to be analyzed changes with the change of fill light color during the image freezing analysis period; at the same time, it is determined whether the change law of the image effect is consistent with the change law of the fill light. In this way, it is determined whether the long-term unchanged image is caused by the freezing of the shooting device or by the fact that the actual scene has not changed.

[0076] It should be noted that in this embodiment, the screen freeze is determined by controlling the fill light to perform fill light, and judging whether the brightness and / or color of the image output by the shooting device changes with the fill light. It is also possible to judge whether the image screen has changed by controlling the pan / tilt to rotate, scale the screen size, etc., so as to judge whether the shooting device has frozen the screen. It is also possible to control the image signal processor to add regular changes to the original image when processing the original image collected by the image sensor by adjusting parameters such as white balance and gain, such as adjusting the brightness and / or color in a specific area of ​​the original image. It is then determined whether the shooting device has frozen the screen by judging whether the specific area of ​​the image to be analyzed output by the shooting device conforms to the brightness and / or color change rules.

[0077] The technical solution of the embodiment of the present invention controls the fill light to perform fill light according to the fill light change rule. During the fill light process, the camera is controlled to perform image processing according to the image processing parameters when the fill light change returns to zero, and the camera is judged whether the image is frozen according to the image to be analyzed output by the camera. The problem in the prior art that when the image does not change for a long time, it is impossible to accurately judge whether it is caused by the freezing of the camera or the unchanged shooting scene, and the accurate judgment of whether the image of the camera is frozen is achieved.

[0078] Embodiment 2

[0079] Figure 2 This is a flow chart of a method for determining whether a screen is frozen provided in the second embodiment of the present invention. Based on the above embodiments, the embodiment of the present invention further specifies the specific process of determining whether the screen of a shooting device is frozen according to an image to be analyzed.

[0080] like Figure 2 As shown, the method includes:

[0081] S210: Control the fill light to perform fill light according to a preset fill light variation rule.

[0082] The fill light process has been described in the above embodiment, and will not be described again in detail in this embodiment.

[0083] S220, determining image processing parameters when a fill light change zeroing condition is met, and controlling the shooting device to perform image processing according to the image processing parameters.

[0084] S230: Determine the image to be analyzed outputted after the shooting device performs image processing during the fill light filling process.

[0085] The process of determining the image processing parameters when the fill light change returns to zero, fixing the image processing parameters for image processing, and determining the image to be analyzed output by the shooting device has been described in the above embodiment, and will not be repeated in this embodiment.

[0086] S240: Determine the image entropy of the image to be analyzed, and determine a change rule of the image entropy based on consecutive images to be analyzed.

[0087] This embodiment takes image entropy as an example to analyze whether the image effect of the image to be analyzed changes with the change rule of fill light.

[0088] Among them, image entropy indicates the amount of information contained in the image. The more information the image contains, the higher the image entropy. When the image entropy is high, the distribution of the grayscale value (or color value) of the pixels in the image is more complex and diverse, and the details are richer.

[0089] In this embodiment, the image entropy of the image to be analyzed is determined. In a specific example, the image to be analyzed can be grayed to obtain a grayscale image, and then the image entropy can be calculated using the following formula: , where H represents the image entropy, L is the maximum possible number of image pixel values ​​in the image to be analyzed (for example, for an 8-bit grayscale image, L=256), is the probability that a pixel with a pixel value of k appears in the image to be analyzed.

[0090] In another specific example, the above method can be used for the three color channels of RGB to calculate the entropy value corresponding to each color channel: , as well as . Then , as well as Find the average value as the final image entropy of the image to be analyzed.

[0091] The image entropy variation law is determined based on the continuous images to be analyzed. Specifically, the image entropy variation curve can be drawn with each image to be analyzed as the horizontal axis and the image entropy corresponding to each image to be analyzed as the vertical axis. The image entropy variation curve represents the image entropy variation law within the frame freezing analysis period.

[0092] S250: Determine whether the image of the shooting device is frozen according to the image entropy change rule and the fill light change rule.

[0093] Correspondingly, the fill light variation rule can also be represented by a curve, with the time in a fill light variation cycle as the horizontal axis and the light brightness and / or light color corresponding to different times as the vertical axis to determine the fill light variation curve.

[0094] Further, S250 may include: if it is determined that the image entropy change rule matches the fill light change rule, determining that the shooting device has not frozen the screen.

[0095] Specifically, to determine whether the image entropy change law and the fill light change law match, the key feature points of the image entropy change curve and the fill light change curve can be determined respectively, such as the starting point, end point, turning point, maximum point, minimum point, etc., and by comparing whether the order, relative position, distance, etc. of these feature points appearing in the image entropy change curve and the fill light change curve are consistent, to determine whether the image entropy change law and the fill light change law match.

[0096] For example, taking the fill light change rule that the fill light brightness is gradually increased from 0 within 20 seconds and then reduced to 0, accordingly, the image to be analyzed acquired within the 20 seconds shows that the image entropy gradually increases from an initial value and then gradually decreases to near the initial value. At this time, the image entropy change rule matches the fill light change rule, and the shooting device does not freeze the screen.

[0097] In this embodiment, when the image entropy change law matches the fill light change law, the picture scene does not change, which may be caused by the fact that the actual scene has not changed. The picture freezing judgment can be performed at every preset time interval to achieve continuous monitoring of the picture freezing of the shooting device.

[0098] Further, S250 may include: if it is determined that the image entropy change law and the fill light change law do not match, and / or the proportion of the image entropy within the image entropy change range to the total image entropy is greater than or equal to a preset proportion threshold, then determining that the picture of the shooting device is frozen.

[0099] In an optional embodiment, the image entropy variation range can be determined by pre-calibration. Specifically, when it is determined that the shooting device is in a normal working state, the images output by the shooting device within a preset time period can be collected, the image entropy of each image can be calculated, and the image entropy variation range can be determined. Furthermore, multiple image entropy variation ranges can be calibrated for different time periods or different scenes, and the corresponding image entropy variation range can be determined according to the time period or scene in which the current picture freezing analysis cycle is located.

[0100] In another optional embodiment, the image entropy variation range can be calculated for images output by a continuous preset number of shooting devices, and the image entropy variation range is determined when it is determined that the screen freeze judgment trigger condition is met, that is, when it is determined that for images output by a continuous preset number of shooting devices, the similarity between two adjacent images is greater than or equal to a similarity threshold.

[0101] In this embodiment, a variation range is set for the image entropy. If, for each image to be analyzed within the screen freeze analysis period, the ratio of the image entropy within the image entropy variation range to the total image entropy is greater than or equal to a preset ratio threshold, that is, most of the image entropy remains within the variation range, indicating that the image entropy of the image to be analyzed remains relatively unchanged for a long time, then it is determined that the screen freeze is caused by an abnormal operation of the shooting device.

[0102] Or, the image entropy change law does not match the fill light change law. Still taking the above fill light change law as an example, that is, the fill light brightness is gradually increased from 0 within 20 seconds, and then the fill light brightness is reduced to 0, if the image entropy changes first and then increases, or continuously decreases or continuously increases, or the image entropy change cycle is 10 seconds, and it first increases and then decreases within the change cycle, etc., then it is considered that the image entropy change law does not match the fill light change law, the shooting equipment is working abnormally, and the picture is frozen.

[0103] Furthermore, this embodiment is explained by taking the calculation of the overall image entropy of the image to be analyzed as an example. It is also possible to determine the fill light area in each image to be analyzed, calculate the image entropy for the fill light area, and then determine the image entropy change law based on the image entropy of the fill light area of ​​the continuous image to be analyzed, and determine whether the shooting device is frozen based on the image entropy change law and the fill light change law. The specific process of calculating the image entropy, determining the image entropy change law, and determining whether the image entropy change law and the fill light change law match is similar to the above embodiment, and this embodiment will not be repeated here.

[0104] The technical solution of this embodiment, on the one hand, controls the fill light to change the brightness or color according to the fill light change law, fills the field of view of the shooting device, and introduces the fill light with regular changes to introduce the changes in fill light brightness and / or color into the image effect of the image output by the shooting device. On the other hand, the image signal sensor of the shooting device is controlled, the image processing parameters when the fill light change returns to zero are fixed, and subsequent image processing is performed during the fill light process, so that the final output image can better present the influence of the change in fill light brightness and / or color on the image effect, thereby improving the accuracy of subsequent picture freezing judgment. For each image to be analyzed within the picture freezing analysis period, the image entropy parameter is used to reflect the change of the continuously acquired image to be analyzed with the fill light change. It is judged whether the image entropy change law of each image to be analyzed is consistent with the fill light change law, so as to judge whether the shooting device is in a normal working state, and it can effectively distinguish whether the image screen does not change for a long time due to the freezing of the shooting device screen or due to the unchanged shooting scene, thereby realizing accurate judgment of whether the picture of the shooting device is frozen.

[0105] Embodiment 3

[0106] Figure 3 A flowchart of another method for determining whether a screen is frozen is provided for Embodiment 3 of the present invention. This embodiment can be applied to the case where the screen of a shooting device is frozen. The method can be executed by a device for determining whether a screen is frozen. The device for determining whether a screen is frozen can be implemented in the form of hardware and / or software. The device for determining whether a screen is frozen can be configured in a server or an image management device and used in conjunction with a moving-point shooting device.

[0107] like Figure 3 As shown, the method includes:

[0108] S310: Control the moving-point shooting device to rotate in the horizontal direction and / or the vertical direction.

[0109] This embodiment is applicable to the judgment of image freezing of a moving-point shooting device. It can be understood that since the shooting device needs to be controlled to rotate in this embodiment, it can only be achieved through the moving-point shooting device.

[0110] Specifically, the server may send a rotation control instruction to the pan-tilt head or mobile device to control the rotation of the moving point shooting device, which is not limited in this embodiment. Meanwhile, the rotation control method based on the pan-tilt head or mobile device may be implemented in a conventional manner, which is not described in detail in this embodiment.

[0111] In this embodiment, it is necessary to ensure that the pan / tilt or mobile device that performs rotation control is in a normal working state. Similarly, it is possible to determine whether the pan / tilt or mobile device is working normally through a heartbeat mechanism or the like.

[0112] In this embodiment, the field of view of the moving-point shooting device is changed by controlling the moving-point shooting device to rotate in the horizontal direction and / or the vertical direction, thereby determining whether the shooting device is in a frozen state.

[0113] S320: Determine the image to be analyzed output by the moving-point shooting device, and judge whether the image of the moving-point shooting device is frozen according to the image to be analyzed.

[0114] Specifically, the similarity between two consecutive images to be analyzed output by the shooting device can be determined, and the similarity calculation process will not be repeated. If the similarity is less than the preset similarity threshold, it means that the moving point shooting device has not frozen the screen. If for a preset number of consecutive images to be analyzed, the ratio of the number of similarities greater than or equal to the similarity threshold to the preset number is greater than or equal to the preset ratio threshold, it is determined that the screen of the moving point shooting device is frozen.

[0115] Furthermore, after determining that the image of the moving-point shooting device is not frozen, it is also necessary to control the moving-point shooting device to reset so as to restore the original position and angle of the moving-point shooting device.

[0116] The technical solution of this embodiment controls the moving point shooting device to rotate in the horizontal direction and / or vertical direction to change the field of view of the moving point shooting device, so that it can be judged whether the moving point shooting device has frozen images according to the similarity between the images to be analyzed output by the shooting device. This solves the problem in the prior art that when the image screen does not change for a long time, it is impossible to accurately judge whether it is caused by the freezing of the shooting device screen or the unchanged shooting scene, and realizes accurate judgment of whether the image of the shooting device is frozen.

[0117] Embodiment 4

[0118] Figure 4 This is a schematic diagram of the structure of a device for determining whether a picture is frozen provided by Embodiment 4 of the present invention. Figure 4 As shown, the device comprises:

[0119] The fill light control module 410 is used to control the fill light to perform fill light according to a preset fill light change rule;

[0120] An image processing parameter control module 420, used to determine image processing parameters when the fill light change zeroing condition is met, and control the shooting device to perform image processing according to the image processing parameters;

[0121] The first freeze screen determination module 430 is used to determine the image to be analyzed output by the shooting device after image processing during the fill light filling process, and determine whether the shooting device has frozen the screen according to the image to be analyzed.

[0122] The technical solution of the embodiment of the present invention controls the fill light to perform fill light according to the fill light change rule. During the fill light process, the camera is controlled to perform image processing according to the image processing parameters when the fill light change returns to zero, and the camera is judged whether the image is frozen according to the image to be analyzed output by the camera. The problem in the prior art that when the image does not change for a long time, it is impossible to accurately judge whether it is caused by the freezing of the camera or the unchanged shooting scene, and the accurate judgment of whether the image of the camera is frozen is achieved.

[0123] Based on the above embodiment, optionally, the fill light change rule includes a light color change rule and / or a light brightness change rule.

[0124] Based on the above embodiment, optionally, the image processing parameter control module 420 includes:

[0125] The fill light change zeroing condition judgment unit is used to determine that the fill light change zeroing condition is met if the color of the fill light is determined to be within the color change range and / or the brightness of the fill light is within the brightness change range according to the fill light change law.

[0126] Based on the above embodiment, optionally, the first picture freezing determination module 430 includes:

[0127] The image acquisition unit to be analyzed is used to determine each image to be analyzed outputted after the shooting device performs image processing during the fill light filling process within the picture freezing analysis period;

[0128] The image freezing analysis period is greater than or equal to the fill light change period of the fill light.

[0129] Based on the above embodiment, optionally, the first picture freezing determination module 430 includes:

[0130] An image entropy variation law determination unit, used to determine the image entropy of the image to be analyzed, and determine the image entropy variation law based on the continuous images to be analyzed;

[0131] The image freezing judgment unit is used to judge whether the image of the shooting device is frozen according to the image entropy change law and the fill light change law.

[0132] Based on the above embodiment, optionally, the picture freezing judgment unit is specifically used to:

[0133] If it is determined that the image entropy change rule matches the fill light change rule, it is determined that the image capturing device is not frozen.

[0134] Based on the above embodiment, optionally, the picture freezing judgment unit is specifically used to:

[0135] If it is determined that the image entropy change law and the fill light change law do not match, and / or the proportion of the image entropy within the image entropy change range to the total image entropy is greater than or equal to a preset proportion threshold, it is determined that the picture of the shooting device is frozen.

[0136] The image freezing determination device provided in the embodiment of the present invention can execute the image freezing determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0137] Embodiment 5

[0138] Figure 5 The fifth embodiment of the present invention provides another structural diagram of a device for determining whether the image is frozen. Figure 5 As shown, the device comprises:

[0139] A rotation control module 510, used to control the moving point shooting device to rotate in the horizontal direction and / or vertical direction;

[0140] The second freeze frame determination module 520 is used to determine the image to be analyzed output by the moving-point shooting device, and determine whether the moving-point shooting device is frozen according to the image to be analyzed.

[0141] The technical solution of this embodiment controls the moving point shooting device to rotate in the horizontal direction and / or vertical direction to change the field of view of the moving point shooting device, so that it can be judged whether the moving point shooting device has frozen images according to the similarity between the images to be analyzed output by the shooting device. This solves the problem in the prior art that when the image screen does not change for a long time, it is impossible to accurately judge whether it is caused by the freezing of the shooting device screen or the unchanged shooting scene, and realizes accurate judgment of whether the image of the shooting device is frozen.

[0142] The image freezing determination device provided in the embodiment of the present invention can execute the image freezing determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] Embodiment 6

[0144] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0145] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0146] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0147] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for determining whether the screen is frozen.

[0148] In some embodiments, the method for determining whether the screen is frozen may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining whether the screen is frozen described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for determining whether the screen is frozen in any other appropriate manner (e.g., by means of firmware).

[0149] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0151] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0153] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0154] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0155] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0156] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining whether a picture is frozen, characterized in that: include: Control the fill light to fill in light according to the preset fill light change rule; Determine image processing parameters when the fill light change zeroing condition is met, and control the shooting device to perform image processing according to the image processing parameters; Wherein, the shooting device is a fixed-point shooting device or a dynamic-point shooting device; An image to be analyzed output by the shooting device after image processing during the fill light filling process is determined, and whether the shooting device is frozen is determined according to the image to be analyzed.

2. The method according to claim 1, characterized in that The fill light variation rule includes a light color variation rule and / or a light brightness variation rule.

3. The method according to claim 2, characterized in that The fill light change zeroing conditions are met, including: If it is determined according to the fill light variation rule that the color of the fill light is within the color variation range, and / or the brightness of the fill light is within the brightness variation range, then it is determined that the fill light variation zeroing condition is met.

4. The method according to claim 1, characterized in that Determine the image to be analyzed output by the camera after image processing during the fill light filling process, including: Determine each image to be analyzed that is output after the camera performs image processing during the fill light filling process within the image freezing analysis period; The image freezing analysis period is greater than or equal to the fill light change period of the fill light.

5. The method according to claim 1, characterized in that Determining whether the image of the shooting device is frozen according to the image to be analyzed includes: Determine the image entropy of the image to be analyzed, and determine the image entropy change law based on the continuous images to be analyzed; Determine whether the image of the shooting device is frozen based on the change rules of image entropy and fill light.

6. The method according to claim 5, characterized in that According to the change rules of image entropy and fill light, determine whether the image of the shooting device is frozen, including: If it is determined that the image entropy change rule matches the fill light change rule, it is determined that the image capturing device is not frozen.

7. The method according to claim 5, characterized in that According to the change law of image entropy and fill light, it is judged whether the image of the shooting device is frozen, and it also includes: If it is determined that the image entropy change law and the fill light change law do not match, and / or the proportion of the image entropy within the image entropy change range to the total image entropy is greater than or equal to a preset proportion threshold, it is determined that the picture of the shooting device is frozen.

8. A method for determining whether a picture is frozen, characterized in that: The method comprises: Controlling the moving point shooting device to rotate in the horizontal direction and / or vertical direction; An image to be analyzed output by the moving-point shooting device is determined, and whether the moving-point shooting device is frozen is determined according to the image to be analyzed.

9. A device for determining whether a picture is frozen, characterized in that: include: A fill light control module is used to control the fill light to perform fill light according to a preset fill light change rule; An image processing parameter control module, used to determine the image processing parameters when the fill light change zeroing condition is met, and control the shooting device to perform image processing according to the image processing parameters; Wherein, the shooting device is a fixed-point shooting device or a dynamic-point shooting device; The first screen freeze judgment module is used to determine the image to be analyzed output by the shooting device after image processing during the fill light filling process, and judge whether the shooting device has a screen freeze according to the image to be analyzed.

10. A device for determining whether a picture is frozen, characterized in that: include: A rotation control module, used to control the moving point shooting device to rotate in the horizontal direction and / or vertical direction; The second screen freeze judgment module is used to determine the image to be analyzed output by the moving point shooting device, and judge whether the screen of the moving point shooting device is frozen according to the image to be analyzed.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for determining whether the screen is frozen as described in any one of claims 1 to 8 is implemented.

12. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the method for determining whether the screen is frozen as described in any one of claims 1 to 8.

Images