Image optimization method and device and electronic equipment

Abstract

The invention discloses an image optimization method and device and electronic equipment. The method comprises the steps that a target image in a video frame is detected, a fuzzy area is obtained, and the target image is any image in the video frame; feature information of the fuzzy area is determined, and the feature information comprises color features and energy values; and performing clear processing on the fuzzy region according to the feature information. The technical problem that accurate monitoring cannot be carried out under the condition that smoke and water vapor exist in the environment where the monitoring video is located is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image optimization method, device and electronic device. Background Art

[0002] At present, although campus catering supervision technology has started, it is mostly limited to the widespread deployment of surveillance cameras in the kitchen. Although this measure has basic monitoring functions, it faces significant challenges. In the kitchen environment, smoke and water vapor often cause the monitoring screen to be blurred, the quality is greatly reduced, and the accuracy and effectiveness of the monitoring are weakened. In this context, it is difficult to achieve panoramic and high-quality monitoring of kitchen operations and ensure the transparency and safety of the food preparation process by traditional monitoring methods alone.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method, device and electronic device for image optimization to at least solve the technical problem that accurate monitoring cannot be performed when smoke and water vapor exist in the environment of the surveillance video.

[0005] According to one aspect of an embodiment of the present application, a method for image optimization is provided, comprising: detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; determining feature information of the blurred area, wherein the feature information includes color features and energy values; and performing clarity processing on the blurred area based on the feature information.

[0006] Optionally, the target image in the video frame is detected to obtain a blurred area, including: obtaining target pixel points of the target image and obtaining background pixel points in the background image, wherein the target pixel point is any pixel point in the target image and the background pixel point is any pixel point in the background image; determining the difference between the target pixel point and the background pixel point; and determining the area formed by the target pixel points whose absolute value of the difference is greater than a preset threshold as a blurred area.

[0007] Optionally, after obtaining the blurred area, the method also includes: determining the area formed by target pixel points whose absolute value of difference is less than or equal to a preset threshold as the background area of ​​the target image; obtaining a first pixel point in the background area, and obtaining a second pixel point in the background image, wherein the background image and the target image have the same size, the position of the first pixel point in the target image and the position of the second pixel point in the background image are the same, the first pixel point is any pixel point in the background area, and the second pixel point is any pixel point in the background image; updating the background image based on the first pixel point, the second pixel point and the background update factor.

[0008] Optionally, determining the characteristic information of the blurred area includes: obtaining the contrast brightness and saturation of the blurred area; determining the color characteristics based on the contrast brightness and saturation; performing discrete wavelet transform on the blurred area to obtain a high-frequency component sub-image, wherein the high-frequency component sub-image includes a first image, a second image and a third image, the first image includes texture information of the blurred area in a horizontal direction, the second image includes texture information of the blurred area in a vertical direction, and the third image includes texture information of the blurred area in a diagonal direction; determining the energy value of the blurred area based on the high-frequency component sub-image; and determining the characteristic information of the blurred area based on the color characteristics and the energy value.

[0009] Optionally, the blurred area is processed clearly based on the feature information, including: determining the image type of the blurred area based on the range of the energy value in the feature information, wherein the image types include water vapor images and smoke images; determining the image depth of the blurred area based on the image type and color characteristics of the blurred area; and processing the blurred area clearly based on the image depth.

[0010] Optionally, determining the image depth of the blurred area includes: determining the relative proportion of water vapor and smoke in the blurred area based on the image type and color characteristics of the blurred area; determining a first coefficient corresponding to the brightness in the color characteristics based on the relative proportion, and determining a second coefficient corresponding to the saturation in the color characteristics; determining the image depth based on the first coefficient, the second coefficient and the color characteristics.

[0011] Optionally, the blurred area is processed for clarity based on the image depth, including: obtaining a blurred image corresponding to the blurred area, the atmospheric light value of the environment in which the target image is located, and the image depth; determining a clarity algorithm based on the blurred image, the atmospheric light value, and the image depth; and processing the blurred area for clarity based on the clarity algorithm.

[0012] According to another aspect of an embodiment of the present application, there is also provided an image optimization device, including: a detection module, used to detect a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; a determination module, used to determine feature information of the blurred area, wherein the feature information includes color features and energy values; and a processing module, used to clearly process the blurred area based on the feature information.

[0013] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions for implementing the following functions: detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; determining feature information of the blurred area, wherein the feature information includes color features and energy values; and clearly processing the blurred area based on the feature information.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned image optimization method by running the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned image optimization method when executed by a processor.

[0016] In an embodiment of the present application, a blurred area is obtained by detecting a target image in a video frame, wherein the target image is any image in the video frame; characteristic information of the blurred area is determined, wherein the characteristic information includes color characteristics and energy values; the blurred area is clearly processed based on the characteristic information, thereby achieving the purpose of enhancing picture clarity, thereby achieving the technical effect of improving video quality and optimizing viewing experience, and further solving the technical problem of being unable to perform accurate monitoring when there is smoke and water vapor in the environment where the surveillance video is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a computer terminal for implementing an image optimization method according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of an image optimization method according to an embodiment of the present application;

[0020] Figure 3 It is a structural diagram of an image optimization device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] In the related art, the monitoring of campus catering only stops at installing a large number of monitors in the kitchen, but how to manage and control these monitors has become a major difficulty. For example, it is inconvenient to view the monitoring video: the monitoring video and video playback can only be viewed locally, and it does not support viewing anytime and anywhere through the Internet. In the face of emergencies, the reaction speed is insufficient, and it is impossible to quickly view the on-site situation, lagging behind the development of the mobile Internet era. The equipment has weak intelligent capabilities and cannot recognize and alarm the entry and exit behaviors, safety behaviors, operating behaviors, and wearing behaviors of people in key areas of the kitchen, and cannot meet regulatory needs. The lack of a back-kitchen display platform makes online supervision difficult. In addition, in view of the defects of the monitoring video in the related art, when smoke and water vapor appear in the kitchen during use, the quality of the monitoring video will be affected, making it difficult to accurately monitor the kitchen. In order to eliminate the problem of monitoring video quality in the specific environment of the kitchen, or to eliminate video quality problems in other environments with smoke and water vapor, the embodiment of the present application provides an image optimization method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0024] The image optimization method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing an image optimization method. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0025] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0026] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the method for image optimization in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned method for image optimization. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0029] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0030] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a method for image optimization. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] Figure 2 is a flow chart of an image optimization method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0032] Step S202, detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame.

[0033] In an optional embodiment of the above step S202, a camera is used to monitor the campus kitchen to obtain a video frame. In another optional embodiment, the video frame may also be video data in other environments collected by the camera. The blurred area detection is performed on any target image in the video frame, which can be mainly achieved by the following methods:

[0034] 1. Method based on the difference between adjacent frames:

[0035] When the background of the video is static, the blurred area can be detected by comparing the difference between the target image and its previous and next frames. Because blur is usually caused by motion or camera shake, the blurred area will show a large difference between adjacent frames. The core idea of ​​this method is to perform a difference between the previous or next frames of the image sequence and the current frame, and then extract the changed area from it, so as to detect the blurred area.

[0036] 2. Algorithms based on image processing:

[0037] Image processing algorithms such as the Laplacian operator, Brenner operator, and Tenengrad gradient function can be used to detect the clarity of an image. These algorithms evaluate the clarity of an image by calculating the edge gradient or gradient change of the image, and then identify the blurred area. For example, the Laplacian operator is a second-order differential operator. After performing a Laplace transform on an image, the edge information of the image can be obtained. Since the edges of a blurred image are not clear, the result of the Laplace transform will show a lower value, so the blurriness of the image can be judged by setting a threshold.

[0038] 3. Method based on background modeling:

[0039] For videos with dynamic backgrounds, background modeling can be used. The Gaussian model is a commonly used model for theoretical derivation of image blur regions. By performing Gaussian modeling on the background of the image and then judging the degree of fit between each pixel of the image and the model, blur detection of partially blurred images can be achieved.

[0040] It should be noted that the above describes several methods of obtaining the blurred area by way of examples. However, when actually determining the blurred area, the blurred area may be determined by the above method or other methods, which is not limited here.

[0041] Step S204, determining feature information of the blurred area, wherein the feature information includes color features and energy values.

[0042] In the above step S204, color feature is one of the basic attributes of the image, which describes the color distribution and changes on the surface of the object in the image. In the detection of blurred areas, color features can provide clues about the color difference between the blurred area and the surrounding clear areas. In the embodiment of the present application, the color features mainly include contrast brightness and saturation. Energy value is a commonly used concept in image processing, which reflects the activity or importance of a specific area or feature in an image. In the detection of blurred areas, energy value can be used to evaluate the clarity and detail richness of the blurred area.

[0043] Step S206, clearing the blurred area according to the feature information.

[0044] In the above step S206, the image depth can be determined according to the feature information obtained above, and then the clarity algorithm is determined according to the image depth, and the blurred area is processed by the clarity algorithm. The influence of water vapor and smoke on the monitoring video can be effectively reduced, so as to more accurately monitor the kitchen and food safety. At the same time, by clearing the blurred image, the food preparation process and sanitary conditions inside the kitchen can be more accurately monitored, thereby enhancing the monitoring effect of campus food safety.

[0045] Through the above steps S202 to S206, the purpose of enhancing the picture clarity is achieved, thereby achieving the technical effect of improving video quality and optimizing viewing experience, and further solving the technical problem of being unable to perform accurate monitoring when there is smoke and water vapor in the environment where the monitoring video is located. The following is an explanation.

[0046] In step S202 of the above-mentioned image optimization method, the target image in the video frame is detected to obtain a blurred area, including: obtaining target pixel points of the target image, and obtaining background pixel points in the background image, wherein the target pixel point is any pixel point in the target image, and the background pixel point is any pixel point in the background image; determining the difference between the target pixel point and the background pixel point; and determining the area formed by the target pixel points whose absolute value of the difference is greater than a preset threshold as a blurred area.

[0047] In some embodiments of the present application, the background image is the same size as the target image, but belongs to the background part in the video frame. In the embodiments of the present application, the background part is formed into a background image separately, for example, by background modeling, background subtraction or deep learning method, so as to be better compared with the target image. For each pair of target pixels and background pixels, the difference between them is calculated. This difference can be a color difference (such as the Euclidean distance between RGB values), a brightness difference or other types of difference metrics. In the embodiments of the present application, the difference between the pixel values ​​of the target pixel and the background pixel is calculated. A preset threshold is set to determine whether the target pixel belongs to the fuzzy area. Traverse all pixels on the target image and calculate the difference between each target pixel and the corresponding background pixel. If the difference between the pixel values ​​of a certain target pixel and the background pixel is greater than the preset threshold, it is considered that the target pixel belongs to a part of the fuzzy area. Connect all such target pixels to form one or more continuous areas, which are the fuzzy areas finally determined.

[0048] Specifically, the target image and the background image are differentially operated, and the differential result is segmented by OTSU threshold to obtain a binary motion mask image and separate the foreground and background pixels:

[0049]

[0050] Among them, object is the binary mask result, F n is the target image, CB is the background image, F n (x,y) represents the target pixel, CB( x ,y) represents the background pixel point, OT represents the above preset threshold, the pixel value 255 represents the foreground target, and 0 represents the background. The foreground target is regarded as the above blurred area.

[0051] In step S202 of the above-mentioned image optimization method, after obtaining the blurred area, the method also includes: determining the area formed by the target pixel points whose absolute value of the difference is less than or equal to the preset threshold as the background area of ​​the target image; obtaining the first pixel point in the background area, and obtaining the second pixel point in the background image, wherein the background image and the target image have the same size, the position of the first pixel point in the target image and the position of the second pixel point in the background image are the same, the first pixel point is any pixel point in the background area, and the second pixel point is any pixel point in the background image; updating the background image based on the first pixel point, the second pixel point and the background update factor.

[0052] In an embodiment of the present application, if the difference between a certain target pixel and the corresponding pixel in the background image is very small (i.e., less than or equal to a preset threshold), it is considered that the target pixel is likely to belong to the background, rather than the main part of the target image, thereby obtaining the background area in the target image. In addition, in order to update the background image, it is necessary to obtain the pixel in the background area of ​​the target image (referred to as the first pixel) and the pixel at the corresponding position in the background image (referred to as the second pixel). It is assumed here that the size of the background image and the target image are the same so that the corresponding pixel can be found directly by the position index. The first pixel is any pixel in the background area (but in actual application, all pixels in the background area will be traversed), and the second pixel is the pixel in the background image with the same position as the first pixel. The background image is updated using the first pixel (from the background area of ​​the target image), the second pixel (from the original background image) and a background update factor. This update process is usually to make the background image more accurately reflect the background information in the current scene, especially when the target image interacts with the background image (such as target movement, occlusion, etc.). The role of the background update factor is to control the weight of new information and old information in the update process. If the background update factor is large, the new information (i.e., the value of the first pixel) has a greater impact on the background image; conversely, if the background update factor is small, the old information (i.e., the value of the second pixel) is retained more. Generally speaking, in dynamic scenes, you may need to choose a larger update factor to quickly adapt to background changes; in static scenes, you can choose a smaller update factor to maintain the stability of the background.

[0053] Specifically, the background image can be adaptively updated by the following formula:

[0054]

[0055] Among them, object( x ,y)=0 represents the background area of ​​the target image, α is the background update factor, CB( x ,y) represents the second pixel (that is, the background pixel mentioned above), F' n (x, y) represents the first pixel point in the background area of ​​the target image. The above formula can be used to update the background image CB.

[0056] In step S204 of the above-mentioned image optimization method, characteristic information of the blurred area is determined, including: obtaining contrast brightness and saturation of the blurred area; determining color characteristics based on the contrast brightness and saturation; performing discrete wavelet transform on the blurred area to obtain a high-frequency component sub-image, wherein the high-frequency component sub-image includes a first image, a second image and a third image, the first image includes texture information of the blurred area in the horizontal direction, the second image includes texture information of the blurred area in the vertical direction, and the third image includes texture information of the blurred area in the diagonal direction; based on the high-frequency component sub-image, determining the energy value of the blurred area; based on the color characteristics and the energy value, determining the characteristic information of the blurred area.

[0057] In the embodiment of the present application, contrast brightness reflects the brightness difference between different areas in the image, while saturation describes the purity and vividness of the color, and the color features are determined based on contrast brightness and saturation. And through discrete wavelet transform, the blurred area is decomposed into high-frequency component sub-images, which contain the texture information of the image in different directions. Specifically: the first image (horizontal direction): contains the texture information of the blurred area in the horizontal direction, such as horizontal edges, stripes, etc. The second image (vertical direction): contains the texture information of the blurred area in the vertical direction, such as vertical edges, columnar textures, etc. The third image (diagonal direction): contains the texture information of the blurred area in the diagonal direction, which usually reflects the oblique features in the image. The energy value in the high-frequency component sub-image reflects the intensity and complexity of the texture information in the image. In the blurred area, due to the loss of texture details, the energy value of the high-frequency component sub-image may be reduced. By calculating the energy values ​​of these sub-images, the degree of reduction of texture information in the blurred area can be quantified, which can be used as an indicator of the degree of blur. Finally, combining color features and energy values ​​can form a comprehensive description of the feature information of the blurred area. This description not only includes the color characteristics of the blurred area, but also reflects the degree of loss of texture information.

[0058] Specifically, when determining the characteristic information of the blurred area, a YUV model and an HSI model are established, and the contrast brightness Y and saturation S of the blurred area are used as color features, and the changes in the contrast brightness Y and saturation S can be measured by component energy:

[0059]

[0060] Among them, E(Y) represents the change of contrast brightness, and E(S) represents the change of saturation.

[0061] By performing a two-bit discrete wavelet transform on the blurred area, three high-frequency component sub-images can be obtained. The three high-frequency component sub-images include texture information in the horizontal, vertical and diagonal directions respectively. The sum of the energy values ​​of the three high-frequency component sub-images constitutes a composite image. The energy of the blurred area of ​​the target image is:

[0062]

[0063] in, Represents the energy value of the fuzzy area, represents the energy value of the first image, represents the energy value of the second image, Represents the energy value of the third image.

[0064] The characteristic information of the blurred area can be determined through the color features and energy values ​​obtained above.

[0065] In step S206 of the above-mentioned image optimization method, the blurred area is clearly processed based on the feature information, including: determining the image type of the blurred area based on the range of the energy value in the feature information, wherein the image types include water vapor images and smoke images; determining the image depth of the blurred area based on the image type and color characteristics of the blurred area; and clearly processing the blurred area based on the image depth.

[0066] In the embodiment of the present application, the energy value in the feature information becomes an important basis for judging the image type of the blurred area. The energy value reflects the intensity of the high-frequency components in the image, and different image types (such as water vapor images, smoke images, etc.) often have different energy distribution characteristics. By analyzing the range to which the energy value belongs, the blurred area can be divided into different image types. For example, a water vapor image may be blurred due to the scattering effect of water molecules, and its energy value may show a certain specific distribution pattern; while a smoke image may show another energy distribution feature due to the occlusion and scattering of smoke particles. Image depth here refers to the degree of blur or the distance of the object from the camera. Different image types and color features can provide clues about image depth. For example, in a water vapor image, the density and distribution range of water vapor may affect the depth perception of the image; and in a smoke image, the concentration and diffusion mode of smoke may also affect the image depth. In addition, color contrast, saturation, etc. in color features may also be related to image depth. By analyzing these features, the image depth of the blurred area can be roughly estimated. Adjust the parameters of the sharpening algorithm according to the image depth to achieve more accurate sharpening processing.

[0067] In the above steps, the image depth of the blurred area is determined, including: determining the relative proportion of water vapor and smoke in the blurred area according to the image type and color characteristics of the blurred area; determining a first coefficient corresponding to the brightness in the color characteristics according to the relative proportion, and determining a second coefficient corresponding to the saturation in the color characteristics; determining the image depth according to the first coefficient, the second coefficient and the color characteristics.

[0068] In the embodiment of the present application, the image type (such as water vapor image, smoke image or a mixture of the two) and color features (such as color distribution, brightness, saturation, etc.) of the blurred area are used to infer the relative content of water vapor and smoke in the blurred area. Different image types and color features can reflect the different performances of water vapor and smoke in the image. For example, water vapor may cause the overall brightness of the image to decrease and the tone to be cold, while smoke may cause the image to appear gray and reduce the saturation of the color. By analyzing these features, the relative proportion of water vapor and smoke in the blurred area can be estimated. Two coefficients are determined according to the relative proportion of water vapor and smoke: the first coefficient is related to the brightness in the color feature, and the second coefficient is related to the saturation in the color feature. The values ​​of these two coefficients depend on the specific proportion of water vapor and smoke in the blurred area and the degree of their influence on the brightness and saturation of the image. For example, if the proportion of water vapor in the blurred area is high, a larger first coefficient will be set to reflect the degree of brightness reduction; and if the proportion of smoke is high, a smaller second coefficient may be set to reflect the degree of saturation reduction. The image depth is calculated using the previously determined first coefficient and second coefficient in combination with the color features of the blurred area (including the original brightness and saturation values). The determination of the image depth may be based on one or more algorithms that estimate the distance or blur degree of the object in the blurred area based on the changes in brightness and saturation and their relationship with the first coefficient and the second coefficient. For example, one possible algorithm is to first adjust the brightness value according to the first coefficient to reflect the effect of water vapor on the image brightness, then adjust the saturation value according to the second coefficient to reflect the effect of smoke on the image color, and finally estimate the image depth in combination with the adjusted brightness and saturation values.

[0069] In the above steps, the blurred area is processed for clarity according to the image depth, including: obtaining the blurred image corresponding to the blurred area, the atmospheric light value and the image depth of the environment in which the target image is located; determining a clarity algorithm according to the blurred image, the atmospheric light value and the image depth; and processing the blurred area for clarity according to the clarity algorithm.

[0070] In an embodiment of the present application, it is first necessary to determine the specific position of the blurred area in the image and extract the blurred image corresponding to the area. At the same time, it is also necessary to obtain the atmospheric light value (usually a global parameter) of the environment in which the target image is located, which represents the intensity and color of the light in the environment. In addition, it is also necessary to determine the relative distance or degree of blur of the object in the blurred area based on the image depth information determined in the previous step. After obtaining the above data, the clear algorithm can be determined, and then the blurred area can be cleared by the clear algorithm.

[0071] Specifically, the blurred area is converted to the YUV or HSI color space, and then the contrast brightness (Y), saturation (S) and energy value of the image (obtained by discrete wavelet transform) are extracted; based on the known sample data and analysis results, a deep learning model or statistical model can be established, through which the contrast brightness, saturation and energy values ​​can be used as input to infer the relative proportion of water vapor and smoke in the blurred area. For example, the model may learn that in a typical smoke image, the brightness value is low and the saturation is high, while in a typical water vapor image, the brightness value is relatively high and the saturation is low. At the same time, the model will also consider the energy value of the image, because smoke and water vapor will have different degrees of influence on the high-frequency components of the image. The blurred area to be analyzed is input into the trained model, and the relative proportion of water vapor and smoke in the blurred area can be obtained through the calculation of the model. In this way, the relative proportion of water vapor and smoke in the blurred image can be inferred based on the contrast brightness, saturation and energy values ​​by using image processing and deep learning technology, and the image is further optimized to reduce or eliminate the blurring effect of smoke and water vapor on the image.

[0072] The image depth model is established by the following formula:

[0073] D(X)=α 0 +α 1 Y+α 2 S+β

[0074] Where D(X) is the image depth, β is the random error, and α 0 is the overall linear coefficient, α 1 is the brightness linear coefficient (i.e. the first coefficient mentioned above), α 2 is the saturation linear coefficient (ie the second coefficient mentioned above).

[0075] The relevant parameter α is obtained by analyzing a large number of smoke images with different concentrations and water vapor images with different concentrations. 1 and α 2, the image is optimized according to the clear algorithm. If smoke and water vapor exist in the blurred image at the same time, the linear relationship between the brightness and saturation of the image change is a fixed ratio. The image energy value is introduced to determine it. The three-dimensional coordinates are constructed by brightness, saturation and energy values. The brightness and saturation can be fixed in the three-dimensional coordinates, thereby fixing the parameter α 2 and α 1 .

[0076] Taking the clarity algorithm further:

[0077]

[0078] Where t(x) = e -μD(X) ; e is a constant value, J( x ) is a clear image, I( x ) is the blurred image, A is the atmospheric light value, and μ represents the parameter.

[0079] In addition, the embodiment of the present application also provides an Internet-based campus food safety monitoring platform, including: a device layer, a capability layer, an application layer, and a user layer. The device layer collects data through sensors, and the application layer analyzes whether there are food safety issues through the collected data. The application layer is used to provide specific applications, including data display, data analysis results and other functions, and the user layer is used for users to log in and query corresponding data; the platform realizes the functions of online supervision of the back kitchen, comprehensive data display, online viewing, video aggregation, etc., creates and uses a unified supervision platform for bright kitchens and stoves, supports the management of merchant information, supports automatic alarms for violations and pushes SMS reminders, can remotely supervise online and issue supervision reports, reduce supervision costs, and improve supervision efficiency; at the same time, through the web management system, it provides regulators with a unified management entrance for catering enterprises, supports the release of food safety information, and uses information management methods to classify and statistically manage various data, greatly reducing the workload of supervisors.

[0080] The above-mentioned security monitoring platform is used to realize the analysis of the following videos, and specifically reflects the kitchen hygiene conditions according to the following situations, such as: abnormal organism (such as mice, etc.) detection, identification of abnormal organisms entering the detection area, if abnormal organisms are detected, an alarm is immediately issued, and the alarm information is pushed to the management personnel simultaneously, effectively ensuring the safety and hygiene of the back kitchen; uncovered trash can detection: supports various types of common trash cans, and supports yellow, green, blue, and red large trash cans to effectively distinguish from ordinary buckets; ground garbage detection: monitors ground garbage and issues an alarm after detecting garbage; chef uniform detection: draws a detection area in the picture, and identifies and alarms people who are not wearing white chef uniforms in the video stream; chef hat detection: draws a detection area in the picture, and identifies and alarms people who are not wearing white chef hats in the video stream; mask wearing detection: monitors the wearing of masks by the back kitchen staff, and immediately marks and reminds those who are not wearing them, effectively ensuring the safety and hygiene of the workplace; smoking / calling recognition detection: automatically identifies people entering the detection area in real time, and immediately generates an alarm signal if it is detected that the person is smoking or calling.

[0081] The identification of the above-mentioned abnormal creatures specifically includes: obtaining multi-angle and multi-feature images of abnormal creatures, and inputting them into a neural network model for training; extracting feature images of abnormal creatures; inputting suspicious images in surveillance videos into the DarkNet53 network, and extracting feature results in the suspicious images; inputting the feature results into the feature space for adjustment to obtain a feature image to be matched. If the similarity between the feature image to be matched and the feature image reaches a preset value, it is considered that an abnormal creature appears in the suspicious image.

[0082] In the feature space, the concept of Gaussian kernel is introduced. Given the preset feature (i.e., the feature corresponding to the abnormal organism) R as a kernel, x is defined as a subset of R, and the kernel function K h (R,x) represents the mapping of x in space h. By calculating the inner product of the kernel function, we can get the similarity between x and R features in the feature space. According to the Gaussian kernel k(R,x):

[0083]

[0084] Among them, σ is an adjustable parameter. Since the feature space of the Gaussian kernel is infinite-dimensional, the weight ratio of the adjustable parameter σ to the space of each dimension is consistent. Therefore, two adjustable parameters c and d are introduced to use different widths to express the feature information in different dimensions:

[0085]

[0086] Finally, c is obtained by linearly superimposing Gaussian kernels of different widths, and d cis the weight coefficient, which controls the relative weight of the Gaussian kernel according to the Gaussian kernels of different widths, and finally synthesizes the accurate feature image to be matched in the feature space.

[0087] Specifically: During the monitoring process, due to the movement of abnormal creatures, the monitored angles are also different, and each angle will present different features. For example, the front angle may show the facial features of abnormal creatures, including the shape and color of eyes, nose and ears; the side angle may show the body outline and tail shape of abnormal creatures; the upper angle may show the overall appearance and color of abnormal creatures; now these abnormal creature images at different angles are input into the neural network model for training. Through this training process, the neural network will learn how to identify and extract various features of abnormal creatures from images at different angles; then, when suspicious images are captured in the surveillance video, these images will be input into the DarkNet53 network for feature extraction. The DarkNet53 network will extract corresponding feature results according to different angles of the image, such as facial features at the front angle, body outline at the side angle, etc.; then, these feature results are input into the feature space for adjustment. In this feature space, the Gaussian kernel function is used to compare the feature similarity at different angles to determine whether the features of abnormal creatures are matched. This process takes into account the feature information at different angles and uses the kernel function to calculate the similarity; finally, by adjusting the weight coefficient, an accurate feature image to be matched is synthesized. This process will comprehensively consider the characteristics of abnormal organisms at various angles and obtain accurate matching results; therefore, based on the shooting factors at different angles, the above technical solution can train the computer to recognize and match the image characteristics of abnormal organisms at different angles, and realize the automatic recognition and detection of abnormal organisms in all directions and at multiple angles.

[0088] In addition, the above security monitoring platform also establishes a security protection module in the capability layer, which mainly involves:

[0089] Identity authentication: includes both host and application aspects. Authentication is required for host operating system login, database login, and application system login. Identifiers and passwords that are too simple can be easily cracked by exhaustive attacks. At the same time, illegal users can eavesdrop on the network and obtain administrator privileges, which can allow illegal access to any resource and unauthorized operations. Therefore, the complexity of the username / password must be increased, and network eavesdropping must be prevented; at the same time, failure handling mechanisms should be considered.

[0090] Access control: includes two aspects: host and application. Access control is mainly to ensure the legal use of host resources and application system resources by users. Illegal users may attempt to impersonate legitimate users to enter the system, and legitimate users with low permissions may also attempt to perform operations of high-privileged users. These behaviors will bring great security risks to the host system and application system. Users must have legitimate user identifiers and operate under the established access control policies to prevent unauthorized and illegal operations.

[0091] Intrusion prevention: The host operating system faces various targeted intrusion threats. Common operating systems have various security vulnerabilities, and the time difference between the discovery of a vulnerability and its exploitation is getting shorter and shorter. This makes the security of the operating system itself a huge security risk to the entire system. Therefore, requirements are put forward for the installation, use, and maintenance of the host operating system to prevent intrusions against the system.

[0092] Malicious code prevention: Malicious code is the biggest hidden danger that harms the computing environment. The current virus threat is very serious. It will quickly spread to other subnets and launch network attacks and data theft. It will occupy a large amount of bandwidth that is very limited for normal business, causing serious degradation of network performance, server crashes, and even network communication interruptions, and information damage or leakage. It seriously affects the normal business. Therefore, it is necessary to deploy malicious code prevention software for defense, and keep the malicious code library updated in a timely manner.

[0093] Data security: mainly refers to the integrity and confidentiality of data. Measures should be taken to ensure the integrity and confidentiality of data during transmission; protect the confidentiality of identification information.

[0094] Figure 3 is a structural diagram of an image optimization device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0095] A detection module 30, configured to detect a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame;

[0096] A determination module 32, used to determine characteristic information of the blurred area, wherein the characteristic information includes color characteristics and energy values;

[0097] The processing module 34 is used to perform clear processing on the blurred area according to the feature information.

[0098] Through the detection module 30, determination module 32 and processing module 34 in the above-mentioned image optimization device, the purpose of enhancing picture clarity is achieved, thereby achieving the technical effect of improving video quality and optimizing viewing experience, and further solving the technical problem of being unable to perform accurate monitoring when there is smoke and water vapor in the environment where the surveillance video is located.

[0099] In the detection module in the above-mentioned image optimization device, the detection module is also used to obtain target pixel points of the target image and background pixel points in the background image, wherein the target pixel point is any pixel point in the target image and the background pixel point is any pixel point in the background image; determine the difference between the target pixel point and the background pixel point; and determine the area formed by the target pixel points whose absolute value of the difference is greater than a preset threshold as a blurred area.

[0100] The above-mentioned image optimization device also includes an update module 36, which is used to determine the area formed by target pixel points whose absolute value of difference is less than or equal to a preset threshold as the background area of ​​the target image; obtain a first pixel point in the background area, and obtain a second pixel point in the background image, wherein the background image and the target image have the same size, the position of the first pixel point in the target image and the position of the second pixel point in the background image are the same, the first pixel point is any pixel point in the background area, and the second pixel point is any pixel point in the background image; update the background image based on the first pixel point, the second pixel point and the background update factor.

[0101] In the determination module in the above-mentioned image optimization device, the determination module is also used to obtain the contrast brightness and saturation of the blurred area; determine the color characteristics based on the contrast brightness and saturation; perform discrete wavelet transform on the blurred area to obtain a high-frequency component sub-image, wherein the high-frequency component sub-image includes a first image, a second image and a third image, the first image includes texture information of the blurred area in the horizontal direction, the second image includes texture information of the blurred area in the vertical direction, and the third image includes texture information of the blurred area in the diagonal direction; determine the energy value of the blurred area based on the high-frequency component sub-image; determine the characteristic information of the blurred area based on the color characteristics and the energy value.

[0102] In the processing module in the above-mentioned image optimization device, the processing module is also used to determine the image type of the blurred area based on the range of the energy value in the feature information, wherein the image types include water vapor images and smoke images; determine the image depth of the blurred area based on the image type and color characteristics of the blurred area; and perform clarity processing on the blurred area based on the image depth.

[0103] In the processing module in the above-mentioned image optimization device, the processing module is also used to determine the relative proportion of water vapor and smoke in the blurred area based on the image type and color characteristics of the blurred area; determine the first coefficient corresponding to the brightness in the color characteristics based on the relative proportion, and determine the second coefficient corresponding to the saturation in the color characteristics; determine the image depth based on the first coefficient, the second coefficient and the color characteristics.

[0104] In the processing module in the above-mentioned image optimization device, the processing module is also used to obtain the blurred image corresponding to the blurred area, the atmospheric light value and the image depth of the environment in which the target image is located; determine the clarity algorithm based on the blurred image, the atmospheric light value and the image depth; and perform clarity processing on the blurred area based on the clarity algorithm.

[0105] It should be noted that Figure 3 The image optimization device shown is used to perform Figure 2 The image optimization method shown in the figure, therefore the relevant explanations in the above-mentioned image optimization method are also applicable to the image optimization device, and will not be repeated here.

[0106] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute program instructions to implement the following functions: detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; determining feature information of the blurred area, wherein the feature information includes color features and energy values; and clearly processing the blurred area based on the feature information.

[0107] It should be noted that the above electronic equipment is used to execute Figure 2 The image optimization method shown in the figure, therefore the relevant explanations and instructions in the above-mentioned image optimization method are also applicable to the electronic device and will not be repeated here.

[0108] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following image optimization method by running the computer program: detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; determining feature information of the blurred area, wherein the feature information includes color features and energy values; and clearly processing the blurred area based on the feature information.

[0109] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The image optimization method shown, therefore the relevant explanations in the above image optimization method are also applicable to the non-volatile storage medium, and will not be repeated here.

[0110] The embodiments of the present application also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the image optimization method in each embodiment of the present application.

[0111] The embodiments of the present application also provide a computer program, which, when executed by a processor, implements the steps of the image optimization method in each embodiment of the present application.

[0112] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0118] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for image optimization, characterized in that: include: Detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; Determining feature information of the blurred area, wherein the feature information includes color features and energy values; The blurred area is clarified according to the feature information.

2. The method according to claim 1, characterized in that Detect the target image in the video frame and obtain the blurred area, including: Acquire a target pixel of the target image, and acquire a background pixel in the background image, wherein the target pixel is any pixel in the target image, and the background pixel is any pixel in the background image; Determine the difference between the target pixel and the background pixel; The area formed by the target pixel points whose absolute value of the difference is greater than a preset threshold is determined as the fuzzy area.

3. The method according to claim 2, characterized in that After obtaining the fuzzy area, the method further includes: Determine an area formed by target pixel points whose absolute value of the difference is less than or equal to the preset threshold as the background area of ​​the target image; Acquire a first pixel point in the background area, and acquire a second pixel point in the background image, wherein the background image and the target image have the same size, the position of the first pixel point in the target image and the position of the second pixel point in the background image are the same, the first pixel point is any pixel point in the background area, and the second pixel point is any pixel point in the background image; The background image is updated according to the first pixel point, the second pixel point and a background update factor.

4. The method according to claim 1, characterized in that Determining characteristic information of the fuzzy area includes: Obtaining the contrast brightness and saturation of the blurred area; Determining the color feature according to the contrast brightness and the saturation; Performing discrete wavelet transform on the blurred area to obtain a high-frequency component sub-image, wherein the high-frequency component sub-image includes a first image, a second image, and a third image, the first image includes texture information of the blurred area in a horizontal direction, the second image includes texture information of the blurred area in a vertical direction, and the third image includes texture information of the blurred area in a diagonal direction; Determining the energy value of the blurred area according to the high-frequency component sub-image; Determine feature information of the blurred area based on the color feature and the energy value.

5. The method according to claim 1, characterized in that: The blurred area is processed clearly according to the characteristic information, including: Determining the image type of the blurred area according to the range to which the energy value in the characteristic information belongs, wherein the image type includes a water vapor image and a smoke image; Determining the image depth of the blurred area according to the image type of the blurred area and the color feature; The blurred area is processed to be clear according to the image depth.

6. The method according to claim 5, characterized in that Determining the image depth of the blurred area includes: Determining the relative proportion of water vapor and smoke in the blurred area according to the image type and the color characteristics of the blurred area; Determine a first coefficient corresponding to brightness in the color feature according to the relative proportion, and determine a second coefficient corresponding to saturation in the color feature; The image depth is determined according to the first coefficient, the second coefficient and the color feature.

7. The method according to claim 5, characterized in that The blurred area is processed clearly according to the image depth, including: Acquire a blurred image corresponding to the blurred area, an atmospheric light value of an environment in which the target image is located, and the image depth; Determining a clear algorithm according to the blurred image, the atmospheric light value and the image depth; The blurred area is subjected to a clearing process according to the clearing algorithm.

8. An image optimization device, characterized in that: include: A detection module, used to detect a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; A determination module, used to determine feature information of the fuzzy area, wherein the feature information includes color features and energy values; A processing module is used to perform clear processing on the blurred area according to the feature information.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is connected to the memory and is used to execute program instructions to implement the following functions: detecting a target image in a video frame to obtain a blurred area, wherein the target image is any image in the video frame; determining feature information of the blurred area, wherein the feature information includes color features and energy values; and performing clear processing on the blurred area based on the feature information.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the image optimization method described in any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the image optimization method described in any one of claims 1 to 7 is implemented.

Images