Image recognition management system based on video monitoring

Abstract

The invention discloses a picture recognition management system based on video monitoring, and relates to the technical field of image management. The image processing unit is used for collecting image features of the basic data and carrying out gray processing on the image features, the image recognition management system based on video monitoring can enhance the image features by adjusting the foreground and background of a gray image, detail optimization can be carried out when character feature recognition blurring exists in a live broadcast image, and the image recognition efficiency is improved. The quality of the live broadcast picture is enhanced, and continuous adjustment is performed on the enhanced image features, so that the situation that the attention of the live broadcast room is not on a product and public opinions are caused due to sudden picture adjustment can be effectively avoided, a good live broadcast picture effect can be ensured, the attention of the live broadcast room can be reduced, and the user experience is improved. And the adjustment sequence of the plurality of image pictures is analyzed under the delimited rule, so that the pictures can be managed in a unified manner, and the value maximization of live broadcast picture optimization is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image management, and in particular to a picture recognition management system based on video monitoring. Background Art

[0002] In the live broadcast industry, when monitoring the video of the live broadcast, multiple images may appear on the same screen. In order to achieve a good picture effect, the images are usually optimized. However, in the existing picture recognition management process, only the beauty effect of the picture is improved. However, due to the lighting, the recognition of human features is unclear, and there is ambiguity in local features, which affects the authenticity of the live broadcast. In addition, the existing picture addition processing is too abrupt, and the rapid picture adjustment will cause the attention of the live broadcast room to be not on the product, triggering public opinion and affecting the live broadcast effect. In addition, the adjustment of the live broadcast picture lacks systematic management, and the value of live broadcast picture optimization is not guaranteed to be maximized. For this reason, the present invention proposes a picture recognition management system based on video monitoring to solve the above-mentioned problems. Summary of the Invention

[0003] (1) Technical problems solved

[0004] In view of the deficiencies in the prior art, the present invention provides a picture recognition and management system based on video monitoring, which solves the problems mentioned in the above background technology.

[0005] (2) Technical solution

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a picture recognition management system based on video monitoring, comprising:

[0007] Image processing unit: collects image features of basic data, performs grayscale processing on the image features, and then enhances the image features by adjusting the foreground and background of the grayscale image, wherein the basic data is specifically a live image screen;

[0008] A hierarchical adjustment unit: establishing a hierarchical management system for image feature enhancement, and continuously adjusting the enhanced image features so that the image features are enhanced in sequence according to time, wherein the image feature enhancement is performed within a time T;

[0009] Picture management unit: At this time, multiple image pictures are displayed on the same screen, and the adjustment order of the multiple image pictures is analyzed according to the demarcation rules, wherein the demarcation rules are specifically the popularity of the live screen.

[0010] As an improved technical solution, the image processing unit performs grayscale processing on the image features, and then enhances the image features by adjusting the foreground and background of the grayscale image.

[0011] S1: Let the live image be t, the original image be a color image, the color of each pixel in the color image is determined by Yj, according to F(t) = αY(j), according to F(t) = αY(j), F(t) is the grayscale live image pixel, α is the weighting coefficient of the color component;

[0012] S2: After the image is grayed, an image segmentation algorithm is used to separate the image foreground and background, extract the human features in the image, and segment the image foreground and background so that the image foreground grayscale is higher than the background grayscale, and then low-light image feature enhancement is performed.

[0013] As an improved technical solution, the foreground grayscale of the image in S2 is higher than the background grayscale, and the specific method of low-light image feature enhancement is as follows:

[0014] S21: Extract the image’s texture features and frequency domain features. The specific formula is as follows:

[0015]

[0016] In the formula, P represents the overall mean of the image, B n The random variable representing the pixel reflection value, F(B n ) represents the image histogram, n represents the number of image pixels, the larger the image mean P is, the brighter the image is, and vice versa, the smaller P is, the darker the image is.

[0017] S22: Then, a Retinex neural network model is constructed to convert the original image from the RGB space to the YCrCb space. The Y component is processed separately using the network model, and only the Cr and Cb components are processed using guided filtering. A multi-scale average algorithm is used to adaptively enhance the low-light image.

[0018] As an improved technical solution, the hierarchical adjustment unit continuously adjusts the enhanced image features so that the image features are enhanced in sequence according to the time sequence.

[0019] P1: First, the initial illumination of the image feature is recorded as ZD, and then the adjusted illumination of the image feature is recorded as TD. Then, the time range YT is set, and the ZD is adjusted to the TD range for calculation. Where K is the curvature of the image feature illumination adjustment;

[0020] P2: Then, based on the curvature K, a coordinate system for illumination adjustment is established with time as the horizontal axis and illumination as the vertical axis. The illumination of the image feature is linearly adjusted within the time range YT according to the value of the coordinate system.

[0021] As an improved technical solution, when linearly adjusting the illumination of the image features within the time range YT according to the numerical values of the coordinate system in P2, first extract the image features in the initial state, and then extract the adjusted image features. After confirmation by the video monitoring client, the adjustment can be carried out.

[0022] As an improved technical solution, the specific method for the screen management unit to analyze the adjustment order of the several image screens according to the defined rules is as follows:

[0023] M1: First, record the screen displayed on the same screen as Rj, where j is the serial number of the image screen. Then, record the live broadcast popularity of the screen Rj in real time and denote it as ERj. Arrange Rj in descending order according to ERj. Among them, the adjustment order of the image features of the screen is carried out according to the arrangement order of Rj.

[0024] M2: If the arrangement order of Rj changes during the process of image adjustment according to the arrangement order of Rj, the process of image adjustment terminates or until the current adjustment is completed. Specifically:

[0025] Set the difference range WE of the live broadcast popularity. Let the live broadcast popularity of the currently adjusted screen be A1, and the live broadcast popularity of the screen that needs to be adjusted first after the arrangement order of Rj changes be An. If any one of An≥L×A1, at this time, terminate the image features of the current adjustment and adjust the image features of An first. If all An<L×A1, at this time, wait until the image features of the current screen are adjusted and then perform the sorting adjustment.

[0026] As an improved technical solution, when the adjustment order of the image features of the screen in M1 is carried out according to the arrangement order of Rj, it is not adjusted in the initial state and is randomly arranged, and the arrangement order of Rj is adjusted after the start time reaches t.

[0027] (III) Beneficial effects

[0028] The present invention provides a screen recognition management system based on video monitoring. Compared with the prior art, it has the following beneficial effects:

[0029] This screen recognition management system based on video monitoring can enhance the image features by adjusting the foreground and background of the grayscale image. When the character features in the live broadcast screen are blurred, it can optimize the details, enhance the quality of the live broadcast screen, and continuously adjust the enhanced image features, which can effectively avoid sudden screen adjustments that may cause the attention in the live broadcast room not to be on the product and trigger public opinion. It can not only ensure a good live broadcast screen effect but also reduce the attention in the live broadcast room, and analyze the adjustment order of several image screens according to the defined rules, which can uniformly manage the screens to maximize the value of optimizing the live broadcast screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a block diagram of a video surveillance-based image recognition management system according to an embodiment of the present application;

[0031] Figure 2 This is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] See also Figure 1-Figure 2 , the embodiment of the present invention provides five technical solutions:

[0034] Example 1

[0035] A picture recognition management system based on video monitoring, wherein the image processing unit collects image features of basic data, performs grayscale processing on the image features, and then enhances the image features by adjusting the foreground and background of the grayscale image. The basic data is specifically a live image. The image processing unit performs grayscale processing on the image features and then enhances the image features by adjusting the foreground and background of the grayscale image in the following manner:

[0036] S1: Let the live image be t, the original image be a color image, the color of each pixel in the color image is determined by Yj, according to F(t) = αY(j), according to F(t) = αY(j), F(t) is the grayscale live image pixel, α is the weighting coefficient of the color component;

[0037] S2: After the image is grayed, an image segmentation algorithm is used to separate the image foreground and background, extract the human features in the image, and segment the image foreground and background so that the image foreground grayscale is higher than the background grayscale, and then perform low-light image feature enhancement. In S2, the image foreground grayscale is higher than the background grayscale, and the specific method for low-light image feature enhancement is as follows:

[0038] S21: Extract the image’s texture features and frequency domain features. The specific formula is as follows:

[0039]

[0040] In the formula, P represents the overall mean of the image, B nThe random variable representing the pixel reflection value, F(B n ) represents the image histogram, n represents the number of image pixels, the larger the image mean P is, the brighter the image is, and vice versa, the smaller P is, the darker the image is.

[0041] S22: Then, a Retinex neural network model is constructed to transform the original image from RGB space to YCrCb space. The network model is used to process the Y component alone, and only the Cr and Cb components are processed by guided filtering. A multi-scale average algorithm is used to adaptively enhance the low-light image. The specific expression for adaptive enhancement of the low-light image is:

[0042]

[0043] Where R(x, y) represents the enhanced image, M(×, y) represents the enhanced reflection image of the Y component, and K(x, y) represents the enhanced illumination image of the Y component. Represents element-wise product operation, C r (x, y) represents the enhanced C r Quantity, C b (x, y) represents the enhanced C b Quantity;

[0044] The expressions of M(x, y) and K(x, y) are:

[0045] M(x, y) = RTN_D θ (Y(x,y))

[0046] K(x, y) = RTN_E α (Y(x,y))

[0047] Where, RTN_D θ Represents the decomposition sub-network model based on Retinex, RTN_E α (Y(x, y)) represents the Retinex-based enhancement subnetwork model, θ and α represent the learnable network parameters, and Y(x, y) represents the Y component of the original input image.

[0048] By constructing the Retinex-Net model, we can achieve good image enhancement effects, retain more image detail information, and optimize the image with dynamic histogram equalization and homomorphic filtering to reduce the degree of distortion in image optimization.

[0049] Hierarchical adjustment unit: Establish a hierarchical management system for image feature enhancement, continuously adjust the enhanced image features so that the image features are enhanced in chronological order, wherein the image feature enhancement is performed within a time T; the specific manner in which the hierarchical adjustment unit continuously adjusts the enhanced image features so that the image features are enhanced in chronological order is:

[0050] P1: First, the initial illumination of the image feature is recorded as ZD, and then the adjusted illumination of the image feature is recorded as TD. Then, the time range YT is set within 1-10s, and the ZD is adjusted to the TD range for calculation. Where K is the curvature of the image feature illumination adjustment;

[0051] A further design may be to set a standard numerical range of curvature, recorded as (AK, BK), where K should be within the range of (AK, BK), otherwise the image adjustment will be too obvious due to the large fluctuation of the curvature.

[0052] P2: Then, based on the curvature K, a coordinate system for illumination adjustment is established with time as the horizontal axis and illumination as the vertical axis. The illumination of the image features is linearly adjusted within the time range YT based on the values ​​in the coordinate system. In P2, when linearly adjusting the illumination of the image features within the time range YT based on the values ​​in the coordinate system, the initial image features are first extracted, and then the adjusted image features are extracted. After confirmation by the video monitoring client, adjustments can be made.

[0053] Image management unit: At this time, multiple images are displayed on the same screen, and the adjustment order of the multiple images is analyzed according to the demarcation rules. The demarcation rules are specifically based on the popularity of the live images. The specific method of analyzing the adjustment order of the multiple images according to the demarcation rules in the image management unit is as follows:

[0054] M1: First, the pictures displayed on the same screen are recorded as Rj, where j is the sequence number of the picture. Then, the live broadcast popularity of picture Rj is recorded in real time and recorded as ERj. Rj is arranged in descending order according to ERj. The order of adjusting the image features of the pictures is carried out according to the arrangement order of Rj. When the order of adjusting the image features of the pictures in M1 is carried out according to the arrangement order of Rj, it is not adjusted in the initial state and the arrangement is random. The arrangement order of Rj is adjusted after the broadcast start time reaches t.

[0055] M2: If the order of Rj changes during the image adjustment process, the image adjustment process is terminated or the current adjustment is completed. Specifically:

[0056] Set the difference range WE of the live broadcast popularity, with the currently adjusted live broadcast popularity as A1. After the arrangement order of Rj changes, the live broadcast popularity of the picture that needs to be adjusted first is An. If any one of An≥L×A1, the current adjustment of the image features is terminated at this time, and the image features of An are adjusted first. If all An<L×A1, wait until the image features of the current picture are adjusted before sorting and adjusting.

[0057] Based on actual analysis, for example, there are 6 pictures on the same screen. At this time, they are divided into R1, R2, R3, R4, R5 and R6 in order of popularity. When R4 is subjected to image feature enhancement processing, the popularity of R5 soars. For example, the popularity of R4 is 10,000 people, and the popularity of R5 soars to 30,000 people. At this time, the coefficient L is 2, and R5 ≥ 4×R4.

[0058] Example 2

[0059] This embodiment is based on the first embodiment and differs from the first embodiment in that it further includes a data storage unit for storing data of the image processing unit, the grading adjustment unit and the picture management unit.

[0060] Example 3

[0061] Compared with the first and second embodiments, the technical solution of this embodiment is to combine the solutions of the first and second embodiments.

[0062] Example 4

[0063] A picture recognition management system based on video monitoring, comprising:

[0064] Image processing unit: collects image features of basic data, performs grayscale processing on the image features, and then enhances the image features by adjusting the foreground and background of the grayscale image, wherein the basic data is specifically a live image screen;

[0065] A hierarchical adjustment unit: establishing a hierarchical management system for image feature enhancement, and continuously adjusting the enhanced image features so that the image features are enhanced in sequence according to time, wherein the image feature enhancement is performed within a time T;

[0066] The image management unit displays multiple images on the same screen and analyzes the adjustment order of the multiple images according to the demarcation rules, wherein the demarcation rules are specifically the popularity of the live images;

[0067] Data storage unit: used to store data of the image processing unit, grading adjustment unit and picture management unit.

[0068] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0069] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0070] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0071] Example 5

[0072] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an electronic device and corresponding embodiments.

[0073] See also Figure 2 , the electronic device 1000 includes a memory 1010 and a processor 1020.

[0074] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0075] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all instructions and data required by the processor during operation. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0076] The memory 1010 stores executable codes. When the executable codes are processed by the processor 1020 , the processor 1020 may execute part or all of the above-mentioned methods.

[0077] The scheme of the present application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. Those skilled in the art should also be aware that the actions and modules involved in the description are not necessarily required for this application. In addition, it is understood that the steps in the method of the embodiment of the present application can be adjusted in sequence, merged and deleted according to actual needs, and the modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0078] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0079] Alternatively, the present application can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or electronic device, server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0080] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the application herein may be implemented as electronic hardware, computer software, or combinations of both.

[0081] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems and methods according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0082] The embodiments of the present application have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A picture recognition management system based on video monitoring, characterized in that: include: Image processing unit: collects image features of basic data, performs grayscale processing on the image features, and then enhances the image features by adjusting the foreground and background of the grayscale image, wherein the basic data is specifically a live image screen; A hierarchical adjustment unit: establishing a hierarchical management system for image feature enhancement, and continuously adjusting the enhanced image features so that the image features are enhanced in sequence according to time, wherein the image feature enhancement is performed within a time T; Picture management unit: At this time, multiple image pictures are displayed on the same screen, and the adjustment order of the multiple image pictures is analyzed according to the demarcation rules, wherein the demarcation rules are specifically the popularity of the live screen.

2. The image recognition management system based on video monitoring according to claim 1, characterized in that: The specific manner in which the image processing unit performs grayscale processing on the image features and then enhances the image features by adjusting the foreground and background of the grayscale image is as follows: S1: Let the live image be t, the original image be a color image, the color of each pixel in the color image is determined by Yj, according to F(t) = αY(j), according to F(t) = αY(j), F(t) is the grayscale live image pixel, α is the weighting coefficient of the color component; S2: After the image is grayed, an image segmentation algorithm is used to separate the image foreground and background, extract the human features in the image, and segment the image foreground and background so that the image foreground grayscale is higher than the background grayscale, and then low-light image feature enhancement is performed.

3. The image recognition management system based on video monitoring according to claim 1, characterized in that: The foreground grayscale of the image in S2 is higher than the background grayscale, and the specific method of low-light image feature enhancement is as follows: S21: Extract the image’s texture features and frequency domain features. The specific formula is as follows: In the formula, P represents the overall mean of the image, B n The random variable representing the pixel reflection value, F(B n ) represents the image histogram, n represents the number of image pixels, the larger the image mean P is, the brighter the image is, and vice versa, the smaller P is, the darker the image is. S22: Then, a Retinex neural network model is constructed to convert the original image from the RGB space to the YCrCb space. The Y component is processed separately using the network model, and only the Cr and Cb components are processed using guided filtering. A multi-scale average algorithm is used to adaptively enhance the low-light image.

4. The image recognition management system based on video monitoring according to claim 1, characterized in that: The specific manner in which the hierarchical adjustment unit continuously adjusts the enhanced image features so that the image features are enhanced in sequence according to the time sequence is: P1: First, the initial illumination of the image feature is recorded as ZD, and then the adjusted illumination of the image feature is recorded as TD. Then, the time range YT is set, and the ZD is adjusted to the TD range for calculation. Where K is the curvature of the image feature illumination adjustment; P2: Then, based on the curvature K, a coordinate system for illumination adjustment is established with time as the horizontal axis and illumination as the vertical axis. The illumination of the image feature is linearly adjusted within the time range YT according to the value of the coordinate system.

5. The image recognition management system based on video monitoring according to claim 4, characterized in that: When linearly adjusting the illumination of the image features in the time range YT according to the numerical value of the coordinate system in P2, the image features of the initial state are first extracted, and then the adjusted image features are extracted. The adjustment can be made after confirmation by the video monitoring client.

6. The image recognition management system based on video monitoring according to claim 1, characterized in that: The specific manner in which the image management unit analyzes the adjustment order of the plurality of image images according to the defined rules is as follows: M1: First, record the images displayed on the same screen as Rj, where j is the sequence number of the image. Then, record the live broadcast popularity of image Rj in real time and record it as ERj. Arrange Rj in descending order according to ERj. The order of adjusting the image features of the images is based on the arrangement order of Rj. M2: If the arrangement order of Rj changes during the process of image adjustment according to the arrangement order of Rj, the process of image adjustment terminates or until the current adjustment is completed. Specifically: Set the difference range WE of the live broadcast popularity. Let the live broadcast popularity of the current adjusted screen be A1, and the live broadcast popularity of the screen to be adjusted first after the arrangement order of Rj changes be An. If any one of An ≥ L × A1, terminate the image features of the current adjustment at this time and adjust the image features of An first. If all An < L × A1, wait until the image features of the current screen are adjusted and then perform the sorting adjustment.

7. The image recognition management system based on video monitoring according to claim 5, characterized in that: When the adjustment order of the image features of the screen in M1 is carried out according to the arrangement order of Rj, it is not adjusted in the initial state and the arrangement is random. The arrangement order of Rj is adjusted after the broadcast time reaches t.

Images