Methods, devices, equipment and media for locating information displayed on the screen in surveillance images

Abstract

This invention discloses a method, apparatus, device, and medium for locating screen display information in a surveillance image. The method includes: extracting a target surveillance image from a video stream and reducing the target surveillance image to a size to be identified; generating at least one OSD (On-Screen Defect) highlighting image that matches the image to be identified; identifying target OSD regions in each OSD highlighting image based on the position of the target pixel value's pixel in each OSD highlighting image, and identifying regions that meet positional and size constraints; and marking the identified target OSD regions in each OSD highlighting image in the image to be identified to obtain the OSD location result matching the image to be identified. The technical solution of this invention can efficiently and accurately locate OSDs in a single frame of a surveillance image without any human resource investment, effectively improving the OSD detection accuracy and significantly reducing the false alarm rate of OSD detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for locating information displayed on a screen in a monitoring image. Background Technology

[0002] Video surveillance is now widely used in society. A key feature of video surveillance is the overlay of OSD (on-screen display) onto the video image, used to record information such as the time and location of the video for evidence preservation. OSDs, with their high contrast and sharp stroke edges, facilitate human visual identification, but also negatively impact image analysis based on video surveillance, leading to numerous false alarms and missed detections.

[0003] To address this issue, the traditional approach is to manually label the OSD (Optical Disk Deposition) location of each camera. Image detection algorithms then proactively avoid OSD locations during image analysis, thus mitigating the negative impact of OSDs on image analysis. Alternatively, OCR (optical character recognition) methods can be used for automatic OSD detection in images.

[0004] The main drawbacks of manually marking OSD locations are: 1. It requires a large amount of manpower; 2. It cannot handle changes in camera OSD locations, requiring maintenance personnel to re-mark OSD locations only after the problem is discovered, resulting in high maintenance costs; 3. For offline video analysis of recorded files, each video file needs to be independently marked with its OSD location, significantly reducing the efficiency of automated processing. Meanwhile, the main drawback of using OCR methods to detect OSD locations is that it may also detect text other than OSDs in the video surveillance, such as slogans on walls or billboards, leading to false OSD detections and making automatic detection difficult. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for locating screen display information in a surveillance image, so as to achieve efficient and accurate location of OSDs included in the surveillance image.

[0006] According to one aspect of the present invention, a method for locating screen display information in a surveillance image is provided, the method comprising:

[0007] Extract the target surveillance image from the video stream and reduce the target surveillance image to the size of the image to be identified;

[0008] Generate at least one OSD highlighting image that matches the image to be identified, wherein different OSD highlighting images are used to highlight OSDs of different brightness, and in the OSD highlighting image, each pixel used to potentially describe the OSD is set to the target pixel value.

[0009] Based on the position of the target pixel value in each OSD highlighted image, the target OSD region that meets the position and size constraints is identified in each OSD highlighted image.

[0010] The target OSD regions identified in each OSD-highlighted image are labeled in the image to be identified, thus obtaining the OSD localization results that match the image to be identified.

[0011] According to another aspect of the present invention, a device for locating screen display information in a monitoring image is also provided, the device comprising:

[0012] The image acquisition module is used to extract the target monitoring image from the video stream and reduce the target monitoring image to the image to be recognized;

[0013] The OSD highlighting image generation module is used to generate at least one OSD highlighting image that matches the image to be identified. Different OSD highlighting images are used to highlight OSDs with different brightness levels. In the OSD highlighting image, each pixel used to potentially describe the OSD is set to the target pixel value.

[0014] The target OSD region recognition module is used to identify target OSD regions that meet the position and size constraints in each OSD highlighting image based on the position of the pixel with the target pixel value in each OSD highlighting image.

[0015] The OSD localization result determination module is used to mark the target OSD regions identified in each OSD highlighted image in the image to be identified, thereby obtaining the OSD localization result that matches the image to be identified.

[0016] According to another aspect of the present invention, an electronic device is also provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the method for locating screen display information in a monitoring image according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the method for locating screen display information in a monitoring image as described in any embodiment of the present invention.

[0021] The technical solution of this invention provides a new way to locate OSDs using only image recognition technology. This method involves reducing the size of a target monitoring image captured from a video stream to an image to be identified; generating at least one OSD highlighting image that matches the image to be identified; identifying target OSD regions that meet position and size constraints in each OSD highlighting image based on the position of the target pixel value; and marking the identified target OSD regions in each OSD highlighting image in the image to be identified, thus obtaining the OSD location result matching the image to be identified. This approach provides a new way to locate OSDs using only image recognition technology. It can efficiently and accurately locate OSDs in a single frame of a monitoring image without any human resource investment, effectively improving the accuracy of OSD detection and significantly reducing the false alarm rate of OSD detection.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a method for locating OSDs in a surveillance image according to Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of another method for locating OSDs in a monitoring image according to Embodiment 2 of the present invention;

[0026] Figure 3 This is a flowchart of another method for locating OSDs in a monitoring image according to Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram illustrating the vertical division of an OSD highlighted image, to which the technical solution of this embodiment of the invention applies;

[0028] Figure 5This is a schematic diagram of the structure of an OSD positioning device in a monitoring image according to Embodiment 4 of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device that implements a method for locating OSDs in a monitoring image according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1 This is a flowchart illustrating a method for locating OSDs in a surveillance image according to Embodiment 1 of the present invention. This embodiment is applicable to situations where OSD locations are identified in a single frame of a surveillance image using image processing technology. The method can be executed by an OSD location device in the surveillance image. This device can be implemented in hardware and / or software and is generally configured in an electronic device with image processing capabilities, such as a terminal device or a server. Figure 1 As shown, the method includes:

[0034] S110. Extract the target monitoring image from the video stream and reduce the target monitoring image to the size of the image to be identified.

[0035] In this embodiment, the video stream can be obtained in real time from a designated surveillance camera. By decoding the video stream, multiple frames of surveillance images can be obtained. The target surveillance image is one frame from these multiple surveillance images.

[0036] In order to reduce the amount of computation and improve the computation response speed, the target monitoring image can first be converted into a grayscale image, and then the width and height of the grayscale image can be reduced to a set number of pixels (e.g., 960 or 640) or less to obtain the image to be identified for subsequent OSD positioning.

[0037] Optionally, the target monitoring image can be a YUV image, where the Y channel is the grayscale channel. By extracting the channel values ​​of the Y channel, the target monitoring image can be converted into a grayscale image.

[0038] Optionally, during the process of reducing the target monitoring image to the image to be identified, the image reduction (or compression) process can be carried out while maintaining the aspect ratio, so as to avoid deformation of the OSD in the target monitoring image.

[0039] S120. Generate at least one OSD highlighting image that matches the image to be identified.

[0040] Different OSD highlighting images are used to highlight OSDs with different brightness levels. In the OSD highlighting images, each pixel used to potentially describe the OSD is set to the target pixel value.

[0041] In this embodiment, considering that when multiple OSDs (e.g., time, location, and camera identification) are added to the same camera on the same monitoring image, the brightness of these multiple OSDs may vary. For example, if a first OSD for recording the current system time is added to the top of the monitoring image, and the brightness of the top image area is relatively high, a low-brightness first OSD, such as a black OSD, can be added to the top of the monitoring image to achieve high contrast between the OSD and the monitoring image. Similarly, if a second OSD for recording location information needs to be added to the bottom of the monitoring image, and the brightness of the bottom image area is relatively low, a high-brightness second OSD, such as a white OSD, can be added to the bottom of the monitoring image to achieve high contrast between the OSD and the monitoring image.

[0042] Furthermore, different OSD highlighting images can be generated for OSDs of different brightness levels included in the image to be identified. Continuing the previous example, if the image to be identified includes both a first OSD with white brightness and a second OSD with black brightness, then two OSD highlighting images corresponding to the first OSD and the second OSD can be generated respectively.

[0043] Specifically, each OSD highlighting image can be understood as an image that emphasizes one or more OSDs in the image to be identified at a set brightness. Optionally, the brightness range of an OSD to be highlighted can be determined first. Then, the pixel values ​​of each pixel in the image to be identified that are within the brightness range are enhanced (or weakened) in brightness, while the pixel values ​​of each pixel that are outside the brightness range are appropriately weakened (or enhanced) in brightness to distinguish the differences between the two types of pixels, thereby obtaining an OSD highlighting image.

[0044] In a specific example, the method of enhancing (or weakening) the brightness of pixel values ​​within a certain brightness range in the image to be identified can be as follows: Set the target pixel value for all pixels within that brightness range (i.e., those used to potentially describe the OSD); correspondingly, set other pixel values ​​that can be significantly distinguished from the target pixel value for pixels outside that brightness range. For example, if the target pixel value is 255, then other pixel values ​​could be 128. The reason for setting other pixel values ​​to 128 is that the image to be identified may contain white or black OSD strokes. By setting other pixel values ​​to 128, it can be ensured that these pixels, which are not used to potentially describe the OSD, have sufficient contrast with both white and black OSD strokes.

[0045] S130. Based on the position of the target pixel value in each OSD highlighting image, identify the target OSD region that meets the position and size constraints in each OSD highlighting image.

[0046] In this embodiment, considering that the OSD included in each OSD highlighting image should be located within a preset image area and should have a preset OSD size, the target OSD area that meets both the position and size constraints is accurately located by analyzing the position of the target pixel value in each OSD highlighting image.

[0047] The target OSD region can be understood as the marked location of an OSD (e.g., time marker information or location marker information) in the OSD highlighting image.

[0048] S140. The target OSD regions identified in each OSD highlighting image are marked in the image to be identified to obtain the OSD localization result that matches the image to be identified.

[0049] As mentioned earlier, since different OSD highlighting images may contain OSDs of varying brightness, by identifying each OSD highlighting image separately, all target OSD regions included in the image to be identified can be obtained. Furthermore, by merging the results of each identified target OSD region in the image to be identified, a complete OSD localization result can be obtained.

[0050] Furthermore, after obtaining the OSD localization result matching the image to be identified, the image to be identified and the OSD localization result can be provided as a whole to the video analysis and processing module. This module then combines the OSD localization result with the image to be identified to perform accurate image analysis, and when specific anomalies are identified, corresponding monitoring and early warnings are issued. When the accuracy of the OSD localization result is guaranteed, the false alarm rate of the video analysis and early warning will be greatly reduced.

[0051] The technical solution of this invention provides a new way to locate OSDs using only image recognition technology. This method involves reducing the size of a target monitoring image captured from a video stream to an image to be identified; generating at least one OSD highlighting image that matches the image to be identified; identifying target OSD regions that meet position and size constraints in each OSD highlighting image based on the position of the target pixel value; and marking the identified target OSD regions in each OSD highlighting image in the image to be identified, thus obtaining the OSD location result matching the image to be identified. This approach provides a new way to locate OSDs using only image recognition technology. It can efficiently and accurately locate OSDs in a single frame of a monitoring image without any human resource investment, effectively improving the accuracy of OSD detection and significantly reducing the false alarm rate of OSD detection.

[0052] Example 2

[0053] Figure 2 This is a flowchart of another method for locating OSDs in a surveillance image provided in Embodiment 2 of the present invention. This embodiment is based on the above embodiments and is optimized. In this embodiment, the operations of "extracting the target surveillance image from the video stream and reducing the target surveillance image to the image to be identified" and "generating at least one OSD highlighting image that matches the image to be identified" are specified.

[0054] Correspondingly, such as Figure 2 As shown, the method includes:

[0055] S210. Capture the target monitoring image from the video stream in real time according to the preset frame extraction frequency.

[0056] In this embodiment, considering that the OSD addition position is generally not frequently changed during the video stream acquisition process of the surveillance camera, it is not necessary to locate the OSD for each frame of the surveillance image in the video stream.

[0057] Correspondingly, a preset frame extraction frequency, such as once per second or once every 5 seconds, can be used to extract the target monitoring image from the video stream in real time for OSD positioning in the image, thereby further reducing the computational load of the computer equipment and improving the response speed of the entire OSD positioning process.

[0058] S220. Convert the target monitoring image into a monitoring grayscale image with a set number of bits, and obtain the aspect ratio of the monitoring grayscale image.

[0059] Specifically, the number of bits in the monitoring grayscale image can be set according to the number of grayscale levels required in the subsequent image recognition process. For example, if 256 grayscale levels are required, the target monitoring image can be converted into an 8-bit monitoring grayscale image.

[0060] As mentioned earlier, in order to ensure that the OSD in the image is not deformed during the image reduction process, the monitoring grayscale image can be reduced to the image to be identified by using a fixed aspect ratio.

[0061] Specifically, the aspect ratio can be obtained from the image description information of the target monitoring image, or by counting the number of row pixels and column pixels in the monitoring grayscale image.

[0062] S230. Determine the image size to be reduced based on the image size of the monitored grayscale image and the aspect ratio.

[0063] Generally, it is necessary to preset the width and height range of the image to be identified, for example, both below 640 pixels. Then, by combining the image size of the monitored grayscale image with the aspect ratio, the actual size of the image to be identified under that aspect ratio can be determined, that is, the image size can be reduced.

[0064] For example, if the image size of the grayscale image being monitored is 1024*768, then the aspect ratio is 4:3. If it is necessary to set both the width and height values ​​of the grayscale image being monitored to below 640 pixels, then an alternative is to reduce the image size to 640*480 while maintaining this aspect ratio.

[0065] Of course, it is understood that, while ensuring the aspect ratio and the width and height range of the image to be recognized, there may be more than one determined size of the reduced image. Those skilled in the art can freely choose according to the actual application scenario, and this embodiment does not impose any restrictions on this.

[0066] It needs to be emphasized again that the aspect ratio of the image to be identified is maintained in the monitoring grayscale image, and the image is reduced to the preset width and height range respectively. This reduces the amount of computation and unifies the pixel size of the OSD characters, avoiding the problem that the pixel height of the OSD characters changes significantly with the image resolution. This provides effective data support for the accurate positioning of the OSD in the future.

[0067] S240. Using the nearest neighbor interpolation algorithm, the monitored grayscale image is reduced to the size of the image to be identified.

[0068] The nearest neighbor interpolation algorithm refers to setting the pixel value of a specific pixel A in the image to be identified to the pixel value of the nearest pixel in the grayscale image that satisfies the mapping relationship with pixel A during the process of mapping a large-size monitoring grayscale image to a small-size image to be identified.

[0069] The advantage of this setting is that it ensures that the brightness values ​​of the strokes of OSD characters will not change due to scaling interpolation, thereby improving the accuracy of OSD position detection.

[0070] S250. In the image to be identified, obtain each target pixel whose pixel value is located in the standard brightness range, and set the pixel value of each target pixel to the average brightness value to obtain a brightness occlusion image.

[0071] In this embodiment, considering that during the OSD overlay process of the surveillance camera, in order to maximize the contrast of the OSD characters for easy human eye recognition, the brightness of the OSD character strokes is always kept as close as possible to 0 or 255. Therefore, in order to effectively remove the influence of a large number of non-OSD character stroke pixels in the image to be recognized, a brightness masking image is first acquired.

[0072] That is, a standard brightness area is determined based on the brightness values ​​of each pixel that is clearly excluded as OSD character strokes. In a specific example, the brightness range [25, 230] is determined as the standard brightness range. Each target pixel whose pixel value is located within the standard brightness range is determined as a non-OSD stroke pixel, and thus, the pixel value of the above-mentioned pixels can be set to 128.

[0073] Generally, OSD strokes in surveillance videos are automatically inverted based on the background color, typically using white (or near-white) or black (or near-black) brightness values, or using both black and white brightness values ​​to generate the outline. This setting ensures that the pixel values ​​of non-OSD strokes are as far away as possible from both the high-brightness (e.g., white) and low-brightness (e.g., black) strokes of the OSD, achieving maximum contrast between OSD and non-OSD strokes.

[0074] S260. After dilating the brightness masking image to obtain the dilated image, the right interval of the standard brightness range is used as the binarization threshold to binarize the dilated image, thereby obtaining an OSD highlighting image corresponding to the high-brightness OSD.

[0075] In this embodiment, by dilating the luminance masking image, the highlight areas can be expanded outwards. This, in turn, highlights the white (or near-white) OSD strokes in the luminance masking image. By using the right interval value of the standard luminance range (e.g., 230 in the previous example) as the binarization threshold, the dilated image is binarized. This allows the luminance values ​​of pixels with values ​​greater than or equal to this right interval value to be set as the target pixel value, and the luminance values ​​of pixels with values ​​less than this right interval value to other pixel values ​​that can be effectively distinguished from the target pixel value. For example, the target pixel value can be 255, and other pixel values ​​can be 0.

[0076] S270. After sequentially performing erosion processing and pixel inversion processing on the brightness masking image to obtain an eroded image, the eroded image is binarized using the right interval value of the standard brightness interval as the binarization threshold to obtain an OSD highlighting image corresponding to the low brightness OSD.

[0077] In this embodiment, by performing erosion processing on the luminance masking image, the low-brightness areas can be expanded outwards. This, in turn, highlights OSD strokes with black (or near-black) brightness in the luminance masking image. The purpose of inverting each pixel after eroding the black OSD strokes is to ensure that each OSD highlighting image uses the target pixel value to represent each pixel potentially describing the OSD, and uses other pixel values ​​that are clearly distinguishable from the target pixel values ​​to represent each pixel not potentially describing the OSD, thus facilitating subsequent processing.

[0078] Similarly, by using the right interval value of the standard brightness range (e.g., 230 in the previous example) as the binarization threshold, the eroded image can be binarized. The brightness values ​​of pixels with values ​​greater than or equal to this right interval value can be set to the target pixel value, while the brightness values ​​of pixels with values ​​less than this right interval value can be set to other pixel values ​​that can be effectively distinguished from the target pixel value. For example, the target pixel value could be 255, and other pixel values ​​could be 0.

[0079] S280. Based on the position of the target pixel value in each OSD highlighting image, identify the target OSD region that meets the position and size constraints in each OSD highlighting image.

[0080] S290. The target OSD regions identified in each OSD highlighting image are marked in the image to be identified to obtain the OSD localization result that matches the image to be identified.

[0081] The technical solution of this invention employs a nearest neighbor interpolation algorithm to reduce the size of the monitored grayscale image to the size of the image to be identified, ensuring that the brightness of OSD strokes remains unchanged. By performing brightness masking on the grayscale image to be identified, pixels that are not OSD strokes can be effectively filtered out, improving the accuracy of subsequent OSD detection. Furthermore, by enhancing the white brightness OSD strokes and the black brightness OSD strokes on the brightness-masked image respectively, OSDs can be effectively detected adaptively regardless of whether the OSD strokes are white brightness, black brightness, or both white and black brightness.

[0082] Example 3

[0083] Figure 3 This is a flowchart of another OSD localization method in a surveillance image provided in Embodiment 3 of the present invention. This embodiment is based on the above embodiments and is optimized. In this embodiment, the operation of "identifying the target OSD region that meets the position and size constraints in each OSD highlighting image according to the position of the pixel point of the target pixel value in each OSD highlighting image" is specified.

[0084] Correspondingly, such as Figure 3 The method may specifically include:

[0085] S310. Extract the target monitoring image from the video stream and reduce the target monitoring image to the size of the image to be identified.

[0086] S320. Generate at least one OSD highlighting image that matches the image to be identified.

[0087] Different OSD highlighting images are used to highlight OSDs with different brightness levels. In the OSD highlighting images, each pixel used to potentially describe the OSD is set to the target pixel value.

[0088] S330. Sequentially acquire one OSD highlighting image as the target OSD highlighting image for the current processing, and divide the target OSD highlighting image into multiple regions along the vertical direction.

[0089] Specifically, in Figure 4 The diagram illustrates a method for dividing an OSD (Optical Display Panel) image into regions along a vertical direction. For example... Figure 4As shown, the target OSD highlighted image can be divided into four regions—Region 1, Region 2, Region 3, and Region 4—along the vertical direction. Using this region division method, a complete image row in the target OSD highlighted image can be divided into four region rows.

[0090] Based on the above embodiments, before dividing the target OSD highlighted image into multiple regions along the vertical direction, it may further include:

[0091] The target OSD highlighted image is subjected to a first-order erosion process and a first-order dilation process in sequence.

[0092] Specifically, the target OSD highlighting image can be subjected to three erosion processes followed by three dilation processes. Through the above settings, the hole defects of OSD strokes can be effectively eliminated, resulting in a target OSD highlighting image with enhanced OSD strokes. Then, the region can be divided in the vertical direction.

[0093] S340. Count the number of pixels containing the target pixel value in each row of each region in the target OSD highlighting image, and mark the potential OSD region rows whose number of pixels exceeds the threshold.

[0094] Specifically, a row within a region of the target OSD highlighted image can be understood as... Figure 4 The area rows in area 1 shown.

[0095] In a specific example, such as Figure 4 As shown, after dividing the target OSD highlighting image into four regions vertically, the rows of each region can be statistically analyzed according to the scanning order. Specifically, the number of pixels with a brightness (pixel value) of 255 in each region row can be recorded. If the number of such pixels exceeds 1 / 4 of the width of the region row, then that region row can be marked as a potential OSD region row.

[0096] In this embodiment, each row of the target OSD highlighting image is divided into multiple region rows, and statistics are performed on a region row basis rather than on the entire image row. This is because OSDs in monitoring images are often concentrated in a specific horizontal region, where the number of pixels corresponding to OSD strokes is significantly higher than in other regions. Regional statistics help distinguish between bright or dark noise points similar to OSD strokes within a data row, improving the accuracy of OSD stroke detection.

[0097] S350. Calculate the row height value of each consecutive potential OSD area row contained in each region, and remove the markers of consecutive potential OSD area rows whose row height value exceeds the row height threshold.

[0098] In this embodiment, the continuity of each potential OSD region row contained in each region can be detected by region. When two or more potential OSD region rows in a certain region are spatially continuous, the two or more potential OSD region rows are determined as continuous potential OSD region rows.

[0099] Furthermore, the markers for each potential OSD region row in a series of consecutive potential OSD region rows where the row height value exceeds the row height threshold are removed. For example, assuming the OSD has a known standard row height value, we can first obtain consecutive potential OSD region rows that exceed four times the standard OSD row height value, and then remove the markers from each potential OSD region row in that series of consecutive potential OSD region rows. This effectively reduces the false detection impact caused by large, continuous bright or dark areas in the surveillance video.

[0100] Based on the above embodiments, before calculating the row height value of each consecutive potential OSD area row contained in each region, the following may also be included:

[0101] For each consecutive potential OSD region row, a second expansion process and a second erosion process are performed in the vertical direction.

[0102] For example, two independent expansion processes can be performed on each row of consecutive potential OSD regions in the vertical direction, followed by two independent erosion processes, in order to solve the problem that OSD strokes may have holes in some rows.

[0103] S360. Based on the currently labeled rows of potential OSD regions, determine the potential OSD rows in the target OSD highlighting image and identify the potential OSD regions that match the potential OSD rows.

[0104] In an optional implementation of this embodiment, determining potential OSD rows in the target OSD highlighting image based on the currently marked potential OSD region rows, and identifying potential OSD regions matching the potential OSD rows, can be as follows:

[0105] The search is performed on a row-by-row basis in the target OSD highlighting image. If any potential OSD region row exists in the currently searched image row, the currently searched image row is identified as a potential OSD row. In the target OSD highlighting image, the number of each consecutive potential OSD row is counted, and each consecutive potential OSD row whose number of rows exceeds a preset row value is identified as a potential OSD region.

[0106] Continuing from the previous example, such as Figure 4As shown, the potential OSD row annotation results of the four regions in each image row can be merged. That is, if there is one labeled potential OSD row in each image row, then that image row is considered a potential OSD row. With the above settings, this can automatically adapt to the situation where OSDs are in different horizontal positions, allowing multiple OSD rows appearing in the monitoring screen at different horizontal positions.

[0107] Furthermore, following the top-to-bottom order, the number of consecutive potential OSD rows is identified. If the number of consecutive potential OSD rows is greater than a preset number of rows, such as 10 rows, then the start and end numbers of this part of the consecutive potential OSD rows are considered to correspond to a target OSD display area. Thus, the above consecutive potential OSD rows can be identified as a target OSD area.

[0108] S370. If the identified potential OSD region is located within a preset image region in the target OSD highlighting image, then the potential OSD region is determined as the target OSD region.

[0109] As mentioned earlier, when adding OSDs to the video stream of a surveillance camera, they are generally added to the upper or lower part of the monitored image. Therefore, the OSD will not be located arbitrarily in the image to be identified. Consequently, only when the identified potential OSD area is located within a preset image area in the target OSD highlighting image will the potential OSD area be determined as the target OSD area.

[0110] In a specific example, the image region can be located at the upper 20% or lower 20% of the image in the target OSD highlighting image.

[0111] S380. Check whether the processing of all OSD highlighted images is complete: if yes, proceed to S390; otherwise, return to S330.

[0112] S390. The target OSD regions identified in each OSD highlighted image are marked in the image to be identified to obtain the OSD localization result that matches the image to be identified.

[0113] The technical solution of this invention can adapt to various types of OSDs, including text, fonts, black brightness, white brightness, and black-and-white brightness outlines, reducing the deployment complexity of OSD detection algorithms. By using techniques such as brightness masking, stroke enhancement, multi-region detection, and position filtering, the accuracy of OSD detection algorithms is effectively improved. Furthermore, the technical solution of this invention can immediately detect the OSD position of an image using only a single frame, offering advantages such as fast response speed and low system footprint. At the same time, it does not require additional parameter configuration, greatly reducing the manpower cost during project deployment.

[0114] Example 4

[0115] Figure 5 This is a schematic diagram of the structure of an OSD (Optical Disk Detection) positioning device in a surveillance image, provided in Embodiment 4 of the present invention. Figure 5 As shown, the device includes: a target OSD region acquisition module 510, an OSD highlighting image generation module 520, a target OSD region recognition module 530, and an OSD localization result determination module 540, wherein:

[0116] The image acquisition module 510 is used to extract the target monitoring image from the video stream and reduce the target monitoring image to the image to be recognized;

[0117] OSD highlighting image generation module 520 is used to generate at least one OSD highlighting image that matches the image to be identified. Different OSD highlighting images are used to highlight OSDs with different brightness. In the OSD highlighting image, each pixel used to potentially describe the OSD is set to the target pixel value.

[0118] The target OSD region recognition module 530 is used to identify target OSD regions that meet the position and size constraints in each OSD highlighting image based on the position of the pixel of the target pixel value in each OSD highlighting image.

[0119] OSD localization result determination module 540 is used to mark the target OSD region identified in each OSD highlighted image in the image to be identified, so as to obtain the OSD localization result matching the image to be identified.

[0120] The technical solution of this invention provides a new way to locate OSDs using only image recognition technology. This method involves reducing the size of a target monitoring image captured from a video stream to an image to be identified; generating at least one OSD highlighting image that matches the image to be identified; identifying target OSD regions that meet position and size constraints in each OSD highlighting image based on the position of the target pixel value; and marking the identified target OSD regions in each OSD highlighting image in the image to be identified, thus obtaining the OSD location result matching the image to be identified. This approach provides a new way to locate OSDs using only image recognition technology. It can efficiently and accurately locate OSDs in a single frame of a monitoring image without any human resource investment, effectively improving the accuracy of OSD detection and significantly reducing the false alarm rate of OSD detection.

[0121] Based on the above embodiments, the image acquisition module 510 can be specifically used for:

[0122] The target monitoring image is converted into a monitoring grayscale image with a set number of bits, and the aspect ratio of the monitoring grayscale image is obtained;

[0123] Based on the image size of the monitored grayscale image and the aspect ratio, determine the image size to be reduced;

[0124] The nearest neighbor interpolation algorithm is used to reduce the size of the monitored grayscale image to that of the image to be identified.

[0125] Based on the above embodiments, the OSD highlighting image generation module 520 can be specifically used for:

[0126] In the image to be identified, each target pixel whose pixel value is within the standard brightness range is obtained, and the pixel value of each target pixel is set to the average brightness value to obtain a brightness occlusion image;

[0127] After dilating the brightness masking image to obtain the dilated image, the right interval of the standard brightness range is used as the binarization threshold to binarize the dilated image, thereby obtaining an OSD highlighting image corresponding to the high brightness OSD.

[0128] After sequentially performing erosion processing and pixel inversion processing on the brightness masking image to obtain an eroded image, the eroded image is binarized using the right interval value of the standard brightness interval as the binarization threshold to obtain an OSD highlighting image corresponding to the low brightness OSD.

[0129] Based on the above embodiments, the target OSD region identification module 530 may specifically include:

[0130] The region segmentation unit is used to acquire the target OSD saturation image currently being processed and to divide the target OSD saturation image into multiple regions along the vertical direction;

[0131] The potential OSD region row marking unit is used to count the number of pixels containing the target pixel value in each row of each region in the target OSD highlighted image, and to mark the potential OSD region rows whose number of pixels exceeds the number threshold.

[0132] The marker removal unit is used to count the row height value of each consecutive potential OSD area row contained in each region, and remove the markers of consecutive potential OSD area rows whose row height value exceeds the row height threshold.

[0133] The potential OSD region identification unit is used to determine potential OSD rows in the target OSD highlighting image based on the currently labeled potential OSD region rows, and to identify potential OSD regions that match the potential OSD rows.

[0134] The target OSD region determination unit is used to determine the potential OSD region as the target OSD region if the identified potential OSD region is located within a preset image region range in the target OSD saturation image.

[0135] Based on the above embodiments, the potential OSD region identification unit can be specifically used for:

[0136] The search is performed on a row-by-row basis in the target OSD highlighting image. If any potential OSD region row exists in the currently searched image row, then the currently searched image row is identified as a potential OSD row.

[0137] In the target OSD highlighting image, the number of rows of each consecutive potential OSD row is counted, and each consecutive potential OSD row with a row number exceeding a preset row value is identified as a potential OSD region.

[0138] Based on the above embodiments, it may also include:

[0139] The erosion and dilation processing unit is used to perform a first number of erosion processes and a first number of dilation processes on the target OSD highlighting image sequentially before dividing the target OSD highlighting image into multiple regions along the vertical direction.

[0140] The expansion and corrosion processing unit is used to perform a second expansion process and a second corrosion process on each consecutive potential OSD region row in the vertical direction before calculating the row height value of each consecutive potential OSD region row contained in each region.

[0141] Based on the above embodiments, the image acquisition module 510 can be further used to: extract the target monitoring image from the video stream in real time according to a preset frame extraction frequency.

[0142] The OSD positioning device in the monitoring image provided in the embodiments of the present invention can execute the OSD positioning method in the monitoring image provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0143] Example 5

[0144] Figure 6A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0145] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0146] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, for example, performing the OSD localization method in a surveillance image as provided in any embodiment of the present invention.

[0148] That is, the target surveillance image is extracted from the video stream and reduced to the image to be identified; at least one OSD highlighting image matching the image to be identified is generated, wherein different OSD highlighting images are used to highlight OSDs of different brightness, and each pixel in the OSD highlighting image is set to the target pixel value; based on the position of the pixel with the target pixel value in each OSD highlighting image, the target OSD region that meets the position and size constraints is identified in each OSD highlighting image; the target OSD region identified in each OSD highlighting image is marked in the image to be identified to obtain the OSD localization result matching the image to be identified.

[0149] In some embodiments, the OSD localization method in a surveillance image, as provided in any embodiment of the present invention, can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the OSD localization method in a surveillance image provided in any embodiment of the present invention described above can be performed. Alternatively, in other embodiments, processor 11 can be configured by any other suitable means (e.g., by means of firmware) to perform the OSD localization method in a surveillance image provided in any embodiment of the present invention.

[0150] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0155] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0156] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0157] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for locating OSD (On-Screen Display Information) in a surveillance image, characterized in that, include: Extract the target surveillance image from the video stream and reduce the target surveillance image to the size of the image to be identified; Generate at least one OSD highlighting image that matches the image to be identified, wherein different OSD highlighting images are used to highlight OSDs of different brightness, and in the OSD highlighting image, each pixel used to potentially describe the OSD is set to the target pixel value. Based on the position of the target pixel value in each OSD highlighted image, the target OSD region that meets the position and size constraints is identified in each OSD highlighted image. The target OSD regions identified in each OSD highlighted image are marked in the image to be identified to obtain the OSD localization results that match the image to be identified; The process of generating at least one OSD highlighting image that matches the image to be identified includes: In the image to be identified, each target pixel whose pixel value is within the standard brightness range is obtained, and the pixel value of each target pixel is set to the average brightness value to obtain a brightness occlusion image; After dilating the brightness masking image to obtain the dilated image, the right interval of the standard brightness range is used as the binarization threshold to binarize the dilated image, thereby obtaining an OSD highlighting image corresponding to the high brightness OSD. After sequentially performing erosion processing and pixel inversion processing on the brightness masking image to obtain an eroded image, the eroded image is binarized using the right interval value of the standard brightness interval as the binarization threshold to obtain an OSD highlighting image corresponding to the low brightness OSD.

2. The method according to claim 1, characterized in that, The target surveillance image is reduced to the image to be identified, including: The target monitoring image is converted into a monitoring grayscale image with a set number of bits, and the aspect ratio of the monitoring grayscale image is obtained; Based on the image size of the monitored grayscale image and the aspect ratio, determine the image size to be reduced; The nearest neighbor interpolation algorithm is used to reduce the size of the monitored grayscale image to that of the image to be identified.

3. The method according to any one of claims 1-2, characterized in that, Based on the position of the target pixel value in each OSD embossed image, the target OSD region that meets the position and size constraints is identified in each OSD embossed image, including: Obtain the target OSD highlighting image currently being processed, and divide the target OSD highlighting image into multiple regions along the vertical direction; The number of pixels containing the target pixel value in each row of each region in the target OSD highlighted image is counted, and the rows of potential OSD regions with a number of pixels exceeding the threshold are marked. Calculate the row height value of each consecutive potential OSD area row contained in each region, and remove the markers of consecutive potential OSD area rows whose row height value exceeds the row height threshold; Based on the currently labeled rows of potential OSD regions, determine the potential OSD rows in the target OSD highlighting image and identify the potential OSD regions that match the potential OSD rows. If the identified potential OSD region is located within a preset image region in the target OSD highlighted image, then the potential OSD region is determined as the target OSD region.

4. The method according to claim 3, characterized in that, Based on the currently labeled rows of potential OSD regions, determine the potential OSD rows in the target OSD highlighting image, and identify the potential OSD regions that match the potential OSD rows, including: The search is performed on a row-by-row basis in the target OSD highlighting image. If any potential OSD region row exists in the currently searched image row, then the currently searched image row is identified as a potential OSD row. In the target OSD highlighting image, the number of rows of each consecutive potential OSD row is counted, and each consecutive potential OSD row with a row number exceeding a preset row value is identified as a potential OSD region.

5. The method according to claim 3, characterized in that, Before dividing the target OSD highlighting image into multiple regions along the vertical direction, the following steps are also included: The target OSD highlighted image is subjected to the first erosion process and the first dilation process in sequence; Before calculating the row height value of each consecutive potential OSD area row contained within each region, the following steps are also included: For each consecutive potential OSD region row, a second expansion process and a second erosion process are performed in the vertical direction.

6. The method according to claim 1, characterized in that, Extracting target surveillance images from the video stream, including: The target monitoring image is captured from the video stream in real time according to the preset frame extraction frequency.

7. A device for locating OSD (On-Screen Display Information) information in a surveillance image, characterized in that, include: The image acquisition module is used to extract the target monitoring image from the video stream and reduce the target monitoring image to the image to be recognized; The OSD highlighting image generation module is used to generate at least one OSD highlighting image that matches the image to be identified. Different OSD highlighting images are used to highlight OSDs with different brightness levels. In the OSD highlighting image, each pixel used to potentially describe the OSD is set to the target pixel value. The target OSD region identification module is used to identify the target OSD region that meets the position and size constraints in each OSD highlighting image based on the position of the pixel with the target pixel value in each OSD highlighting image. The OSD localization result determination module is used to mark the target OSD regions identified in each OSD highlighted image in the image to be identified, and obtain the OSD localization result that matches the image to be identified. The OSD highlighting image generation module is specifically used for: In the image to be identified, each target pixel whose pixel value is within the standard brightness range is obtained, and the pixel value of each target pixel is set to the average brightness value to obtain a brightness occlusion image; After dilating the brightness masking image to obtain the dilated image, the right interval of the standard brightness range is used as the binarization threshold to binarize the dilated image, thereby obtaining an OSD highlighting image corresponding to the high brightness OSD. After sequentially performing erosion processing and pixel inversion processing on the brightness masking image to obtain an eroded image, the eroded image is binarized using the right interval value of the standard brightness interval as the binarization threshold to obtain an OSD highlighting image corresponding to the low brightness OSD.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for locating OSD (On-Screen Display Information) in a monitoring image according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for locating OSD (On-Screen Display Information) in a monitoring image according to any one of claims 1-6.

Images