A real-time optimization method for monitoring images based on multi-target positioning analysis

By calibrating multiple shooting units in video surveillance, acquiring the target background image and analysis model, fitting out the video stream and path of the target monitoring object, the accuracy problem of multi-objective positioning analysis is solved, and an intelligent and accurate monitoring picture is achieved optimization.

CN118631973BActive Publication Date: 2024-11-22TIANJIAN GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411099747.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-22
Estimated Expiration
2044-08-12

AI Technical Summary

Technical Problem

In video surveillance, there is a problem of inaccurate positioning during multi-object positioning analysis, especially in complex scenarios. The prior art fails to effectively utilize the co-prosperity processing of multiple shooting units, resulting in inaccurate and unintelligent positioning analysis.

Method used

A real-time optimization method for monitoring screens based on multi-objective positioning analysis is proposed. By receiving screen optimization instructions, multiple shooting units are calibrated, target background images are obtained, target monitoring proportion is obtained using the analysis model set and initial images, and target monitoring is fitted. The fitted video stream of the object and the monitoring target path are combined with the video segments of multiple shooting units for inclusive analysis to improve the accuracy and intelligence of positioning analysis.

Benefits of technology

Accurate and intelligent positioning analysis of multiple targets is achieved, the accuracy and intelligence of monitoring picture optimization is improved, the changes in the target background image are adapted to dynamically confirm the target monitoring object, and the dynamic movement of the target monitoring object is considered.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118631973B_ABST
    Figure CN118631973B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and discloses a real-time optimization method for monitoring images based on multi-target positioning analysis, including: based on multiple calibrated shooting units, real-time shooting of monitoring areas to obtain multiple video sequences, obtaining target background images based on multiple video sequences, obtaining target images using initial images and target background images, obtaining one or more target monitoring ratios based on target images and analysis model sets, respectively summarizing the target monitoring ratios using target monitoring objects to obtain one or more target monitoring ratio sets, respectively identifying the main shooting units of one or more target monitoring objects from the one or more target monitoring ratio sets, confirming the reception of the picture integration instruction from the monitoring picture integration unit, and fitting the fitting video stream and monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit. The main purpose of the present invention is to achieve accurate and intelligent positioning analysis of multiple targets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a real-time optimization method for monitoring images based on multi-target positioning analysis, and belongs to the technical field of image processing. Background Art

[0002] With the rapid development of video surveillance technology, multi-target positioning and tracking technology has been widely used in video surveillance. However, when dealing with the positioning of multiple targets in complex scenes, there is often an inaccurate positioning problem. Therefore, how to improve the accuracy of multi-target positioning has become an urgent problem to be solved.

[0003] Currently, most of the methods rely on manual processing of multiple videos to achieve positioning analysis of multiple targets.

[0004] Although the above method can realize the positioning analysis of multiple targets, it does not consider the use of multiple shooting units for co-processing when positioning and analyzing multiple targets and optimizing the monitoring screen, which may lead to inaccurate and unintelligent problems when positioning and analyzing multiple targets. Summary of the invention

[0005] The present invention provides a monitoring picture real-time optimization method, device and computer-readable storage medium based on multi-target positioning analysis, the main purpose of which is to achieve accurate and intelligent positioning analysis of multiple targets.

[0006] To achieve the above object, the present invention provides a monitoring screen real-time optimization method based on multi-target positioning analysis, comprising:

[0007] Receiving a picture optimization instruction, and confirming a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: a plurality of shooting units for shooting, a monitoring area to be monitored, and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit, and a data result feedback unit;

[0008] Performing a calibration operation on the plurality of shooting units to obtain a plurality of calibration shooting units with serial numbers identified thereon, shooting a monitoring area in real time based on the plurality of calibration shooting units to obtain a plurality of video sequences, and acquiring a target background image based on the plurality of video sequences;

[0009] Extracting video sequences from the multiple video sequences in sequence, and performing the following operations on the extracted video sequences:

[0010] The video frame number of the video sequence is obtained, and the initial images are sequentially extracted from the extracted video sequence using the video frame number, and the following operations are performed on the extracted initial images:

[0011] Confirming receipt of a screen division instruction from a monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using an initial image and a target background image, and obtaining one or more target monitoring ratios that identify a target monitoring object based on the target image and the analysis model set, wherein the analysis model set includes a plurality of analysis models;

[0012] The target monitoring proportions are respectively summarized by using the target monitoring objects to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instruction from the monitoring screen integration unit is confirmed to be received, and the fitting video stream and the monitoring target path of the target monitoring object are fitted based on the screen integration instruction and the main shooting unit;

[0013] The step of fitting the video stream and the monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit includes:

[0014] Using the picture integration instruction and the division time threshold corresponding to the main shooting unit to obtain multiple video segments, and using the pre-built video fitting model and the multiple video segments to fit a fitting video stream;

[0015] Extract video segments from the fitted video stream in sequence, and perform the following operations on the extracted video segments:

[0016] The target monitoring images are sequentially extracted from the extracted video segments using the video frame number, and the following operations are performed on the extracted target monitoring images:

[0017] The monitoring center is calculated based on the extracted target monitoring image and the pre-constructed center extraction relation, wherein the monitoring center is composed of the monitoring row coordinates and the monitoring column coordinates, the monitoring centers are summarized to obtain a monitoring center set, and the monitoring target path is fitted using the pre-constructed path fitting relation and the monitoring center set;

[0018] The center extraction relationship is as follows:

[0019]

[0020] Among them, H0 represents the monitoring row coordinate, H1 represents the monitoring column coordinate, and h b0 represents the coordinate value of the bth row in the target monitoring image, h 10 represents the center value of the row coordinates in the target monitoring image, h 11 represents the center value of the column coordinates in the target monitoring image, h b1 represents the coordinate value of the bth column in the target monitoring image, δ is the preset coefficient, S represents the total number of S pixels in the target monitoring image, and v b0represents the number of pixels in the bth row of the target monitoring image, v b1 Indicates the number of pixels in the bth column of the target monitoring image, max indicates the maximum value, and [] indicates rounding operation;

[0021] The path fitting relationship is as follows:

[0022]

[0023] Among them, f represents the monitoring target path, are the coefficients generated during fitting, and c represents the independent variable;

[0024] The data result feedback unit is used to send the fitted video stream and the monitoring target path to the initiator of the picture optimization instruction to achieve real-time optimization of the monitoring picture.

[0025] Optionally, acquiring a target background image based on the multiple video sequences includes:

[0026] Acquire a monitoring background picture set based on a preset background extraction frame number and a plurality of video sequences, wherein the monitoring background picture set includes a plurality of monitoring background pictures, and the monitoring background pictures correspond to the calibration shooting units one by one;

[0027] Extract monitoring background images from the monitoring background image set in sequence, and perform the following operations on the extracted monitoring background images:

[0028] Performing a grayscale operation on the extracted monitoring background image to obtain a grayscale image, and performing an equalization operation on the grayscale image to obtain an equalized image;

[0029] Summarizing the balanced images to obtain a balanced image set, and obtaining a grayscale variance value set based on the balanced image set, wherein the balanced image set corresponds to the grayscale variance value set one-to-one, and the grayscale variance value set includes a plurality of grayscale variance values;

[0030] Sorting the grayscale variance values ​​in the grayscale variance value set in descending order to obtain a grayscale variance sequence;

[0031] The grayscale variance values ​​are sequentially extracted from the grayscale variance sequence, and the following operations are performed on the extracted grayscale variance values:

[0032] The grayscale variance values ​​that have not been extracted in the grayscale variance value set are used to obtain the mean of the grayscale variance values, thereby obtaining the grayscale variance mean;

[0033] Comparing the grayscale variance mean with a preset grayscale variance threshold;

[0034] If the grayscale variance mean is greater than the grayscale variance threshold, the extracted grayscale variance value is eliminated from the grayscale variance value set to obtain an updated variance value set, the updated variance value set is the grayscale variance value set, and the step of performing a sorting operation on the grayscale variance values ​​in the grayscale variance value set in descending order is returned until the grayscale variance mean is less than or equal to the grayscale variance threshold;

[0035] If the grayscale variance means in the grayscale variance value set are all less than or equal to the grayscale variance threshold, the target background image is obtained using the balanced image set.

[0036] Optionally, the obtaining the target background image by using the balanced image set includes:

[0037] Extracting gray value groups in sequence using the balanced image set, wherein the gray value group includes a plurality of gray values, and the pixel coordinates of the balanced image corresponding to the gray values ​​in the gray value group are all the same;

[0038] The grayscale variance group and the preset image division window are used to calculate the target background grayscale value. The calculation formula is as follows:

[0039]

[0040] Among them, H represents the gray value of the target background, n represents the total number of pixels in the image division window, and x i represents the gray value in the gray value group corresponding to the i-th pixel in the image division window, a represents the preset coefficient, m represents that there are m gray values ​​in the gray value group, and j and k are used for counting;

[0041] The target background image is obtained based on the target background grayscale value.

[0042] Optionally, the acquiring the target image by using the initial image and the target background image includes:

[0043] The pre-built Canny algorithm and the initial image are used to obtain the initial target image, and the first monitoring image is calculated based on the initial target image and the initial image. The calculation formula is as follows:

[0044] J(y,z)=C(y,z)-B(y,z)

[0045] Wherein, J(y,z) represents the first monitoring image, C(y,z) represents the initial image, B(y,z) represents the initial target image, and (y,z) represents the pixel at the yth row and the zth column in the initial image;

[0046] Acquire a monitoring mean based on the first monitoring image, wherein the monitoring mean is the mean of the grayscale values ​​in the first monitoring image;

[0047] Extract pixel points from the first monitoring image in sequence, and perform the following operations on the extracted pixel points:

[0048] Compare the grayscale value corresponding to the pixel with the monitored mean value;

[0049] If the grayscale value corresponding to the pixel is greater than or equal to the monitored mean, the pixel is marked as 1;

[0050] If the grayscale value corresponding to the pixel is less than the monitored mean, the pixel is marked as 0;

[0051] A first identification image is obtained based on the identified pixel points, a second identification image is obtained using the initial image and the target background image, and a plurality of reference pixel point groups are obtained based on the first identification image and the second identification image, wherein the reference pixel point groups are composed of a first pixel point in the first identification image and a second pixel point in the second identification image, and the pixel coordinates corresponding to the first pixel point and the second pixel point are the same;

[0052] The following operations are performed on each of the multiple reference pixel groups:

[0053] If the identifiers corresponding to the first pixel point and the second pixel point are both 1, the first pixel point in the first identification image is identified as a target object point;

[0054] The target object points are aggregated to obtain a target object point set, and a target image is acquired based on the target object point set.

[0055] Optionally, obtaining one or more target monitoring proportions identifying the target monitoring objects based on the target image and the analysis model set includes:

[0056] An analysis model is randomly extracted from the analysis model set, one or more target monitoring objects are obtained based on the extracted analysis model and the target image, a marking operation is performed on the one or more target monitoring objects using a pre-constructed marking sequence to obtain one or more marked detection objects, and the extracted analysis model is eliminated from the analysis model set, and the included analysis model set is summarized to obtain an updated model set;

[0057] Taking the updated model set as the analysis model set, returning to the step of randomly extracting an analysis model from the analysis model set, respectively aggregating the identified detection objects according to the identification sequence, and obtaining one or more identified detection object groups;

[0058] Extracting identification detection object groups from the one or more identification detection object groups in sequence, and performing the following operations on the extracted identification detection object groups:

[0059] Randomly extract an analysis model from the update model set, and obtain a similarity evaluation value using the extracted analysis model and the identified detection object group;

[0060] Comparing the similarity evaluation value with a preset similarity threshold;

[0061] If the similarity evaluation value is greater than or equal to the similarity threshold, one or more identified detection objects are aggregated to obtain a target monitoring object set;

[0062] If the similarity evaluation value is less than the similarity threshold, the updated model set is used to obtain an updated identification detection object group;

[0063] In a combined manner, a plurality of update groups are obtained by using the update identification detection object group;

[0064] For each of the multiple update groups, perform the following operations:

[0065] Acquire update evaluation values ​​based on update grouping, summarize the update evaluation values ​​to obtain an update evaluation value set, and use the update evaluation value set to identify a target monitoring object, wherein the target monitoring object is an identified detection object corresponding to the largest update evaluation value in the update evaluation value set;

[0066] The target monitoring objects are aggregated to obtain a target monitoring object set, and the target monitoring object set is used to obtain one or more target monitoring proportions that identify the target monitoring objects.

[0067] Optionally, the obtaining one or more target monitoring proportions identifying the target monitoring objects by using the target monitoring object set includes:

[0068] Extract target monitoring objects from the target monitoring object set in sequence, and perform the following operations on the extracted target monitoring objects:

[0069] Acquire the target pixel count based on the extracted target monitoring object;

[0070] The target monitoring ratio is obtained by using the number of target pixels and the pixels included in the initial image;

[0071] The target monitoring proportions are summarized to obtain one or more target monitoring proportions that identify the target monitoring objects.

[0072] Optionally, the main shooting unit that identifies one or more target monitoring objects from the one or more target monitoring proportion sets includes:

[0073] Acquire multiple divided monitoring ratio sets using a preset divided time threshold, one or more target monitoring ratio sets, and multiple calibration shooting units;

[0074] Using multiple partition monitoring proportion sets to obtain multiple partition monitoring means, wherein the partition monitoring means correspond to the partition monitoring proportion sets one by one, and the partition monitoring means are the means of the target monitoring proportions in the partition monitoring proportion sets;

[0075] Using the target monitoring objects corresponding to multiple divided monitoring means, the divided monitoring means are summarized respectively to obtain a divided monitoring mean set, and the main shooting unit is obtained based on the divided monitoring mean set, wherein the main shooting unit is the shooting unit corresponding to the largest divided monitoring mean in the divided monitoring mean set.

[0076] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0077] at least one processor; and,

[0078] a memory communicatively connected to the at least one processor; wherein,

[0079] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the above-mentioned real-time optimization method of monitoring images based on multi-target positioning analysis.

[0080] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned real-time optimization method of monitoring images based on multi-target positioning analysis.

[0081] Compared with the problem described in the background technology, the present invention first receives a picture optimization instruction, and confirms a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: multiple shooting units for shooting, a monitoring area to be monitored and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit and a data result feedback unit. It can be seen that the present invention combines the actual situation in life and formulates a monitoring environment that is more in line with actual life. In addition, the present invention confirms that a calibration operation is performed on multiple shooting units to obtain multiple calibrated shooting units, wherein the serial numbers of the calibrated shooting units are identified, and the monitoring area is photographed in real time based on the multiple calibrated shooting units to obtain multiple video sequences, and the target background image is obtained based on the multiple video sequences. Then, by calibrating the multiple shooting units, multiple calibrated shooting units are obtained, thereby realizing that the videos shot by the multiple calibrated shooting units can be analyzed together, thereby improving the intelligence of positioning analysis of multiple targets, and when obtaining the target background image, it is considered that the target background image may change with time and that the target background image may change through two different methods. The formula is used to jointly fit a target background image that conforms to the actual situation, thereby improving the accuracy of the target background image, and thereby improving the accuracy of positioning analysis of multiple targets, confirming the reception of a screen division instruction from the monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using the initial image and the target background image, and obtaining one or more target monitoring ratios based on the target image and the analysis model set, wherein the target monitoring object of the target monitoring ratio is identified, and the analysis model set includes multiple models for analysis. It can be seen that the embodiment of the present invention takes into account the randomness of the target monitoring object, and therefore, dynamically confirms the target monitoring object, and thereby improves the accuracy of positioning analysis of multiple targets by clarifying the form of the target monitoring object. The target monitoring objects are used to summarize the target monitoring proportions respectively to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instructions received from the monitoring screen integration unit are confirmed. Based on the screen integration instructions and the main shooting unit, the fitting video stream and the monitoring target path of the target monitoring object are fitted. It can be seen that the embodiment of the present invention takes into account the dynamic movement of the target monitoring object. Therefore, the main shooting unit is identified in multiple shooting units, and the fitting video stream is fitted with the video segment corresponding to the main shooting unit, and the monitoring target path is fitted. Therefore, the real-time optimization method, device, electronic device and computer-readable storage medium of the monitoring screen based on multi-target positioning analysis proposed in the present invention mainly aim to achieve accurate and intelligent positioning analysis of multiple targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1A schematic diagram of a flow chart of a method for real-time optimization of monitoring images based on multi-target positioning analysis provided by an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of the structure of an electronic device for implementing the method for real-time optimization of monitoring images based on multi-target positioning analysis provided by an embodiment of the present invention.

[0084] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0085] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0086] The embodiment of the present application provides a method for real-time optimization of monitoring images based on multi-target positioning analysis. The execution subject of the real-time optimization method for monitoring images based on multi-target positioning analysis includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the real-time optimization method for monitoring images based on multi-target positioning analysis can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0087] Embodiment 1:

[0088] Reference Figure 1 FIG. 1 is a flow chart of a method for real-time optimization of monitoring images based on multi-target positioning analysis provided by an embodiment of the present invention. In this embodiment, the method for real-time optimization of monitoring images based on multi-target positioning analysis includes:

[0089] S1. Receive a picture optimization instruction, and confirm a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: multiple shooting units for shooting, a monitoring area to be monitored, and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit, and a data result feedback unit.

[0090] It should be explained that, generally, the picture optimization instruction is an instruction issued by a person who needs to locate and analyze multiple targets. The real-time picture optimization environment is a necessary environment constructed for real-time optimization of the monitoring picture, and the real-time picture optimization environment includes: multiple shooting units for shooting, the monitoring area to be monitored and the picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit and a data result feedback unit. The monitoring area refers to the area that needs to be monitored. For the application of specific units, please refer to the subsequent embodiments.

[0091] For example, Xiao Zhang, as the person in charge of a factory, is mainly responsible for ensuring the safety of various confidential places in the factory, where each confidential place is a monitoring area. Therefore, Xiao Zhang issues a picture optimization instruction and places a shooting unit with a rotating unit at different locations in the factory, through which different monitoring pictures can be collected. The pictures monitored by multiple shooting units can fit the position of suspicious persons in the monitoring pictures at different locations and times, so as to improve the intelligent monitoring of suspicious persons.

[0092] S2. Perform a calibration operation on multiple shooting units to obtain multiple calibration shooting units with serial numbers identified, shoot a monitoring area in real time based on the multiple calibration shooting units to obtain multiple video sequences, and acquire a target background image based on the multiple video sequences.

[0093] It should be understood that the calibration shooting unit refers to a shooting unit after the shooting units in the multiple shooting units are all subjected to coordinate transformation, and the coordinate transformation means: the pictures that can be shot by the multiple shooting units at different positions are all transformed into the same coordinate, thereby improving the accuracy of the picture processing that can be generated by using the multiple shooting units, and the coordinate transformation technology is a prior art and will not be repeated here. For example: in a monitoring environment, there are 10 shooting units evenly distributed on a circle, the angle between the shooting units is 36 degrees, and these 10 shooting units are located at different positions in the monitoring environment. The coordinate transformation is performed on these 10 shooting units, so that these 10 shooting units use unified coordinates to shoot the pictures, and the 10 shooting units are identified according to the preset serial number, wherein any one shooting unit is used as the starting point of identification, and the shooting unit is identified as the calibration shooting unit with serial number 1, and the remaining shooting units are identified in sequence to obtain 10 calibration shooting units.

[0094] It should be explained that the video sequence refers to a video sequence with a time relationship obtained by shooting the monitored area using a calibrated shooting unit. In actual applications, the shooting unit corresponds to the video sequence one by one.

[0095] Further, acquiring a target background image based on the multiple video sequences includes:

[0096] Acquire a monitoring background picture set based on a preset background extraction frame number and a plurality of video sequences, wherein the monitoring background picture set includes a plurality of monitoring background pictures, and the monitoring background pictures correspond to the calibration shooting units one by one;

[0097] Extract monitoring background images from the monitoring background image set in sequence, and perform the following operations on the extracted monitoring background images:

[0098] Performing a grayscale operation on the extracted monitoring background image to obtain a grayscale image, and performing an equalization operation on the grayscale image to obtain an equalized image;

[0099] Summarizing the balanced images to obtain a balanced image set, and obtaining a grayscale variance value set based on the balanced image set, wherein the balanced image set corresponds to the grayscale variance value set one-to-one, and the grayscale variance value set includes a plurality of grayscale variance values;

[0100] Sorting the grayscale variance values ​​in the grayscale variance value set in order from large to small to obtain a grayscale variance sequence;

[0101] The grayscale variance values ​​are sequentially extracted from the grayscale variance sequence, and the following operations are performed on the extracted grayscale variance values:

[0102] The grayscale variance values ​​that have not been extracted in the grayscale variance value set are used to obtain the mean of the grayscale variance values, thereby obtaining the grayscale variance mean;

[0103] Comparing the grayscale variance mean with a preset grayscale variance threshold;

[0104] If the grayscale variance mean is greater than the grayscale variance threshold, the extracted grayscale variance value is eliminated from the grayscale variance value set to obtain an updated variance value set, the updated variance value set is the grayscale variance value set, and the step of performing a sorting operation on the grayscale variance values ​​in the grayscale variance value set in descending order is returned until the grayscale variance mean is less than or equal to the grayscale variance threshold;

[0105] If the grayscale variance means in the grayscale variance value set are all less than or equal to the grayscale variance threshold, the target background image is obtained using the balanced image set.

[0106] It is understandable that when multiple shooting units are used for shooting, some of the shooting units may be affected by factors such as lighting and environment. For example, when multiple shooting units are used to monitor the same area, sudden lighting may cause one or more of the multiple shooting units to produce a surveillance image that cannot express the scene in the environment.

[0107] Furthermore, the monitoring background image in the video sequence will change with the changes in the environment of the monitoring area. Therefore, obtaining the target background image in the video sequence obtained by shooting the monitoring area in real time can adapt to the changes in the environment, thereby improving the accuracy of the analysis of the monitoring image. For example, when the natural environment is used as the monitoring area, the lighting conditions in the monitoring area are constantly changing over time. If only the images collected from the natural environment in a certain period of time are used as the target background image, it may lead to inaccurate analysis results when analyzing the area.

[0108] It should be understood that, optionally, a monitoring probe is used as a shooting unit, and generally, a shooting unit can shoot more pictures per second. Therefore, the background extraction frame number is used here to process the monitored pictures to increase the acquisition speed of the target background image and ensure the timeliness of the target background image. For example: when multiple shooting units are shooting in real time, their frame rate is 24 frames, and the preset background extraction frame number is 7200 frames, that is, the shooting unit can shoot 24 images per second, and the background extraction frame number obtains a monitoring background picture set from multiple video sequences once every 5 minutes.

[0109] It is understood that the grayscale operation refers to the operation of converting a color image into a grayscale image, and the equalization operation refers to the operation of enhancing the contrast and visual effect of an image by redistributing the pixel values ​​of the image, and the grayscale operation and the equalization operation are both prior arts and will not be described in detail here. The grayscale variance value refers to the variance of multiple grayscale values ​​of the equalized image in the equalized image set.

[0110] It should be understood that when the monitoring background picture is affected, its main manifestation is that there is an obvious difference between the monitoring background picture and the monitoring area, which is manifested here as: the variance value of the monitoring background picture exists in the monitoring background picture concentration, which is significantly higher than the variance values ​​of other monitoring background pictures. Then, by sorting, comparing and eliminating the grayscale variance values, the accuracy of acquiring the target background image can be improved.

[0111] For example, there is one shooting unit among the 10 shooting units, which happens to be illuminated by light when acquiring the monitoring background picture set, so the monitoring background picture corresponding to the shooting unit has a clearly illuminated part. If the monitoring background picture set is not processed, the target background image acquired through the monitoring background picture set will have a large error.

[0112] It should be explained that the method of obtaining the target background image by using the balanced image set includes:

[0113] Extracting gray value groups in sequence using the balanced image set, wherein the gray value group includes a plurality of gray values, and the pixel coordinates of the balanced image corresponding to the gray values ​​in the gray value group are all the same;

[0114] The grayscale variance group and the preset image division window are used to calculate the target background grayscale value. The calculation formula is as follows:

[0115]

[0116] Among them, H represents the gray value of the target background, n represents the total number of pixels in the image division window, and x irepresents the gray value in the gray value group corresponding to the i-th pixel in the image division window, a represents the preset coefficient, m represents that there are m gray values ​​in the gray value group, and j and k are used for counting;

[0117] The target background image is obtained based on the target background grayscale value.

[0118] It should be explained that the gray value group is a set of gray values ​​of each balanced image at each pixel coordinate in the balanced atlas. For example, if the balanced atlas includes 10 balanced images, and each balanced image includes 50 pixels, a total of 50 gray value groups can be extracted from the balanced atlas, where the first gray value group is a set of gray values ​​of the first pixel of each balanced image in the 10 balanced images.

[0119] It should be understood that the image division window is a preset window for calculating local features of the image. Optionally, a 3×3 window is selected as the image division window. Other technologies can achieve the same effect, which will not be repeated here. The purpose of the preset image division window is to achieve local analysis of grayscale values ​​through the image division window to solve the problem of image exposure that may occur due to graying and equalization of the monitoring background image, thereby improving the accuracy of obtaining the target background image.

[0120] Furthermore, obtaining a target background image using a target background grayscale value means converting a grayscale image corresponding to the target background grayscale value into a color image, and the technology of obtaining a target background image using a target background grayscale value is prior art and will not be described in detail here.

[0121] S3. Confirm receipt of the screen division instruction from the monitoring screen processing unit, obtain an analysis model set based on the screen division instruction, obtain a target image using an initial image and a target background image, and obtain one or more target monitoring ratios that identify the target monitoring object based on the target image and the analysis model set, wherein the analysis model set includes multiple analysis models.

[0122] It is understandable that the number of video frames is the number of frames in the initial image. Optionally, the number of video frames is 24 frames per second. Other technologies can achieve the same effect, which will not be repeated here. The analysis model set refers to a set of multiple models for monitoring images for analysis. Optionally, the analysis model set refers to a set of multiple models for image analysis.

[0123] Furthermore, the step of obtaining the target image by using the initial image and the target background image includes:

[0124] The pre-built Canny algorithm and the initial image are used to obtain the initial target image, and the first monitoring image is calculated based on the initial target image and the initial image. The calculation formula is as follows:

[0125] J(y,z)=C(y,z)-B(y,z)

[0126] Wherein, J(y,z) represents the first monitoring image, C(y,z) represents the initial image, B(y,z) represents the initial target image, and (y,z) represents the pixel at the yth row and the zth column in the initial image;

[0127] Acquire a monitoring mean based on the first monitoring image, wherein the monitoring mean is the mean of the grayscale values ​​in the first monitoring image;

[0128] Extract pixel points from the first monitoring image in sequence, and perform the following operations on the extracted pixel points:

[0129] Compare the grayscale value corresponding to the pixel with the monitored mean value;

[0130] If the grayscale value corresponding to the pixel is greater than or equal to the monitored mean, the pixel is marked as 1;

[0131] If the grayscale value corresponding to the pixel is less than the monitored mean, the pixel is marked as 0;

[0132] A first identification image is obtained based on the identified pixel points, a second identification image is obtained using the initial image and the target background image, and a plurality of reference pixel point groups are obtained based on the first identification image and the second identification image, wherein the reference pixel point groups are composed of a first pixel point in the first identification image and a second pixel point in the second identification image, and the pixel coordinates corresponding to the first pixel point and the second pixel point are the same;

[0133] The following operations are performed on each of the multiple reference pixel groups:

[0134] If the identifiers corresponding to the first pixel point and the second pixel point are both 1, the first pixel point in the first identification image is identified as a target object point;

[0135] The target object points are aggregated to obtain a target object point set, and a target image is acquired based on the target object point set.

[0136] It should be understood that the Canny algorithm is a prior art and will not be described in detail here. The initial target image refers to an image that is obtained by graying the initial image and then equalizing it. Ideally, the initial target image obtained using the Canny algorithm and the initial image is an image that only includes the target monitoring object. For example, when one or more people enter the monitoring area, an image that only includes one or more people can be extracted through the initial image and the Canny algorithm. The purpose of marking pixels is to distinguish between images including the target monitoring object and images in the background. For example, in the first monitoring image, all pixels including the target monitoring object are marked as 1, and all pixels not including the target monitoring object are marked as 0, so that only pixels including the target monitoring object can be extracted from the first monitoring image, thereby improving the accuracy of identifying the target monitoring object. The technology for performing marking operations on pixels in an image is a prior art and will not be described in detail here.

[0137] Furthermore, the first identification image refers to an initial image after the identification value is added. The second identification image is obtained in the same manner as the first identification image and can achieve the same effect, which will not be described in detail.

[0138] It is understandable that the total number and position of the pixels included in the first identification image and the second identification image are consistent. The reference pixel group is two pixels in the same position in the first identification image and the second identification image, and the number of reference pixel groups is related to the resolution of the image. The target object point set includes a set of multiple pixels used to represent the target monitoring object. The target image refers to an image used to represent the target monitoring object. For example: when one or more suspicious persons appear in the monitoring environment, the target image refers to the image of one or more suspicious persons.

[0139] It should be explained that the step of obtaining one or more target monitoring ratios that identify target monitoring objects based on the target image and the analysis model set includes:

[0140] An analysis model is randomly extracted from the analysis model set, one or more target monitoring objects are obtained based on the extracted analysis model and the target image, a marking operation is performed on the one or more target monitoring objects using a pre-constructed marking sequence to obtain one or more marked detection objects, and the extracted analysis model is eliminated from the analysis model set, and the included analysis model set is summarized to obtain an updated model set;

[0141] Taking the updated model set as the analysis model set, returning to the step of randomly extracting an analysis model from the analysis model set, respectively aggregating the identified detection objects according to the identification sequence, and obtaining one or more identified detection object groups;

[0142] Extracting identification detection object groups from the one or more identification detection object groups in sequence, and performing the following operations on the extracted identification detection object groups:

[0143] Randomly extract an analysis model from the update model set, and obtain a similarity evaluation value using the extracted analysis model and the identified detection object group;

[0144] Comparing the similarity evaluation value with a preset similarity threshold;

[0145] If the similarity evaluation value is greater than or equal to the similarity threshold, one or more identified detection objects are aggregated to obtain a target monitoring object set;

[0146] If the similarity evaluation value is less than the similarity threshold, the updated model set is used to obtain an updated identification detection object group;

[0147] In a combined manner, a plurality of update groups are obtained by using the update identification detection object group;

[0148] For each of the multiple update groups, perform the following operations:

[0149] Acquire update evaluation values ​​based on update grouping, summarize the update evaluation values ​​to obtain an update evaluation value set, and use the update evaluation value set to identify a target monitoring object, wherein the target monitoring object is an identified detection object corresponding to the largest update evaluation value in the update evaluation value set;

[0150] The target monitoring objects are aggregated to obtain a target monitoring object set, and the target monitoring object set is used to obtain one or more target monitoring proportions that identify the target monitoring objects.

[0151] It is understandable that the analysis model refers to a model used to process and identify images. Optionally, the analysis model set includes: YOLOv3, YOLOv5, convolutional neural network, etc. Different models have different structures and constituent parameters. Therefore, when different analysis models are used to analyze the same image, they may produce different results. The identification sequence refers to a sequence with a certain order used to distinguish target monitoring objects. Optionally, the identification is performed from left to right. For example: the target image includes 10 target monitoring objects, and these 10 target monitoring objects are identified from left to right in sequence, then the target monitoring object on the far left is target monitoring object No. 1, and the target monitoring object on the far right is target monitoring object No. 10, and the technology for performing identification operations on target monitoring objects is the prior art and will not be repeated here.

[0152] It should be understood that the identification detection object group refers to the target monitoring objects with the same identification obtained after analyzing the same target image by two different analysis models. The similarity evaluation value is a numerical value used to evaluate the similarity of two target monitoring objects in the identification detection object group. When the similarity evaluation value is greater than or equal to the similarity threshold, it can be determined that the two target monitoring objects in the identification detection object group are consistent.

[0153] It should be explained that for every analysis model in the update model set, a group of identification detection objects can be obtained when the target image is detected, wherein a group of identification detection objects is composed of one or more identification detection objects. The updated identification detection object group includes: using the update model set and one or more identification detection object groups obtained before the update model set, and the identification values ​​corresponding to the identification detection objects included in the updated identification detection object group are consistent. For example: the updated identification model set includes 8 analysis models, and the identification detection object group obtained before the update model set includes 2 identification detection objects, and the number of target monitoring objects is 10, then the updated identification detection object group includes 10 identification detection objects with consistent identification values. Multiple update groups are obtained by combining these 10 identification detection objects, wherein each update group includes only two identification detection objects.

[0154] It is understandable that the update evaluation value is obtained in the same manner as the similarity evaluation value, and can achieve the same effect. By comparing the update evaluation values ​​corresponding to the update groups, the purpose is to confirm the identification order or value corresponding to the identification detection object in the updated identification detection object group, so as to analyze according to each target monitoring object to improve the accuracy of the analysis of the target monitoring object. For example: when the first analysis model is used to identify the target image, it identifies the two target monitoring objects as one identification detection object, and when the second analysis model is used to identify the target image, it can identify the two target monitoring objects as the first identification detection object and the second identification detection object.

[0155] Further, the method of obtaining one or more target monitoring proportions that identify the target monitoring objects by using the target monitoring object set includes:

[0156] Extract target monitoring objects from the target monitoring object set in sequence, and perform the following operations on the extracted target monitoring objects:

[0157] Acquire the target pixel count based on the extracted target monitoring object;

[0158] The target monitoring ratio is obtained by using the number of target pixels and the pixels included in the initial image;

[0159] The target monitoring proportions are summarized to obtain one or more target monitoring proportions that identify the target monitoring objects.

[0160] It should be explained that the target pixel number refers to the number of pixels in the image corresponding to the target monitoring object. The target monitoring ratio refers to the ratio of the target pixel number to the target monitoring ratio.

[0161] S4. Summarize the target monitoring proportions by using the target monitoring objects to obtain one or more target monitoring proportion sets, identify the main shooting units of one or more target monitoring objects from the one or more target monitoring proportion sets, confirm receipt of the screen integration instruction from the monitoring screen integration unit, and fit the fitting video stream and the monitoring target path of the target monitoring object based on the screen integration instruction and the main shooting unit.

[0162] It is understandable that when multiple calibration shooting units are used to shoot the same monitoring area, the position and angle of the calibration shooting unit in the actual position and the behavior of the target monitoring object may cause different target monitoring ratios when shooting at different times and locations using calibration shooting units in different positions. For example: there are three calibration shooting units, and the corresponding monitoring areas are the same, but the three calibration shooting units are located in different positions. When there are two target monitoring objects in the monitoring area, two target monitoring ratio sets can be obtained through these two target monitoring objects, and each target monitoring ratio set includes three target monitoring ratios.

[0163] It should be understood that the main shooting unit that identifies one or more target monitoring objects from the one or more target monitoring proportion sets includes:

[0164] Acquire multiple divided monitoring ratio sets using a preset divided time threshold, one or more target monitoring ratio sets, and multiple calibration shooting units;

[0165] Using multiple partition monitoring proportion sets to obtain multiple partition monitoring means, wherein the partition monitoring means correspond to the partition monitoring proportion sets one by one, and the partition monitoring means are the means of the target monitoring proportions in the partition monitoring proportion sets;

[0166] Using the target monitoring objects corresponding to multiple divided monitoring means, the divided monitoring means are summarized respectively to obtain a divided monitoring mean set, and the main shooting unit is obtained based on the divided monitoring mean set, wherein the main shooting unit is the shooting unit corresponding to the largest divided monitoring mean in the divided monitoring mean set.

[0167] It should be explained that one or more target monitoring proportion sets are the proportions of one or more target monitoring objects in the images captured by different shooting units. Therefore, one or more target monitoring proportion sets can be divided by dividing the time threshold and multiple calibration shooting units to obtain a set of target monitoring proportions of each calibration shooting unit within the divided time threshold. This set is the divided monitoring proportion set, and the target monitoring object corresponding to the divided monitoring proportion set is the same target monitoring object. For example: the division time threshold is once every 2 seconds, there are 3 calibration shooting units, the shooting time is 4 seconds, each calibration shooting unit can shoot 24 pictures per second, and the number of target monitoring objects is two, then the number in the target monitoring proportion set is two, and through the division of the target monitoring proportion set, we can obtain: two divided monitoring proportion sets for each of the 3 calibration shooting units.

[0168] It should be understood that, within the divided time threshold, the larger the divided monitoring mean value of a target monitoring object corresponding to a certain calibration shooting unit is, the more information the target monitoring object has within the time corresponding to the calibration shooting unit and the divided time threshold. Therefore, using the calibration shooting unit as the main shooting unit can obtain information that is more consistent with the target monitoring object, thereby improving the accuracy of monitoring the target monitoring object.

[0169] Furthermore, the fitting of the video stream and the monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit includes:

[0170] Using the picture integration instruction and the division time threshold corresponding to the main shooting unit to obtain multiple video segments, and using the pre-built video fitting model and the multiple video segments to fit a fitting video stream;

[0171] Extract video segments from the fitted video stream in sequence, and perform the following operations on the extracted video segments:

[0172] The target monitoring images are sequentially extracted from the extracted video segments using the video frame number, and the following operations are performed on the extracted target monitoring images:

[0173] The monitoring center is calculated based on the extracted target monitoring image and the pre-constructed center extraction relationship, wherein the monitoring center is composed of the monitoring row coordinates and the monitoring column coordinates. The monitoring centers are summarized to obtain a monitoring center set, and the monitoring target path is fitted using the pre-constructed path fitting relationship and the monitoring center set.

[0174] It is understandable that the main shooting unit used to represent the main information of each target monitoring object may be different in different time periods. Therefore, it is necessary to divide the video shot by each calibration shooting unit according to the division time threshold when the main shooting unit corresponding to each target monitoring unit changes, so as to obtain multiple video segments. For example: there are 3 calibration shooting units and two target monitoring objects. At the first division time threshold, the main shooting unit of the first target monitoring object changes, and at the second division time threshold, the main shooting unit of the second target monitoring object changes. Then, it is necessary to divide the videos shot by the three calibration shooting units at the time corresponding to the first division time threshold and the time corresponding to the second division time threshold to obtain multiple video segments. Optionally, Adobe Premiere Pro is selected as the video fitting model, and other technologies can achieve the same effect, which will not be repeated here. The video fitting model can fit the video segments expressing each target monitoring unit into a video in chronological order, and the video is the fitted video stream.

[0175] It should be understood that the center extraction relationship is as follows:

[0176]

[0177] Among them, H0 represents the monitoring row coordinate, H1 represents the monitoring column coordinate, and h b0 represents the coordinate value of the bth row in the target monitoring image, h 10 represents the center value of the row coordinates in the target monitoring image, h 11 represents the center value of the column coordinates in the target monitoring image, h b1 represents the coordinate value of the bth column in the target monitoring image, δ is the preset coefficient, S represents the total number of S pixels in the target monitoring image, and v b0 represents the number of pixels in the bth row of the target monitoring image, v b1 Indicates the number of pixels in the bth column of the target monitoring image, max indicates the maximum value, and [] indicates rounding operation.

[0178] It should be explained that in real life, the position of the target monitoring object is extremely random and uncertain. Therefore, in the embodiment of the present invention, the center value of the row coordinates in the target monitoring image refers to: the midpoint of the row coordinates in the target monitoring image. The center value of the column coordinates in the target monitoring image is obtained in the same way as the center value of the row coordinates in the target monitoring image, and can achieve the same effect, which will not be repeated here.

[0179] Furthermore, the path fitting relationship is as follows:

[0180]

[0181] Among them, f represents the monitoring target path, are the coefficients generated during fitting, and c represents the independent variable.

[0182] It is understandable that the monitoring target path is obtained by fitting a set of discrete points that change over time to represent the path walked by the target monitoring object. Optionally, the monitoring target path is fitted using the least squares method, and during the fitting, the independent variable can be time. Other methods can achieve the same effect, which will not be repeated here.

[0183] S5. Use the data result feedback unit to send the fitted video stream and the monitoring target path to the initiator of the picture optimization instruction to achieve real-time optimization of the monitoring picture.

[0184] Exemplarily, three target monitoring objects appear in the monitoring area. When the three target monitoring objects all leave the monitoring area, three fitting video streams and three monitoring target paths can be generated, wherein the fitting video streams correspond one-to-one to the target monitoring objects, and the monitoring target paths correspond one-to-one to the target monitoring objects. The three fitting video streams and the three monitoring target paths are sent to the initiator of the picture optimization instruction through the data result feedback unit.

[0185] Compared with the problem described in the background technology, the present invention first receives a picture optimization instruction, and confirms a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: multiple shooting units for shooting, a monitoring area to be monitored and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit and a data result feedback unit. It can be seen that the present invention combines the actual situation in life and formulates a monitoring environment that is more in line with actual life. In addition, the present invention confirms that a calibration operation is performed on multiple shooting units to obtain multiple calibrated shooting units, wherein the serial numbers of the calibrated shooting units are identified, and the monitoring area is photographed in real time based on the multiple calibrated shooting units to obtain multiple video sequences, and the target background image is obtained based on the multiple video sequences. Then, by calibrating the multiple shooting units, multiple calibrated shooting units are obtained, thereby realizing that the videos shot by the multiple calibrated shooting units can be analyzed together, thereby improving the intelligence of positioning analysis of multiple targets, and when obtaining the target background image, it is considered that the target background image may change with time and that the target background image may change through two different methods. The formula is used to jointly fit a target background image that conforms to the actual situation, thereby improving the accuracy of the target background image, and thereby improving the accuracy of positioning analysis of multiple targets, confirming the reception of a screen division instruction from the monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using the initial image and the target background image, and obtaining one or more target monitoring ratios based on the target image and the analysis model set, wherein the target monitoring object of the target monitoring ratio is identified, and the analysis model set includes multiple models for analysis. It can be seen that the embodiment of the present invention takes into account the randomness of the target monitoring object, and therefore, dynamically confirms the target monitoring object, and thereby improves the accuracy of positioning analysis of multiple targets by clarifying the form of the target monitoring object. The target monitoring objects are used to summarize the target monitoring proportions respectively to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instructions received from the monitoring screen integration unit are confirmed. Based on the screen integration instructions and the main shooting unit, the fitting video stream and the monitoring target path of the target monitoring object are fitted. It can be seen that the embodiment of the present invention takes into account the dynamic movement of the target monitoring object. Therefore, the main shooting unit is identified in multiple shooting units, and the fitting video stream is fitted with the video segment corresponding to the main shooting unit, and the monitoring target path is fitted. Therefore, the real-time optimization method, device, electronic device and computer-readable storage medium of the monitoring screen based on multi-target positioning analysis proposed in the present invention mainly aim to achieve accurate and intelligent positioning analysis of multiple targets.

[0186] Embodiment 2:

[0187] like Figure 2, which is a schematic diagram of the structure of an electronic device for implementing a method for real-time optimization of monitoring images based on multi-target positioning analysis provided by an embodiment of the present invention.

[0188] The electronic device 1 may include a processor 10, a memory 11, a bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a real-time optimization program for monitoring images based on multi-target positioning analysis.

[0189] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of a real-time optimization program for monitoring images based on multi-target positioning analysis, but may also be used to temporarily store data that has been output or is to be output.

[0190] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as a real-time optimization program for monitoring images based on multi-target positioning analysis, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0191] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0192] Figure 2 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0193] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0194] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0195] Optionally, the electronic device 1 may further include a user interface, which may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0196] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0197] The monitoring screen real-time optimization program based on multi-target positioning analysis stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0198] Receiving a picture optimization instruction, and confirming a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: a plurality of shooting units for shooting, a monitoring area to be monitored, and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit, and a data result feedback unit;

[0199] Performing a calibration operation on the plurality of shooting units to obtain a plurality of calibration shooting units with serial numbers identified thereon, shooting a monitoring area in real time based on the plurality of calibration shooting units to obtain a plurality of video sequences, and acquiring a target background image based on the plurality of video sequences;

[0200] Extracting video sequences from the multiple video sequences in sequence, and performing the following operations on the extracted video sequences:

[0201] The video frame number of the video sequence is obtained, and the initial images are sequentially extracted from the extracted video sequence using the video frame number, and the following operations are performed on the extracted initial images:

[0202] Confirming receipt of a screen division instruction from a monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using an initial image and a target background image, and obtaining one or more target monitoring ratios that identify a target monitoring object based on the target image and the analysis model set, wherein the analysis model set includes a plurality of analysis models;

[0203] The target monitoring proportions are respectively summarized by using the target monitoring objects to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instruction from the monitoring screen integration unit is confirmed to be received, and the fitting video stream and the monitoring target path of the target monitoring object are fitted based on the screen integration instruction and the main shooting unit;

[0204] The step of fitting the video stream and the monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit includes:

[0205] Using the picture integration instruction and the division time threshold corresponding to the main shooting unit to obtain multiple video segments, and using the pre-built video fitting model and the multiple video segments to fit a fitting video stream;

[0206] Extract video segments from the fitted video stream in sequence, and perform the following operations on the extracted video segments:

[0207] The target monitoring images are sequentially extracted from the extracted video segments using the video frame number, and the following operations are performed on the extracted target monitoring images:

[0208] The monitoring center is calculated based on the extracted target monitoring image and the pre-constructed center extraction relation, wherein the monitoring center is composed of the monitoring row coordinates and the monitoring column coordinates, the monitoring centers are summarized to obtain a monitoring center set, and the monitoring target path is fitted using the pre-constructed path fitting relation and the monitoring center set;

[0209] The center extraction relationship is as follows:

[0210]

[0211] Among them, H0 represents the monitoring row coordinate, H1 represents the monitoring column coordinate, and h b0 represents the coordinate value of the bth row in the target monitoring image, h 10 represents the center value of the row coordinates in the target monitoring image, h 11 represents the center value of the column coordinates in the target monitoring image, h b1 represents the coordinate value of the bth column in the target monitoring image, δ is the preset coefficient, S represents the total number of S pixels in the target monitoring image, and v b0 represents the number of pixels in the bth row of the target monitoring image, v b1 Indicates the number of pixels in the bth column of the target monitoring image, max indicates the maximum value, and [] indicates rounding operation;

[0212] The path fitting relationship is as follows:

[0213]

[0214] Among them, f represents the monitoring target path, are the coefficients generated during fitting, and c represents the independent variable;

[0215] The data result feedback unit is used to send the fitted video stream and the monitoring target path to the initiator of the picture optimization instruction to achieve real-time optimization of the monitoring picture.

[0216] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0217] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0218] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0219] Receiving a picture optimization instruction, and confirming a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: a plurality of shooting units for shooting, a monitoring area to be monitored, and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit, and a data result feedback unit;

[0220] Performing a calibration operation on the plurality of shooting units to obtain a plurality of calibration shooting units with serial numbers identified thereon, shooting a monitoring area in real time based on the plurality of calibration shooting units to obtain a plurality of video sequences, and acquiring a target background image based on the plurality of video sequences;

[0221] Extracting video sequences from the multiple video sequences in sequence, and performing the following operations on the extracted video sequences:

[0222] The video frame number of the video sequence is obtained, and the initial images are sequentially extracted from the extracted video sequence using the video frame number, and the following operations are performed on the extracted initial images:

[0223] Confirming receipt of a screen division instruction from a monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using an initial image and a target background image, and obtaining one or more target monitoring ratios that identify a target monitoring object based on the target image and the analysis model set, wherein the analysis model set includes a plurality of analysis models;

[0224] The target monitoring proportions are respectively summarized by using the target monitoring objects to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instruction from the monitoring screen integration unit is confirmed to be received, and the fitting video stream and the monitoring target path of the target monitoring object are fitted based on the screen integration instruction and the main shooting unit;

[0225] The step of fitting the video stream and the monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit includes:

[0226] Using the picture integration instruction and the division time threshold corresponding to the main shooting unit to obtain multiple video segments, and using the pre-built video fitting model and the multiple video segments to fit a fitting video stream;

[0227] Extract video segments from the fitted video stream in sequence, and perform the following operations on the extracted video segments:

[0228] The target monitoring images are sequentially extracted from the extracted video segments using the video frame number, and the following operations are performed on the extracted target monitoring images:

[0229] The monitoring center is calculated based on the extracted target monitoring image and the pre-constructed center extraction relation, wherein the monitoring center is composed of the monitoring row coordinates and the monitoring column coordinates, the monitoring centers are summarized to obtain a monitoring center set, and the monitoring target path is fitted using the pre-constructed path fitting relation and the monitoring center set;

[0230] The center extraction relationship is as follows:

[0231]

[0232] Among them, H0 represents the monitoring row coordinate, H1 represents the monitoring column coordinate, and h b0 represents the coordinate value of the bth row in the target monitoring image, h 10 represents the center value of the row coordinates in the target monitoring image, h 11 represents the center value of the column coordinates in the target monitoring image, h b1 represents the coordinate value of the bth column in the target monitoring image, δ is the preset coefficient, S represents the total number of S pixels in the target monitoring image, and v b0 represents the number of pixels in the bth row of the target monitoring image, v b1 Indicates the number of pixels in the bth column of the target monitoring image, max indicates the maximum value, and [] indicates rounding operation;

[0233] The path fitting relationship is as follows:

[0234]

[0235] Among them, f represents the monitoring target path, are the coefficients generated during fitting, and c represents the independent variable;

[0236] The data result feedback unit is used to send the fitted video stream and the monitoring target path to the initiator of the picture optimization instruction to achieve real-time optimization of the monitoring picture.

[0237] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0238] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0239] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A real-time optimization method for monitoring images based on multi-target positioning analysis, characterized in that: The method comprises: Receiving a picture optimization instruction, and confirming a monitoring environment based on the picture optimization instruction, wherein the monitoring environment includes: a plurality of shooting units for shooting, a monitoring area to be monitored, and a picture optimization system, wherein the picture optimization system includes: a monitoring picture processing unit, a monitoring picture integration unit, and a data result feedback unit; Performing a calibration operation on the plurality of shooting units to obtain a plurality of calibration shooting units with serial numbers identified thereon, shooting a monitoring area in real time based on the plurality of calibration shooting units to obtain a plurality of video sequences, and acquiring a target background image based on the plurality of video sequences; Extracting video sequences from the multiple video sequences in sequence, and performing the following operations on the extracted video sequences: The video frame number of the video sequence is obtained, and the initial images are sequentially extracted from the extracted video sequence using the video frame number, and the following operations are performed on the extracted initial images: Confirming receipt of a screen division instruction from a monitoring screen processing unit, obtaining an analysis model set based on the screen division instruction, obtaining a target image using an initial image and a target background image, and obtaining one or more target monitoring ratios that identify a target monitoring object based on the target image and the analysis model set, wherein the analysis model set includes a plurality of analysis models; The target monitoring proportions are respectively summarized by using the target monitoring objects to obtain one or more target monitoring proportion sets, and the main shooting units of one or more target monitoring objects are respectively identified from the one or more target monitoring proportion sets, and the screen integration instruction from the monitoring screen integration unit is confirmed to be received, and the fitting video stream and the monitoring target path of the target monitoring object are fitted based on the screen integration instruction and the main shooting unit; The step of fitting the video stream and the monitoring target path of the target monitoring object based on the picture integration instruction and the main shooting unit includes: Using the picture integration instruction and the division time threshold corresponding to the main shooting unit to obtain multiple video segments, and using the pre-built video fitting model and the multiple video segments to fit a fitting video stream; Extract video segments from the fitted video stream in sequence, and perform the following operations on the extracted video segments: The target monitoring images are sequentially extracted from the extracted video segments using the video frame number, and the following operations are performed on the extracted target monitoring images: The monitoring center is calculated based on the extracted target monitoring image and the pre-constructed center extraction relation, wherein the monitoring center is composed of the monitoring row coordinates and the monitoring column coordinates, the monitoring centers are summarized to obtain a monitoring center set, and the monitoring target path is fitted using the pre-constructed path fitting relation and the monitoring center set; The center extraction relationship is as follows: Among them, H0 represents the monitoring row coordinate, H1 represents the monitoring column coordinate, and h b0 represents the coordinate value of the bth row in the target monitoring image, h 10 represents the center value of the row coordinates in the target monitoring image, h 11 represents the center value of the column coordinates in the target monitoring image, h b1 represents the coordinate value of the bth column in the target monitoring image, δ is the preset coefficient, S represents the total number of S pixels in the target monitoring image, and v b0 represents the number of pixels in the bth row of the target monitoring image, v b1 Indicates the number of pixels in the bth column of the target monitoring image, max indicates the maximum value, and [] indicates rounding operation; The path fitting relationship is as follows: f=θ0+θ1c+θ2c 2 +θ3c 3 +θ4c 4 Among them, f represents the monitoring target path, θ0, θ1, θ2, θ3, and θ4 are the coefficients generated during fitting, and c represents the independent variable; The data result feedback unit is used to send the fitted video stream and the monitoring target path to the initiator of the picture optimization instruction to achieve real-time optimization of the monitoring picture.

2. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 1 is characterized in that: The acquiring a target background image based on the multiple video sequences comprises: Acquire a monitoring background picture set based on a preset background extraction frame number and a plurality of video sequences, wherein the monitoring background picture set includes a plurality of monitoring background pictures, and the monitoring background pictures correspond to the calibration shooting units one by one; Extract monitoring background images from the monitoring background image set in sequence, and perform the following operations on the extracted monitoring background images: Performing a grayscale operation on the extracted monitoring background image to obtain a grayscale image, and performing an equalization operation on the grayscale image to obtain an equalized image; Summarizing the balanced images to obtain a balanced image set, and obtaining a grayscale variance value set based on the balanced image set, wherein the balanced image set corresponds to the grayscale variance value set one-to-one, and the grayscale variance value set includes a plurality of grayscale variance values; Sorting the grayscale variance values ​​in the grayscale variance value set in descending order to obtain a grayscale variance sequence; The grayscale variance values ​​are sequentially extracted from the grayscale variance sequence, and the following operations are performed on the extracted grayscale variance values: The grayscale variance values ​​that have not been extracted in the grayscale variance value set are used to obtain the mean of the grayscale variance values, thereby obtaining the grayscale variance mean; Comparing the grayscale variance mean with a preset grayscale variance threshold; If the grayscale variance mean is greater than the grayscale variance threshold, the extracted grayscale variance value is eliminated from the grayscale variance value set to obtain an updated variance value set, the updated variance value set is the grayscale variance value set, and the step of performing a sorting operation on the grayscale variance values ​​in the grayscale variance value set in descending order is returned until the grayscale variance mean is less than or equal to the grayscale variance threshold; If the grayscale variance means in the grayscale variance value set are all less than or equal to the grayscale variance threshold, the target background image is obtained using the balanced image set.

3. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 2 is characterized in that: The method of obtaining the target background image by using the balanced image set includes: Extracting gray value groups in sequence using the balanced image set, wherein the gray value group includes a plurality of gray values, and the pixel coordinates of the balanced image corresponding to the gray values ​​in the gray value group are all the same; The grayscale variance group and the preset image division window are used to calculate the target background grayscale value. The calculation formula is as follows: Among them, H represents the gray value of the target background, n represents the total number of pixels in the image division window, and x i represents the gray value in the gray value group corresponding to the i-th pixel in the image division window, a represents the preset coefficient, m represents that there are m gray values ​​in the gray value group, and j and k are used for counting; The target background image is obtained based on the target background grayscale value.

4. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 1 is characterized in that: The method of obtaining a target image by using the initial image and the target background image comprises: The pre-built Canny algorithm and the initial image are used to obtain the initial target image, and the first monitoring image is calculated based on the initial target image and the initial image. The calculation formula is as follows: J(y,z)=C(y,z)-B(y,z) Wherein, J(y,z) represents the first monitoring image, C(y,z) represents the initial image, B(y,z) represents the initial target image, and (y,z) represents the pixel at the yth row and the zth column in the initial image; Acquire a monitoring mean based on the first monitoring image, wherein the monitoring mean is the mean of the grayscale values ​​in the first monitoring image; Extract pixel points from the first monitoring image in sequence, and perform the following operations on the extracted pixel points: Compare the grayscale value corresponding to the pixel with the monitored mean value; If the grayscale value corresponding to the pixel is greater than or equal to the monitored mean, the pixel is marked as 1; If the grayscale value corresponding to the pixel is less than the monitored mean, the pixel is marked as 0; A first identification image is obtained based on the identified pixel points, a second identification image is obtained using the initial image and the target background image, and a plurality of reference pixel point groups are obtained based on the first identification image and the second identification image, wherein the reference pixel point groups are composed of a first pixel point in the first identification image and a second pixel point in the second identification image, and the pixel coordinates corresponding to the first pixel point and the second pixel point are the same; The following operations are performed on each of the multiple reference pixel groups: If the identifiers corresponding to the first pixel point and the second pixel point are both 1, the first pixel point in the first identification image is identified as a target object point; The target object points are aggregated to obtain a target object point set, and a target image is acquired based on the target object point set.

5. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 1 is characterized in that: The step of obtaining one or more target monitoring proportions identifying target monitoring objects based on the target image and the analysis model set includes: An analysis model is randomly extracted from the analysis model set, one or more target monitoring objects are obtained based on the extracted analysis model and the target image, a marking operation is performed on the one or more target monitoring objects using a pre-constructed marking sequence to obtain one or more marked detection objects, and the extracted analysis model is eliminated from the analysis model set, and the included analysis model set is summarized to obtain an updated model set; Taking the updated model set as the analysis model set, returning to the step of randomly extracting an analysis model from the analysis model set, respectively aggregating the identified detection objects according to the identification sequence, and obtaining one or more identified detection object groups; Extracting identification detection object groups from the one or more identification detection object groups in sequence, and performing the following operations on the extracted identification detection object groups: Randomly extract an analysis model from the update model set, and obtain a similarity evaluation value using the extracted analysis model and the identified detection object group; Comparing the similarity evaluation value with a preset similarity threshold; If the similarity evaluation value is greater than or equal to the similarity threshold, one or more identified detection objects are aggregated to obtain a target monitoring object set; If the similarity evaluation value is less than the similarity threshold, the updated model set is used to obtain an updated identification detection object group; In a combined manner, a plurality of update groups are obtained by using the update identification detection object group; For each of the multiple update groups, perform the following operations: Acquire update evaluation values ​​based on update grouping, summarize the update evaluation values ​​to obtain an update evaluation value set, and use the update evaluation value set to identify a target monitoring object, wherein the target monitoring object is an identified detection object corresponding to the largest update evaluation value in the update evaluation value set; The target monitoring objects are aggregated to obtain a target monitoring object set, and the target monitoring object set is used to obtain one or more target monitoring proportions that identify the target monitoring objects.

6. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 5, characterized in that: The step of obtaining one or more target monitoring proportions that identify the target monitoring objects by using the target monitoring object set includes: Extract target monitoring objects from the target monitoring object set in sequence, and perform the following operations on the extracted target monitoring objects: Acquire the target pixel count based on the extracted target monitoring object; The target monitoring ratio is obtained by using the number of target pixels and the pixels included in the initial image; The target monitoring proportions are summarized to obtain one or more target monitoring proportions that identify the target monitoring objects.

7. The real-time optimization method for monitoring images based on multi-target positioning analysis according to claim 1 is characterized in that: The main shooting unit that respectively identifies one or more target monitoring objects from the one or more target monitoring proportion sets includes: Acquire multiple divided monitoring ratio sets using a preset divided time threshold, one or more target monitoring ratio sets, and multiple calibration shooting units; Using multiple partition monitoring proportion sets to obtain multiple partition monitoring means, wherein the partition monitoring means correspond to the partition monitoring proportion sets one by one, and the partition monitoring means are the means of the target monitoring proportions in the partition monitoring proportion sets; Using the target monitoring objects corresponding to multiple divided monitoring means, the divided monitoring means are summarized respectively to obtain a divided monitoring mean set, and the main shooting unit is obtained based on the divided monitoring mean set, wherein the main shooting unit is the shooting unit corresponding to the largest divided monitoring mean in the divided monitoring mean set.