A video analysis method, apparatus, device and medium

Abstract

Embodiments of the present application provide a video analysis method, device, equipment and medium to solve the problem of wasting the computing power of the chips of the backend equipment in the prior art. In the method, a plurality of first video frames are extracted from a video stream including a target object according to a currently saved frame extraction rate and an end position of the last extraction; each first video frame is divided into a plurality of sub-regions, and sub-regions in which the target object appears in each first video frame are counted; and a sub-image of each sub-region in which the target object appears in each first video frame is analyzed. First, each sub-region in which the target object appears in each first video frame is determined, and then the sub-image corresponding to each sub-region is analyzed and processed, which can reduce the waste of the computing power of the backend equipment, especially in the case of a scene in which the target is relatively discrete or appears infrequently in the front-end code stream, the waste of the computing power of the backend equipment can be effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a video analysis method, apparatus, device, and medium. Background Technology

[0002] In the current field of video surveillance, such as Figure 1 As shown, backend devices such as network video recorders (NVRs) can connect to multiple frontend devices. Different channels are allocated to access the frontend streams of different frontend devices, and then different types of analysis can be performed on the frontend streams corresponding to different channels, generating analysis results that are then reported. Currently, the allocation method for analyzing the frontend streams corresponding to channels is based on binding channels to analyzers, such as... Figure 2 As shown, channel 1 corresponds to analyzer 1. Each analyzer decodes and analyzes the front-end bitstream corresponding to the channel to obtain the target and attributes of each front-end bitstream, and then reports the obtained target and attributes. Furthermore, once a certain analyzer is enabled for a certain channel, it will always occupy that analyzer to perform real-time analysis of the front-end bitstream corresponding to the channel.

[0003] Furthermore, due to hardware limitations imposed by the chip specifications of each backend device, the total analysis capability is capped, meaning the total computing power is limited. Currently, when allocating computing power, backend devices allocate power based on the resolution of video frames in the frontend stream. Generally, the higher the resolution of video frames in the frontend stream, the more computing power the backend device allocates to that video frame; conversely, the lower the resolution of video frames in the frontend stream, the less computing power the backend device allocates to that video frame.

[0004] However, for some high-resolution video frames, even if the frame lacks or has few valid targets, the backend device will still allocate significant computing power to it because it analyzes the entire frame. Therefore, in scenarios where the front-end stream contains high-resolution video frames but targets are scattered or appear infrequently, analyzing the entire frame by the backend device would severely waste its computing power. Summary of the Invention

[0005] This application provides a video analysis method, apparatus, device, and medium to solve the problem of wasting the computing power of backend device chips in the prior art.

[0006] In a first aspect, embodiments of this application provide a video analysis method, the method comprising:

[0007] Based on the currently saved frame extraction rate and the end position of the last extraction, extract multiple first video frames from the video stream including the target object.

[0008] Each first video frame is divided into multiple sub-regions, and the sub-regions in which the target object appears in each first video frame are counted; the sub-images of the sub-regions in which the target object appears in each first video frame are analyzed.

[0009] Secondly, embodiments of this application also provide a video analysis device, the device comprising:

[0010] The extraction module is used to extract multiple first video frames from the video stream including the target object based on the currently saved extraction frame rate and the end position of the last extraction.

[0011] The statistics module is used to divide each first video frame into multiple sub-regions and count the sub-regions in which the target object appears in each first video frame.

[0012] The analysis module is used to analyze the sub-images of the sub-regions where the target object appears in each first video frame.

[0013] Thirdly, embodiments of this application also provide an electronic device, which includes at least a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the video analysis method as described in any of the preceding claims.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the video analysis method as described in any of the preceding claims.

[0015] In this embodiment, multiple first video frames are extracted from the video stream including the target object based on the currently saved frame extraction rate and the end position of the last extraction. Each first video frame is divided into multiple sub-regions, and the sub-regions in which the target object appears in each first video frame are statistically analyzed. The sub-images of the sub-regions in which the target object appears in each first video frame are analyzed. This method first determines each sub-region in which the target object appears in each first video frame, and then analyzes and processes the sub-images corresponding to each sub-region. This can reduce the waste of computing power on the backend device, especially in scenarios where the target is relatively discrete or appears infrequently in the frontend bitstream. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram illustrating the connection method between front-end and back-end devices;

[0018] Figure 2 This is a schematic diagram showing the correspondence between analyzers and channels in the backend equipment;

[0019] Figure 3 A schematic diagram of a video analysis process provided in an embodiment of this application;

[0020] Figure 4 A schematic diagram of a video frame marked by a front-end device provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram illustrating the mapping of a target object to grid coordinates, as provided in an embodiment of this application.

[0022] Figure 6 This is a schematic diagram of a sub-region where a target object appears, provided in an embodiment of this application.

[0023] Figure 7 A schematic diagram illustrating a scoring result corresponding to a sub-region, provided in an embodiment of this application;

[0024] Figure 8 A schematic diagram of an analysis region with better performance is provided for an embodiment of this application;

[0025] Figure 9 A schematic diagram of a hotspot distribution map provided in an embodiment of this application;

[0026] Figure 10 This application provides a schematic diagram of a hotspot distribution map for a certain time period as an embodiment of the present application.

[0027] Figure 11 A schematic diagram of a security area provided in an embodiment of this application;

[0028] Figure 12 A schematic diagram of an occlusion area provided in an embodiment of this application;

[0029] Figure 13 A schematic diagram of a video analysis process provided in an embodiment of this application;

[0030] Figure 14 This is a schematic diagram of a video analysis device provided in an embodiment of this application;

[0031] Figure 15 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, a further detailed description will be provided below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0033] This application provides a video analysis method, apparatus, device, and medium. The method extracts multiple first video frames from a video stream including a target object based on the currently saved frame extraction rate and the end position of the previous extraction. Each first video frame is divided into multiple sub-regions, and the sub-regions in which the target object appears in each first video frame are statistically analyzed. The sub-images of the sub-regions in which the target object appears in each first video frame are analyzed. This method first determines each sub-region in which the target object appears in each first video frame, and then analyzes and processes the sub-images corresponding to each sub-region. This can reduce the waste of computing power on backend devices, especially in scenarios where the target is relatively discrete or appears infrequently in the frontend bitstream, effectively reducing the waste of computing power on backend devices.

[0034] Example 1:

[0035] Figure 3 A schematic diagram of a video analysis process provided in this application embodiment, the process including:

[0036] S301: Extract multiple first video frames from the video stream including the target object based on the currently saved frame extraction rate and the end position of the last extraction.

[0037] The video analysis method provided in this application is applied to an electronic device, which can be a backend device, such as an NVR or a Digital Video Recorder (DVR). This application uses an electronic device as an example of a backend device for illustration. The backend device supports multiple frontend device connections, meaning it connects to multiple frontend devices. These frontend devices can be image acquisition devices, such as cameras or Internet Protocol Cameras (IPCs). This application primarily describes the analysis process of the video stream from a single frontend device connected to the backend device. For scenarios with multiple frontend devices connected to the backend device, the analysis process for the video stream from each frontend device is similar and will not be repeated.

[0038] Most front-end devices currently support basic motion detection, which involves dividing video frames into different blocks. When the difference in pixel values ​​within a block reaches a motion detection threshold, a motion detection event is triggered, and the front-end device begins recording. However, in basic motion detection, environmental factors such as numerical values, wind, light and shadow, and insects can also cause the difference in pixel values ​​to reach the motion detection threshold, triggering a motion detection event and affecting the accuracy of the detection. Intelligent motion detection, on the other hand, is target-oriented and can perform low-power algorithm analysis on top of basic motion detection to identify moving object categories, such as people or vehicles. Therefore, to improve the accuracy of motion detection and to identify target objects, in this embodiment, the front-end device connected to the back-end device can support intelligent motion detection. Through intelligent motion detection, the front-end device can mark target information of target objects in the acquired video frames. Target information may include one or more of the following: target area coordinates, target identity document (ID) identifier, target type, etc. The video frames marked by the front-end device can be as follows: Figure 4 As shown, the labeled video frames include the target objects, the bounding box corresponding to each target object, and the target ID identifier corresponding to each target object, for example... Figure 4 The target ID identifiers include 108969, 108967, 108972, and 108973.

[0039] After performing intelligent motion detection on the video frames in the acquired video stream, the front-end device sends the intelligently motion-detected video stream (including tagged video frames) to the back-end device. Upon receiving the video stream from the front-end device, the back-end device filters the video stream to include the target object. For a given front-end device, its installation location and shooting angle are usually fixed; therefore, the field of view is the same for each video frame in the video stream acquired by the front-end device.

[0040] The backend device can store the frame extraction rate and the end position of the last extraction. The end position of the last extraction may include the last video frame extracted from the video stream and / or the capture time corresponding to that last video frame. The frame extraction rate is used to determine the number of frames to be extracted from the video stream containing the target object from the capture time of the last video frame corresponding to the end position of the last extraction to the current time. For the backend device, after acquiring the video stream containing the target object from the end position of the last extraction to the current time, it can determine the number of first video frames to be extracted based on the currently stored frame extraction rate and the end position of the last extraction, and extract multiple first video frames of that number from the video stream. Each first video frame may include one or more target objects, including but not limited to people and / or vehicles.

[0041] The aforementioned first video frame can be an unprocessed video frame containing the target object, or it can be a video frame obtained after processing a video frame containing the target object. For example, the process of processing a video frame containing the target object to obtain the first video frame can be described in the following ways: The first way includes: compressing the video frame containing the target object to determine the first video frame; the second way includes: extracting the target object from the video frame containing the target object and mapping it to grid coordinates to determine the first video frame; the third way includes: compressing the video frame containing the target object, extracting the target object included in the compressed video frame, and mapping all extracted target objects to grid coordinates to determine the first video frame.

[0042] The process of the third method described above is explained in detail below: Assuming the video frame resolution in the video stream containing the target object is W×H, for each video frame in the video stream, the video frame is compressed and split into an M×N grid. The target object is extracted from the compressed and split video frame, and then, combined with the target object's position in the video frame, the target object is compressed into grid coordinates to determine its position. The data on the corresponding grid is then updated to obtain the first video frame containing the target object. For example, the process of compressing the target object into grid coordinates and determining its position in grid coordinates can satisfy the following formula:

[0043]

[0044] Where (x1, y1) are the top-left corner coordinates of the target object in the video frame of the video stream, (x2, y2) are the bottom-right corner coordinates of the target object in the video frame of the video stream, (m1, n1) are the top-left corner coordinates of the target object in grid coordinates, and (m2, n2) are the bottom-right corner coordinates of the target object in grid coordinates. W×H is the resolution of the video frame, and M×N is the size of the video frame mapped to grid coordinates. For each pixel occupied by the target object in the video frame, the corresponding position of that pixel in grid coordinates can be determined according to the above formula. The above formula only shows the mapping process of the top-left and bottom-right corner coordinates of the target object; the mapping process of other pixels is not described here. Figure 5 To illustrate the mapping of the target object to grid coordinates, the target object with the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) in the video frame is extracted, and the extracted target object is mapped to grid coordinates to obtain the first video frame including the target object.

[0045] In this embodiment, the first video frame determined by the third method described above is used as an example for illustration.

[0046] S302: Divide each first video frame into multiple sub-regions and count the sub-regions in which the target object appears in each first video frame.

[0047] For each of the multiple first video frames, the backend device can divide the first video frame into multiple sub-regions in the same or different ways; each sub-region may include a set number of grids. This embodiment does not limit the value of the set number, for example, but not limited to, a set number of 1. The processing of multiple sub-regions divided into the same or different ways for each first video frame is similar. This embodiment mainly describes multiple sub-regions divided into the same way for each first video frame; multiple sub-regions divided in different ways will not be described in detail.

[0048] After dividing the first video frame into multiple sub-regions, for each sub-region in the first video frame, it is determined whether a target object appears in that sub-region. If so, the sub-region is determined as the sub-region where the target object appears, and the sub-regions where the target object appears in the first video frame are counted. Figure 6 The sub-regions where the target object appears in the first video frame are shown, where each grid represents a sub-region, and a black grid indicates that the target object appears in that sub-region.

[0049] In one possible implementation, the sub-regions where the target object appears in all first video frames can be determined by taking the union of the sub-regions in each first video frame. For example, suppose three first video frames are extracted, each containing four sub-regions. In the first first video frame, the target object appears in sub-regions 1 and 2; in the second first video frame, the target object appears in sub-region 1; and in the third first video frame, the target object appears in sub-regions 1, 2, and 3. Then, in these three first video frames, the sub-regions where the target object appears are determined to be sub-regions 1, 2, and 3. Therefore, the sub-regions in which the target object appears in each first video frame include sub-regions 1, 2, and 3.

[0050] S303: Analyze the sub-image of the sub-region where the target object appears in each first video frame.

[0051] In this step, based on the sub-region where the target object appears in each first video frame, the position of the target object in all first video frames is determined, and the position of the target object in all first video frames is determined as the sub-region where the target object appears in each first video frame; for each sub-region where the target object appears in each first video frame, the sub-image corresponding to the position of the sub-region in the first video frame can be extracted according to the position of the sub-region in the first video frame, and then the sub-image is analyzed.

[0052] For example, when analyzing a sub-image, the category of the target object included in the sub-image can be identified first. Based on the identified target object category, an algorithm for analyzing the sub-image is determined, and the algorithm is used to perform feature analysis on the sub-image to obtain and save the features of the target object included in the sub-image. Simultaneously, the location information corresponding to the sub-image and the target thumbnail corresponding to the sub-image are also saved. The location information corresponding to the sub-image can be a sub-region corresponding to the sub-image; the target thumbnail can be a sub-image captured from the first video frame based on the location information corresponding to the sub-image. For example, if the target object in the sub-image is identified as a person, a face recognition algorithm can be used to perform facial feature analysis on the target object to determine its features, such as the target object's Euler attitude angles (including pitch, yaw, roll, etc.) and attribute information (including expression, age, clothing color, whether wearing glasses, whether wearing a hat, etc.).

[0053] Furthermore, when performing feature analysis on sub-images, the computing power allocated to the sub-image by the backend device can also be used for analysis. Specifically, the size of the sub-image used for analysis in each first video frame can be determined, and the total computing power of the backend device can be allocated according to the proportion of the corresponding sub-image size in each first video frame. The computing power allocated by the backend device to the sub-image corresponding to each first video frame can be determined, and the computing power allocated by the backend device to the sub-image can be used to analyze the sub-image. Since the computing power of the backend device is reasonably scheduled and allocated according to the determined sub-image containing the target object in each first video frame in this embodiment, the situation of backend device chips running idle and wasting computing power is reduced in scenarios where the target is relatively discrete or appears infrequently in the front-end bitstream.

[0054] In this embodiment, by determining the sub-region in which the target object appears in each first video frame and analyzing and processing the sub-image corresponding to each sub-region, the waste of computing power of the back-end device can be reduced. This is especially effective in reducing the waste of computing power of the back-end device in scenarios where the target is relatively discrete or appears infrequently in the front-end bitstream.

[0055] Example 2:

[0056] Based on the above embodiments, in this embodiment of the application, the method further includes:

[0057] For each sub-image in any first video frame, the score corresponding to the sub-image in the first video frame is determined based on the features of the target object contained in the sub-image obtained by analyzing the sub-image and the pre-saved sub-scores corresponding to each feature.

[0058] For each of the multiple sub-regions, the score result corresponding to that sub-region is determined based on the score of the sub-image corresponding to that sub-region in each first video frame;

[0059] Based on the sub-regions whose scores are greater than the threshold, the corresponding target region image is determined in each first video frame, and the target region image in each first video frame is analyzed.

[0060] The backend device stores sub-scores for each feature of the target object. Sub-scores for different features can be the same or different. Taking a person as an example, when the target object's features include clothing color, whether it wears glasses, or whether it wears a hat—features that don't affect the shooting effect—the sub-scores for that feature can be the same regardless of whether the analyzed features are identical. For example, the sub-scores for yellow or green clothing colors are the same. However, when the target object's features include Euler pose angles, which affect the shooting effect, the sub-scores for that feature can be different depending on the shooting effect. For example, when the front-end device captures the target object's face directly, the sub-score for that feature can be higher; when the front-end device captures the top of the target object's head, the sub-score for that feature can be lower.

[0061] The backend device can select any one of the multiple first video frames for processing.

[0062] In one possible implementation, when determining the score corresponding to each sub-image in any first video frame, for each sub-image, the score corresponding to the sub-image in the first video frame can be determined by summing or weighting the sub-scores of each feature of the target object contained in the sub-image.

[0063] In another possible implementation, when determining the score corresponding to each sub-image in any first video frame, the score for each sub-image can be determined based on the sub-scores of each feature of the target object contained in the sub-image and the sub-score corresponding to the image sharpness of the sub-image. Specifically, the backend device can store scoring criteria, which include the sub-scores corresponding to image sharpness and the sub-scores corresponding to each of the aforementioned features. For each sub-image in any first video frame, the backend device can identify the sharpness of the target thumbnail corresponding to the stored sub-image, and determine the sharpness sub-score corresponding to the sub-image based on the determined sharpness of the target thumbnail and the stored sub-scores corresponding to image sharpness. Then, based on the sub-scores of the sharpness and the sub-scores of each feature, the sum or weighted sum of the sub-scores of the sharpness and the sub-scores of each feature is determined as the score corresponding to the sub-image in the first video frame.

[0064] After obtaining the score corresponding to each sub-image in any video frame, for each sub-region corresponding to the sub-image, the score of the sub-image corresponding to that sub-region in each first video frame can be determined. Then, based on the scores of the sub-image corresponding to that sub-region in each first video frame, the sum, weighted sum, or average of the scores of the sub-image corresponding to that sub-region in each first video frame can be determined as the score result of that sub-region. Figure 7 The scoring results corresponding to the sub-regions are shown, where each grid represents a sub-image corresponding to a sub-region, and the value in each grid represents the scoring result of the corresponding sub-region.

[0065] After obtaining the score for each sub-region, the score for each sub-region is compared with a threshold stored in the backend device. Sub-regions with scores greater than the threshold are identified. Target regions are then determined based on the sub-images corresponding to these sub-regions. Based on these target regions, a corresponding target region image is determined in each first video frame. The target region images in each first video frame are then analyzed to obtain the analysis results. The process of analyzing the target region images is similar to the process of analyzing each sub-image described above. The threshold stored in the backend device can be pre-set or it can be the average score of all sub-regions, with this average value stored as the threshold in the backend device.

[0066] After identifying sub-regions where the score exceeds the threshold, these sub-regions can be considered as the best-performing analysis points or regions. This allows for prioritizing the analysis of these best-performing points or regions in subsequent video analyses. This can be illustrated with specific scenarios, such as... Figure 8 The diagram illustrates the optimal analysis area. After identifying sub-regions where the score exceeds a threshold, the gate entrance is designated as the optimal analysis area. The gate entrance provides better capture of target objects, leading to more accurate analysis. Therefore, this optimal analysis area can be prioritized for subsequent video analysis. By adjusting the analysis area based on the feedback mechanism and score results, and designating areas with scores above the threshold as optimal, the diagram prioritizes areas with higher scores in future analyses. This provides a basis for scheduling and target prioritization, facilitating the optimization of subsequent backend device computing power scheduling strategies.

[0067] Because the sub-region scores in the target region image are higher than the threshold, the analysis effect on the target object is better. Therefore, when determining the sub-regions used for analyzing the target object, the sub-regions in the target region image obtained in the previous analysis can also be combined to improve the analysis effect on the target object in this analysis.

[0068] For example, when counting the sub-regions where the target object appears in each first video frame, the following method can be used: Based on the first target sub-region in the saved previous target region image and the second target sub-regions where the target object appears in each first video frame, determine the sub-region where the target object appears in each first video frame; wherein, the sub-region where the target object appears in each first video frame includes the first target sub-region and the second target sub-region. That is, when determining the sub-region where the target object appears in each first video frame, the obtained previous target region image can be used as a reference. The sub-region in the previous target region image can be used as the first target sub-region, and the sub-regions where the target object appears in each first video frame can be used as the second target sub-region. The union of the first and second target sub-regions can be used to determine the sub-region where the target object appears in each first video frame.

[0069] In this embodiment of the application, by referring to the historical scoring results of the sub-regions, regions with higher priority scores are selected for analysis, which can reduce the number of times different regions in the video frame are analyzed and improve the efficiency and effectiveness of the analysis.

[0070] Example 3:

[0071] To prevent missed detections, based on the above embodiments, in this embodiment, the currently saved frame extraction rate is determined according to the following method:

[0072] Based on the saved previous frame rate, obtain multiple second video frames contained in each of the preset multiple time periods; divide each second video frame into multiple sub-regions;

[0073] For each time period, based on the number of sub-regions in which the target object appears in each second video frame within that time period, determine the first number of sub-regions in which the target object appears in that time period and the first number of times the target object appears in each sub-region; based on the first number and the first number, determine the average frequency corresponding to that time period; based on the first number and the total number of sub-regions contained in the second video frame, determine the distribution ratio corresponding to that time period.

[0074] The current frame rate is determined based on at least one adjustment amount from the average frequency and distribution ratio corresponding to each time period.

[0075] In the previous frame extraction cycle, the backend device, based on its stored previous frame extraction frame rate, extracts a corresponding number of second video frames from the video stream containing the target object within each of the preset multiple time periods. For each of the extracted second video frames, the backend device can divide it into multiple sub-regions using the same or different partitioning methods. Typically, the backend device partitions each second video frame using the same method, which can be similar to the partitioning method used for the first video frame. The number of preset multiple time periods is a positive integer greater than 1; for example, the number of preset multiple time periods could be 3.

[0076] For example, each time period can be of the same length but with different corresponding times. For instance, each time period could be 16:00-16:05, 16:05-16:10, or 16:10-16:15. Alternatively, each time period can be of different lengths but with different corresponding times. For instance, each time period could be 16:00-16:01, 16:01-16:06, or 16:06-16:16. There are no specific restrictions here.

[0077] For each time period, the number of sub-regions in which the target object appears in each second video frame within that time period is counted to determine the total number of such sub-regions in each second video frame. Based on the number and location of these sub-regions, the total number of sub-regions in all second video frames where the target object appears can be determined by taking the union of the sub-regions in each first video frame. This total number is then defined as the first count of sub-regions in which the target object appears in that time period. For each sub-region where the target object appears, the total number of target objects in that sub-region across all second video frames is counted to determine the first count of the target object's appearance in that sub-region.

[0078] In one possible implementation, the first count and the first quantity can be determined as follows: For each time period, a heatmap of the target object is determined based on the sub-regions where the target object appears in multiple second video frames corresponding to that time period, and the number of times the target object appears in each sub-region. The values ​​on the sub-regions in the heatmap represent the number of times the target object appears in that sub-region, i.e., the first count of (all) target objects appearing in that sub-region, and the number and distribution of sub-regions in the heatmap are the same as in the second video frames. For each sub-region in the heatmap with a non-zero value, the first count of the target object appearing in that sub-region is determined based on the corresponding value in the heatmap; the first quantity of sub-regions where the target object appears is determined by counting the non-zero values ​​in each sub-region in the heatmap.

[0079] Specifically, the second video frame can be a video frame mapped to grid coordinates. Each grid in the second video frame is considered a sub-region, and the heat map is also mapped to grid coordinates, with each grid representing a sub-region. For each sub-region in the heat map, all second video frames are traversed. Whenever a target object appears in the corresponding sub-region of a second video frame, the value corresponding to that sub-region is updated. For example, an update can be performed by incrementing the value by 1. Once the values ​​corresponding to each sub-region in the heat map have been updated, the heat map is obtained. It is understandable that for sub-regions where no target object appears, the value corresponding to that sub-region may not be updated, or it may be set to 0. Figure 9 This is a schematic diagram of a hotspot distribution map, determined based on multiple second video frames within a certain time period. Each grid in the map represents a sub-region, and the numerical value corresponding to each sub-region indicates the number of target objects that appeared in that sub-region within that time period. Figure 9 Taking a sub-region with a value of 10 as an example, 10 means that the target object appeared 10 times in the sub-region during the time period. Therefore, for each sub-region, the first appearance number of the target object in the sub-region can be determined based on the value in the grid corresponding to the sub-region. The first number can be obtained by counting the number of sub-regions with values ​​in the hotspot distribution map (i.e., the sub-regions where the target object appears).

[0080] After obtaining the first occurrence count of the target object in each sub-region, for each time period, based on the first occurrence count of the target object in each sub-region within that time period, the first occurrence counts for each sub-region are summed to determine the total frequency of the target object's appearance within that time period. The ratio of the total frequency to the first occurrence count in the sub-regions where the target object appears within that time period is determined as the average frequency for that time period. For example, the average frequency for that time period satisfies the formula:

[0081] Let's take a hotspot distribution map as an example to illustrate, such as... Figure 10 The hotspot distribution map shown is for a specific time period. This hotspot distribution map consists of a 4×4 grid. Since each grid corresponds to a sub-region, and the value in each grid represents the number of times a target object has appeared in that sub-region, the number of grids in this hotspot distribution map where the target object appears is 5, meaning the first occurrence is 5. The values ​​corresponding to the grids for the target object in this hotspot distribution map are 10, 10, 20, 20, and 20 respectively. This means that the first occurrence of the target object in each sub-region during this time period is 10, 10, 20, 20, and 20 respectively. Adding these first occurrences of the target object in each sub-region, the total frequency of the target object during this time period is determined to be 80. Based on the total frequency and the first occurrence of the target object during this time period, the average frequency for this time period is determined to be... Where A represents the average frequency corresponding to that time period.

[0082] After obtaining the first number of sub-regions where the target object appears, for each time period, based on the first number of sub-regions where the target object appears in that time period and the total number of sub-regions contained in each second video frame, the ratio of the first number to the total number of contained sub-regions is determined as the distribution ratio corresponding to that time period. For example, the distribution ratio corresponding to this time period satisfies the formula: Because the same division method was used for each second video frame when dividing it into multiple sub-regions, the total number of sub-regions contained in each second video frame is the same.

[0083] return Figure 10 Taking a hotspot distribution map as an example, since this hotspot distribution map includes a 4×4 grid, and the number and distribution of sub-regions in the second video frame are the same as those in the hotspot distribution map, meaning the second video frame corresponding to this hotspot distribution map also includes a 4×4 grid, the total number of sub-regions contained in the second video frame is 16. Based on the first number of sub-regions where the target object appears within this time period and the total number of sub-regions contained in the second video frame, the distribution ratio corresponding to this time period is determined as follows: Where B represents the distribution proportion corresponding to that time period.

[0084] After obtaining the average frequency and distribution ratio for each time period, the average frequency and distribution ratio are used as adjustment values. Multiple preset time periods are sorted chronologically. Based on the change in at least one of the adjustment values ​​in the average frequency and distribution ratio for each time period after chronological sorting, the frame rate is adjusted. The adjusted frame rate is then used as the current frame rate and saved. For example, for each time period after chronological sorting, if the corresponding average frequency or distribution ratio increases sequentially, it indicates that the activity of the target object is increasing. The previous frame rate can be appropriately increased, and the current frame rate can be determined and saved to prevent missed detections. Conversely, if the corresponding average frequency or distribution ratio decreases sequentially, it indicates that the activity of the target object is decreasing. The previous frame rate can be appropriately decreased, and the current frame rate can be determined and saved to reduce the waste of computing power on the backend devices.

[0085] Taking the preset time periods of 16:00-16:01, 16:01-16:06, and 16:06-16:16 as examples, the average frequency and distribution ratio for each time period are statistically analyzed. For instance, the average frequency for the 16:00-16:01 time period is 16, with a distribution ratio of 0.3125; the average frequency for 16:01-16:06 is 20, with a distribution ratio of 0.4569; and the average frequency for 16:06-16:16 is 30, with a distribution ratio of 0.7865. After sorting each time period in chronological order, it is found that both the average frequency and the corresponding distribution ratio of the target object are increasing. This indicates that there are more and more active target objects in the monitoring scene captured by the front-end device, and the range of their activity is expanding. Therefore, the frame rate can be appropriately increased to determine the current frame rate and prevent missed detections. Similarly, when the average frequency and distribution ratio are gradually decreasing, the frame rate can be appropriately reduced to reduce the waste of computing power of the backend equipment.

[0086] In this embodiment, the activity of the target object is inferred based on the average frequency and distribution ratio corresponding to each time period. When it is determined that the activity of the target object increases, the frame rate is appropriately increased to prevent missed detections; when it is determined that the activity of the target object decreases, the frame rate is appropriately decreased to reduce the waste of computing power of the backend device.

[0087] Example 4:

[0088] Based on the above embodiments, in this embodiment, the currently saved frame rate is determined according to any one of the adjustment amounts from the average frequency and distribution ratio corresponding to each time period, including:

[0089] Based on the adjustment amount corresponding to each time period, determine whether the adjustment amounts corresponding to multiple time periods in chronological order should decrease or increase sequentially.

[0090] If the adjustment amount decreases sequentially across multiple time periods, the previous frame rate is reduced to obtain the current frame rate and saved; if the adjustment amount increases sequentially across multiple time periods, the previous frame rate is increased to obtain the current frame rate and saved.

[0091] For any adjustment value among average frequency or distribution ratio, multiple preset time periods are sorted chronologically. The adjustment values ​​for each sorted time period are compared to determine whether the adjustment values ​​for the multiple time periods decrease or increase sequentially according to the chronological order. For example, for the average frequency adjustment value, multiple preset time periods are sorted chronologically to determine whether the average frequency decreases or increases sequentially after sorting. Similarly, for the distribution ratio adjustment value, multiple preset time periods are sorted chronologically to determine whether the distribution ratio decreases or increases sequentially after sorting.

[0092] When it is determined that any adjustment value corresponding to multiple time periods decreases sequentially according to time sequence, that is, when it is determined that the average frequency or distribution ratio corresponding to multiple time periods decreases sequentially, the adjustment value corresponding to the previous frame rate can be reduced to determine the current frame rate and save the current frame rate in the backend device.

[0093] When it is determined that any adjustment value corresponding to multiple time periods increases sequentially in chronological order (i.e., when the average frequency or distribution ratio corresponding to multiple time periods increases sequentially), the adjustment value corresponding to the previous frame rate can be increased to determine the current frame rate, which is then stored in the backend device. There are no restrictions on the specific values ​​of the adjustment values ​​corresponding to decreasing and increasing the frame rate; these values ​​can be the same or different.

[0094] Understandably, if it's determined that the adjustment amount isn't decreasing or increasing sequentially, the overall change in the adjustment amount can be determined, i.e., whether the adjustment amount shows an overall decreasing trend or an overall increasing trend. When the adjustment amount shows an overall decreasing trend, the previous frame rate can be reduced, the current frame rate can be determined, and saved; when the adjustment amount shows an overall increasing trend, the previous frame rate can be increased, the current frame rate can be determined, and saved.

[0095] Example 5:

[0096] Based on the above embodiments, in this embodiment, the currently saved frame rate is determined according to the average frequency and distribution ratio corresponding to each time period, including:

[0097] Based on the average frequency corresponding to each time period, determine whether the average frequency of multiple time periods decreases or increases sequentially according to the chronological order.

[0098] Based on the distribution ratio corresponding to each time period, determine whether the distribution ratios of multiple time periods decrease or increase sequentially according to their chronological order.

[0099] If at least one of the adjustment values ​​in the average frequency and distribution ratio corresponding to multiple time periods decreases sequentially, the previous frame rate is reduced to obtain the current frame rate and saved; if at least one of the adjustment values ​​in the average frequency and distribution ratio corresponding to multiple time periods increases sequentially, the previous frame rate is increased to obtain the current frame rate and saved.

[0100] Based on the average frequency corresponding to each time period, multiple preset time periods are sorted in chronological order to determine whether the average frequency of the multiple time periods decreases or increases sequentially after being sorted in chronological order.

[0101] Based on the distribution ratio corresponding to each time period, multiple preset time periods are sorted in chronological order to determine whether the distribution ratios of the multiple time periods decrease or increase sequentially after being sorted in chronological order.

[0102] When at least one adjustment value among the average frequency and distribution ratio corresponding to multiple time periods is determined to decrease sequentially in chronological order, the corresponding adjustment value of the previous frame rate can be reduced to determine the current frame rate, which is then stored in the backend device. The situation where at least one adjustment value decreases sequentially in chronological order includes several scenarios: the average frequency among the adjustment values ​​corresponding to multiple time periods ordered in chronological order decreases sequentially; or, the distribution ratio among the adjustment values ​​corresponding to multiple time periods ordered in chronological order decreases sequentially; or, both the average frequency and distribution ratio among the adjustment values ​​corresponding to multiple time periods ordered in chronological order decrease sequentially.

[0103] When at least one adjustment value among the average frequency and distribution ratio corresponding to multiple time periods is determined to increase sequentially in chronological order, the previous frame rate can be increased by the corresponding adjustment value to determine the current frame rate, which is then stored in the backend device. The situation where at least one adjustment value increases sequentially in chronological order includes several scenarios: the average frequency among the adjustment values ​​corresponding to multiple time periods increases sequentially; or, the distribution ratio among the adjustment values ​​corresponding to multiple time periods increases sequentially; or, both the average frequency and distribution ratio among the adjustment values ​​corresponding to multiple time periods increase sequentially.

[0104] There are no restrictions on the specific values ​​of the adjustment values ​​corresponding to decreasing the frame rate and increasing the frame rate. The adjustment values ​​corresponding to decreasing the frame rate and increasing the frame rate can be the same or different.

[0105] Understandably, if it's determined that the average frequency or distribution ratio doesn't decrease or increase sequentially, the overall change in the average frequency or distribution ratio can be determined, i.e., whether the average frequency or distribution ratio shows an overall decreasing or increasing trend. When the average frequency or distribution ratio doesn't show an overall decreasing trend, the previous frame rate can be reduced, the current frame rate can be determined, and saved. Conversely, when the average frequency or distribution ratio doesn't show an overall increasing trend, the previous frame rate can be increased, the current frame rate can be determined, and saved.

[0106] Example 6:

[0107] Based on the above embodiments, in this embodiment, the current frame rate obtained by reducing the previous frame rate includes:

[0108] Determine the first target difference between the time period ranked first and the time period ranked last;

[0109] Based on the first target difference and the correspondence between the saved difference and the frame rate adjustment value, determine the first target frame rate adjustment value corresponding to the first target difference;

[0110] Based on the previous frame rate and the first target frame rate adjustment value, the previous frame rate is reduced.

[0111] When it is determined that the frame rate of the previous frame extraction should be reduced, a first target difference between the adjustment amounts corresponding to the first and last time periods is determined based on the absolute value of the difference between the adjustment amounts corresponding to the first time period and the last time period, which are sorted chronologically. The adjustment amount includes the average frequency and distribution ratio, and the first target difference includes the difference in average frequency and the difference in distribution ratio.

[0112] The backend device stores the correspondence between the average frequency difference, the distribution ratio difference, and the frame rate adjustment value. Based on the average frequency difference and the distribution ratio difference in the first target difference, as well as the stored correspondence between the average frequency difference, the distribution ratio difference, and the frame rate adjustment value, the first target frame rate adjustment value corresponding to the first target difference is determined.

[0113] For example, after determining the first target frame rate adjustment value, the previous frame rate is subtracted from the first target frame rate adjustment value based on the previous frame rate and the first target frame rate adjustment value, and the previous frame rate is reduced to determine and save the current frame rate.

[0114] Example 7:

[0115] Based on the above embodiments, in this embodiment, the current frame rate obtained by increasing the previous frame rate includes:

[0116] Determine the second target difference between the time period ranked first and the time period ranked last;

[0117] Based on the second target difference and the correspondence between the saved difference and the frame rate adjustment value, determine the second target frame rate adjustment value corresponding to the second target difference;

[0118] Based on the previous frame rate and the second target frame rate adjustment value, the previous frame rate is increased.

[0119] When it is determined that the frame rate of the previous frame extraction should be increased, a second target difference between the adjustment amounts corresponding to the first and last time periods is determined based on the absolute value of the difference between the adjustment amounts corresponding to the first time period and the last time period, which are sorted chronologically. The adjustment amounts include average frequency and distribution ratio, and the first target difference includes the difference in average frequency and the difference in distribution ratio.

[0120] The backend device stores the correspondence between the average frequency difference, the distribution ratio difference, and the frame rate adjustment value. Based on the average frequency difference and the distribution ratio difference in the second target difference, as well as the stored correspondence between the average frequency difference, the distribution ratio difference, and the frame rate adjustment value, the second target frame rate adjustment value corresponding to the second target difference is determined.

[0121] For example, after determining the second target frame rate adjustment value, the previous frame rate and the second target frame rate adjustment value are added together to increase the previous frame rate, and the current frame rate is determined and saved.

[0122] Example 8:

[0123] To make the backend device analysis more accurate, based on the above embodiments, in this embodiment, before counting the sub-regions where the target object appears in each first video frame, the following steps are included:

[0124] Determine whether the sub-region in which the target object appears in each first video frame is located within a preset region; wherein, the preset region includes a set safe region and / or occlusion region;

[0125] If not, continue counting the sub-regions where the target object appears in each first video frame.

[0126] The backend device stores a preset area. There is no need to count the target objects appearing in the preset area. Therefore, before counting the sub-area where the target object appears in each first video frame, it is determined whether the sub-area where the target object appears in each first video frame is located in the preset area. If it is, there is no need to count the target object. If not, the step of counting the sub-area where the target object appears in each first video frame continues.

[0127] For example, the preset area includes a set safe area and / or an obstruction area.

[0128] When the preset area includes a safe area, such as Figure 11 A schematic diagram of a safe zone is shown, with the area within the white box representing the safe zone. In scenarios involving monitoring abnormal traffic, such as illegal crossing of fences or jaywalking, safe zones can be established. For target objects appearing within a safe zone, their traffic behavior is regulated, eliminating the need for backend equipment to perform analysis and reducing the analytical burden on backend equipment when target object traffic is high. Therefore, in this embodiment, the step of determining whether the sub-region in which the target object appears in each first video frame is located within the established safe zone, and if not, continuing to count the sub-regions in which the target object appears in each first video frame, is included.

[0129] When the preset area includes an obstructed area, such as Figure 12 A schematic diagram of the occlusion area is shown, with the area within the white box representing the occlusion area. When a target object appears within the occlusion area, it is usually impossible to capture the object. Although in some cases a portion of the target object may be captured, the image quality is often poor due to the obstruction. Therefore, target objects within the occlusion area are typically not statistically analyzed. This makes the analysis results of the backend device more accurate and reliable, and also helps optimize the timing of target object capture, reducing the waste of computing power on the backend device. Therefore, in this embodiment, the step of determining whether the sub-region in which the target object appears in each first video frame is located within the set occlusion area is included. If not, the step of statistically analyzing the sub-region in which the target object appears in each first video frame continues.

[0130] When the preset area includes a safe area and an occlusion area, determine whether the sub-region in which the target object appears in each first video frame is located within the set safe area and occlusion area. If not, continue to count the sub-region in which the target object appears in each first video frame.

[0131] In this embodiment, based on the set security area, there is no need to analyze the security area; instead, the focus is on monitoring scenarios that may have anomalies, making the backend device's analysis more targeted and reducing the computing power of the backend device. Based on the set occlusion area, there is no need to analyze the occlusion area, making the backend device more accurate in analyzing, capturing, and selecting target objects.

[0132] The above embodiments are described below with a specific example. See [link to example]. Figure 13 The video analytics workflow diagram includes the following steps:

[0133] S1301: Obtain the video stream containing the target object.

[0134] The video stream is acquired in the front-end device. This video stream can be a video stream obtained after intelligent motion detection. In other words, the video frames in the video stream can be video frames marked with target objects.

[0135] S1302: Extract multiple video frames from the video stream based on the frame extraction rate.

[0136] S1303: Determine whether the target object is located in the set safe area. If yes, execute S1309; otherwise, execute S1304.

[0137] When the target object is located in the designated safe zone, the processing of the target object ends.

[0138] S1304: Determine whether the target object is located in the set occlusion area. If yes, execute S1309; otherwise, execute S1305.

[0139] When the target object is located within the set occlusion area, the processing of the target object ends.

[0140] S1305: For each of the preset multiple time periods, determine the hotspot distribution map based on the sub-region where the target object is located in multiple video frames, and update the data corresponding to each sub-region in the hotspot distribution map.

[0141] In the hotspot distribution map, the data corresponding to each sub-region represents the frequency of the target object appearing in that sub-region.

[0142] S1306: Based on the frequency statistics of sub-regions appearing in the hotspot distribution map, determine the corresponding sub-image in the video frame and perform analysis.

[0143] S1307: Based on the scoring results determined after analyzing the sub-images, sub-images with scoring results higher than the threshold are identified as target region images.

[0144] In the next analysis, sub-regions within the target region image can be selected for analysis.

[0145] S1308: Adjust the frame rate based on the hotspot distribution map corresponding to different time periods, determine the next frame rate, and save it.

[0146] The next frame rate is used to extract the corresponding number of video frames from the video stream containing the target object during the next analysis.

[0147] S1309: Processing terminated.

[0148] Example 9:

[0149] Based on the same technical concept and the above embodiments, this application provides a video analysis device. Figure 14 This is a schematic diagram of a video analysis device provided in an embodiment of this application, as shown below. Figure 14 As shown, the device includes:

[0150] The extraction module 1401 is used to extract multiple first video frames from the video stream including the target object based on the currently saved extraction frame rate and the end position of the last extraction.

[0151] The statistics module 1402 is used to divide each first video frame into multiple sub-regions and count the sub-regions in which the target object appears in each first video frame.

[0152] Analysis module 1403 is used to analyze the sub-image of the sub-region where the target object appears in each first video frame.

[0153] In one possible implementation, the analysis module 1403 is further configured to, for each sub-image in any first video frame, determine the score corresponding to the sub-image in the first video frame based on the features of the target object contained in the sub-image obtained by analyzing the sub-image and the pre-saved sub-score corresponding to each feature; for each sub-region in multiple sub-regions, determine the score result corresponding to the sub-region based on the score of the sub-image corresponding to the sub-region in each first video frame; and determine the corresponding target region image in each first video frame based on the sub-region whose score result is greater than a threshold, and analyze the target region image in each first video frame.

[0154] In one possible implementation, the extraction module 1401 is specifically configured to: acquire multiple second video frames contained in each of a preset multiple time periods based on the saved previous extraction frame rate; divide each second video frame into multiple sub-regions; for each time period, determine a first number of sub-regions in which the target object appears and a first count of the target object in each sub-region based on the number of sub-regions in which the target object appears in each second video frame in that time period; determine the average frequency corresponding to that time period based on the first count and the first number; determine the distribution ratio corresponding to that time period based on the first number and the total number of sub-regions contained in the second video frame; and determine the currently saved extraction frame rate based on at least one adjustment amount in the average frequency and distribution ratio corresponding to each time period.

[0155] In one possible implementation, the extraction module 1401 is specifically configured to: determine whether the average frequency of multiple time periods decreases or increases sequentially according to the average frequency corresponding to each time period; determine whether the distribution ratio of multiple time periods decreases or increases sequentially according to the distribution ratio corresponding to each time period; if at least one adjustment amount among the average frequency and distribution ratio of multiple time periods decreases sequentially, the previous extraction frame rate is reduced to obtain the current extraction frame rate and saved; if at least one adjustment amount among the average frequency and distribution ratio of multiple time periods increases sequentially, the previous extraction frame rate is increased to obtain the current extraction frame rate and saved.

[0156] In one possible implementation, the extraction module 1401 is specifically used to determine the first target difference between the time period ranked first and the time period ranked last; determine the first target frame rate adjustment value corresponding to the first target difference based on the first target difference and the correspondence between the saved difference and the frame rate adjustment value; and reduce the frame rate of the previous extraction frame based on the previous extraction frame rate and the first target frame rate adjustment value.

[0157] In one possible implementation, the extraction module 1401 is specifically used to determine the second target difference between the time period ranked first and the time period ranked last; determine the second target frame rate adjustment value corresponding to the second target difference based on the second target difference and the correspondence between the saved difference and the frame rate adjustment value; and increase the frame rate of the previous extraction frame based on the previous extraction frame rate and the second target frame rate adjustment value.

[0158] In one possible implementation, the statistics module 1402 is specifically used to determine whether the sub-region in which the target object appears in each first video frame is located within a preset area before counting the sub-regions in which the target object appears in each first video frame; wherein, the preset area includes a set safe area and / or occlusion area; if not, the step of counting the sub-regions in which the target object appears in each first video frame continues.

[0159] Example 10:

[0160] Based on the same technical concept, this application also provides an electronic device. Figure 15 This application provides a schematic diagram of an electronic device structure, such as... Figure 15 As shown, it includes: processor 1501, communication interface 1502, memory 1503 and communication bus 1504, wherein processor 1501, communication interface 1502 and memory 1503 communicate with each other through communication bus 1504.

[0161] The memory 1503 stores a computer program that, when executed by the processor 1501, causes the processor 1501 to implement any of the above embodiments.

[0162] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0163] Communication interface 1502 is used for communication between the above-mentioned electronic device and other devices.

[0164] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0165] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0166] Example 11:

[0167] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, it causes the electronic device to implement any of the above embodiments.

[0168] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A video analysis method, characterized in that, The method includes: Based on the currently saved frame extraction rate and the end position of the last extraction, extract multiple first video frames from the video stream including the target object. Each first video frame is divided into multiple sub-regions, and the sub-regions in which the target object appears in each first video frame are counted; the sub-images in the sub-regions in which the target object appears in each first video frame are analyzed; The currently saved frame rate is determined according to the following method: Based on the saved previous frame rate, obtain multiple second video frames contained in each of the preset multiple time periods; divide each second video frame into multiple sub-regions; For each time period, based on the number of sub-regions of the target object appearing in each second video frame within that time period, a first number of sub-regions of the target object appearing in that time period and the first number of times the target object appears in each sub-region are determined; based on the first number and the first quantity, the average frequency corresponding to that time period is determined; based on the first quantity and the total number of sub-regions contained in each second video frame, the distribution ratio corresponding to that time period is determined; The currently saved frame rate is determined based on at least one adjustment amount from the average frequency and distribution ratio corresponding to each time period.

2. The method according to claim 1, characterized in that, The method further includes: For each sub-image in any first video frame, the score corresponding to the sub-image in the first video frame is determined based on the features of the target object contained in the sub-image obtained by analyzing the sub-image and the pre-saved sub-scores corresponding to each feature. For each of the plurality of sub-regions, the score result corresponding to the sub-region is determined based on the score of the sub-image corresponding to the sub-region in each first video frame; Based on the sub-regions whose scores are greater than a threshold, a corresponding target region image is determined in each first video frame, and the target region image in each first video frame is analyzed.

3. The method according to claim 1, characterized in that, The currently saved frame rate is determined based on the average frequency and distribution ratio corresponding to each time period, including: Based on the average frequency corresponding to each time period, determine whether the average frequency corresponding to the multiple time periods in chronological order decreases or increases sequentially. Based on the distribution ratio corresponding to each time period, determine whether the distribution ratios of the multiple time periods in chronological order decrease or increase sequentially. If at least one of the adjustment amounts in the average frequency and distribution ratio corresponding to the multiple time periods decreases sequentially, the previous frame rate is reduced to obtain the current frame rate and saved; if at least one of the adjustment amounts in the average frequency and distribution ratio corresponding to the multiple time periods increases sequentially, the previous frame rate is increased to obtain the current frame rate and saved.

4. The method according to claim 3, characterized in that, The process of reducing the previous frame rate to obtain the current frame rate includes: Determine the first target difference between the time period ranked first and the time period ranked last; Based on the first target difference and the correspondence between the saved difference and the frame rate adjustment value, the first target frame rate adjustment value corresponding to the first target difference is determined; Based on the previous frame rate and the first target frame rate adjustment value, the previous frame rate is reduced.

5. The method according to claim 3, characterized in that, The process of increasing the previous frame rate to obtain the current frame rate includes: Determine the second target difference between the time period ranked first and the time period ranked last; Based on the second target difference and the correspondence between the saved difference and the frame rate adjustment value, determine the second target frame rate adjustment value corresponding to the second target difference; Based on the previous frame rate and the second target frame rate adjustment value, the previous frame rate is increased.

6. The method according to claim 1, characterized in that, The step of counting the sub-regions in which the target object appears in each first video frame includes: Determine whether the sub-region in which the target object appears in each first video frame is located within a preset region; wherein, the preset region includes a set safe region and / or an occlusion region; If not, continue with the step of counting the sub-regions where the target object appears in each first video frame.

7. A video analysis device, characterized in that, The device includes: The extraction module is used to extract multiple first video frames from the video stream including the target object based on the currently saved extraction frame rate and the end position of the last extraction. The statistics module is used to divide each first video frame into multiple sub-regions and count the sub-regions in which the target object appears in each first video frame; The analysis module is used to analyze the sub-images of the sub-regions in which the target object appears in each first video frame; Specifically, the extraction module is used to: acquire multiple second video frames contained in each of a preset multiple time periods based on the saved previous extraction frame rate; divide each second video frame into multiple sub-regions; for each time period, determine a first number of sub-regions containing the target object and the first occurrence of the target object in each sub-region based on the number of sub-regions containing the target object in each second video frame within that time period; determine the average frequency corresponding to that time period based on the first occurrence and the first number; determine the distribution ratio corresponding to that time period based on the first number and the total number of sub-regions contained in each second video frame; and determine the currently saved extraction frame rate based on at least one adjustment amount among the average frequency and distribution ratio corresponding to each time period.

8. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the steps of the video analysis method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the video analysis method according to any one of claims 1-6.

Images