Monitoring processing method, device, computer equipment, storage medium and program product

Abstract

The present application relates to a monitoring processing method, device, computer equipment, storage medium and program product. The method includes: obtaining cached videos of multiple monitoring points in a monitoring area, determining at least one segment of interest in the cached videos of multiple monitoring points, filtering and processing the at least one segment of interest, and generating an event video of the target object; the target object exists in each video frame in the segment of interest, and the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and filtering according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame. In the above method, an event video showing the event that the user is concerned about is obtained from a large amount of video, which improves the effectiveness and simplicity of the event video.

Description

Technical Field

[0001] The present application relates to the field of monitoring and processing technology, and in particular to a monitoring and processing method, device, computer equipment, storage medium and program product. Background Art

[0002] With the development of science and technology, monitoring equipment is increasingly used in people's lives and production.

[0003] Monitoring equipment usually needs to communicate with cameras deployed at different monitoring points to obtain and display monitoring videos collected at different monitoring points. In related technologies, monitoring equipment collects monitoring videos from different monitoring points on a large screen for synchronous or cyclic display.

[0004] However, the related art cannot accurately and completely display the surveillance video of the object of interest. Summary of the invention

[0005] Based on this, it is necessary to provide a monitoring and processing method, device, computer equipment, medium and program product to address the above technical problems.

[0006] In a first aspect, an embodiment of the present application provides a monitoring processing method, the method comprising:

[0007] Obtain cached videos of multiple monitoring points in the monitoring area;

[0008] In the cached videos of multiple monitoring points, at least one segment of interest is determined; each video frame in the segment of interest has a target object;

[0009] At least one segment of interest is screened to generate an event video of the target object; the event video is used to record the entire event from the appearance to the disappearance of the target object in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0010] In an embodiment of the present application, cached videos of multiple monitoring points in a monitoring area are obtained, and at least one segment of interest is determined in the cached videos of the multiple monitoring points, and the at least one segment of interest is screened to generate an event video of the target object; the target object exists in each video frame in the segment of interest; the event video is used to record the whole process event from the appearance to the disappearance of the target object in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the result of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame. In the above method, by determining the segment of interest in which the target object exists in all video frames in the cached videos of multiple monitoring points, and further screening the segment of interest to generate an event video that records the whole process event from the appearance to the disappearance of the target object in the monitoring area, it is achieved that an event video that can accurately and completely display the event of interest to the user is obtained from a large amount of video, thereby improving the effectiveness and simplicity of the obtained event video.

[0011] In one of the embodiments, determining at least one interesting segment in cached videos of multiple monitoring points includes:

[0012] Inputting the received indication text of the target object and the video frames in each cached video into the language image multimodal model to detect the existence status of the target object in each video frame;

[0013] The video frames in each cached video that are continuous in time sequence and contain the target object are divided into an interesting segment to obtain at least one interesting segment.

[0014] In an embodiment of the present application, the received indication text of the target object and the video frames in each cached video are input into the language image multimodal model, and the existence state of the target object in each video frame is detected, so as to divide the video frames in each cached video that are time-sequentially continuous and contain the target object into an interesting segment, and obtain at least one interesting segment. In the above method, the target object is detected on the video frame based on the language image multimodal model, so that the detection is not limited to a single object. The user can input the indication text of the target object to be detected according to the actual scene requirements, so as to flexibly adjust the target object to be detected, thereby improving the flexibility of target detection.

[0015] In one embodiment, the method further comprises:

[0016] Sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative texts of the reference objects in each sample image are used as training samples to train a language image multimodal model.

[0017] In the embodiment of the present application, sample images of different types of reference objects in the historical cached videos of the monitoring area and the indicative text of the reference objects in each sample image are used as training samples to train a language image multimodal model. In the above method, the historical cached videos of the monitoring area are used to determine the training samples, and the language image multimodal model is trained to detect the newly acquired cached videos in the monitoring area, which improves the adaptability between the language multimodal model and the monitoring scene, and correspondingly improves the accuracy of the detection results.

[0018] In one embodiment, sample images of different types of reference objects in the historical cached video of the monitoring area and the indicative text of the reference object in each sample image are used as training samples to train a language image multimodal model, including:

[0019] Inputting each instruction text and sample image into an initial multimodal model, and extracting a text feature vector of each instruction text and an image feature vector of each sample image by the initial multimodal model;

[0020] The vector similarity between each text feature vector and each image feature vector is obtained, and the model parameters in the initial multimodal model are adjusted to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, thereby obtaining a language-image multimodal model.

[0021] In an embodiment of the present application, each indicative text and sample image is input into an initial multimodal model, and the text feature vector of each indicative text and the image feature vector of each sample image are extracted by the initial multimodal model, and the vector similarity between each text feature vector and each image feature vector is obtained, and the model parameters in the initial multimodal model are adjusted to maximize the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, and a language image multimodal model is obtained. Each group of training samples includes multiple sample images and indicative texts of the same reference object. In the above method, the training of the language image multimodal model is realized, and the target object can be detected based on the indicative text of the target object, so that the detection is not limited to a single object, and the flexibility of target detection is improved.

[0022] In one embodiment, screening and processing are performed on at least one segment of interest to generate an event video of a target object, including:

[0023] The video frames at the same acquisition time in all the segments of interest are divided into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time;

[0024] Selecting at least one spare video frame from each set of candidate video frames;

[0025] An event video of the target object is generated based on the spare video frames and the remaining video frames in all the segments of interest.

[0026] In an embodiment of the present application, video frames at the same acquisition time in all the segments of interest are divided into a candidate video frame set, at least one spare video frame is selected from each candidate video frame set, and an event video of the target object is generated based on each spare video frame and the remaining video frames in all the segments of interest; the candidate video frame set includes at least two video frames at the same acquisition time. In the above method, at least one spare video is selected from each candidate video frame set to streamline multiple video frames at the same acquisition time, reduce data redundancy, and thus improve the simplicity of the obtained event video.

[0027] In one of the embodiments, screening at least one spare video frame from each candidate video frame set includes:

[0028] For each candidate video frame set, obtaining a picture quality order of each video frame in the candidate video frame set;

[0029] According to the picture quality order, at least one video frame whose order meets a preset requirement is determined in the candidate video frame set as a spare video frame.

[0030] In the embodiment of the present application, for each candidate video frame set, the picture quality order of each video frame in the candidate video frame set is obtained, and according to the picture quality order, at least one video frame whose order meets the preset requirements is determined in the candidate video frame set as a spare video frame. In the above method, based on the picture quality order of each video frame, the candidate video frame set is screened to obtain spare video frames whose order meets the preset requirements, which improves the picture quality of the screened spare video frames, and can correspondingly improve the picture quality of the temporal video generated based on the spare video frames.

[0031] In one embodiment, obtaining the picture quality order of each video frame in the candidate video frame set includes:

[0032] Obtaining at least one picture parameter of a target object in each video frame in the candidate video frame set;

[0033] According to various picture parameters, the video frames are sorted from high to low in parameter value to obtain the video frame arrangement order corresponding to each picture parameter;

[0034] The picture quality order of each video frame is determined according to the video frame arrangement order corresponding to each picture parameter.

[0035] In the embodiment of the present application, at least one picture parameter of a target object in each video frame in the candidate video frame set is obtained, and the parameter values ​​of each video frame are sorted from high to low according to each picture parameter, so as to obtain the video frame arrangement order corresponding to each picture parameter, and determine the picture quality order of each video frame according to the video frame arrangement order corresponding to each picture parameter. In the above method, the final picture quality order is determined based on the video frame arrangement order corresponding to each picture parameter, taking into account the influence of multiple picture parameters on the picture quality of the video frame, improving the accuracy of the video frame quality assessment, and thus improving the accuracy and reliability of the obtained picture quality order.

[0036] In one embodiment, the picture parameters include the number, confidence and volume of the target object; and according to the arrangement order of the video frames corresponding to the picture parameters, determining the picture quality order of each video frame includes:

[0037] For each video frame in the candidate video frame set, determine the quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine the confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine the volume sequence number in the video frame arrangement sequence corresponding to the volume;

[0038] Obtain the total value of the number sequence number, confidence sequence number and volume sequence number of the video frame;

[0039] The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

[0040] In the embodiment of the present application, the picture parameters include the number, confidence and volume of the target object. For each video frame in the candidate video frame set, the number sequence number is determined in the video frame arrangement order corresponding to the number, the confidence sequence number is determined in the video frame arrangement order corresponding to the confidence, and the volume sequence number is determined in the video frame arrangement order corresponding to the volume, so as to obtain the total sequence number of the number sequence number, the confidence sequence number and the volume sequence number of each video frame, and the total sequence number of each video frame is sorted from small to large to obtain the picture quality order of each video frame. In the above method, in the case of multiple picture parameters included, the sequence number of each video frame in the video frame arrangement order of the corresponding picture parameter is determined respectively to obtain the total value of the sequence number corresponding to each picture parameter, and the picture quality order is determined based on the total sequence number of each video frame. The picture quality order is determined by using simple sequence number calculation and sorting to determine the picture quality order, which improves the convenience of quality sorting, reduces the sorting time, and correspondingly improves the efficiency of determining the picture quality order.

[0041] In one embodiment, generating an event video of a target object according to each spare video frame and remaining video frames in all interested segments includes:

[0042] Divide the video frames corresponding to the same monitoring point and having a continuous time sequence among all the spare video frames and the remaining video frames into a segment to be processed;

[0043] Generate event videos based on the segments to be processed.

[0044] In the embodiment of the present application, among all spare video frames and remaining video frames, the video frames corresponding to the same monitoring point and having a continuous time sequence are divided into a to-be-processed segment, so as to generate an event video according to each to-be-processed segment. In the above method, the scattered spare video frames and remaining video frames are divided into at least one to-be-processed segment according to the monitoring point and time sequence, so as to realize the integration, and then regenerate the event video according to each to-be-processed segment, thereby simplifying the generation of event videos and improving the generation efficiency.

[0045] In one embodiment, generating an event video according to each to-be-processed segment includes:

[0046] When the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen pictures, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed in different picture areas of the split-screen picture to obtain the event video.

[0047] In the embodiment of the present application, when the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen images, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed through different screen areas of the split-screen images to obtain the event video. In the above method, the video frames at the same acquisition time are synchronously displayed through different screen areas of the split-screen images to synchronously display the target objects from different perspectives, thereby improving the comprehensiveness of the obtained event video.

[0048] In one embodiment, generating an event video according to each to-be-processed segment includes:

[0049] When the number of video frames at the same acquisition time in all the segments to be processed at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority;

[0050] Extracting substitute video frames at the same acquisition time from the cached video of the substitute monitoring point;

[0051] The substitute video frames and the video frames in each to-be-processed segment are sorted in chronological order, and the substitute video frames at the same acquisition moment and the video frames corresponding to the same acquisition moment in each to-be-processed segment are synchronously displayed in different screen areas of the split-screen image to obtain the event video.

[0052] In an embodiment of the present application, when the number of video frames at the same acquisition time in all the to-be-processed segments at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority, and a substitute video frame at the same acquisition time is extracted from the cached video of the substitute monitoring point, so that the substitute video frame and the video frames in each to-be-processed segment are sequentially sorted in chronological order, and the substitute video frame at the same acquisition time and each video frame corresponding to the same acquisition time in each to-be-processed segment are synchronously displayed through different screen areas of the split-screen image to obtain an event video. In the above method, in order to cope with the situation where the number of spare video frames obtained by screening is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the monitoring point priority, so as to determine the corresponding substitute video frame to complete the screen area of ​​the split-screen image, thereby improving the signal richness of the obtained event video and the uniformity of the screen display between each video frame.

[0053] In one embodiment, generating an event video according to each to-be-processed segment further includes:

[0054] Determine the initial segment with the earliest timing among all the segments to be processed;

[0055] Update the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment;

[0056] An event video is generated based on the updated initial segment and the remaining segments to be processed.

[0057] In the embodiment of the present application, the earliest initial segment in the sequence is determined among all the segments to be processed, and the initial segment is updated according to the cached video of the initial monitoring point, and the event video is generated according to the updated initial segment and the remaining segments to be processed; wherein the initial monitoring point is the monitoring point corresponding to the initial segment. In the above method, the cached video based on the initial monitoring point realizes the update of the initial segment, which can improve the reliability of the initial segment, and correspondingly improve the reliability of the event video generated based on the initial segment.

[0058] In one embodiment, updating the initial segment according to the cached video of the initial monitoring point includes:

[0059] In the cached video of the initial monitoring point, determine a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame;

[0060] The target cache fragment and the initial fragment are spliced ​​in time sequence to obtain an updated initial fragment.

[0061] In the embodiment of the present application, in the cached video of the initial monitoring point, a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame is determined, and the target cached segment is spliced ​​with the initial segment in time sequence to obtain an updated initial segment. In the above method, the initial segment is expanded by splicing the target cached segment with the initial segment in time sequence, and a part of the video segment before the target object appears in the monitoring area is added to the initial segment, so that the event video obtained based on the updated initial segment can include the scene where the target object appears for the first time in the monitoring, thereby improving the integrity of the event recorded in the event video.

[0062] In one embodiment, the method further comprises:

[0063] When a target object is detected, an alarm prompt is triggered that the target object appears in the monitoring area; and / or the event video is sent to an external terminal.

[0064] In the embodiment of the present application, when the target object is detected, an alarm prompt of the target object appearing in the monitoring area is triggered; and / or the event video is sent to an external terminal. In the above method, an alarm prompt of the target object appearing in the monitoring area is triggered, or the event video is sent to an external terminal, thereby realizing monitoring feedback and improving monitoring timeliness.

[0065] In a second aspect, an embodiment of the present application further provides a monitoring processing device, the device comprising:

[0066] A cache acquisition module is used to acquire cached videos of multiple monitoring points in the monitoring area;

[0067] A segment determination module is used to determine at least one segment of interest in the cached videos of multiple monitoring points; each video frame in the segment of interest has a target object;

[0068] The screening and processing module is used to screen and process at least one segment of interest to generate an event video of the target object; the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0069] In a third aspect, an embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in the monitoring processing method provided in any one of the embodiments of the first aspect are implemented.

[0070] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the monitoring processing method provided in any one of the embodiments of the first aspect above are implemented.

[0071] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps in the monitoring processing method provided in any one of the embodiments of the first aspect above.

[0072] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of an application scenario of a monitoring processing method in an embodiment;

[0074] Figure 2 A schematic diagram of a monitoring process in one embodiment;

[0075] Figure 3 A schematic diagram of a process for determining a fragment of interest in one embodiment;

[0076] Figure 4 A schematic diagram of a process for generating an event video in one embodiment;

[0077] Figure 5 A schematic diagram of a process for generating an event video in another embodiment;

[0078] Figure 6 A schematic diagram of a process of selecting spare video frames in one embodiment;

[0079] Figure 7 A schematic diagram of a flow chart for determining a picture quality order in one embodiment;

[0080] Figure 8 A schematic diagram of a flow chart for determining a picture quality order in another embodiment;

[0081] Fig. 9 A schematic diagram of a process for generating an event video in another embodiment;

[0082] Fig.10 A schematic diagram of a process for generating an event video in another embodiment;

[0083] Fig.11 A schematic diagram of a video frame in an event video in an embodiment;

[0084] Fig.12 A schematic diagram of a process for generating an event video in another embodiment;

[0085] Fig.13 A schematic diagram of a process of updating an initial segment in one embodiment;

[0086] Fig.14 A schematic flow chart of a monitoring processing method in another embodiment;

[0087] Fig.15 A structural block diagram of a monitoring processing device in an embodiment;

[0088] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0090] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the term "including" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0091] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0092] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, 3 and / or 4 may represent: 3 exists alone, 3 and 4 exist at the same time, and 4 exists alone. In the description of the embodiments of the present application, the term "plurality" refers to more than two (including two), unless otherwise clearly and specifically defined.

[0093] Monitoring equipment usually needs to communicate with cameras installed at different monitoring points to obtain and display monitoring videos collected at different monitoring points.

[0094] In the related art, monitoring equipment can usually display the monitoring videos of different monitoring points on a large screen synchronously or in a loop. The on-duty personnel need to pay attention to multiple monitoring screens on the large screen and observe the abnormal conditions in each monitoring screen with the naked eye to take corresponding measures. This kind of monitoring equipment cannot accurately and completely display the monitoring videos of the objects that need to be paid attention to, and the personnel cannot pay attention to all monitoring screens at the same time, so they cannot find abnormal conditions in time, which not only wastes manpower but also has poor monitoring effects.

[0095] To this end, the integrated camera in the related technology can automatically monitor whether there are moving objects in the target area and track the moving objects. However, in order to avoid the loss of object tracking due to occlusion by objects in the environment, the coverage is often expanded by adding cameras to reduce the probability of object tracking loss. Then, the increase in cameras leads to too many monitoring screens for people to pay attention to, and it cannot meet the growing demand for monitoring scenes.

[0096] Therefore, the related technology cannot accurately and completely display the surveillance video of the object of concern, which not only reduces the surveillance effect, but also fails to meet the growing demand for surveillance scenarios.

[0097] Based on this, the present application provides a monitoring processing method, which determines that interesting segments of the target object exist in all video frames in the cached videos of multiple monitoring points, and further filters and processes the interesting segments to generate an event video that records the entire process of the target object from appearance to disappearance in the monitoring area, so as to obtain a monitoring video that can accurately and completely display the object of concern from massive videos, thereby improving the effectiveness and conciseness of the obtained event video.

[0098] The monitoring processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, multiple cameras 102 communicate with the monitoring host 104 to realize the linkage of multiple cameras 102. Multiple cameras 102 are arranged at multiple monitoring points in the monitoring area. Each camera 102 is used to collect video data in the monitoring area, and transmit the collected real-time video stream data to the monitoring host 104. The monitoring host 104 receives the real-time video stream data transmitted by each camera 102 accordingly, and caches the real-time video stream data to obtain cached videos of multiple monitoring points. The monitoring host 104 determines that there is at least one interesting segment of the target object in the cached videos of multiple monitoring points, and then screens and processes the at least one interesting segment to generate an event video that records the entire process of the target object from appearance to disappearance in the monitoring area.

[0099] It should be noted that the monitoring processing method provided in the embodiments of the present application can be used in a variety of monitoring scenarios. For example, it can be applied to employee management, such as monitoring employees' violations such as not coming to work on time, sleeping on duty, and leaving their jobs. It can also be applied to dangerous situation monitoring, such as emergencies such as fires, floods, and gas leaks.

[0100] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the computer device to which the embodiment of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0101] In one embodiment, the present application provides a monitoring processing method, which can extract event videos recording events of interest to users from massive video data, such as Figure 2 As shown, the above method is applied to Figure 1 Taking the monitoring host in as an example, the method includes the following steps:

[0102] S210: Obtain cached videos of multiple monitoring points in the monitoring area.

[0103] The monitoring point is the location where the camera is set, and the cached videos of multiple monitoring points are the video data collected by the cameras at the corresponding monitoring points. For example, each monitoring point can be set with one camera, or multiple cameras, such as three cameras facing different directions.

[0104] Optionally, multiple cameras are set at multiple monitoring points in the monitoring area, each camera is used to collect video data of the covered area, and transmit the collected real-time video stream data to the monitoring host. The monitoring host receives the real-time video stream data transmitted by each camera in response, and caches the real-time video stream data to obtain cached videos of multiple monitoring points.

[0105] Exemplarily, the video frames in the cached video may be any one of a grayscale image, an RGB image, an infrared image, or a depth image, or a combination of at least two of them.

[0106] S220: Determine at least one segment of interest in the cached videos of the multiple monitoring points; each video frame in the segment of interest has a target object.

[0107] The target object is a pre-set object of interest or concern. Optionally, the target object may be an object, such as a person, an animal, a vehicle, etc., or an event object, such as a fire, etc.

[0108] Optionally, after the monitoring host obtains the cached videos of multiple monitoring points, it can detect the existence status of the target object in each video frame, that is, whether the target object is included, for the cached videos of each monitoring point, and take the video frames in which the target object exists and are time-continuous as a segment of interest, so as to determine at least one segment of interest from the cached videos of multiple monitoring points.

[0109] For example, the monitoring host is equipped with a target detection model for identifying the target object. The monitoring host can input each video frame in the cached video into the target detection model to determine the existence state of the target object in each video frame to obtain the video frame where the target object exists, thereby obtaining the interesting segment. This can greatly improve the detection efficiency and accuracy, and also reduce the probability of target object tracking loss.

[0110] S230, screening at least one segment of interest to generate an event video of the target object; the event video is used to record the entire event process from the appearance to the disappearance of the target object in the monitoring area; the screening process includes sorting the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0111] In actual applications, a large number of cameras are set up in the monitoring area. The amount of cached video data obtained by the monitoring host is very large, and the amount of data of the interested segments determined therefrom is also very large. Therefore, the multiple interested segments determined can be screened and processed to obtain a streamlined event video of the target object.

[0112] Optionally, the filtering process can be performed on the segments of interest, such as the monitoring host can filter out some segments of interest from multiple segments of interest to generate event videos of the target object based on the filtered out segments of interest; the filtering process can also be performed on the video frames in the segments of interest, such as the monitoring host can filter out some video frames from multiple segments of interest to generate event videos of the target object based on the filtered out some video frames.

[0113] Exemplarily, the monitoring host can filter out an interesting segment in which the target object appears earliest, an interesting segment in which the target object appears last, and at least one interesting segment in which the target object appears earliest and last from multiple interesting segments, and obtain the screen display parameters related to the target object in each video frame for the video frames at the same acquisition time in the at least three interesting segments, and sort the picture quality of multiple video frames at the same acquisition time according to the screen display parameters, retain one video frame with the best picture quality at the same acquisition time, that is, the video frame with the highest picture quality ranking, and then sort all the processed video frames in time sequence to obtain the event video of the target object. The monitoring host can directly obtain the screen display parameters related to the target object in each video frame for the video frames at the same acquisition time in all interesting segments, and sort the picture quality of multiple video frames at the same acquisition time according to the screen display parameters, retain one video frame with the best picture quality at the same acquisition time, that is, the video frame with the highest picture quality ranking, and then sort all the processed video frames in time sequence to obtain the event video of the target object.

[0114] In an embodiment of the present application, cached videos of multiple monitoring points in a monitoring area are obtained, and at least one segment of interest is determined in the cached videos of the multiple monitoring points, and the at least one segment of interest is screened to generate an event video of the target object; the target object exists in each video frame in the segment of interest; the event video is used to record the whole process event from the appearance to the disappearance of the target object in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the result of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame. In the above method, by determining the segment of interest in which the target object exists in all video frames in the cached videos of multiple monitoring points, and further screening the segment of interest to generate an event video that records the whole process event from the appearance to the disappearance of the target object in the monitoring area, it is achieved that an event video that can accurately and completely display the event of interest to the user is obtained from a large amount of video, thereby improving the effectiveness and simplicity of the obtained event video.

[0115] To improve the flexibility of target object detection, in one embodiment, as Figure 3 As shown, the above S220, determining at least one interesting segment in the cached videos of multiple monitoring points, includes:

[0116] S310: Input the received indication text of the target object and the video frames in each cached video into the language image multimodal model to detect the existence status of the target object in each video frame.

[0117] The target object indication text is the text information describing the target object input by the user to the monitoring host. For example, the target object indication text may be "a person wearing a hat", "a truck with a blue license plate", etc. The language image multimodal model is a detection model that combines object detection and phrase grounding. It can recognize and detect the object described by the input text in the image based on the understanding of the input text.

[0118] Optionally, the monitoring host may receive an indication text of a target object input by a user, and input the indication text and video frames in each cached video into a pre-trained language image multimodal model, which detects whether the target object indicated by the indication text exists in each video frame.

[0119] Among them, the language image multimodal model includes two functional modules, namely a semantic recognition module and a target detection module. The semantic recognition module can perform semantic recognition on the input instruction text to determine the target object indicated by the instruction text, and inform the target detection module to detect whether there is a target object indicated by the instruction text in the video frame.

[0120] S320: Divide the video frames in each cached video that are continuous in time sequence and contain the target object into an interesting segment, and obtain at least one interesting segment.

[0121] Optionally, after the monitoring host obtains the existence status of the target object in each video frame based on the language image multimodal model detection, it can divide the video frames that are continuous in time sequence and contain the target object into an interesting segment for each cached video, thereby obtaining at least one interesting segment in the cached videos of multiple monitoring points.

[0122] In an embodiment of the present application, the received indication text of the target object and the video frames in each cached video are input into the language image multimodal model, and the existence state of the target object in each video frame is detected, so as to divide the video frames in each cached video that are time-sequentially continuous and contain the target object into an interesting segment, and obtain at least one interesting segment. In the above method, the target object is detected on the video frame based on the language image multimodal model, so that the detection is not limited to a single object. The user can input the indication text of the target object to be detected according to the actual scene requirements, so as to flexibly adjust the target object to be detected, thereby improving the flexibility of target detection.

[0123] In order to improve the adaptability between the language multimodal model and the monitoring scenario, in one embodiment, the method further includes:

[0124] Sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative texts of the reference objects in each sample image are used as training samples to train a language image multimodal model.

[0125] The sample images with different types of reference objects are video frames with reference objects in the historical cached video. The historical cached video can be a cached video of any monitoring point in the monitoring area, a cached video of one monitoring point, or a cached video of multiple monitoring points. There is at least one type of reference object in each sample image. The reference object indication text is used to indicate the reference object.

[0126] Exemplarily, different types of reference objects may include vehicles, people, animals, etc. They may also be refined into people not wearing work clothes, blue trucks, etc.

[0127] Optionally, the monitoring host may pre-receive video frames containing different types of reference objects obtained by the user based on historical cached videos of the monitoring area as sample images, and receive indication text of the reference objects in each sample image input by the user, so as to train an initial multimodal model using each sample image and each indication text as training samples, thereby training a language image multimodal model.

[0128] In the embodiment of the present application, sample images of different types of reference objects in the historical cached videos of the monitoring area and the indicative text of the reference objects in each sample image are used as training samples to train a language image multimodal model. In the above method, the historical cached videos of the monitoring area are used to determine the training samples, and the language image multimodal model is trained to detect the newly acquired cached videos in the monitoring area, which improves the adaptability between the language multimodal model and the monitoring scene, and correspondingly improves the accuracy of the detection results.

[0129] Next, the training process of the language-image multimodal model is introduced. In one embodiment, Figure 4 As shown, the above-mentioned sample images of different types of reference objects in the historical cached video of the monitoring area and the indicative text of the reference object in each sample image are used as training samples to train a language image multimodal model, including:

[0130] S410: Input each instruction text and sample image into an initial multimodal model, and extract a text feature vector of each instruction text and an image feature vector of each sample image through the initial multimodal model.

[0131] Optionally, the initial multimodal model includes a text encoder (Text Encoder) and an image encoder (ImageEncoder). The monitoring host can input each indication text and sample image into the initial multimodal model, and the text encoder maps the indication text to a feature space to obtain a text feature vector of each indication text, and the image encoder maps the sample image to the same feature space to obtain an image feature vector of each sample image.

[0132] In practical applications, the monitoring host can pre-divide all sample images and instruction texts into multiple groups of training samples, each group of training samples includes multiple sample images and corresponding instruction texts, and input a group of training samples for training each time, thereby realizing multiple batches of model training. For example, the user selects 1000 sample images and corresponding 1000 instruction texts from the historical cache video, and the monitoring host can divide these 100 sample images and corresponding 100 instruction texts into 50 groups of training samples in groups of 20, each group of training samples includes 20 sample images and corresponding 20 instruction texts.

[0133] S420. Obtain the vector similarity between each text feature vector and each image feature vector, and adjust the model parameters in the initial multimodal model to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, thereby obtaining a language-image multimodal model.

[0134] The vector similarity between the text feature vector and the image feature vector can be represented by the vector inner product of the two. The training cutoff condition can be that all training samples have completed training, or that the model loss function has converged, or that the vector similarity between the text feature vector and the image feature vector of the same reference object is the largest among the vector similarities obtained from several consecutive training samples. At least one of the conditions can be satisfied.

[0135] Optionally, after obtaining the text feature vector and the image feature vector, the monitoring host can obtain the vector inner product between each text feature vector and each image feature vector as the vector similarity, and after obtaining the vector similarities of each training sample in the current group, the model parameters in the initial multimodal model are adjusted, and the next group of training samples are input, so that the vector similarity between the text feature vector and the image feature vector of the same reference object in the next group of training samples increases. During the trial adjustment process, if the vector similarity of the same reference object in the next group of training samples output by the model decreases after the parameter adjustment, the monitoring host readjusts the model parameters, such as adjusting in the opposite direction of the previous adjustment, until the obtained vector similarity increases. The initial multimodal model is trained in this way until the training cutoff condition is reached, and the initial multimodal model after the last parameter adjustment is used as the language image multimodal model.

[0136] It should be noted that when the language image multimodal model is applied, the video frame detected with the target object is the video frame in which the vector similarity between the image feature vector and the text feature vector of the indicative text of the target object is the largest and greater than a threshold.

[0137] In an embodiment of the present application, all sample images and indicative texts are divided into multiple groups of training samples. For each group of training samples, each indicative text and sample image is input into an initial multimodal model. The text feature vector of each indicative text and the image feature vector of each sample image are extracted by the initial multimodal model, and the vector similarity between each text feature vector and each image feature vector is obtained. The model parameters in the initial multimodal model are adjusted to maximize the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, and a language image multimodal model is obtained. Each group of training samples includes multiple sample images and indicative texts of the same reference object. In the above method, the training of the language image multimodal model is realized, and the target object can be detected based on the indicative text of the target object, so that the detection is not limited to a single object, and the flexibility of target detection is improved.

[0138] There are video frames with the same acquisition time in multiple segments of interest. Based on this, in one embodiment, Figure 5 As shown, the above S230 performs screening processing on at least one interesting segment to generate an event video of the target object, including:

[0139] S510: Divide the video frames at the same acquisition time in all the segments of interest into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time.

[0140] Among them, one candidate video frame set corresponds to one acquisition time, and different candidate video frame sets correspond to different acquisition times.

[0141] Optionally, after the monitoring host obtains at least one segment of interest from multiple cached videos, it can divide the video frames with the same acquisition time into the same set according to the acquisition time of each video frame in all the segments of interest, and obtain a candidate video frame set including at least two video frames with the same acquisition time, thereby obtaining at least one candidate video frame set among all the segments of interest.

[0142] S520: Filter at least one spare video frame from each candidate video frame set.

[0143] Optionally, for each candidate video frame set, the monitoring host may filter at least one video frame in the candidate video frame set according to a preset filtering condition as a spare video frame to obtain a spare video frame corresponding to each candidate video frame set.

[0144] Exemplarily, the preset filtering conditions include filtering at least one video frame with the best picture quality / meeting the requirements in the candidate video frame set as a backup video frame; the preset filtering conditions may also include determining a target video frame whose timing is before and continuous with the corresponding acquisition time of the candidate video frame set in all the segments of interest, and selecting a video frame with a different monitoring point from the target video frame in the candidate video frame set as a backup video frame.

[0145] S530: Generate an event video of the target object according to the spare video frames and the remaining video frames in all the interested segments that are not classified into the candidate video frame set.

[0146] Optionally, when one spare video frame is selected from each candidate video frame set, the monitoring host may sequentially sort the spare video frames and the remaining video frames in all the interested segments that are not divided into the candidate video frame set in time sequence to generate the event video of the target object. When less than one spare video frame is selected from each candidate video frame set, the monitoring host may sequentially sort the spare video frames and all the remaining video frames in time sequence, and for multiple spare video frames at the same acquisition time, the multiple spare video frames may be synchronously thumbnailed in the same video frame to generate the event video of the target object.

[0147] In an embodiment of the present application, all video frames at the same acquisition time in all segments of interest are divided into a candidate video frame set, at least one spare video frame is selected from each candidate video frame set, and an event video of the target object is generated based on each spare video frame and the remaining video frames in all segments of interest that are not divided into the candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time. In the above method, at least one spare video is selected from each candidate video frame set to streamline multiple video frames at the same acquisition time, reduce data redundancy, and thus improve the simplicity of the obtained event video.

[0148] To improve the picture quality of the spare video frame, in one embodiment, Figure 6 As shown, the above S520, selecting at least one spare video frame from each candidate video frame set, includes:

[0149] S610 . For each candidate video frame set, obtain a picture quality order of each video frame in the candidate video frame set.

[0150] Optionally, for each candidate video frame set, the monitoring host may determine the picture quality of each video frame in the candidate video frame set, and sort all the video frames in the candidate video frame set from high to low in picture quality according to the picture quality of each video frame, to obtain the picture quality order of each video frame in the candidate video frame set.

[0151] It should be noted that the picture quality of the video frame can also be determined based on the resolution of the video frame. The higher the resolution, the higher the picture quality of the video frame. For each candidate video frame set, the monitoring host can obtain the resolution of each video frame in the candidate video frame set to sort the video frames according to the resolution and obtain the picture quality order of each video frame in the candidate video frame set.

[0152] S620: Determine, according to the picture quality order, at least one video frame in the candidate video frame set whose order meets a preset requirement as a spare video frame.

[0153] Optionally, after obtaining the picture quality order of each video frame in the candidate video frame set, the monitoring host selects at least one video frame whose order meets the preset requirements from the candidate video frame set as a backup video frame according to the picture quality order. Exemplarily, when the picture quality is sorted from high to low, for each candidate video frame set, the monitoring host can determine the video frame with the first picture quality order as the backup video frame of the corresponding candidate video frame set.

[0154] In the embodiment of the present application, for each candidate video frame set, the picture quality order of each video frame in the candidate video frame set is obtained, and according to the picture quality order, at least one video frame whose order meets the preset requirements is determined in the candidate video frame set as a spare video frame. In the above method, based on the picture quality order of each video frame, the candidate video frame set is screened to obtain spare video frames whose order meets the preset requirements, which improves the picture quality of the screened spare video frames, and can correspondingly improve the picture quality of the temporal video generated based on the spare video frames.

[0155] The picture quality can be determined based on the picture parameters of the target object in the video frame. Figure 7 As shown, the step of obtaining the picture quality order of each video frame in the candidate video frame set in S610 includes:

[0156] S710: Obtain at least one picture parameter of a target object in each video frame in the candidate video frame set.

[0157] The target object's picture parameter represents a picture display parameter related to the target object in the video frame, which can be used to evaluate the picture quality of the video frame. Exemplarily, the target object's picture parameter may include the number, probability value (ie, confidence), area or volume of the target object in the video frame.

[0158] Optionally, the monitoring host may obtain at least one screen display parameter related to the target object in each video frame in the candidate video frame set as the screen parameter of the target object. Exemplarily, the monitoring host may obtain the area of ​​the target object in the video frame as the screen parameter of the target object, or may obtain the number, confidence, and volume of the target object in the video frame as the screen parameter of the target object.

[0159] S720 , sort the video frames from high to low according to the various picture parameters, to obtain the video frame arrangement sequence corresponding to the various picture parameters.

[0160] Optionally, the monitoring host can sort the video frames in the candidate video frame set according to the parameter values ​​of each picture parameter from high to low, so as to obtain the arrangement order of the video frames corresponding to different picture parameters. Among them, the more the number of target objects in the video frame, the higher the confidence, the larger the area or the larger the volume, the higher the ranking.

[0161] Exemplarily, at least one image parameter of the target object includes the number, confidence and volume of the target object. The monitoring host can sort the video frames in the candidate video frame set according to the number, confidence and volume of the target objects, and obtain the video frame arrangement order corresponding to the number, confidence and volume of the target objects as shown in the following table.

[0162]

[0163] As shown in the above table, the candidate video frame set A includes video frames A1-A8. Video frames A1-A8 are sorted from most to least according to the number of target objects in the video frames to obtain a set of video frame arrangement orders of corresponding numbers. Video frames A1-A8 are sorted from high to low according to the confidence of the target objects in the video frames to obtain a set of video frame arrangement orders of corresponding confidences. Video frames A1-A8 are sorted from large to small according to the volume of the target objects in the video frames to obtain a set of video frame arrangement orders of corresponding volumes.

[0164] S730 , determining a picture quality order of each video frame according to the video frame arrangement order corresponding to each picture parameter.

[0165] Optionally, after obtaining the video frame arrangement order corresponding to each picture parameter, the monitoring host determines the picture quality order of each video frame according to the video frame arrangement order corresponding to each picture parameter. The more the number of target objects in the video frame, the higher the confidence, the larger the area or the larger the volume, the higher the picture quality of the video frame, and the higher the ranking.

[0166] For example, when one picture parameter is included, the monitoring host directly uses the video arrangement order corresponding to the picture parameter as the picture quality order of each video frame in the candidate video frame set. When multiple picture parameters are included, the monitoring host determines the picture quality order of each video frame by comprehensively considering the video frame arrangement order corresponding to each picture parameter, such as determining the picture quality order of each video frame according to the priority of each parameter and the video frame arrangement order corresponding to each picture parameter. Continuing with the example in the above table, the monitoring host can obtain the first video frames in the order of the number, confidence and volume of the video frames, which are A6, A8 and A7 respectively, and sort them in the order of priority confidence > volume > number to obtain A8-A7-A6; then obtain the second video frames in the order of the number, confidence and volume of the video frames, which are A2, A5 and A2 respectively, and sort them in the order of priority confidence > volume > number to obtain A2-A5; then obtain the third video frames in the order of the number, confidence and volume of the video frames, which are A8, A6 and A6 respectively, A8 and A6 have been sorted before, and do not need to be sorted again this time; in this way, until all the video frames are sorted, the sorted splicing A8-A7-A6- A2-A5-… is obtained as the picture quality order of each video frame.

[0167] In the embodiment of the present application, at least one picture parameter of a target object in each video frame in the candidate video frame set is obtained, and each video frame is sorted according to each picture parameter to obtain the video frame arrangement order corresponding to each picture parameter, so as to determine the picture quality order of each video frame according to the video frame arrangement order corresponding to each picture parameter. In the above method, the final picture quality order is determined based on the video frame arrangement order corresponding to each picture parameter, taking into account the influence of multiple picture parameters on the picture quality of the video frame, improving the accuracy of the video frame quality assessment, and thus improving the accuracy and reliability of the obtained picture quality order.

[0168] In the case where the picture parameters include the number, confidence and volume of target objects, such as Figure 8 As shown, the above S730, according to the video frame arrangement order corresponding to each picture parameter, determines the picture quality order of each video frame, including:

[0169] S810. For each video frame in the candidate video frame set, determine a quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine a confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine a volume sequence number in the video frame arrangement sequence corresponding to the volume.

[0170] During the above-mentioned processing, the monitoring host sorts the video frames in the candidate video frame set according to the number of target objects existing in the video frames to obtain the video frame arrangement order corresponding to the number, sorts the video frames in the candidate video frame set according to the confidence of the target objects in the video frames to obtain the video frame arrangement order corresponding to the confidence, and sorts the video frames in the candidate video frame set according to the volume of the target objects in the video frames to obtain the video frame arrangement order corresponding to the volume.

[0171] Optionally, after obtaining the video frame arrangement sequences corresponding to the quantity, confidence, and volume of the target object, for each video frame in the candidate video frame set, the monitoring host can obtain the sequence number of the video frame in the video frame arrangement sequence corresponding to the quantity, that is, the quantity sequence number, obtain the sequence number of the video frame in the video frame arrangement sequence corresponding to the confidence, that is, the confidence sequence number, and obtain the sequence number of the video frame in the video frame arrangement sequence corresponding to the volume, that is, the volume sequence number. For example, as shown in the table above, for video frame A1, the corresponding quantity sequence number is 4, the confidence sequence number is 5, and the volume sequence number is 4; for video frame A2, the corresponding quantity sequence number is 2, the confidence sequence number is 4, and the volume sequence number is 2.

[0172] S820, obtaining the total value of the quantity serial number, confidence serial number and volume serial number of each video frame.

[0173] Optionally, after obtaining the quantity sequence number, confidence sequence number and volume sequence number of each video frame in the candidate video frame set, the monitoring host adds the quantity sequence number, confidence sequence number and volume sequence number of the video frame to obtain the total sequence number value corresponding to the video frame. For example, as shown in the above table, for video frame A1, the total sequence number value of the quantity sequence number, confidence sequence number and volume sequence number is 4+5+4=13; for video frame A2, the total sequence number value of the quantity sequence number, confidence sequence number and volume sequence number is 2+4+2=8.

[0174] S830, sort the total values ​​of the sequence numbers of the video frames from small to large to obtain the picture quality order of the video frames.

[0175] Optionally, after obtaining the total serial number value of each video frame in the candidate video frame set, the monitoring host can sort the total serial number values ​​of each video frame from small to large, and use the sorted order as the picture quality order of each video frame. For example, as shown in the above table, the candidate video frame set A includes video frames A1-A8, and the total serial number values ​​of video frames A1-A8 are 13, 8, 19, 21, 15, 7, 16, 9, respectively. Then, all the total serial number values ​​are sorted from small to large to obtain the picture quality order of each video frame in the candidate video frame set. In the candidate video frame set A, the serial number of video frame A1 in the picture quality sorting is 1, which is the first place, indicating that its picture quality is the highest. The serial number of video frame A4 in the picture quality sorting is 8, which is the last place, indicating that its picture quality is the worst.

[0176] In an embodiment of the present application, the picture parameters include the number, confidence and volume of the target object. For each video frame in the candidate video frame set, the number sequence number is determined in the video frame arrangement order corresponding to the number, the confidence sequence number is determined in the video frame arrangement order corresponding to the confidence, and the volume sequence number is determined in the video frame arrangement order corresponding to the volume, so as to obtain the total sequence number of the number sequence number, the confidence sequence number and the volume sequence number of each video frame, and the total sequence number of each video frame is sorted from small to large to obtain the picture quality order of each video frame. In the above method, in the case of multiple picture parameters included, the sequence number of each video frame in the video frame arrangement order of the corresponding picture parameter is determined respectively to obtain the total value of the sequence number corresponding to each picture parameter, and the picture quality order is determined based on the total sequence number of each video frame. The picture quality order is determined by using simple sequence number calculation and sorting to determine the picture quality order, thereby improving the convenience of quality sorting, reducing sorting time, and correspondingly improving the efficiency of determining the picture quality order. In order to simplify the generation process of event videos, in one of the embodiments, such as Fig. 9 As shown, the above S530, generating an event video of the target object according to each spare video frame and the remaining video frames in all the interested segments, includes:

[0177] S910: Divide the video frames corresponding to the same monitoring point and having a continuous time sequence among all the spare video frames and the remaining video frames into a segment to be processed.

[0178] Optionally, after the monitoring host obtains the backup video frames of each candidate video frame set, it can group all the backup video frames and the remaining video frames that correspond to the same monitoring point and are time-continuous together as a fragment to be processed, so as to divide all the backup video frames and the remaining video frames into at least one fragment to be processed.

[0179] S920: Generate an event video according to each segment to be processed.

[0180] Optionally, when a candidate video frame set corresponds to one backup video frame, after the monitoring host obtains each segment to be processed, it can splice each segment to be processed in time sequence to generate an event video of the target object; when a candidate video frame set corresponds to multiple backup video frames, after the monitoring host obtains each segment to be processed, it can splice each segment to be processed in time sequence, and synchronously display thumbnails of multiple backup video frames corresponding to the same acquisition moment in the same video frame, so as to generate an event video of the target object.

[0181] In the embodiment of the present application, the video frames corresponding to the same monitoring point and continuous in time sequence in all spare video frames and remaining video frames are divided into a to-be-processed segment, so as to generate an event video according to each to-be-processed segment; each to-be-processed segment includes video frames with the same monitoring point and continuous in time sequence. In the above method, the scattered spare video frames and remaining video frames are divided into at least one to-be-processed segment according to the monitoring point and time sequence, so as to realize the integration, so as to regenerate the event video according to each to-be-processed segment, thereby simplifying the generation of event videos and improving the generation efficiency.

[0182] In order to synchronously display the video frames at the same acquisition time, in one embodiment, the above S920, generating an event video according to each to-be-processed segment, includes:

[0183] When the number of video frames at the same acquisition time is equal to the number of split-screen images at any time, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed in different screen areas of the split-screen images to obtain the event video.

[0184] The number of split screen images is preset and equal to the number of spare video frames to be screened in the candidate video frame set. For example, the preset number of split screen images is 4, and the first four video frames in the candidate video frame set are determined as spare video frames.

[0185] The split screen image includes image areas in the number of split screen images. One image area in the split screen image is used to display one video frame, and multiple image areas can synchronously display multiple video frames at the same acquisition time.

[0186] In the actual screening process, the number of spare video frames screened out from the candidate video frame set is not necessarily equal to the required number, that is, the number of spare video frames screened out is not necessarily equal to the number of split-screen images. For example, the preset number of split-screen images is 4, and the candidate video frame set includes only 2 video frames, and accordingly only 2 spare video frames can be screened out; and when the candidate video frame set includes ≥ 4 video frames, 4 spare video frames can be screened out.

[0187] Optionally, in all the segments to be processed, the monitoring host obtains the number of video frames at the same acquisition time at any time, and compares the number of video frames at the same acquisition time with the preset number of split-screen images, so as to generate an event video according to the comparison result. Wherein, in the case where the number of video frames at the same acquisition time is equal to the number of split-screen images, the monitoring host sorts the video frames in all the segments to be processed in chronological order, and synchronously displays the video frames at the same acquisition time through each screen area of ​​the split-screen image, so as to synchronously display multiple video frames at the same acquisition time through the split-screen image, thereby obtaining an event video in which the video frames in all the segments to be processed are sorted in chronological order, and multiple video frames at the same acquisition time are synchronously displayed through the split-screen image.

[0188] In the embodiment of the present application, when the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen images, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed through different screen areas of the split-screen images to obtain the event video. In the above method, the video frames at the same acquisition time are synchronously displayed through different screen areas of the split-screen images to synchronously display the target objects from different perspectives, thereby improving the comprehensiveness of the obtained event video.

[0189] In the case where the number of the selected spare video frames is less than the number of split-screen images, in one embodiment, Fig.10 As shown, the above S920, generating an event video according to each to-be-processed segment, includes:

[0190] S1010: When the number of video frames at the same acquisition time in all the segments to be processed at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to a preset monitoring point priority.

[0191] Among them, the number of video frames at the same acquisition time in all the segments to be processed is less than the number of split-screen pictures, indicating that the number of spare video frames obtained by screening is less than the number of split-screen pictures, and some picture areas in the corresponding split-screen pictures have no video frames that can be displayed. Exemplarily, the number of preset split-screen pictures is 4, and the split-screen pictures correspondingly include 4 picture areas. When 2 spare video frames are screened out from the candidate video frame set, there are 2 picture areas in the split-screen pictures that can display the 2 video frames at the same acquisition time, i.e., the 2 spare video frames, but there are no video frames that can be displayed in the other 2 picture areas in the split-screen pictures.

[0192] Optionally, when the number of video frames at the same acquisition time in all the segments to be processed at any time is less than the number of split-screen images, the monitoring host determines the required number of substitute monitoring points from all monitoring points according to the preset monitoring point priority. The required number is the difference between the number of video frames at the same acquisition time and the number of split-screen images, that is, the number of screen areas that need to be filled to display video frames. Exemplarily, continuing with the above example, the preset number of split-screen images is 4, and the number of video frames at the same acquisition time is 2. The monitoring host determines the top 2 priority monitoring points from all monitoring points as substitute monitoring points according to the preset monitoring point priority.

[0193] It should be noted that the backup monitoring point is different from the monitoring point of each video frame of the same acquisition. For example, the monitoring area includes 5 monitoring points S1-S5, and the number of video frames at the same acquisition time is 2, corresponding to monitoring points S1 and S3 respectively. The preset monitoring point priorities are monitoring points S1, S2, S3, S4 and S5 in descending order. When the required number is 2, the monitoring host can determine S2 and S4 as backup monitoring points.

[0194] S1020: extract substitute video frames at the same acquisition time from the cached videos of the substitute monitoring points.

[0195] Optionally, after determining the substitute monitoring point, the monitoring host extracts the video frame with the same acquisition time from the cached video of the substitute monitoring point as the substitute video frame. Exemplarily, continuing the above example, the number of video frames with the same acquisition time T1 in all the segments to be processed is less than the number of split-screen images, and S2 and S4 are determined as substitute monitoring points. The monitoring host extracts the video frame with the same acquisition time T1 from the cached video frames of the substitute monitoring points S2 and S4, respectively, as the substitute video frame, and the two substitute monitoring points correspondingly obtain two substitute video frames with the same acquisition time T1.

[0196] S1030, sorting the substitute video frames and the video frames in each to-be-processed segment in chronological order, and synchronously displaying the substitute video frames at the same acquisition moment and the video frames corresponding to the same acquisition moment in each to-be-processed segment in different picture areas of the split-screen picture to obtain the event video.

[0197] Optionally, the monitoring host sorts all the substitute video frames and the video frames in each segment to be processed in chronological order, and synchronously displays the substitute video frames at the same acquisition moment and the video frames corresponding to the same acquisition moment in the segment to be processed through each screen area of ​​the split-screen image, so as to synchronously display multiple video frames at the same acquisition moment through the split-screen image, thereby obtaining an event video in which all the substitute video frames and the video frames in all the segments to be processed are sorted in chronological order, and multiple video frames at the same acquisition moment are synchronously displayed through the split-screen image.

[0198] For example, Fig.11 As shown, it is a video frame at a certain acquisition time T in the obtained event video, in which 4 video frames at the same acquisition time T are synchronously displayed through a 2*2 split screen. The upper left screen area and the upper right screen area in the split screen display two video frames at the same acquisition time T in each to-be-processed segment (with target objects), and the lower left screen area and the lower right screen area display two video frames at the same acquisition time T of two substitute monitoring points (without target objects). Each screen area can also simultaneously display the device information of the camera to which the displayed video frame belongs. The device information may include the camera code, and may also include the camera setting position or monitoring position.

[0199] In an embodiment of the present application, when the number of video frames at the same acquisition time in all the to-be-processed segments at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority, and a substitute video frame at the same acquisition time is extracted from the cached video of the substitute monitoring point, so that the substitute video frame and the video frames in each to-be-processed segment are sequentially sorted in chronological order, and the substitute video frame at the same acquisition time and each video frame corresponding to the same acquisition time in each to-be-processed segment are synchronously displayed through different screen areas of the split-screen image to obtain an event video. In the above method, in order to cope with the situation where the number of spare video frames obtained by screening is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the monitoring point priority, so as to determine the corresponding substitute video frame to complete the screen area of ​​the split-screen image, thereby improving the signal richness of the obtained event video and the uniformity of the screen display between each video frame.

[0200] To improve the integrity of the event video, in one embodiment, Fig.12 As shown, the above S920, generating an event video according to each to-be-processed segment, further includes:

[0201] S1210: Determine an initial segment with the earliest timing among all segments to be processed.

[0202] The earliest initial segment in time sequence is the segment to be processed that includes the video frame at the earliest acquisition time among all the segments to be processed.

[0203] Optionally, after obtaining all the segments to be processed, the monitoring host can select the segment to be processed that includes the video frame at the earliest acquisition time among all the segments to be processed as the earliest initial segment in the time sequence. The earliest initial segment in the time sequence can be one or more.

[0204] S1220, updating the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment.

[0205] Optionally, after obtaining the initial segment, the monitoring host can determine that the monitoring point corresponding to the initial segment is the initial monitoring point, and update the corresponding initial segment according to the cached video of the initial monitoring point to obtain an updated initial segment. Exemplarily, the earliest initial segment in time sequence includes initial segment 1 and initial segment 2, initial segment 1 corresponds to initial monitoring point S1, and initial segment 2 corresponds to initial monitoring point S2. The monitoring host can update initial segment 1 based on the cached video of initial monitoring point S1 to obtain an updated initial segment, and update initial segment 2 according to the cached video of initial monitoring point S2 to obtain an updated initial segment 2.

[0206] S1230: Generate an event video according to the updated initial segment and the remaining segments to be processed.

[0207] The remaining to-be-processed segments are the other to-be-processed segments except the initial segment among all the to-be-processed segments obtained above.

[0208] Optionally, after obtaining the updated initial segment, the monitoring host may generate an event video based on the updated initial segment and the remaining segments to be processed.

[0209] Exemplarily, when the number of video frames at the same acquisition time in all the to-be-processed segments at any time is equal to the number of split-screen images, the video frames in all the to-be-processed segments are sequentially sorted in time sequence, and the video frames at the same acquisition time are synchronously displayed through different screen areas of the split-screen images to obtain the event video; when the number of video frames at the same acquisition time in all the to-be-processed segments at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority; a substitute video frame at the same acquisition time is extracted from the cached video of the substitute monitoring point; the substitute video frame and the video frames in each to-be-processed segment are sequentially sorted in time sequence, and the substitute video frame at the same acquisition time and the video frames corresponding to the same acquisition time in each to-be-processed segment are synchronously displayed through different screen areas of the split-screen images to obtain the event video. The specific process can be referred to the aforementioned embodiment, which will not be described in detail here.

[0210] In the embodiment of the present application, the earliest initial segment in the sequence is determined among all the segments to be processed, and the initial segment is updated according to the cached video of the initial monitoring point, and the event video is generated according to the updated initial segment and the remaining segments to be processed; wherein the initial monitoring point is the monitoring point corresponding to the initial segment. In the above method, the cached video based on the initial monitoring point realizes the update of the initial segment, which can improve the reliability of the initial segment, and correspondingly improve the reliability of the event video generated based on the initial segment.

[0211] Updating the initial segment may be expanding the initial segment. In one embodiment, Fig.13 As shown, in the above S1220, the initial segment is updated according to the cached video of the initial monitoring point, including:

[0212] S1310: In the cached video of the initial monitoring point, determine a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame.

[0213] The first video frame in the initial segment is the earliest video frame captured in the initial segment.

[0214] Optionally, for the initial segment to be corrected, the monitoring host may determine a portion of the cached segment in the cached video at the initial monitoring point that is continuous in timing with the first video frame in the initial segment and is located before the first video frame. Exemplarily, the monitoring host may obtain a cached segment of a preset duration / preset number of frames in the cached video at the initial monitoring point that is continuous in timing with the first video frame and is located before the first video frame as the target cached segment.

[0215] S1320: splice the target cache segment and the initial segment in time sequence to obtain an updated initial segment.

[0216] Optionally, after obtaining the target cache segment corresponding to the initial segment, the monitoring host can splice the target cache segment with the initial segment in time sequence, specifically by placing the target cache segment before the initial segment to expand the initial segment to obtain an updated initial segment.

[0217] In the embodiment of the present application, in the cached video of the initial monitoring point, a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame is determined, and the target cached segment is spliced ​​with the initial segment in time sequence to obtain an updated initial segment. In the above method, the initial segment is expanded by splicing the target cached segment with the initial segment in time sequence, and a part of the video segment before the target object appears in the monitoring area is added to the initial segment, so that the event video obtained based on the updated initial segment can include the scene where the target object appears for the first time in the monitoring, thereby improving the integrity of the event recorded in the event video.

[0218] To improve the timeliness of monitoring, in one embodiment, the method further includes:

[0219] When a target object is detected, an alarm prompt is triggered that the target object appears in the monitoring area; and / or the event video is sent to an external terminal.

[0220] Optionally, the monitoring host can detect the existence of the target object in each video frame frame by frame for each cached video of each monitoring point to obtain the segment of interest, and at the same time, when the target object is detected in the video frame, trigger an alarm prompt of the presence of the target object in the monitoring area. There are many ways to give an alarm prompt. For example, the monitoring host can start the alarm device to sound an alarm, or call the on-site voice alarm player to notify the alarm voice of the presence of the target object in the monitoring area, and can also send an alarm prompt message to the management platform or group associated with the monitoring host.

[0221] Optionally, after obtaining the event video of the target object, the monitoring host can send the event video to an external terminal associated with the monitoring host. Exemplarily, the monitoring host can package the obtained event video and send it to the management platform or group, and can also send the real-time data stream of the event video to the monitoring main screen, and present the event video in real time through the monitoring main screen.

[0222] In the embodiment of the present application, when the target object is detected, an alarm prompt of the target object appearing in the monitoring area is triggered; and / or the event video is sent to an external terminal. In the above method, an alarm prompt of the target object appearing in the monitoring area is triggered, or the event video is sent to an external terminal, thereby realizing monitoring feedback and improving monitoring timeliness.

[0223] In order to facilitate the understanding of those skilled in the art, the monitoring processing method provided by this application is described in detail below. Fig.14 As shown, the method may include:

[0224] S1401, using sample images of different types of reference objects in historical cached videos of the monitoring area and indicative texts of the reference objects in each sample image as training samples, training to obtain a language image multimodal model, and obtaining cached videos of multiple monitoring points in the monitoring area;

[0225] S1402, inputting the received indication text of the target object and the video frames in each cached video into the language image multimodal model, and detecting the existence status of the target object in each video frame;

[0226] S1403, dividing the video frames in each cached video that are continuous in time sequence and contain the target object into an interesting segment, to obtain at least one interesting segment;

[0227] S1404, dividing all video frames collected at the same time in the interested segment into a candidate video frame set; the candidate video frame set includes at least two video frames collected at the same time;

[0228] S1405, obtaining at least one picture parameter of a target object in each video frame in the candidate video frame set;

[0229] S1406, sorting the video frames in the candidate video frame set according to the various picture parameters from high to low, to obtain the video frame arrangement order corresponding to each picture parameter;

[0230] S1407, determining the picture quality order of each video frame in the candidate video frame set according to the video frame arrangement order corresponding to each picture parameter;

[0231] S1408. Determine, according to the picture quality order, at least one video frame whose order meets a preset requirement in the candidate video frame set as a spare video frame;

[0232] S1409: Divide the video frames corresponding to the same monitoring point and having a continuous time sequence among all the spare video frames and the remaining video frames in all the interested segments into one segment to be processed;

[0233] S1410, determining the initial segment with the earliest timing among all the segments to be processed;

[0234] S1411, in the cached video of the initial monitoring point, determine a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame;

[0235] S1412, splicing the target cache segment and the initial segment in time sequence to obtain an updated initial segment;

[0236] S1413, when the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen images, the video frames in all the segments to be processed are sorted in time sequence, and the video frames at the same acquisition time are synchronously displayed in different screen areas of the split-screen images to obtain the event video;

[0237] S1414, when the number of video frames at the same acquisition time in all the to-be-processed segments at any time is less than the number of split-screen images, determine a substitute monitoring point from all monitoring points according to a preset monitoring point priority;

[0238] S1415, extracting a substitute video frame at the same acquisition time from the cached video of the substitute monitoring point;

[0239] S1416. Sort the substitute video frames and the video frames in each to-be-processed segment in chronological order, and synchronously display the substitute video frames at the same acquisition time and the video frames corresponding to the same acquisition time in each to-be-processed segment in different screen areas of the split screen to obtain the event video.

[0240] It should be noted that for the description in the above S1401-S1416, reference may be made to the relevant description in the above embodiment, and the effects are similar, which will not be repeated in this embodiment.

[0241] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0242] In one embodiment, Fig.15 As shown, a monitoring processing device is provided, including: a cache acquisition module 1501, a fragment determination module 1502 and a screening processing module 1503; wherein,

[0243] The cache acquisition module 1501 is used to acquire cache videos of multiple monitoring points in the monitoring area;

[0244] The segment determination module 1502 is used to determine at least one segment of interest in the cached videos of multiple monitoring points; each video frame in the segment of interest has a target object;

[0245] The screening processing module 1503 is used to screen at least one segment of interest to generate an event video of the target object; the event video is used to record the entire event process from the appearance to the disappearance of the target object in the monitoring area; the screening process includes sorting the picture quality of the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0246] In one embodiment, the fragment determination module 1502 includes:

[0247] The target detection submodule is used to input the received indication text of the target object and the video frames in each cached video into the language image multimodal model to detect the state of the target object in each video frame;

[0248] The segment determination submodule is used to divide the video frames in each cached video that are continuous in time sequence and contain the target object into an interesting segment to obtain at least one interesting segment.

[0249] In one embodiment, the above device further comprises:

[0250] The model training module is used to use sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative text of the reference objects in each sample image as training samples to train a language image multimodal model.

[0251] In one embodiment, the model training module includes:

[0252] A feature acquisition module, used for inputting each instruction text and sample image into an initial multimodal model, and extracting a text feature vector of each instruction text and an image feature vector of each sample image by the initial multimodal model;

[0253] The parameter updating module is used to obtain the vector similarity between each text feature vector and each image feature vector, and adjust the model parameters in the initial multimodal model to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached to obtain a language image multimodal model.

[0254] In one embodiment, the screening processing module 1503 includes:

[0255] A candidate submodule is used to divide the video frames at the same acquisition time in all the interested segments into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time;

[0256] A spare submodule, used for selecting at least one spare video frame from each candidate video frame set;

[0257] The generating submodule is used to generate an event video of the target object according to each spare video frame and the remaining video frames in all the interested segments.

[0258] In one embodiment, the standby submodule includes:

[0259] A picture quality unit, used for obtaining, for each candidate video frame set, a picture quality order of each video frame in the candidate video frame set;

[0260] The spare determination unit is used to determine at least one video frame whose sequence meets a preset requirement in the candidate video frame set as a spare video frame according to the picture quality sequence.

[0261] In one embodiment, the picture quality unit includes:

[0262] A parameter subunit, used to obtain at least one picture parameter of a target object in each video frame in the candidate video frame set;

[0263] The sorting subunit is used to sort the video frames from high to low according to the various picture parameters, so as to obtain the arrangement order of the video frames corresponding to the various picture parameters;

[0264] The quality subunit is used to determine the picture quality order of each video frame according to the video frame arrangement order corresponding to each picture parameter.

[0265] In one embodiment, the image parameters include the number, confidence and volume of target objects; the quality subunit is further configured to:

[0266] For each video frame in the candidate video frame set, determine the quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine the confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine the volume sequence number in the video frame arrangement sequence corresponding to the volume;

[0267] Obtain the total number of the quantity number, confidence number and volume number of each video frame;

[0268] The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

[0269] In one embodiment, the generating submodule includes:

[0270] A processing unit is used to divide the video frames corresponding to the same monitoring point and having a continuous time sequence among all the spare video frames and the remaining video frames into a segment to be processed;

[0271] The event video unit is used to generate an event video according to each segment to be processed.

[0272] In one embodiment, the event video unit includes:

[0273] The first generating sub-unit is used for, when at any moment the number of video frames at the same acquisition moment in all the segments to be processed is equal to the number of split-screen pictures, to sort the video frames in all the segments to be processed in chronological order, and to synchronously display the video frames at the same acquisition moment through different picture areas of the split-screen pictures to obtain the event video.

[0274] In one embodiment, the event video unit includes:

[0275] The substitute point subunit is used to determine the substitute monitoring point from all monitoring points according to the preset monitoring point priority when the number of video frames at the same acquisition time in all the segments to be processed at any time is less than the number of split-screen images;

[0276] A video extraction subunit, used to extract a substitute video frame at the same acquisition time from the cached video of the substitute monitoring point;

[0277] The second generating sub-unit is used to sort the substitute video frames and the video frames in each to-be-processed segment in chronological order, and synchronously display the substitute video frames at the same acquisition moment and the video frames corresponding to the same acquisition moment in each to-be-processed segment through different picture areas of the split-screen picture to obtain the event video.

[0278] In one of the embodiments, the event video unit further includes:

[0279] An initial subunit, used to determine the earliest initial segment in the sequence among all segments to be processed;

[0280] An updating subunit, used to update the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment;

[0281] The video subunit is used to generate an event video according to the updated initial segment and the remaining segments to be processed.

[0282] In one embodiment, the updating subunit is used to:

[0283] In the cached video of the initial monitoring point, determine a target cached segment that is continuous in time sequence with the first video frame in the initial segment and is located before the first video frame; splice the target cached segment and the initial segment in time sequence to obtain an updated initial segment.

[0284] In one embodiment, the above device further comprises:

[0285] An alarm water supply module is used to trigger an alarm prompt of the presence of a target object in the monitoring area when a target object is detected; and / or,

[0286] The external communication module is used to send event videos to external terminals.

[0287] Each module in the above monitoring and processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0288] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.16 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for controlling the heating of a pole piece is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0289] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0290] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0291] Acquire cached videos of multiple monitoring points in a monitoring area; determine at least one segment of interest in the cached videos of multiple monitoring points; each video frame in the segment of interest has a target object; perform screening processing on at least one segment of interest to generate an event video of the target object; the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; the screening processing includes sorting the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0292] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0293] The received indication text of the target object and the video frames in each cached video are input into the language image multimodal model to detect the existence status of the target object in each video frame; the video frames in each cached video that are continuous in time sequence and contain the target object are divided into an interesting segment to obtain at least one interesting segment.

[0294] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0295] Sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative texts of the reference objects in each sample image are used as training samples to train a language image multimodal model.

[0296] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0297] Each indicator text and sample image is input into the initial multimodal model, and the text feature vector of each indicator text and the image feature vector of each sample image are extracted by the initial multimodal model; the vector similarity between each text feature vector and each image feature vector is obtained, and the model parameters in the initial multimodal model are adjusted to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, thereby obtaining a language image multimodal model.

[0298] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0299] The video frames at the same acquisition time in all the segments of interest are divided into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time; at least one spare video frame is selected in each candidate video frame set; and an event video of the target object is generated according to each spare video frame and the remaining video frames in all the segments of interest.

[0300] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0301] For each candidate video frame set, the picture quality order of each video frame in the candidate video frame set is obtained; according to the picture quality order, at least one video frame whose order meets the preset requirements is determined in the candidate video frame set as a spare video frame.

[0302] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0303] Obtain at least one picture parameter of a target object in each video frame in the candidate video frame set; sort the video frames from high to low according to the picture parameters to obtain the video frame arrangement sequence corresponding to each picture parameter; determine the picture quality sequence of each video frame according to the video frame arrangement sequence corresponding to each picture parameter.

[0304] In one embodiment, the image parameters include the number, confidence and volume of the target objects; when the processor executes the computer program, the following steps are also implemented:

[0305] For each video frame in the candidate video frame set, determine the quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine the confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine the volume sequence number in the video frame arrangement sequence corresponding to the volume;

[0306] Obtain the total number of the quantity number, confidence number and volume number of each video frame;

[0307] The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

[0308] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0309] Among all the spare video frames and the remaining video frames, the video frames corresponding to the same monitoring point and having a continuous time sequence are divided into a segment to be processed; and an event video is generated according to each segment to be processed.

[0310] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0311] When the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen pictures, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed in different picture areas of the split-screen picture to obtain the event video.

[0312] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0313] When the number of video frames with the same acquisition time in all the segments to be processed at any moment is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority; the substitute video frames with the same acquisition time are extracted from the cached video of the substitute monitoring point; the substitute video frames and the video frames in each segment to be processed are sorted in chronological order, and the substitute video frames with the same acquisition time and the video frames corresponding to the same acquisition time in each segment to be processed are synchronously displayed through different picture areas of the split-screen image to obtain the event video.

[0314] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0315] Determine the earliest initial segment in all the segments to be processed; update the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment; generate an event video according to the updated initial segment and the remaining segments to be processed.

[0316] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0317] In the cached video of the initial monitoring point, determine a target cached segment that is continuous in time sequence with the first video frame in the initial segment and is located before the first video frame; splice the target cached segment and the initial segment in time sequence to obtain an updated initial segment.

[0318] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0319] When a target object is detected, an alarm prompt is triggered that the target object appears in the monitoring area; and / or the event video is sent to an external terminal.

[0320] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0321] Acquire cached videos of multiple monitoring points in a monitoring area; determine at least one segment of interest in the cached videos of multiple monitoring points; each video frame in the segment of interest has a target object; perform screening processing on at least one segment of interest to generate an event video of the target object; the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; the screening processing includes sorting the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0322] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0323] The received indication text of the target object and the video frames in each cached video are input into the language image multimodal model to detect the existence status of the target object in each video frame; the video frames in each cached video that are continuous in time sequence and contain the target object are divided into an interesting segment to obtain at least one interesting segment.

[0324] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0325] Sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative texts of the reference objects in each sample image are used as training samples to train a language image multimodal model.

[0326] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0327] Each indicator text and sample image is input into the initial multimodal model, and the text feature vector of each indicator text and the image feature vector of each sample image are extracted by the initial multimodal model; the vector similarity between each text feature vector and each image feature vector is obtained, and the model parameters in the initial multimodal model are adjusted to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, thereby obtaining a language image multimodal model.

[0328] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0329] The video frames at the same acquisition time in all the segments of interest are divided into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time; at least one spare video frame is selected in each candidate video frame set; and an event video of the target object is generated according to each spare video frame and the remaining video frames in all the segments of interest.

[0330] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0331] For each candidate video frame set, the picture quality order of each video frame in the candidate video frame set is obtained; according to the picture quality order, at least one video frame whose order meets the preset requirements is determined in the candidate video frame set as a spare video frame.

[0332] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0333] Obtain at least one picture parameter of a target object in each video frame in the candidate video frame set; sort the video frames from high to low according to the picture parameters to obtain the video frame arrangement sequence corresponding to each picture parameter; determine the picture quality sequence of each video frame according to the video frame arrangement sequence corresponding to each picture parameter.

[0334] In one embodiment, the image parameters include the number, confidence and volume of the target objects; when the computer program is executed by the processor, the following steps are also implemented:

[0335] For each video frame in the candidate video frame set, determine the quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine the confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine the volume sequence number in the video frame arrangement sequence corresponding to the volume;

[0336] Obtain the total number of the quantity number, confidence number and volume number of each video frame;

[0337] The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

[0338] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0339] Among all the spare video frames and the remaining video frames, the video frames corresponding to the same monitoring point and having a continuous time sequence are divided into a segment to be processed; and an event video is generated according to each segment to be processed.

[0340] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0341] When the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen pictures, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed in different picture areas of the split-screen picture to obtain the event video.

[0342] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0343] When the number of video frames with the same acquisition time in all the segments to be processed at any moment is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority; the substitute video frames with the same acquisition time are extracted from the cached video of the substitute monitoring point; the substitute video frames and the video frames in each segment to be processed are sorted in chronological order, and the substitute video frames with the same acquisition time and the video frames corresponding to the same acquisition time in each segment to be processed are synchronously displayed through different picture areas of the split-screen image to obtain the event video.

[0344] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0345] Determine the earliest initial segment in all the segments to be processed; update the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment; generate an event video according to the updated initial segment and the remaining segments to be processed.

[0346] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0347] In the cached video of the initial monitoring point, determine a target cached segment that is continuous in time sequence with the first video frame in the initial segment and is located before the first video frame; splice the target cached segment and the initial segment in time sequence to obtain an updated initial segment.

[0348] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0349] When a target object is detected, an alarm prompt is triggered that the target object appears in the monitoring area; and / or the event video is sent to an external terminal.

[0350] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0351] Acquire cached videos of multiple monitoring points in a monitoring area; determine at least one segment of interest in the cached videos of multiple monitoring points; each video frame in the segment of interest has a target object; perform screening processing on at least one segment of interest to generate an event video of the target object; the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; the screening processing includes sorting the video frames at the same acquisition time according to the picture parameters of the video frames at the same acquisition time in the segment of interest, and screening according to the results of the picture quality sorting; the picture parameters are the picture display parameters related to the target object in the video frame.

[0352] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0353] The received indication text of the target object and the video frames in each cached video are input into the language image multimodal model to detect the existence status of the target object in each video frame; the video frames in each cached video that are continuous in time sequence and contain the target object are divided into an interesting segment to obtain at least one interesting segment.

[0354] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0355] Sample images of different types of reference objects in the historical cached videos of the monitored area and the indicative texts of the reference objects in each sample image are used as training samples to train a language image multimodal model.

[0356] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0357] Each indicator text and sample image is input into the initial multimodal model, and the text feature vector of each indicator text and the image feature vector of each sample image are extracted by the initial multimodal model; the vector similarity between each text feature vector and each image feature vector is obtained, and the model parameters in the initial multimodal model are adjusted to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until the training cutoff condition is reached, thereby obtaining a language image multimodal model.

[0358] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0359] The video frames at the same acquisition time in all the segments of interest are divided into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time; at least one spare video frame is selected in each candidate video frame set; and an event video of the target object is generated according to each spare video frame and the remaining video frames in all the segments of interest.

[0360] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0361] For each candidate video frame set, the picture quality order of each video frame in the candidate video frame set is obtained; according to the picture quality order, at least one video frame whose order meets the preset requirements is determined in the candidate video frame set as a spare video frame.

[0362] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0363] Obtain at least one picture parameter of a target object in each video frame in the candidate video frame set; sort the video frames from high to low according to the picture parameters to obtain the video frame arrangement sequence corresponding to each picture parameter; determine the picture quality sequence of each video frame according to the video frame arrangement sequence corresponding to each picture parameter.

[0364] In one embodiment, the image parameters include the number, confidence and volume of the target objects; when the computer program is executed by the processor, the following steps are also implemented:

[0365] For each video frame in the candidate video frame set, determine the quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine the confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine the volume sequence number in the video frame arrangement sequence corresponding to the volume;

[0366] Obtain the total number of the quantity number, confidence number and volume number of each video frame;

[0367] The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

[0368] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0369] Among all the spare video frames and the remaining video frames, the video frames corresponding to the same monitoring point and having a continuous time sequence are divided into a segment to be processed; and an event video is generated according to each segment to be processed.

[0370] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0371] When the number of video frames at the same acquisition time in all the segments to be processed at any time is equal to the number of split-screen pictures, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed in different picture areas of the split-screen picture to obtain the event video.

[0372] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0373] When the number of video frames with the same acquisition time in all the segments to be processed at any moment is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to the preset monitoring point priority; the substitute video frames with the same acquisition time are extracted from the cached video of the substitute monitoring point; the substitute video frames and the video frames in each segment to be processed are sorted in chronological order, and the substitute video frames with the same acquisition time and the video frames corresponding to the same acquisition time in each segment to be processed are synchronously displayed through different picture areas of the split-screen image to obtain the event video.

[0374] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0375] Determine the earliest initial segment in all the segments to be processed; update the initial segment according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment; generate an event video according to the updated initial segment and the remaining segments to be processed.

[0376] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0377] In the cached video of the initial monitoring point, determine a target cached segment that is continuous in time sequence with the first video frame in the initial segment and is located before the first video frame; splice the target cached segment and the initial segment in time sequence to obtain an updated initial segment.

[0378] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0379] When a target object is detected, an alarm prompt is triggered that the target object appears in the monitoring area; and / or the event video is sent to an external terminal.

[0380] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0381] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0382] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A monitoring processing method, characterized in that: The method comprises: Obtain cached videos of multiple monitoring points in the monitoring area; In the cached videos of the plurality of monitoring points, a plurality of segments of interest are determined; each video frame in the segments of interest has a target object; Screening and processing the multiple interesting segments to generate an event video of the target object; the event video is used to record the entire process of the target object from appearance to disappearance in the monitoring area; The screening process of the plurality of interested segments to generate the event video of the target object includes: Divide the video frames at the same acquisition time in all the segments of interest into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time; Selecting at least one spare video frame from each of the candidate video frame sets; Generate an event video of the target object according to each of the spare video frames and remaining video frames in all the interested segments that are not divided into the candidate video frame set; The step of selecting at least one spare video frame from each of the candidate video frame sets comprises: For each of the candidate video frame sets, obtaining a picture quality order of each video frame in the candidate video frame set; According to the picture quality order, determining at least one video frame whose order meets a preset requirement in the candidate video frame set as the spare video frame; The obtaining the picture quality order of each video frame in the candidate video frame set includes: Acquire picture parameters of multiple target objects in each video frame in the candidate video frame set; the picture parameters are picture display parameters related to the target object in the video frame; According to each of the picture parameters, the video frames are sorted from high to low in parameter value to obtain an arrangement order of the video frames corresponding to each of the picture parameters; The picture quality order of each of the video frames is determined according to the video frame arrangement order corresponding to each of the picture parameters.

2. The method according to claim 1, characterized in that Determining a plurality of interesting segments in the cached videos of the plurality of monitoring points includes: Inputting the received indication text of the target object and the video frames in each of the cached videos into a language image multimodal model to detect the existence status of the target object in each of the video frames; The video frames in each of the cached videos that are continuous in time sequence and contain the target object are divided into one of the interest segments, to obtain a plurality of the interest segments.

3. The method according to claim 2, characterized in that The method further comprises: The sample images of different types of reference objects in the historical cached video of the monitoring area and the indicative text of the reference object in each of the sample images are used as training samples to train the language image multimodal model.

4. The method according to claim 3, characterized in that: The method of using sample images containing reference objects in the historical cached video of the monitoring area and the indicative text of the reference objects in each of the sample images as training samples to train the language image multimodal model includes: Inputting each of the instruction texts and the sample image into an initial multimodal model, and extracting a text feature vector of each of the instruction texts and an image feature vector of each of the sample images by the initial multimodal model; The vector similarity between each of the text feature vectors and each of the image feature vectors is obtained, and the model parameters in the initial multimodal model are adjusted to increase the vector similarity between the text feature vector and the image feature vector of the same reference object until a training cutoff condition is reached, thereby obtaining the language image multimodal model.

5. The method according to claim 1, characterized in that The picture parameters include the number, confidence and volume of the target object; and determining the picture quality order of each video frame according to the video frame arrangement order corresponding to each of the picture parameters includes: For each video frame in the candidate video frame set, determine a quantity sequence number in the video frame arrangement sequence corresponding to the quantity, determine a confidence sequence number in the video frame arrangement sequence corresponding to the confidence, and determine a volume sequence number in the video frame arrangement sequence corresponding to the volume; Obtaining a total value of the quantity sequence number, the confidence sequence number, and the volume sequence number of each of the video frames; The total values ​​of the sequence numbers of the video frames are sorted from small to large to obtain the picture quality order of the video frames.

6. The method according to claim 1, characterized in that The step of generating the event video of the target object according to the spare video frames and the remaining video frames in all the interested segments that are not divided into the candidate video frame set comprises: Dividing the video frames corresponding to the same monitoring point and having a continuous time sequence among all the spare video frames and the remaining video frames into a segment to be processed; The event video is generated according to each of the to-be-processed segments.

7. The method according to claim 6, characterized in that The step of generating the event video according to each of the to-be-processed segments includes: When the number of video frames at the same acquisition time in all the segments to be processed is equal to the number of split-screen pictures at any moment, the video frames in all the segments to be processed are sorted in chronological order, and the video frames at the same acquisition time are synchronously displayed through different picture areas of the split-screen pictures to obtain the event video.

8. The method according to claim 7, characterized in that The step of generating the event video according to each of the to-be-processed segments includes: When the number of video frames at the same acquisition time in all the to-be-processed segments at any time is less than the number of split-screen images, a substitute monitoring point is determined from all monitoring points according to a preset monitoring point priority; Extracting the substitute video frame at the same acquisition time from the cached video of the substitute monitoring point; The substitute video frames and the video frames in each of the to-be-processed segments are sorted in chronological order, and the substitute video frames at the same acquisition moment and the video frames corresponding to the same acquisition moment in each of the to-be-processed segments are synchronously displayed through different picture areas of the split-screen picture to obtain the event video.

9. The method according to claim 6, characterized in that The step of generating the event video according to each of the to-be-processed segments further includes: Determine the initial segment with the earliest timing among all the segments to be processed; The initial segment is updated according to the cached video of the initial monitoring point; the initial monitoring point is the monitoring point corresponding to the initial segment; The event video is generated according to the updated initial segment and the remaining segments to be processed.

10. The method according to claim 9, characterized in that The updating of the initial segment according to the cached video of the initial monitoring point includes: In the cached video of the initial monitoring point, determine a target cached segment whose timing is continuous with the first video frame in the initial segment and is located before the first video frame; The target cache segment and the initial segment are spliced ​​in time sequence to obtain the updated initial segment.

11. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the target object is detected, an alarm prompt indicating the target object appears in the monitoring area is triggered; and / or the event video is sent to an external terminal.

12. A monitoring processing device, characterized in that: The device comprises: A cache acquisition module is used to acquire cached videos of multiple monitoring points in the monitoring area; A segment determination module is used to determine a plurality of segments of interest in the cached videos of the plurality of monitoring points; each video frame in the segments of interest has a target object; A screening and processing module, used for screening and processing the plurality of interested segments to generate an event video of the target object; the event video is used for recording the entire process of the target object from appearance to disappearance in the monitoring area; The screening processing module comprises: A candidate submodule is used to divide the video frames at the same acquisition time in all the segments of interest into a candidate video frame set; the candidate video frame set includes at least two video frames at the same acquisition time; A spare submodule, used for selecting at least one spare video frame from each of the candidate video frame sets; A generating submodule, configured to generate an event video of the target object according to each of the spare video frames and the remaining video frames in all the interested segments that are not classified into the candidate video frame set; The standby submodule comprises: A picture quality unit, configured to obtain, for each of the candidate video frame sets, a picture quality order of each video frame in the candidate video frame set; a spare determination unit, configured to determine, according to the picture quality order, at least one video frame whose order meets a preset requirement in the candidate video frame set as the spare video frame; The picture quality unit comprises: A parameter subunit, used to obtain picture parameters of multiple target objects in each video frame in the candidate video frame set; the picture parameters are picture display parameters related to the target object in the video frame; A sorting subunit, used to sort the video frames from high to low according to the picture parameters, to obtain the video frame arrangement sequence corresponding to the picture parameters; The quality subunit is used to determine the picture quality order of each of the video frames according to the arrangement order of the video frames corresponding to each of the picture parameters.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images