Video search method, device, electronic device, and storage medium

Abstract

The application relates to a video search method and device, an electronic device and a storage medium, wherein the video search method comprises the following steps: obtaining a to-be-searched video and a candidate video; obtaining corresponding video detection data based on the to-be-searched video and the candidate video, wherein the video detection data comprises a preset event detected based on the to-be-searched video and the candidate video and a video channel where the preset event is located; determining a first similarity of the to-be-searched video and the candidate video according to the video detection data; and if the first similarity is greater than a first preset threshold, determining the candidate video as a target video. Through the application, the problem of low accuracy of video search in the prior art is solved, and the technical effect of improving the accuracy of video search is achieved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance, and in particular to a video search method, device, electronic device and storage medium. Background Art

[0002] In the internet technology sector, user-selected video clips are compared against historical videos uploaded to the platform, and similar videos are retrieved based on the similarity of features within the video objects. However, in the security video surveillance sector, existing video retrieval technology remains limited to image-based search, failing to capture the plot and semantic features of the video.

[0003] In addition, current solutions rely solely on the similarity of targets in videos, and only rely on the similarity of videos for analysis and comparison, resulting in low accuracy of video search.

[0004] With regard to the low accuracy of video search in related technologies, no effective solution has been proposed so far. Summary of the Invention

[0005] In this embodiment, a video search method, device, electronic device, and storage medium are provided to solve the problem of low accuracy of video search in related technologies.

[0006] In a first aspect, a video search method is provided in this embodiment. The video search method includes:

[0007] Get the videos to be searched and candidate videos;

[0008] Obtaining corresponding video detection data based on the video to be searched and the candidate videos, the video detection data including a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located;

[0009] A first similarity between the video to be searched and the candidate video is determined according to the video detection data, and if the first similarity is greater than a first preset threshold, the candidate video is determined as a target video.

[0010] In one embodiment, the video detection data includes first detection data and second detection data, the first detection data corresponds to the video to be searched, and the second detection data corresponds to the candidate video, and determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: based on the first detection data, counting the first distribution data of the preset event in the video to be searched on different video channels; based on the second detection data, counting the second distribution data of the preset event in the candidate video on different video channels; and determining the first similarity based on the first distribution data and the second distribution data.

[0011] In one embodiment, determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: determining the first occurrence time of the preset event in the video to be searched and the corresponding video channel based on the first detection data to obtain first timing data; determining the second occurrence time of the preset event in the candidate video and the corresponding video channel based on the second detection data to obtain second timing data; and determining the first similarity based on the first timing data and the second timing data.

[0012] In one embodiment, determining the first similarity between the video to be searched and the candidate video based on the video detection data also includes: determining the target type, target attributes and sensor data of the preset event in the video to be searched based on the first detection data to obtain first business data; determining first target data based on the first time series data and the first business data; determining the target type, target attributes and sensor data of the preset event in the candidate video based on the second detection data to obtain second business data; determining second target data based on the second time series data and the second business data; and determining the first similarity based on the first target data and the second target data.

[0013] In one embodiment, after determining the first similarity between the video to be searched and the candidate video based on the video detection data, the method further includes: selecting one or more first image frames based on the video to be searched; selecting one or more second image frames based on the candidate video; determining a second similarity based on the first image frame and the second image frame; determining a target similarity based on the first similarity and the second similarity; if the target similarity is greater than a second preset threshold, determining the candidate video as the target video.

[0014] In one embodiment, determining the image similarity based on the first image frame and the second image frame includes: performing target detection based on the first image frame to obtain first feature information, the first feature information including first feature vector data, first motion trajectory data and first target posture data; performing target detection based on the second image frame to obtain second feature information, the second feature information including second feature vector data, second motion trajectory data and second target posture data; and determining the second similarity based on the first feature information and the second feature information.

[0015] In one embodiment, determining the target similarity based on the first similarity and the second similarity includes: obtaining a similarity parameter, wherein the similarity parameter is used to determine a weight of the first similarity and the second similarity; and determining the target similarity according to the similarity parameter, the first similarity, and the second similarity.

[0016] In a second aspect, a video search device is provided in this embodiment, including:

[0017] An acquisition module is used to obtain videos to be searched and candidate videos;

[0018] a processing module, configured to obtain corresponding video detection data based on the video to be searched and the candidate videos, wherein the video detection data includes a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located;

[0019] A calculation module is used to determine a first similarity between the video to be searched and the candidate video according to the video detection data, and if the first similarity is greater than a first preset threshold, determine the candidate video as a target video.

[0020] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video search method described in the first aspect when executing the computer program.

[0021] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the video search method described in the first aspect is implemented.

[0022] Compared with the related art, the video search method provided in this embodiment obtains a video to be searched and a candidate video; based on the video to be searched and the candidate video, corresponding video detection data is obtained, and the video detection data includes a preset event detected based on the video to be searched and the candidate video and a video channel where the preset event is located; according to the video detection data, a first similarity between the video to be searched and the candidate video is determined, and if the first similarity is greater than a first preset threshold, the candidate video is determined as the target video, which solves the problem of low accuracy of video search in the prior art and achieves the technical effect of improving the accuracy of video search.

[0023] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 1 is a hardware structure block diagram of a terminal of the video search method of this embodiment;

[0026] Figure 2 is a flow chart of the video search method of this embodiment;

[0027] Figure 3 is a schematic diagram of a video search method according to another embodiment of the present application;

[0028] Figure 4 It is a structural block diagram of the video search device of this embodiment. DETAILED DESCRIPTION

[0029] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0030] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0031] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the video search method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0032] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the video search method in this embodiment. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and such remote memory may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] Currently, in the internet sector, feature comparison is performed against historical videos uploaded to the platform based on user-selected video clips. This primarily involves comparing the feature similarity of objects within the video. This approach has yet to be applied in the security video surveillance field. Existing search methods in this field remain at the image-based search stage, failing to capture the plot and semantic features of the video. Current solutions rely solely on the similarity of objects within the video, or search based on spliced ​​videos, without leveraging other business data or overlaying data from IoT sensors. This technology remains a niche area in the security video surveillance field.

[0035] This embodiment provides a video search method. By storing various structured intelligence and business data in auxiliary frames in the video, the temporal correlation between multiple channels is also used as a dimension of video retrieval and matching during video search. Video search is performed in a multi-faceted manner to achieve higher search accuracy and matching degree, which can better reflect the plot and semantic features of the video and is closer to real business scenario applications.

[0036] In this embodiment, a video search method is provided. Figure 2 Flowchart of the video search method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0037] Step S201: Obtain the video to be searched and the candidate videos.

[0038] Specifically, the to-be-searched video is a video clip currently available to the user that contains some of the content they need and serves as the basis for the video search. The candidate videos are the objects of the video search, and there are generally one or more candidate videos. The to-be-searched video and candidate videos are pre-acquired to prepare for the next step of the video search process. In security monitoring scenarios, the to-be-searched video is typically a video clip captured by a network video recorder (NVR). The corresponding candidate videos are pre-stored historical video clips captured by the NVR and used for comparison with the to-be-searched video.

[0039] Step S202: Obtain corresponding video detection data based on the video to be searched and the candidate videos. The video detection data includes a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located.

[0040] Specifically, a preset event is an intelligent detection event set by the camera based on preset business requirements. This event generally includes the detection of a preset target, such as a human or a motor vehicle. This event also includes target attribute information detected based on the video frame, such as the person's or vehicle's appearance. Furthermore, it includes other data collected through sensor information, such as real-time temperature and humidity. In the security monitoring field, network video recorders typically include multiple video channels, each with a video stream. When detecting these preset events, the video channel where the preset event occurred can also be determined.

[0041] Step S203: determining a first similarity between the video to be searched and the candidate video according to the video detection data; if the first similarity is greater than a first preset threshold, determining the candidate video as a target video.

[0042] Specifically, based on the video detection data, the video to be searched and the candidate videos are compared from multiple dimensions including preset events and corresponding video channels, and a first similarity is calculated. When the first similarity is greater than a first preset threshold, the candidate video is determined as the target video, that is, the video data that is desired to be retrieved based on the video to be retrieved and contains all the content required by the user.

[0043] Existing "video search by video" technology is primarily used in the internet search field. For example, in film and television search, a short video or GIF of a few seconds duration is used to search for the source of a video, identifying the source of the clip, i.e., the corresponding longer film video data. In these application scenarios, the video to be searched is limited by the amount of information available, and video search can only be performed based on object recognition and behavioral analysis within the image frame. In security surveillance scenarios, however, smart camera front-ends are typically equipped with other IoT sensors, such as temperature and humidity sensors, in addition to image sensors, based on specific business needs. Therefore, in security scenarios, when performing "video search by video," IoT sensor information can be used to obtain more information in addition to the video itself, thereby improving search accuracy and speed. Furthermore, video data in security scenarios is mostly multi-channel data collected by network video recorders. By analyzing the information content of different video channels, it is possible to further mine video information, analyze the temporal patterns of intelligently detected events, and use these patterns to conduct video search, thereby improving the accuracy of video retrieval. Through the above steps, the video search method of this embodiment fully utilizes the intelligent detection capabilities of the smart camera front end in security scenarios. When performing video search and comparison, the similarity of the video is calculated with the help of video channel information and preset events detected by the smart camera front end. Compared with the existing image search that only relies on the target features of the video image frame, the accuracy of the video search is greatly improved.

[0044] In one embodiment, the video detection data includes first detection data and second detection data, the first detection data corresponds to the video to be searched, and the second detection data corresponds to the candidate video, and determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: based on the first detection data, counting the first distribution data of the preset event in the video to be searched on different video channels; based on the second detection data, counting the second distribution data of the preset event in the candidate video on different video channels; and determining the first similarity based on the first distribution data and the second distribution data.

[0045] Specifically, the preset event is a detection event set by the user of the intelligent camera front end according to actual business needs. The preset event can be implemented with the help of the intelligent detection algorithm built into the intelligent camera front end. The preset events include but are not limited to target detection, temperature detection, humidity detection, violation detection, human body part detection, etc. When the intelligent camera front end detects a preset event, it will record the video channel information where the preset event occurs and the preset event type. The video channel information can be the video channel number. By counting the preset events across channels of the video to be searched and the candidate videos within the same time range, the distribution of the preset events on different video channels is determined according to the statistical results, and the similarity between the video to be searched and the candidate videos is determined based on the distribution. The video search method of this embodiment uses statistical rules to count the preset event types, quantities and corresponding video channels detected by the intelligent camera front end, thereby improving the accuracy of video search.

[0046] In one specific embodiment, for the video to be searched, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4; and the candidate video includes two videos. For the first candidate video, the human target is also first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4; and for the second candidate video, the human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 5; when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0047] In one embodiment, determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: determining the first occurrence time of the preset event in the video to be searched and the corresponding video channel based on the first detection data to obtain first timing data; determining the second occurrence time of the preset event in the candidate video and the corresponding video channel based on the second detection data to obtain second timing data; and determining the first similarity based on the first timing data and the second timing data.

[0048] Specifically, when a preset event is detected in the intelligent camera front end, the time of occurrence of the preset event is recorded, and corresponding time series data is generated based on the video to be searched and the candidate videos. The similarity between the video to be searched and the candidate videos is determined based on the time series data. For example: for the video to be searched, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4; and the candidate videos include two videos. For the first candidate video, a human target is also first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4; and for the second candidate video, a vehicle target is first detected on the video channel numbered 3, and then a human target is detected on the video channel numbered 4; then, when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0049] In another embodiment, when the video to be searched and the candidate video detect the same preset event, the closer the occurrence time of the preset event is, the higher the similarity is. For example: for the video to be searched, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4 after an interval of 10 seconds; and the candidate video includes two videos. For the first candidate video, a human target is also first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4 after an interval of 8 seconds; and for the second candidate video, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4 after an interval of 20 seconds; when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0050] In this embodiment, by combining the preset event dimension, video channel dimension, and time sequence dimension, the video comparison is performed in multiple dimensions, which further improves the calculation accuracy of the similarity calculation and improves the accuracy of the video search.

[0051] In one embodiment, determining the first similarity between the video to be searched and the candidate video based on the video detection data also includes: determining the target type, target attributes and sensor data of the preset event in the video to be searched based on the first detection data to obtain first business data; determining first target data based on the first time series data and the first business data; determining the target type, target attributes and sensor data of the preset event in the candidate video based on the second detection data to obtain second business data; determining second target data based on the second time series data and the second business data; and determining the first similarity based on the first target data and the second target data.

[0052] Specifically, the target type is the type of person detected based on the video image frame, such as vehicle, person, or animal. The target attribute is the attribute information of the target. For example, if the target type is a person, the corresponding target attributes include clothing color, whether the person is wearing glasses, etc. If the target type is a vehicle, the corresponding target attributes include vehicle brand, vehicle color, and license plate number. Sensor refers to sensor information collected by the sensors supporting the intelligent camera front end, such as temperature and humidity. For example: for the video to be searched, a person in a red shirt is first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 4 after an interval of 10 seconds; and the candidate videos include two videos. For the first candidate video, a person in a red shirt is also first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 4 after an interval of 10 seconds; and for the second candidate video, a person in a white shirt is first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 4 after an interval of 10 seconds; when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0053] In this embodiment, the similarity between the video to be searched and the candidate videos is calculated from multiple dimensions by combining the preset event dimension, video channel dimension, timing dimension and business information dimension, which further improves the calculation accuracy of the similarity calculation and improves the accuracy of video search.

[0054] In one embodiment, after determining the first similarity between the video to be searched and the candidate video based on the video detection data, the method further includes: selecting one or more first image frames based on the video to be searched; selecting one or more second image frames based on the candidate video; determining a second similarity based on the first image frame and the second image frame; determining a target similarity based on the first similarity and the second similarity; if the target similarity is greater than a second preset threshold, determining the candidate video as the target video.

[0055] Specifically, when selecting image frames, similar frames can be quickly located based on the frame numbers recorded when a preset event detection is triggered. Feature extraction is performed based on the selected first and second frames to determine the similarity of the frames. The searched video and candidate videos are then compared based on their video content to improve the accuracy of the video search.

[0056] In one embodiment, determining the image similarity based on the first image frame and the second image frame includes: performing target detection based on the first image frame to obtain first feature information, the first feature information including first feature vector data, first motion trajectory data and first target posture data; performing target detection based on the second image frame to obtain second feature information, the second feature information including second feature vector data, second motion trajectory data and second target posture data; and determining the second similarity based on the first feature information and the second feature information.

[0057] Specifically, based on the first image frame and the second image frame, the feature vectors, motion trajectories and postures of the detected targets in the search video and the candidate video can be compared, and the second similarity can be calculated based on the similarity of the feature vectors, motion trajectories and postures. Preferably, the similarity can be calculated through neural network calculation.

[0058] In one embodiment, determining the target similarity based on the first similarity and the second similarity includes: obtaining a similarity parameter, wherein the similarity parameter is used to determine a weight of the first similarity and the second similarity; and determining the target similarity according to the similarity parameter, the first similarity, and the second similarity.

[0059] Specifically, the first similarity and the second similarity are weighted and calculated according to a preset similarity parameter. The similarity parameter can be set according to the actual business scenario based on prior knowledge. In one embodiment, the target similarity is: Z = X × a + Y × (1-), where Z is the target similarity, X is the first similarity, Y is the second similarity, and a is the similarity parameter.

[0060] In one specific embodiment, a video search method is provided. The video search method includes two stages: a real-time data storage stage and a video search stage.

[0061] The real-time data storage stage includes: when the front-end camera triggers an intelligent event, it will add a structured auxiliary frame to the code stream information, and the auxiliary frame will carry the corresponding frame number. Some actual business-related cameras or cameras connected to other sensors will also carry business feature data, such as thermal imaging cameras will carry temperature data, judicial cameras can carry temperature and humidity information and other POS overlay data, etc.

[0062] After receiving these streams, the network video recorder will store the structured information corresponding to the video data in the database, including time and channel information. The storage format is shown in Table 1:

[0063] Table 1 Event information storage table

[0064]

[0065] The video search stage includes: extracting auxiliary frames and target features from the video clips to be searched and the candidate video clips. The historical video feature extraction can refer to the frame numbers recorded in Table 1 for rapid positioning.

[0066] The features of the objects detected in the search video are compared with those in the candidate videos. This comparison can combine the target features with some motion trajectory or posture features. Specifically, the feature vectors, motion trajectory, and posture of the objects in the video clips are compared. This comparison process can be performed within a neural network. The neural network calculates a similarity value, which is used as the similarity 1 and is denoted as X.

[0067] Compare the cross-channel intelligent event sequence information and business information sequence information in the database, as follows:

[0068] First, the first comparison dimension includes the comparison of preset events across channels. For example, a person is first detected on the video channel numbered 3, a vehicle is then detected on the video channel numbered 4, and then other objects are detected on other channels. By interconnecting the channel information, the patterns of preset events occurring across the entire channel can be quantitatively compared.

[0069] Secondly, channel information can be superimposed on the basis of the first comparison dimension, and the comparison can be performed through the temporal dimension of "channel + event", and the evaluation results of the first comparison dimension can be added or subtracted. For example: a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4; and the candidate video includes two videos. For the first candidate video, the human target is also first detected on the video channel numbered 3, and then the vehicle target is detected on the video channel numbered 4; and for the second candidate video, the human target is first detected on the video channel numbered 3, and then the vehicle target is detected on the video channel numbered 5; then, when calculating the similarity of the videos, it is considered that the similarity between the search video and the first candidate video is greater than the similarity between the search video and the second candidate video.

[0070] Secondly, based on the second comparison dimension, the intervals between the occurrences of each preset event can be calculated. The closer the intervals are, the more points will be added to the evaluation results of the second comparison dimension. For example: for the video to be searched, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4 after an interval of 10 seconds; and the candidate videos include two videos. For the first candidate video, a human target is also first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 4 after an interval of 10 seconds; and for the second candidate video, a human target is first detected on the video channel numbered 3, and then a vehicle target is detected on the video channel numbered 5 after an interval of 20 seconds; then when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0071] Finally, based on the third comparison dimension, attributes and other business feature fields can be added. If the attributes and business features are also consistent, additional points will be added to the evaluation results of the third comparison dimension. For example: for the video to be searched, a person in a red shirt is first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 4 after an interval of 10 seconds; and the candidate videos include two videos. For the first candidate video, a person in a red shirt is also first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 4 after an interval of 10 seconds; and for the second candidate video, a person in a white shirt is first detected on the video channel numbered 3, and then a green vehicle is detected on the video channel numbered 5 after an interval of 10 seconds; when calculating the similarity of the videos, it is considered that the similarity between the video to be searched and the first candidate video is greater than the similarity between the video to be searched and the second candidate video.

[0072] According to the above four comparison dimensions, quantitative modeling and calculation are performed to obtain the similarity 2, which is recorded as Y.

[0073] Similarity 1 and Similarity 2 are weighted, for example, based on the business scenario of the feature, to obtain the empirical weight coefficient a. The total similarity Z = X × a + Y × (1-a). If the total similarity does not reach the total threshold, the comparison fails. If the total similarity reaches the total threshold, the comparison succeeds. The matched cross-channel video clips are spliced ​​and reported, along with the similarity and other information.

[0074] It should be emphasized that the candidate video segments may be scattered across various channels, and the video segments of each matched video channel need to be spliced ​​together. For example, the 10:00-10:10 video segment of video channel 1 and the 10:10-10:20 video segment of video channel 2 need to be spliced ​​together, and then the attributes and business information of the corresponding moments of the video segments at both ends and the calculated total threshold need to be reported.

[0075] In addition, the specific comparison strategy can be customized according to the business. It can be based on the timing information of a certain target in the video, such as a person or a car, or a combination of the two, with the default parameter timing arrangement and comparison of all targets.

[0076] In one embodiment, Figure 3 is a schematic diagram of a video search method according to another embodiment of the present application, such as Figure 3 As shown, first, the auxiliary frame data and target feature extraction are obtained, and then the target feature comparison is performed on all channel recordings within the specified time range to calculate similarity 1. The auxiliary information comparison is performed on all channel recordings within the specified time range to calculate similarity 2. The total similarity is calculated based on similarity 1 and similarity 2. If the total similarity is less than the total threshold, it is determined that the target video does not exist in the current candidate video; if the target similarity is greater than the total threshold, the comparison is determined to be successful, and the current candidate video is determined to be the target video and reported.

[0077] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. The video search method of the embodiment of the present application, based on the comparison of video image frames, combines the timing characteristics between multiple channels of the target triggering preset event and other business data, and the timing characteristics of IoT sensor data between multiple channels to perform multi-dimensional video comparison.

[0078] For security monitoring scenarios, the system comprehensively analyzes the triggering conditions of all preset events using global multi-video channel data, and calculates video similarity based on the regular evaluation criteria for the occurrence of preset events. In addition, it combines the business data carried in the auxiliary frames of the video or the information superimposed by the sensors to improve the accuracy of video search.

[0079] This embodiment also provides a video search device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0080] Figure 4 This is a structural block diagram of the video search device of this embodiment. Figure 4 As shown, the device includes:

[0081] An acquisition module 10 is used to acquire the video to be searched and the candidate videos;

[0082] A processing module 20 is configured to obtain corresponding video detection data based on the video to be searched and the candidate videos, wherein the video detection data includes a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located;

[0083] The calculation module 30 is configured to determine a first similarity between the video to be searched and the candidate video according to the video detection data, and determine the candidate video as a target video if the first similarity is greater than a first preset threshold.

[0084] The processing module 20 is also used to obtain video detection data, which includes first detection data and second detection data. The first detection data corresponds to the video to be searched, and the second detection data corresponds to the candidate video. The processing module 20 counts the first distribution data of the preset event in the video to be searched on different video channels based on the first detection data; counts the second distribution data of the preset event in the candidate video on different video channels based on the second detection data; and determines the first similarity based on the first distribution data and the second distribution data.

[0085] The calculation module 30 is also used to determine the first occurrence time of the preset event in the video to be searched and the corresponding video channel based on the first detection data to obtain first timing data; determine the second occurrence time of the preset event in the candidate video and the corresponding video channel based on the second detection data to obtain second timing data; and determine the first similarity based on the first timing data and the second timing data.

[0086] The calculation module 30 is also used to determine the target type, target attributes and sensor data of the preset event in the video to be searched based on the first detection data to obtain first business data; determine the first target data based on the first time series data and the first business data; determine the target type, target attributes and sensor data of the preset event in the candidate video based on the second detection data to obtain second business data; determine the second target data based on the second time series data and the second business data; and determine the first similarity based on the first target data and the second target data.

[0087] The calculation module 30 is also used to select one or more first image frames based on the video to be searched; select one or more second image frames based on the candidate video; determine a second similarity based on the first image frame and the second image frame; determine a target similarity based on the first similarity and the second similarity; if the target similarity is greater than a second preset threshold, determine the candidate video as the target video.

[0088] The calculation module 30 is also used to perform target detection based on the first image frame to obtain first feature information, wherein the first feature information includes first feature vector data, first motion trajectory data, and first target posture data; perform target detection based on the second image frame to obtain second feature information, wherein the second feature information includes second feature vector data, second motion trajectory data, and second target posture data; and determine the second similarity based on the first feature information and the second feature information.

[0089] The calculation module 30 is further configured to obtain a similarity parameter, which is used to determine a weight between the first similarity and the second similarity; and determine the target similarity according to the similarity parameter, the first similarity, and the second similarity.

[0090] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0091] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0092] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0093] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0094] S1, obtain the video to be searched and the candidate videos.

[0095] S2. Obtain corresponding video detection data based on the video to be searched and the candidate videos, where the video detection data includes a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located.

[0096] S3: Determine a first similarity between the video to be searched and the candidate video according to the video detection data. If the first similarity is greater than a first preset threshold, determine the candidate video as a target video.

[0097] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0098] In addition, in combination with the video search method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program that, when executed by a processor, implements any of the video search methods in the above embodiments.

[0099] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0100] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0101] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0102] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0103] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A video search method, characterized in that: include: Get the videos to be searched and candidate videos; Obtaining corresponding video detection data based on the video to be searched and the candidate videos, the video detection data including a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located; determining a first similarity between the video to be searched and the candidate video according to the video detection data, and determining the candidate video as a target video if the first similarity is greater than a first preset threshold; The first similarity is determined by counting preset events across channels of the video to be searched and the candidate video within the same time period, and determining the distribution of the preset events on different video channels based on the statistical results.

2. The video search method according to claim 1, wherein the video detection data includes first detection data and second detection data, the first detection data corresponds to the video to be searched, and the second detection data corresponds to the candidate video, characterized in that: Determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: Based on the first detection data, first distribution data of the preset event in the video to be searched on different video channels is counted; Based on the second detection data, collecting second distribution data of the preset event in the candidate video on different video channels; The first similarity is determined according to the first distribution data and the second distribution data.

3. The video search method according to claim 2, wherein: Determining the first similarity between the video to be searched and the candidate video based on the video detection data includes: Determine, based on the first detection data, a first occurrence time of the preset event in the video to be searched and the corresponding video channel to obtain first time series data; Based on the second detection data, determining a second occurrence time of the preset event in the candidate video and the corresponding video channel to obtain second time series data; The first similarity is determined according to the first time series data and the second time series data.

4. The video search method according to claim 3, wherein: Determining the first similarity between the video to be searched and the candidate video based on the video detection data further includes: Determine, based on the first detection data, a target type, a target attribute, and sensor data of the preset event in the video to be searched, and obtain first service data; Determine first target data based on the first time series data and the first business data; Determine, based on the second detection data, a target type, a target attribute, and sensor data of the preset event in the candidate video to obtain second service data; Determine second target data based on the second time series data and the second business data; The first similarity is determined according to the first target data and the second target data.

5. The video search method according to claim 1, wherein: After determining the first similarity between the video to be searched and the candidate video according to the video detection data, the method further includes: Selecting one or more first image frames according to the video to be searched; selecting one or more second image frames according to the candidate video; determining a second similarity based on the first image frame and the second image frame; determining a target similarity based on the first similarity and the second similarity; If the target similarity is greater than a second preset threshold, the candidate video is determined as the target video.

6. The video search method according to claim 5, characterized in that: Determining the image similarity based on the first image frame and the second image frame includes: Performing target detection based on the first image frame to obtain first feature information, the first feature information including first feature vector data, first motion trajectory data, and first target posture data; Performing target detection based on the second image frame to obtain second feature information, the second feature information including second feature vector data, second motion trajectory data, and second target posture data; The second similarity is determined according to the first feature information and the second feature information.

7. The video search method according to claim 5, characterized in that: Determining the target similarity based on the first similarity and the second similarity includes: Obtaining a similarity parameter, where the similarity parameter is used to determine a weight between the first similarity and the second similarity; The target similarity is determined according to the similarity parameter, the first similarity, and the second similarity.

8. A video search device, characterized in that: include: An acquisition module is used to obtain videos to be searched and candidate videos; a processing module, configured to obtain corresponding video detection data based on the video to be searched and the candidate videos, wherein the video detection data includes a preset event detected based on the video to be searched and the candidate videos and a video channel where the preset event is located; a calculation module, configured to determine a first similarity between the video to be searched and the candidate video based on the video detection data, and determine the candidate video as a target video if the first similarity is greater than a first preset threshold; The first similarity is determined by counting preset events across channels of the video to be searched and the candidate video within the same time period, and determining the distribution of the preset events on different video channels based on the statistical results.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the video search method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video search method according to any one of claims 1 to 7 are implemented.

Images