Video data logic processing method, electronic equipment and storage medium

Abstract

The invention discloses a video data logic processing method, electronic equipment and a storage medium, and relates to a data processing technology, and the method comprises the steps: obtaining video stream data collected by target video equipment at a first collection time, inputting the video stream data to a video detection model, obtaining target information in the video stream data and category information and attribute information respectively corresponding to the target information; associating the target information with the category information, the attribute information, the equipment information of the target video equipment and the first acquisition time to obtain target associated information; and constructing a first target index database under the first acquisition time for the video stream data based on the target associated information. According to the method and the device, the processing efficiency of the video data can be improved, and the video stream data can be stored according to the target index mode, so that all information associated with the keyword can be obtained through keyword search during subsequent query, and the beneficial effect of improving the query efficiency of the video data is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method for logically processing video data, an electronic device, and a storage medium. Background Art

[0002] With the development of science and technology, intelligent monitoring has been widely used in many fields and has become a mainstream monitoring method. Generally, after obtaining video stream data, the video stream data will be stored, which is helpful for viewing the video data.

[0003] The existing video storage method is generally to store a large amount of captured original video data in a fixed location based on a time series method. Therefore, when there is a query demand, it is necessary to query the video streams stored in the database one by one to obtain the target video; further, the target video is played to obtain the target information from the target video by frame-by-frame analysis. This method of storing video data is not convenient for viewing, and the query workload is large, resulting in low query efficiency. Summary of the invention

[0004] The present application provides a video data logical processing method, electronic device and storage medium, which can improve the existing solution for processing video data.

[0005] In a first aspect, the present application provides a method for logically processing video data, comprising:

[0006] Acquire video stream data collected by the target video device at a first collection time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information respectively corresponding to the target information;

[0007] Associating the target information with the category information, the attribute information, the device information of the target video device and the first acquisition time to obtain target association information;

[0008] A first target index database at the first acquisition time is constructed for the video stream data based on the target association information.

[0009] In a second aspect, the present application provides a video data logical processing device, the device comprising:

[0010] A data acquisition module, used to acquire video stream data acquired by a target video device at a first acquisition time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information respectively corresponding to the target information;

[0011] An information association module, used to associate the target information with the category information, the attribute information, the device information of the target video device and the first acquisition time to obtain target association information;

[0012] A database construction module is used to construct a first target index database at the first acquisition time for the video stream data based on the target association information.

[0013] In a third aspect, the present application further provides an electronic device, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data logical processing method described in any embodiment of the present application.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video data logical processing method described in any embodiment of the present application when executed.

[0018] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the video data logical processing method described in any embodiment of the present application.

[0019] The video data logical processing scheme provided by the embodiment of the present application first obtains the target information in the video stream data and the category information and attribute information corresponding to the target information through the video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, the target association information is obtained, and the first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be queried to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of the video data.

[0020] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the video data logical processing device, or may be packaged separately from the processor of the video data logical processing device, and this application does not limit this.

[0021] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0023] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of a method for logically processing video data provided by an embodiment of the present application;

[0026] Figure 2 is another flowchart of the video data logical processing method provided by an embodiment of the present application;

[0027] Figure 3 It is another flowchart of the method for logically processing video data provided by the embodiment of the present application;

[0028] Figure 4 It is another flowchart of the method for logically processing video data provided by the embodiment of the present application;

[0029] Figure 5 It is another flowchart of the method for logically processing video data provided by the embodiment of the present application;

[0030] Figure 6is a structural diagram of a first target index database provided in an embodiment of the present application;

[0031] Figure 7 is another structural diagram of the first target index database provided in an embodiment of the present application;

[0032] Figure 8 It is another flowchart of the method for logically processing video data provided by the embodiment of the present application;

[0033] Fig. 9 is a structural diagram of a multi-link index database provided in an embodiment of the present application;

[0034] Fig.10 It is a structural schematic diagram of a video data logical processing device provided in an embodiment of the present application;

[0035] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the present application will be clearly and completely described below in conjunction with the drawings in the present embodiment. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0039] Figure 1A flow chart of a method for logically processing video data provided in an embodiment of the present application. This embodiment can be applied to processing video data collected by a video device, building an index database, and facilitating data storage and query. The method can be executed by a video data logical processing device, which can be implemented in the form of hardware and / or software and integrated in an electronic device that executes the method. Preferably, the electronic device in the embodiment of the present application can be a server, or a computer device, etc.

[0040] refer to Figure 1 The video data logical processing method of this embodiment includes but is not limited to the following steps:

[0041] S10, obtaining video stream data collected by the target video device at the first collection time, inputting the video stream data into a video detection model, and obtaining target information in the video stream data and category information and attribute information corresponding to the target information.

[0042] In an embodiment of the present application, a video processing system may be integrated in the server, and the video processing system may be connected to a plurality of video devices, so that video stream data collected by the plurality of video devices may be obtained in real time. The target video device is any one of the plurality of video devices. The video processing system obtains the video stream data collected by any one of the plurality of video devices at a first collection time. The first collection time is the time when the target video device collects the video stream of a certain area or a certain target, and the first collection time may be a time point or a time period. Exemplarily, the first collection time may be xx:00 on ...

[0043] The video processing system provided in this embodiment is built with a pre-trained video detection model, which is trained based on sample video stream data, annotated targets in the sample video stream data, and category labels and attribute labels of the annotated targets. Therefore, the video detection model is used to perform video detection on the input video stream data to obtain target information in the video stream data output by the video detection model. The current target information is used to indicate the content contained in the video stream. It is easy to think that in a video stream, when the target video device is fixed, the target information in the video stream data collected by the target video device in the first acquisition time will change. For example, taking the first acquisition time of 30 seconds as an example, an adult may pass by in 0 to 5 seconds; a kitten may be photographed in 5 to 15 seconds; multiple cars may be photographed passing by in 15 to 30 seconds, etc.; the content contained in the specific target information is not limited to the current example.

[0044] In this embodiment, the category information is used to indicate the class to which the current target information belongs. For example, the current category information can be divided into the first category and the second category according to whether the video stream contains a person; it can also be divided according to the content contained in the video stream, such as being divided into three categories including only a person, only an object, and both a person and an object. The basis for dividing the specific category information and the number of category divisions are not limited here.

[0045] Attribute information is used to indicate the attributes corresponding to the current target information in the corresponding category. For example, if the target information is a person photographed, the corresponding attribute information may include color information, appearance information, clothing information, height information, etc.; if the target information is an animal photographed, the corresponding attribute information may be the animal's breed, hair color, hair length, etc.; if the target information is a vehicle, the corresponding attribute information may be the vehicle's color, vehicle driving status, vehicle size, and whether it is a new energy vehicle, etc. The specific attribute content contained in different categories of information is not limited here.

[0046] For a preferred embodiment, please refer to Figure 2 , Figure 2 It is another flow chart of the method for logically processing video data provided by an embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiment, and the specific optimization is to provide a detailed explanation of the implementation process of the above-mentioned "inputting video stream data into the video detection model, obtaining target information in the video stream data and category information and attribute information corresponding to the target information respectively".

[0047] Specifically, the video detection model provided in this embodiment includes a feature extraction layer, a target detection layer, a target classification layer and an attribute recognition layer. The method of this embodiment includes but is not limited to the following steps:

[0048] S110 , extracting features from the video stream data based on the feature extraction layer to obtain video features.

[0049] The video processing system inputs the video stream data into the video detection model. The feature extraction layer extracts image frames from the video stream data at time intervals (e.g., 30 frames per second), adjusts each frame to a uniform size (e.g., 224*224), and performs normalization (subtracting the mean and standard deviation). Furthermore, the feature extraction layer inputs each processed frame into a pre-trained convolutional neural network (e.g., ResNet, Inception) to obtain the feature vector output by the last convolutional layer in the convolutional neural network, which is usually a high-dimensional vector (e.g., 2048 dimensions). The feature extraction layer aggregates the features of all frames into a feature sequence, and processes the feature sequence through a recurrent neural network (RNN) or a long short-term memory network (LSTM) to obtain the video features output by the feature extraction layer. The function formula in the feature extraction layer can be expressed as follows:

[0050]

[0051] Among them, H t represents the video features output by the feature extraction layer, F t Represents the feature vector of the current frame, F t-i represents the feature vector of the previous n frames, φ(·) represents the mapping function of the current frame feature, Aggregation function representing historical frame features; w i Represents the weights of different historical frames, which can be learned through training; T t represents the time information of the current frame; θ(·) represents the time processing function, which is used to convert the time information into features; α, β, γ represent the preset hyperparameters.

[0052] Optionally, the mapping function φ(·) of the current frame feature can be a simple linear transformation or a neural network; the aggregation function of the historical frame feature is It can be an average aggregation function, a weighted aggregation function, etc., and the specific implementation method is not limited here.

[0053] Therefore, for the case where the mapping function φ(·) of the current frame feature is a simple linear transformation, for the feature vector F of the current frame with dimension d t , the feature map can be expressed as:

[0054] φ(F t )=WF t +b (2)

[0055] Among them, W represents the weight matrix, the size of which is m*d (mapping d-dimensional features to m-dimensional); b represents the bias vector, the size of which is m; and m represents the feature dimension after mapping.

[0056] For the case where the mapping function φ(·) of the current frame feature is a neural network, the fully connected neural network may include one or more hidden layers, each of which uses a nonlinear activation function (such as ReLU). Therefore, the feature map can be expressed as:

[0057] φ(F t )=MLP(F t )+Activation(W2*Activation(W1F t +b1)+b2) (3)

[0058] Among them, W1 and W2 represent weight matrices, b1 and b2 represent bias vectors, and Activation(·) represents a nonlinear activation function, such as ReLU function, Sigmoid function, etc.

[0059] S120: Perform target detection on the video features based on the target detection layer to obtain target information.

[0060] The target detection layer performs target detection on the video features to obtain the targets contained in the video stream data output by the target detection layer. Specifically, region-based methods (such as FasterR-CNN) or single-stage methods (such as YOLO) are often used. Taking FasterR-CNN as an example, a region proposal network (RPN) is used to generate region proposals that may contain targets. In one embodiment, the feature map is F, and the RPN predicts the target score and bounding box regression parameters of each position by sliding a small window on the feature map. For each position (i, j), the target score s ij and bounding box regression parameters The calculation can be expressed as:

[0061] s ij =f socre (F(i,j)) (4)

[0062]

[0063] Among them, f socre (·) and f reg (·) are two different convolution operations.

[0064] Furthermore, the video processing system classifies the region proposals and regresses more accurately the bounding boxes to obtain the target information contained in each frame of video data.

[0065] S130: performing category recognition on the target information based on the target classification layer to obtain category information corresponding to the target information.

[0066] The target classification layer classifies the target information contained in the video stream data and obtains the category information of the target information contained in the video stream data output by the target classification layer. Specifically, N targets are detected, and the feature representation of each target is T n (1,2,...,N). Use the fully connected layer and softmax function to calculate the probability that each target belongs to a different category. The number of categories is E, and the probability P that target n belongs to category E is n,E The calculation formula is as follows:

[0067]

[0068] Among them, z n,E =W c T n +b c , W c represents the weight matrix of the fully connected layer, b c Represents the bias vector.

[0069] S140: Perform attribute recognition on the target information based on the attribute recognition layer to obtain attribute information corresponding to the target information.

[0070] The attribute recognition layer recognizes the attributes of the target contained in the video stream data, such as the color and shape of the target, and obtains the attribute information of the target contained in the video stream data output by the attribute recognition layer. Specifically, the feature of the target is represented by A T , the number of attribute categories is M T , the attribute probability is calculated through the fully connected layer and activation function. The probability that the target has attribute m is Q m , the specific calculation formula is as follows:

[0071] Q m =ε(V m A T +d m ) (7)

[0072] Among them, V m represents the weight matrix of the attribute fully connected layer, d m represents the bias vector, and ε represents the activation function, such as the sigmoid function.

[0073] This embodiment processes the video stream data through a pre-trained video detection model. Due to the high performance of the video detection model, the video detection model can quickly process a large amount of video data in a short time, so that the video data can be processed quickly, and the processing efficiency of the video data is improved. At the same time, the video detection model has a unified standard, so in the process of processing video data, each video data is processed according to a unified standard, which ensures the uniformity of the video data standard and improves the classification accuracy of the video data.

[0074] S20: Associating the target information with the category information, the attribute information, the device information of the target video device and the first acquisition time to obtain target association information.

[0075] In this embodiment, the device information of the target video device may include the device number, device type, device location, device name, etc. In this embodiment, the purpose of obtaining the device information is to help locate the data source of the target information. For example, to clarify the location where the target information is generated, etc. The purpose of including the first acquisition time in the target association information is to clarify the generation time of the target information.

[0076] The target association information may be associated in a preset order or in accordance with a pre-defined rule. In one embodiment, the preset order is collection time, category information, target information, attribute information, and device information. Specifically, assuming that the collection time is 22:00 on October 22, 2024, the category information is human, the target is user A, and the attribute information is X (currently X can be represented as a set, and the specific subsets included can be X1, X2, ..., X n ), the device information is device Y. Therefore, the target association information can be expressed as {(collection time: 22:00 on October 22, 2024)-(category information: person)-(target: user A)-(attribute information: X=X1, X2, ..., X n )-(device information: device Y)}, etc. The specific method of obtaining the target related information is not limited here.

[0077] In a preferred embodiment, the target information provided by this embodiment includes first information and / or second information; the category information and attribute information corresponding to the first information are the first category and the first attribute, respectively, and the category information and attribute information corresponding to the second information are the second category and the second attribute, respectively.

[0078] In this embodiment, the first information may be the category information that the target only includes people, and the second information may be the category information that the target does not include people. Accordingly, when the target information only includes the first information, the target-related information is obtained from the first-related information corresponding to the first information; when the target information only includes the second information, the target-related information is obtained from the second-related information corresponding to the second information. In one possible manner, the target information includes both the first information and the second information. For example, if the first acquisition time is 10 seconds, there are time periods when people appear and time periods when no one appears. In this case, the target-related information is obtained from the first-related information and the second-related information.

[0079] Specifically, when the target information corresponds to different types, in order to accurately mark the information of different types of targets, it is necessary to mark and associate different types of targets in different ways. Then, when the target information includes the first information and / or the second information, the implementation of step S20 can be:

[0080] When the target information includes the first information, it is necessary to generate metadata information in advance according to the first attribute and the first preset information; finally, the first information is associated with the first category, the metadata information, the device information and the first collection time to obtain the first associated information.

[0081] In this embodiment, metadata information is an abstract description data that is used to deepen the understanding of the target, realize data association and retrieval, and has the characteristics of flexibility and customization. Therefore, it is not limited to the original directly measurable attributes of the target (such as height, gender, etc.), but can also generate extended attributes based on the measured attribute information and combined with the first preset information. For example, it may also include information on human behavior patterns (such as walking speed, posture characteristics, etc.), social attributes (such as occupation, whether carrying specific items, etc.). These information can be combined with the first preset information to generate metadata information, which can more accurately describe the state and characteristics of a person in the video scene, so as to facilitate subsequent accurate annotation association, such as judging whether the person is a person of a specific identity, whether his behavior meets the requirements of a certain scene, etc.

[0082] When the target information includes the second information, the second information can be directly associated with the second category, the second attribute, the device information and the first acquisition time to obtain the second associated information. For the second category of target information, the video processing system can directly annotate and associate the target according to the target category information and attribute information, as well as the device information of the target video device and the first acquisition time to obtain the second associated information. The annotation process is the same as the first associated information, which will not be described in detail here.

[0083] Finally, the first associated information and / or the second associated information are integrated to obtain the target associated information, that is, the first associated information and / or the second associated information are integrated according to the information content actually contained in the target associated information to obtain the target associated information.

[0084] Another preferred embodiment, the embodiment of the present application is to explain in detail the process of "generating metadata information according to the first attribute and the first preset information" in the above embodiment. Please refer to the following method for specific implementation:

[0085] According to the attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix, the matching degree between the first attribute information and the first preset attribute information is determined; the matching degree between the first attribute information and the first preset attribute information is weighted to obtain weighted attribute information between the first attribute information and the first preset attribute information; metadata information is generated based on the weighted attribute information between the first attribute information and the first preset attribute information.

[0086] Specifically, the video processing system converts the first attribute information into a corresponding attribute vector. For example, the attribute vector can be expressed as Among them, a i represents the attribute value of the i-th attribute information of the target, and n is the dimension of the attribute. For example, the attribute information of user A includes the clothes, clothes color, clothes style, etc. The attribute vector The elements in correspond to the attribute values ​​of the above attribute information.

[0087] The first preset information is an attribute information matrix. Therefore, the first preset information can represent:

[0088]

[0089] Wherein, k represents different categories of the first preset information, and m represents the attribute dimension under each category (m may be equal to or different from n). For example, the first preset information may include weight rules for target attributes in different scenarios, association rules between attributes, etc.

[0090] Furthermore, the video processing system performs attribute matching and weighted calculation, as follows:

[0091] The video processing system calculates the matching degree between the first attribute information and the first preset information, specifically: for the attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix of the first preset information The matching degree s between the first attribute information and the first preset information j The calculation formula is:

[0092]

[0093] Wherein, σ and η represent preset adjustment coefficients.

[0094] Furthermore, the video processing system determines the matching degree w between the first attribute information and the first preset information. j Perform weighted processing to obtain weighted attribute information w j , the specific weighted formula is as follows:

[0095]

[0096] Furthermore, the video processing system generates metadata information according to the weighted attribute information. Specifically, the data vector of the metadata information is represented as It can be calculated by the following formula:

[0097]

[0098] in, Represents a function that transforms a preset attribute vector. For example, if it is a rule vector about attribute weights, Can be a vector of attributes that apply weights to the target Functions on (· denotes the multiplication of corresponding elements.) The size of q depends on the definition dimension of the metadata information.

[0099] Furthermore, the generated metadata information may need to be further integrated and adjusted. For example, if the metadata information contains different types of information (such as text descriptions, numerical features, etc.), it can be integrated into a unified format through a specific mapping function. Let the integration function be g, then the final metadata information can be expressed as Representation converts numerical features into specific text descriptions, concatenates or encodes information of different dimensions, and other operations to meet the subsequent use requirements of metadata information.

[0100] The embodiment of the present application obtains metadata information based on the first attribute information and the first preset information to provide a data basis for the subsequent process, so that the detected target, the category information corresponding to the target, the attribute information or metadata information corresponding to the target, as well as the device information and the acquisition time are annotated and associated and stored in the index database. In subsequent queries, only a certain keyword needs to be searched to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0101] S30: construct a first target index database at a first acquisition time for the video stream data based on the target association information.

[0102] In this embodiment, the obtained target association information is imported into a database, and the target association information is indexed and stored based on a preset index strategy, so as to obtain a first target index database constructed for the video stream data obtained at the first acquisition time. By constructing the first target index database, when it is subsequently applied to data query, it is only necessary to query a certain keyword to obtain all information associated with the keyword, without having to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0103] Among them, the above-mentioned preset index strategy can be to create an index based on the first collection time on the target association information table, so that when querying the target association information within a specific time period, the relevant records can be quickly located; it can also be to create an index based on the category ID on the target association information table, which is helpful to quickly filter out relevant records when querying all target association information belonging to a certain category; it can also be to create an index based on the device ID on the target association information table in order to facilitate the query of all target association information collected by a certain device. The specific method of establishing the preset index strategy is not limited here.

[0104] Another preferred embodiment, please refer to Figure 3 , Figure 3 This is another flow chart of the video data logical processing method provided by the embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiment, and the specific optimization is to explain the implementation process of the above step S30 in detail. When the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information; specifically, in Figure 3 In the above step S130, the above step S130 can be implemented by the following steps S310 to S330:

[0105] S310: Divide the first associated information according to a preset level to construct a first index database.

[0106] For the category information being the first category and the first associated information corresponding to the first category information, the video processing system obtains the metadata information, the device information and the first acquisition time in the first associated information, and constructs a first index tree according to the preset hierarchy based on the first category and the metadata information, the device information and the first acquisition time in the first associated information. Further, the video processing system determines the constructed first index tree as the first index database of the first category. For specific implementation methods, please refer to Figure 4 The corresponding implementation method.

[0107] Among them, the above-mentioned preset levels can be to divide the first information, metadata information, device information and the first acquisition time into layers, such as constructing an index tree with the first acquisition time as the first level, the first information and metadata information as the second level and the device information as the third level, etc. The specific way of establishing the preset levels is not limited to the current example.

[0108] For details, please refer to Figure 4 , Figure 4It is another flow chart of the method for logically processing video data provided by the embodiment of the present application. The embodiment of the present application is to explain in detail the implementation process of the above step S310. Specifically, the implementation process of the above step S310 is as follows: Steps S311 to S313:

[0109] S311, taking the first acquisition time as the root node and the first category as the first-layer child node; determining the first node weight between each target in the first category and the first acquisition time according to the frequency of occurrence of each target in the first category in the first acquisition time; and constructing the connection between the root node and the first-layer child node based on the first node weight corresponding to each target in the first category.

[0110] Specifically, the target information of this embodiment includes multiple targets, with the first collection time as the root node and the first category as the first-layer child node. According to the frequency of occurrence of each target in the first category in the first collection time, the first node weight between each target in the first category and the first collection time is determined, specifically:

[0111] In the embodiment of the present application, the first acquisition time is represented by T1, and the first category set is represented by C1={c 11 ,c 12 ,...,c 1n}. For objective c 1i (i=1,2,...,n) The frequency of occurrence f in the first acquisition time T1 1i . Compute Objective C 1i The first node weight w 1i The formula can be expressed as:

[0112]

[0113] Further, the video processing system constructs a connection between the first layer sub-nodes and the second layer sub-nodes according to the first node weight between each target in the first category and the first acquisition time, wherein the larger the first node weight is, the closer the position of the target information in the first layer sub-nodes is to the root node (acquisition time). The embodiment of the present application can sort the first node weight between each target in the first category and the first acquisition time according to the weight size through a sorting algorithm, and then establish a connection from the root node to each sub-node in the first layer sub-node.

[0114] S312, taking the metadata information corresponding to each target in the first category as the second-layer child node; determining the second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; and constructing the connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category.

[0115] The metadata information of each target of the first category is used as the second-layer child node, and the second node weight between each target of the first category and its metadata information is determined according to the information type of the metadata information of each target of the first category, specifically:

[0116] The first category of metadata information set in the embodiment of the present application is M 1i ={m 1i1 ,m 1i2 ,...,m 1ik}, k represents the number of metadata information in the metadata information set, and the metadata information type set is Type = {t1, t2, ..., t m}. For metadata information m 1ij (j=1,2,...,k), whose information type is t l (l∈{1,2,..m}), where m represents the number of information types.

[0117] Furthermore, the video processing system calculates the frequency of occurrence of the metadata information type in all metadata information of the target. 1il , the specific formula is as follows:

[0118]

[0119] Wherein, Q(·) represents an indicator function, which is used to indicate the number of belonging relationships.

[0120] Furthermore, the video processing system calculates the second node weight between the target and its metadata information, and the specific formula is as follows:

[0121]

[0122] Furthermore, the video processing system connects the node of the metadata information to the node of the corresponding target according to the second node weight between the target and its metadata information. The higher the second node weight is, the closer the metadata information is to the target (first layer subnode) in the second layer subnode. Similarly, the second node weights between the target and its metadata information can be sorted according to the weight size through a sorting algorithm, and then connections are established from the first layer subnode to each subnode in the second layer subnode.

[0123] S313. Use the device information corresponding to each target in the first category as a leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct a first index database.

[0124] The device information of each target of the first category is taken as a leaf-layer child node.

[0125] In the present application, target c 1i Metadata information m 1ij The corresponding device information set is

[0126] D 1ij ={d 1ij1 ,d 1ij2 ,...,d 1ijp}. For nodes with the same device information metadata information, connect them to the same leaf-level child node (device information node). In one embodiment, the connection can be made through a hash table, using device information as a hash key, and nodes with the same device information metadata information are stored in the same hash bucket, and each hash bucket corresponds to a leaf node of device information.

[0127] The embodiment of the present application labels and associates the detected targets, the category information corresponding to the targets, the metadata information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, only a certain keyword needs to be searched to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0128] S320: Divide the second association information according to a preset level to construct a second index database.

[0129] For target information of the second category, the second associated information corresponding to the target information of the second category. Therefore, the video processing system obtains the attribute information, device information and the first acquisition time in the second associated information, and constructs a second index tree according to the target of the second category, and the attribute information, device information and the first acquisition time in the second associated information. Further, the video processing system determines the constructed second index tree as the second index database of the target information of the second category. The specific implementation method can be referred to Figure 5 The corresponding implementation method.

[0130] For details, please refer to Figure 5 , Figure 5 It is another flow chart of the method for logically processing video data provided by the embodiment of the present application. The embodiment of the present application is to explain in detail the implementation process of the above step S320. Specifically, the implementation process of the above step S310 is as follows: Steps S321 to S323:

[0131] S321, taking the first acquisition time as the root node and the second category as the first-layer child node; determining the third node weight between each target in the second category and the first acquisition time according to the frequency of occurrence of each target in the second category in the first acquisition time; and constructing a connection between the root node and the first-layer child node based on the third node weight of each target in the second category.

[0132] Optionally, with the first acquisition time as the root node and the second category as the first-layer child node, the video processing system determines the third node weight between each target of the second category and the first acquisition time according to the frequency of occurrence of each target of the second category in the first acquisition time. Further, the video processing system constructs a connection between the root node and the first-layer child node according to the third node weight of each target of the second category, as specifically described in the embodiment of step S311.

[0133] S322. Take the attribute information corresponding to each target in the second category as the second-layer sub-node; determine the fourth node weight between each target in the second category and the corresponding attribute information according to the similarity between the attribute information corresponding to each target in the second category and the second preset information; and construct the connection between the first-layer sub-node and the second-layer sub-node based on the fourth node weight of each target in the second category.

[0134] Further, with each first attribute information of the second category as a second-layer child node, the video processing system calculates the similarity between each first attribute information of the second category and the second preset attribute information, which can be specifically calculated by the cosine similarity formula and the Euclidean distance formula, which will not be described in detail in the embodiments of the present application. Further, the video processing system determines the fourth node weight between each target of the second category and its attribute information according to the similarity between each first attribute information of the second category and the second preset attribute information. Further, the video processing system constructs a connection between the first-layer child nodes and the second-layer child nodes according to the fourth node weight of each target of the second category. The principle refers to the embodiment of step S312.

[0135] S323. Use the device information corresponding to each target in the second category as a leaf-layer sub-node; connect the second-layer sub-nodes with the same device information to the same leaf-layer sub-node to construct a second index database.

[0136] Furthermore, with the device information of each target of the second category as a leaf-layer sub-node, the video processing system connects the second-layer sub-nodes with the same device information to the same leaf-layer sub-node to construct a second index database. The principle refers to the embodiment of step S313.

[0137] The embodiment of the present application labels and associates the detected targets, the category information corresponding to the targets, the attribute information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, only a certain keyword needs to be searched to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0138] S330: Construct a first target index database at a first acquisition time for the video stream data according to the first index database and / or the second index database.

[0139] The video processing system fuses the first index database and the second index database to generate a first final index database of the target at the first acquisition time. Specifically, the video processing system uses the acquisition time as the standard, and fuses the first index database and the second index database of the same acquisition time according to the acquisition time to obtain the first final index database at the same acquisition time.

[0140] Please refer to Figure 6 , Figure 6 It is a structural diagram of the first target index database provided by an embodiment of the present application; in one embodiment, the first acquisition time is A, the first category includes A1, the second category includes A2, the metadata information of each target in the first category includes A3, A4 and A5, the device information of each target in the first category includes A6 and A7, each first attribute information of the second category includes A8, A9 and A10, and the device information of each target in the second category includes A11. For the first index database of the first category, the acquisition time A is the root node, the target A1 is the first-layer child node, the metadata information A3, A4 and A5 are the second-layer child nodes, the device information A6 and A7 are the leaf-layer child nodes, the node weight between the target A1 and the acquisition time A is 3, the node weights between the target A1 and the metadata information A3, A4 and A5 are 1, 1.5 and 1 respectively, the device information of the metadata information A3 and A4 is the same as A6, and the device information of the metadata information A5 is A7. Therefore, the first index database of the first category refers to Figure 6 shown.

[0141] Please refer to Figure 7 , Figure 7This is another structural diagram of the first target index database provided by an embodiment of the present application. For the second index database of the second category, the acquisition time A is the root node, the target A2 is the first-layer child node, the attribute information A8, A9 and A10 are the second-layer child nodes, the device information A11 is the leaf-layer child node, the node weight between the target A2 and the acquisition time A is 2, the node weights between the target A2 and the attribute information A8, A9 and A10 are 2, 1 and 1.5 respectively, the device information of the attribute information A8, A9 and A10 is the same as A11, and the second index database of the second category of targets is constructed. Furthermore, the first index database and the second index database are merged based on time, and the obtained first target index database is as follows: Figure 7 shown.

[0142] The embodiment of the present application annotates and associates the detected targets, the category information corresponding to the targets, the attribute information or metadata information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, only a certain keyword needs to be searched to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0143] The video data logical processing method provided in the present embodiment first obtains the target information in the video stream data and the category information and attribute information respectively corresponding to the target information through the video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, the target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be queried to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of the video data.

[0144] Another preferred embodiment, please refer to Figure 8 , Figure 8 This is another flow chart of the video data logical processing method provided by the embodiment of the present application. After obtaining the first target index database, the solution provided by the embodiment of the present application can also perform the following steps S40 to S60:

[0145] S40: Acquire a second target index database obtained at a second acquisition time.

[0146] The first target index database and the second target index database in the embodiment of the present application have the same database structure. The first acquisition time and the second acquisition time can be different acquisition times of the same video device, the same acquisition time of different video devices, or different acquisition times of different video devices.

[0147] Optionally, the video processing system obtains a second target index database obtained at a second acquisition time.

[0148] S50, determining the node similarity between the first child node in the first target index database and the second child node in the second target index database; the category information to which the first child node and the second child node belong is the same, and the number of child node layers of the first child node in the first target index database is the same as the number of child node layers of the second child node in the second target index database.

[0149] First, the video processing system traverses the first final index data first target index database library and the second final index database second target index database to find the first child node and the second child node pair that satisfy the same category information and the same number of sub-node layers, that is, the category information of the first child node in the first final index data first target index database library is the same as the category information of the second child node in the second final index database second target index database, and the number of sub-node layers of the first child node in the first final index data first target index database library is the same as the number of sub-node layers of the second child node in the second final index database second target index database.

[0150] In the embodiment of the present application, the first child node N1 is in the first target index database library, the second child node N2 is in the second target index database, the category to which the first child node N1 and the second child node N2 belong is C, and the number of child node layers is L. Therefore, the node similarity between the first child node in the first target index database library and the second child node in the second target index database is determined, and the specific formula is:

[0151] The node similarity formula for the first category is:

[0152] Sim(N1,N2)=R1*Sim attr (N1,N2)+R2*Sim device (N1,N2) (15)

[0153] The similarity formula for the second category of nodes is:

[0154] Sim(N1,N2)=R3*Sim meta (N1,N2)+R2*Sim device (N1,N2) (16)

[0155] Where R1, R2, and R3 are preset weights, R1+R2+R3=1, Sim(N1, N2) represents the node similarity between the first child node and the second child node, Sim attr (N1, N2) represents attribute similarity, Sim meta (N1, N2) represents metadata similarity, Sim device (N1, N2) represents the device similarity.

[0156] For attribute similarity Sim attr (N1,N2):

[0157] The target attribute vector corresponding to the first child node N1 is The target attribute vector corresponding to the second child node N2 is Consider the importance weight vector of the attribute Therefore, the attribute similarity Sim attr The calculation formula of (N1, N2) can be expressed as:

[0158]

[0159] Therefore, the attribute similarity is more discriminative in the interval (0,1).

[0160] For metadata similarity Sim meta (N1,N2):

[0161] The source data set corresponding to the first child node N1 is M1={m 11 ,m 12 ,...,m 1k}, the source data set corresponding to the second child node N2 is M2 = {m 21 ,m 22 ,...,m 2k}, therefore, the metadata similarity Sim meta The calculation formula of (N1, N2) can be expressed as:

[0162]

[0163] Among them, v i represents the preset weight, TF_IDF(·) is used to calculate the importance of metadata in its respective node metadata set. Therefore, for TF_IDF(m 1i ,M1)=TF(m 1i ,M1)*IDF(m 1i ,M1):

[0164]

[0165] Among them, n m1iIndicates m 1i The number of times it appears in M1, N represents the total number of targets, Indicates that it contains m 1i The target number.

[0166] For device similarity Sim device (N1,N2):

[0167] The target attribute vector corresponding to the first child node N1 is D1=(d 11 ,d 12 ,..,d 1p ), the target attribute vector corresponding to the second child node N2 is D2=(d 21 ,d 22 ,..,d 2p ), the similarity is measured by calculating the matching degree of device information. Therefore, the device similarity Sim device The calculation formula of (N1, N2) can be expressed as:

[0168]

[0169] Wherein, Q(·) represents an indicator function, which is used to indicate the number of belonging relationships.

[0170] S60: Connect the first subnode in the first target index database and the second subnode in the second target index database based on node similarity to construct a multi-link index database at the first collection time and the second collection time.

[0171] The database structures of the first target index database and the second target index database are the same.

[0172] Furthermore, the similarity threshold μ (0<μ<1), for each pair of the first child node and the second child node that satisfy the same category information and the same number of child node layers, if the similarity Sim(N1, N2) ≥ μ between the two, a connection is established between the two to construct a multi-link index database at different times.

[0173] In one embodiment, the first target index database is the final index database at the first acquisition time A, and the second target index database is the final index database at the second acquisition time B. For node A1 in the first target index database and node B1 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A8 in the first target index database and node B9 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A10 in the first target index database and node B8 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. Therefore, the final multi-link index database is as follows: Figure 4As shown, Figure 4 It is a schematic diagram of the structure of the multi-link index database provided in an embodiment of the present application.

[0174] Please refer to Fig. 9 , Fig. 9 It is a structural diagram of a multi-link index database provided in an embodiment of the present application. In one embodiment, the first target index database library is the final index database at the first acquisition time A, and the second target index database is the final index database at the second acquisition time B. For node A1 in the first target index database library and node B1 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A8 in the first target index database library and node B9 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A10 in the first target index database library and node B8 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. Therefore, the final multi-link index database is as follows Fig. 9 shown.

[0175] The embodiment of the present application connects the nodes in multiple final index databases whose similarities meet preset conditions to construct a multi-link index database. In subsequent queries, only a certain keyword needs to be queried to obtain all information associated with the keyword. There is no need to watch the video data analysis again, which improves the query efficiency of the video data.

[0176] Fig.10 1 is a schematic diagram of a structure of a video data logical processing device provided in an embodiment of the present application, and the device is suitable for executing the video data logical processing method provided in an embodiment of the present application. Fig.10 As shown, the device may specifically include: a data acquisition module 410, an information association module 420 and a database construction module 430, wherein:

[0177] The data acquisition module 410 is used to acquire the video stream data acquired by the target video device at the first acquisition time, input the video stream data into the video detection model, and obtain the target information in the video stream data and the category information and attribute information respectively corresponding to the target information;

[0178] An information association module 420, configured to associate the target information with the category information, the attribute information, the device information of the target video device and the first acquisition time to obtain target association information;

[0179] The database construction module 430 is used to construct a first target index database at the first acquisition time for the video stream data based on the target association information.

[0180] The video data logical processing device provided in the embodiment of the present application first obtains the target information in the video stream data and the category information and attribute information respectively corresponding to the target information through a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, the target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be queried to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of the video data.

[0181] In one embodiment, the target information includes first information and / or second information; the category information and attribute information corresponding to the first information are respectively the first category and the first attribute, and the category information and attribute information corresponding to the second information are respectively the second category and the second attribute.

[0182] The information association module 420 is specifically used to generate metadata information based on the first attribute and the first preset information; associate the first information with the first category, the metadata information, the device information and the first collection time to obtain first association information; associate the second information with the second category, the second attribute, the device information and the first collection time to obtain second association information; obtain the target association information based on the first association information and / or the second association information.

[0183] In one embodiment, when the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information.

[0184] The database construction module 430 is used to divide the first associated information according to a preset level to construct a first index database; divide the second associated information according to a preset level to construct a second index database; based on the first index database, and / or the second index database, construct the first target index database for the video stream data at the first acquisition time.

[0185] In one embodiment, the target information includes multiple targets.

[0186] The database construction module 430 is specifically used to take the first acquisition time as the root node and the first category as the first-layer child node; determine the first node weight between each target in the first category and the first acquisition time according to the frequency of occurrence of each target in the first category within the first acquisition time; construct the connection between the root node and the first-layer child node based on the first node weight corresponding to each target in the first category; take the metadata information corresponding to each target in the first category as the second-layer child node; determine the second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; construct the connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category; take the device information corresponding to each target in the first category as the leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct the first index database.

[0187] In one embodiment, the database construction module 430 is specifically used to take the first acquisition time as the root node and the second category as the first-layer child node; determine the third node weight between each target in the second category and the first acquisition time according to the frequency of occurrence of each target in the second category in the first acquisition time; construct a connection between the root node and the first-layer child node based on the third node weight of each target in the second category; take the attribute information corresponding to each target in the second category as the second-layer child node; determine the fourth node weight between each target in the second category and the corresponding attribute information according to the similarity between the attribute information corresponding to each target in the second category and the second preset information; construct a connection between the first-layer child node and the second-layer child node based on the fourth node weight of each target in the second category; take the device information corresponding to each target in the second category as the leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct the second index database.

[0188] In one embodiment, the device further comprises: a database acquisition module, a similarity determination module and a node connection module, wherein:

[0189] A database acquisition module, used to acquire a second target index database of the target at a second acquisition time;

[0190] a similarity determination module, configured to determine a node similarity between a first child node in the first target index database and a second child node in the second target index database; the first child node and the second child node have the same category information, and the number of child node layers of the first child node in the first target index database is the same as the number of child node layers of the second child node in the second target index database;

[0191] A node connection module is used to connect the first subnode in the first target index database and the second subnode in the second target index database based on the node similarity to construct a multi-link index database at the first acquisition time and the second acquisition time; wherein the database structure of the first target index database and the second target index database is the same.

[0192] In one embodiment, the first preset attribute information is an attribute information matrix; the information association module 920 includes a matching degree determination unit, a weighted processing unit and a metadata generation unit, wherein:

[0193] a matching degree determining unit, configured to determine a matching degree between the first attribute information and the first preset attribute information according to an attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix;

[0194] a weighted processing unit, configured to perform weighted processing on the matching degree between the first attribute information and the first preset attribute information to obtain weighted attribute information between the first attribute information and the first preset attribute information;

[0195] The metadata generating unit is configured to generate the metadata information based on weighted attribute information between the first attribute information and the first preset attribute information.

[0196] In one embodiment, the video detection model includes a feature extraction layer, a target detection layer, a target classification layer and an attribute recognition layer.

[0197] The data acquisition module 410 is specifically used to perform feature extraction on the video stream data based on the feature extraction layer to obtain video features; perform target detection on the video features based on the target detection layer to obtain the target information; perform category recognition on the target information based on the target classification layer to obtain category information corresponding to the target information; perform attribute recognition on the target information based on the attribute recognition layer to obtain attribute information corresponding to the target information.

[0198] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0199] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data logical processing method described in any embodiment of the present application.

[0200] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video data logical processing method described in any embodiment of the present application when executed.

[0201] Reference below Fig.11 , Fig.11 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and shows a schematic diagram of the structure of a computer system 500 suitable for implementing the electronic device in an embodiment of the present application. Fig.11 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0202] like Fig.11 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0203] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0204] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0205] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the above.

[0206] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] The modules and / or units involved in the embodiments described in the present application may be implemented by software or hardware. The modules and / or units described may also be arranged in a processor, for example, it may be described as follows: a processor includes a data acquisition module, an information association module, and a database construction module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0208] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: obtaining video stream data collected by the target video device at the first collection time, inputting the video stream data into the video detection model, obtaining the target information in the video stream data and the category information and attribute information corresponding to the target information respectively; associating the target information with the category information, the attribute information, the device information of the target video device and the first collection time to obtain target association information; and constructing a first target index database at the first collection time for the video stream data based on the target association information.

[0209] According to the technical solution of this embodiment, firstly, the target information in the video stream data and the category information and attribute information respectively corresponding to the target information are obtained through a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, the target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be queried to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of the video data.

[0210] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for logically processing video data, characterized in that: include: Acquire video stream data collected by the target video device at a first collection time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information respectively corresponding to the target information; Associating the target information with the category information, the attribute information, the device information of the target video device and the first acquisition time to obtain target association information; A first target index database at the first acquisition time is constructed for the video stream data based on the target association information.

2. The video data logical processing method according to claim 1, characterized in that: The target information includes first information and / or second information; the category information and attribute information corresponding to the first information are respectively a first category and a first attribute, and the category information and attribute information corresponding to the second information are respectively a second category and a second attribute; The step of associating the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information includes: Generate metadata information according to the first attribute and first preset information; associate the first information with the first category, the metadata information, the device information and the first collection time to obtain first association information; Associating the second information with the second category, the second attribute, the device information, and the first collection time to obtain second associated information; The target association information is obtained according to the first association information and / or the second association information.

3. The video data logical processing method according to claim 2, characterized in that: When the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information; Constructing a first target index database at the first acquisition time for the video stream data based on the target association information includes: Dividing the first associated information according to a preset level to construct a first index database; Dividing the second association information according to a preset level to construct a second index database; According to the first index database, and / or the second index database, the first target index database is constructed for the video stream data at the first acquisition time.

4. The video data logical processing method according to claim 3, characterized in that: The target information includes a plurality of targets; and the first associated information is divided according to a preset level to construct a first index database, including: Taking the first acquisition time as a root node and the first category as a first-layer child node; determining a first node weight between each target in the first category and the first acquisition time according to the occurrence frequency of each target in the first category in the first acquisition time; and building a connection between the root node and the first-layer child nodes based on the first node weight corresponding to each target in the first category; Taking metadata information corresponding to each target in the first category as a second-layer child node; determining a second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; and constructing a connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category; The device information corresponding to each target in the first category is used as a leaf-layer sub-node; the second-layer sub-nodes with the same device information are connected to the same leaf-layer sub-node to construct the first index database.

5. The video data logical processing method according to claim 3, characterized in that: The target information includes a plurality of targets; and the second associated information is divided according to a preset level to construct a second index database, including: Taking the first acquisition time as a root node and the second category as a first-layer child node; determining a third node weight between each target in the second category and the first acquisition time according to the occurrence frequency of each target in the second category in the first acquisition time; and constructing a connection between the root node and the first-layer child node based on the third node weight of each target in the second category; The attribute information corresponding to each target in the second category is used as a second-layer sub-node; according to the similarity between the attribute information corresponding to each target in the second category and the second preset information, a fourth node weight between each target in the second category and the corresponding attribute information is determined; and a connection between the first-layer sub-node and the second-layer sub-node is established based on the fourth node weight of each target in the second category; The device information corresponding to each target in the second category is used as a leaf-layer sub-node; the second-layer sub-nodes with the same device information are connected to the same leaf-layer sub-node to construct the second index database.

6. The video data logical processing method according to claim 1, characterized in that: After obtaining the first target index database, the method further includes: Acquire a second target index database obtained at a second acquisition time; Determine the node similarity between a first child node in the first target index database and a second child node in the second target index database; the category information to which the first child node and the second child node belong is the same, and the number of child node layers of the first child node in the first target index database is the same as the number of child node layers of the second child node in the second target index database; Connecting a first subnode in the first target index database and a second subnode in the second target index database based on the node similarity to construct a multi-link index database at the first acquisition time and at the second acquisition time; The database structures of the first target index database and the second target index database are the same.

7. The video data logical processing method according to claim 2, characterized in that: The first preset attribute information is an attribute information matrix; The generating metadata information according to the first attribute and the first preset information includes: determining a matching degree between the first attribute information and the first preset attribute information according to an attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix; Performing weighted processing on the matching degree between the first attribute information and the first preset attribute information to obtain weighted attribute information between the first attribute information and the first preset attribute information; The metadata information is generated based on weighted attribute information between the first attribute information and the first preset attribute information.

8. The video data logical processing method according to claim 1, characterized in that: The video detection model includes a feature extraction layer, a target detection layer, a target classification layer and an attribute recognition layer; The step of inputting the video stream data into a video detection model to obtain target information in the video stream data and category information and attribute information respectively corresponding to the target information includes: Performing feature extraction on the video stream data based on the feature extraction layer to obtain video features; Performing target detection on the video features based on the target detection layer to obtain the target information; Performing category identification on the target information based on the target classification layer to obtain category information corresponding to the target information; Attributes of the target information are identified based on the attribute identification layer to obtain attribute information corresponding to the target information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data logical processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for logically processing video data as described in any one of claims 1 to 8 is implemented.

Images