Video data logical processing method, electronic device and storage medium

Abstract

The application discloses a video data logical processing method, an electronic device and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: acquiring video stream data collected by a target video device at a first collection time; inputting the video stream data into a video detection model to obtain target information in the video stream data and category information and attribute information corresponding to the target information respectively; associating the target information with the category information, the attribute information, device information of the target video device and the first collection time to obtain target association information; and constructing a first target index database of the video stream data at the first collection time based on the target association information. The application can improve the processing efficiency of video data, and can also enable the video stream data to be stored in a target index mode, so that all information associated with a keyword can be obtained through keyword search during subsequent query, and the beneficial effect of improving the query efficiency of video data is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method for logically processing video data, an electronic device, and a storage medium. Background Art

[0002] With the development of science and technology, intelligent monitoring has been widely used in many fields and has become a mainstream monitoring method. Generally, after obtaining video stream data, the video stream data will be stored to facilitate viewing of the video data.

[0003] Existing video storage methods typically store large amounts of captured raw video data in a fixed location based on a time series. Consequently, when a query is needed, the database must be queried one by one from all stored video streams to retrieve the target video. The target video is then played back to extract the target information through frame-by-frame parsing. This method of storing video data is inconvenient for viewing, and the query workload is high, resulting in low query efficiency. Summary of the Invention

[0004] The present application provides a method for logically processing video data, an electronic device, and a storage medium, which can improve existing solutions for processing video data.

[0005] In a first aspect, the present application provides a method for logically processing video data, comprising:

[0006] Obtaining video stream data collected by a target video device at a first collection time, inputting the video stream data into a video detection model, and obtaining target information in the video stream data and category information and attribute information corresponding to the target information;

[0007] Associating the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information;

[0008] A first target index database at the first acquisition time is constructed for the video stream data based on the target association information.

[0009] In a second aspect, the present application provides a device for logically processing video data, the device comprising:

[0010] A data acquisition module is used to acquire video stream data acquired by a target video device at a first acquisition time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information corresponding to the target information;

[0011] an information association module, configured to associate the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information;

[0012] A database construction module is used to construct a first target index database at the first acquisition time for the video stream data based on the target association information.

[0013] In a third aspect, the present application further provides an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data logical processing method described in any embodiment of the present application.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video data logical processing method described in any embodiment of the present application when executed.

[0018] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the video data logical processing method described in any embodiment of the present application.

[0019] The video data logical processing solution provided by the embodiment of the present application first obtains the target information in the video stream data and the category information and attribute information corresponding to the target information through a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be searched to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of video data.

[0020] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the video data logical processing device, or may be packaged separately from the processor of the video data logical processing device, and this application does not limit this.

[0021] The descriptions of the second, third, fourth and fifth aspects of this application can refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth and fifth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0023] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 This is a flow chart of a method for logically processing video data provided by an embodiment of the present application;

[0026] Figure 2 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application;

[0027] Figure 3 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application;

[0028] Figure 4 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application;

[0029] Figure 5 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application;

[0030] Figure 6This is a schematic diagram of the structure of the first target index database provided in an embodiment of the present application;

[0031] Figure 7 is another structural diagram of the first target index database provided in an embodiment of the present application;

[0032] Figure 8 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application;

[0033] Figure 9 This is a schematic diagram of the structure of a multi-link index database provided in an embodiment of the present application;

[0034] Figure 10 This is a structural diagram of a video data logical processing device provided by an embodiment of the present application;

[0035] Figure 11 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the present invention, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this embodiment. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the present application, not all of the structures.

[0039] Figure 1This is a flow chart of a method for logically processing video data provided in an embodiment of the present application. This embodiment is applicable to processing video data collected by a video device, constructing an index database, and facilitating data storage and query. The method can be performed by a video data logical processing device, which can be implemented in hardware and / or software and integrated into the electronic device that performs the method. Preferably, the electronic device in the embodiment of the present application can be a server, a computer, or the like.

[0040] refer to Figure 1 The video data logical processing method of this embodiment includes but is not limited to the following steps:

[0041] S10: Obtain video stream data collected by the target video device at the first collection time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information corresponding to the target information.

[0042] In an embodiment of the present application, a video processing system may be integrated into the server, and the video processing system may be connected to a plurality of video devices, so that the video stream data collected by the plurality of video devices may be obtained in real time. The target video device is any one of the plurality of video devices. The video processing system obtains the video stream data collected by any one of the plurality of video devices at a first collection time. The first collection time is the time when the target video device collects the video stream of a certain area or a certain target, and the first collection time may be a time point or a time period. For example, the first collection time may be xx:00 on xx / xx / 20xx, or the first collection time may be A:00 on xx / xx / 20xx - B:00 on xx / xx / 20xx, etc. The specific time corresponding to the first collection time is not limited here.

[0043] The video processing system provided in this embodiment is equipped with a pre-trained video detection model. The video detection model is trained based on sample video stream data, labeled targets in the sample video stream data, and the category labels and attribute labels of the labeled targets. Therefore, the video detection model performs video detection on the input video stream data to obtain target information in the video stream data output by the video detection model. The current target information is used to indicate the content contained in the video stream. It is easy to imagine that in a video stream, when the target video device is fixed, the target information in the video stream data collected by the target video device in the first acquisition time will change. For example, taking the first acquisition time as 30 seconds, an adult may pass by in the 0-5 second range; a kitten may be captured in the 5-15 second range; and multiple cars may pass by in the 15-30 second range. The content contained in the specific target information is not limited to the current example.

[0044] In this embodiment, the category information is used to indicate the class to which the current target information belongs. For example, the current category information can be divided into a first category and a second category based on whether the video stream contains people. Alternatively, the category information can be divided based on the content of the video stream, such as into categories containing only people, only objects, and both people and objects. The specific basis for dividing the category information and the number of categories are not limited herein.

[0045] Attribute information is used to indicate the attributes corresponding to the current target information in the corresponding category. For example, if the target information is a person, the corresponding attribute information may include color information, appearance information, clothing information, height information, etc.; if the target information is an animal, the corresponding attribute information may include the animal's breed, hair color, hair length, etc.; if the target information is a vehicle, the corresponding attribute information may include the vehicle's color, driving status, size, and whether it is a new energy vehicle, etc. The specific attribute content contained in different categories of information is not limited here.

[0046] For a preferred embodiment, please refer to Figure 2 , Figure 2 It is another flow chart of the method for logically processing video data provided by an embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiment. The specific optimization is to provide a detailed explanation of the implementation process of the above-mentioned "inputting video stream data into the video detection model to obtain target information in the video stream data and category information and attribute information corresponding to the target information respectively".

[0047] Specifically, the video detection model provided in this embodiment includes a feature extraction layer, a target detection layer, a target classification layer, and an attribute recognition layer. The method of this embodiment includes but is not limited to the following steps:

[0048] S110 , performing feature extraction on the video stream data based on the feature extraction layer to obtain video features.

[0049] The video processing system inputs the video stream data into the video detection model. The feature extraction layer extracts image frames from the video stream data at time intervals (e.g., 30 frames per second), resizes each frame to a uniform size (e.g., 224*224), and performs normalization (subtracting the mean and standard deviation). Furthermore, the feature extraction layer inputs each processed frame into a pre-trained convolutional neural network (e.g., ResNet, Inception) to obtain the feature vector output by the last convolutional layer in the convolutional neural network. This feature vector is usually a high-dimensional vector (e.g., 2048 dimensions). The feature extraction layer aggregates the features of all frames into a feature sequence and processes the feature sequence through a recurrent neural network (RNN) or a long short-term memory network (LSTM) to obtain the video features output by the feature extraction layer. The function formula in the feature extraction layer can be expressed as follows:

[0050]

[0051] Among them, H t represents the video features output by the feature extraction layer, F t Represents the feature vector of the current frame, F t-i represents the feature vector of the previous n frames, φ(·) represents the mapping function of the current frame feature, Aggregation function representing historical frame features; w i Represents the weights of different historical frames, which can be learned through training; T t represents the time information of the current frame; θ(·) represents the time processing function, which is used to convert the time information into features; α, β, γ represent the preset hyperparameters.

[0052] Optionally, the mapping function φ(·) of the current frame feature can be a simple linear transformation or a neural network; the aggregation function of the historical frame feature is It can be an average aggregation function, a weighted aggregation function, etc., and the specific implementation method is not limited here.

[0053] Therefore, for the case where the mapping function φ(·) of the current frame feature is a simple linear transformation, for the feature vector F of the current frame with dimension d t , the feature map can be expressed as:

[0054] φ(F t )=WF t +b (2)

[0055] Among them, W represents the weight matrix, the size of which is m*d (mapping d-dimensional features to m-dimensional); b represents the bias vector, the size of which is m; and m represents the feature dimension after mapping.

[0056] For the case where the mapping function φ(·) of the current frame feature is a neural network, the fully connected neural network can include one or more hidden layers, each of which uses a nonlinear activation function (such as ReLU). Therefore, the feature map can be expressed as:

[0057] φ(F t )=MLP(F t )+Activation(W2*Activation(W1F t +b1)+b2) (3)

[0058] Among them, W1 and W2 represent weight matrices, b1 and b2 represent bias vectors, and Activation(·) represents a nonlinear activation function, such as the ReLU function and the Sigmoid function.

[0059] S120: Perform target detection on the video features based on the target detection layer to obtain target information.

[0060] The target detection layer performs target detection on the video features to obtain the targets contained in the video stream data output by the target detection layer. Specifically, a region-based method (such as FasterR-CNN) or a single-stage method (such as YOLO) is often used. Taking FasterR-CNN as an example, a region proposal network (RPN) is used to generate region proposals that may contain targets. In one embodiment, the feature map is F, and the RPN predicts the target score and bounding box regression parameters of each position by sliding a small window on the feature map. For each position (i, j), the target score s ij and bounding box regression parameters The calculation can be expressed as:

[0061] s ij =f socre (F(i,j)) (4)

[0062]

[0063] Among them, f socre (·) and f reg (·) are two different convolution operations.

[0064] Furthermore, the video processing system classifies the region proposals and regresses more accurately the bounding boxes to obtain the target information contained in each frame of video data.

[0065] S130 , performing category recognition on the target information based on the target classification layer to obtain category information corresponding to the target information.

[0066] The target classification layer classifies the target information contained in the video stream data and obtains the category information of the target information contained in the video stream data output by the target classification layer. Specifically, N targets are detected, and the feature representation of each target is T n (1,2,...,N). Use the fully connected layer and softmax function to calculate the probability that each target belongs to a different category. The number of categories is E, and the probability P of target n belonging to category E is n,E The calculation formula is as follows:

[0067]

[0068] Among them, z n,E =W c T n +b c , W c represents the weight matrix of the fully connected layer, b c Represents the bias vector.

[0069] S140: Perform attribute recognition on the target information based on the attribute recognition layer to obtain attribute information corresponding to the target information.

[0070] The attribute recognition layer identifies the attributes of the target contained in the video stream data, such as the color and shape of the target, and obtains the attribute information of the target contained in the video stream data output by the attribute recognition layer. Specifically, the feature of the target is represented by A T , the number of attribute categories is M T , the attribute probability is calculated through the fully connected layer and activation function. The probability that the target has attribute m is Q m , the specific calculation formula is as follows:

[0071] Q m =ε(V m A T +d m ) (7)

[0072] Among them, V m Represents the weight matrix of the attribute fully connected layer, d m represents the bias vector, and ε represents the activation function, such as the sigmoid function.

[0073] This embodiment processes video stream data using a pre-trained video detection model. Due to the high performance of the video detection model, it can quickly process large amounts of video data in a short period of time, thereby rapidly processing the video data and improving processing efficiency. Furthermore, the video detection model is standardized, so during the video data processing process, each video data is processed according to the same standard, ensuring the uniformity of the video data standard and improving the accuracy of video data classification.

[0074] S20: Associating the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information.

[0075] In this embodiment, the device information of the target video device may include the device number, device type, device location, device name, etc. In this embodiment, the purpose of obtaining device information is to facilitate locating the data source of the target information. For example, it can clarify the location where the target information was generated. The purpose of including the first acquisition time in the target-related information is to clarify the time when the target information was generated.

[0076] The target association information may be associated in a preset order or in accordance with a pre-defined rule. In one embodiment, the preset order is collection time, category information, target information, attribute information, and device information. Specifically, assuming that the collection time is 22:00 on October 22, 2024, the category information is human, the target is user A, and the attribute information is X (currently X can be represented as a set, and the specific subsets included can be X1, X2, ..., X n ), the device information is device Y. Therefore, the target association information can be expressed as {(collection time: 22:00 on October 22, 2024)-(category information: person)-(target: user A)-(attribute information: X=X1, X2, ..., X n )-(device information: device Y)}, etc. The specific method of obtaining target association information is not limited here.

[0077] In a preferred embodiment, the target information provided by this embodiment includes first information and / or second information; the category information and attribute information corresponding to the first information are the first category and the first attribute respectively, and the category information and attribute information corresponding to the second information are the second category and the second attribute respectively.

[0078] In this embodiment, the first information can be categorized as indicating that the target only contains people, while the second information can be categorized as indicating that the target does not contain people. Accordingly, when the target information only contains the first information, the target-related information is derived from the first-related information corresponding to the first information; when the target information only contains the second information, the target-related information is derived from the second-related information corresponding to the second information. In one possible scenario, where the target information contains both the first and second information, for example, if the first acquisition time is 10 seconds and there are periods of time when people are present and periods when no one is present, the target-related information is derived from the first and second-related information.

[0079] Specifically, when the target information corresponds to different types, in order to accurately label the information of different types of targets, it is necessary to label and associate the different types of targets in different ways. Then, when the target information includes the first information and / or the second information, the above step S20 can be implemented as follows:

[0080] When the target information includes the first information, metadata information needs to be generated in advance according to the first attribute and the first preset information; finally, the first information is associated with the first category, metadata information, device information and the first collection time to obtain first associated information.

[0081] In this embodiment, metadata information is abstract descriptive data used to deepen understanding of the target, enable data association and retrieval, and is flexible and customizable. Therefore, it is not limited to the target's original, directly measurable attributes (such as height and gender). Based on the measured attribute information, extended attributes can be generated by combining the first preset information. For example, it may also include information on the person's behavioral patterns (such as walking speed and posture characteristics) and social attributes (such as occupation and whether they are carrying specific items). This information, combined with the first preset information to generate metadata, can more accurately portray the state and characteristics of a person in the video scene, facilitating subsequent accurate labeling and association, such as determining whether the person has a specific identity or whether their behavior meets the requirements of a specific scene.

[0082] When the target information includes second information, the second information can be directly associated with the second category, second attribute, device information, and first acquisition time to obtain second associated information. For the second category of target information, the video processing system can directly annotate and associate the target based on the target category information and attribute information, as well as the device information of the target video device and the first acquisition time, to obtain the second associated information. The annotation process is similar to that for the first associated information and is not further described here.

[0083] Finally, the first association information and / or the second association information are combined to obtain the target association information. That is, the first association information and / or the second association information are integrated according to the information content actually contained in the target association information to obtain the target association information.

[0084] Another preferred embodiment of the present application is to explain in detail the process of "generating metadata information according to the first attribute and the first preset information" in the above embodiment. Please refer to the following for specific implementation methods:

[0085] Determine the degree of match between the first attribute information and the first preset attribute information based on the attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix; perform weighted processing on the degree of match between the first attribute information and the first preset attribute information to obtain weighted attribute information between the first attribute information and the first preset attribute information; and generate metadata information based on the weighted attribute information between the first attribute information and the first preset attribute information.

[0086] Specifically, the video processing system converts the first attribute information into a corresponding attribute vector. For example, the attribute vector can be expressed as Among them, a i Represents the attribute value of the i-th attribute information of the target, and n is the dimension of the attribute. For example, the attribute information of user A includes the clothes, clothes color, clothes style, etc. The attribute vector The elements in correspond to the attribute values ​​of the above attribute information.

[0087] The first preset information is an attribute information matrix. Therefore, the first preset information can represent:

[0088]

[0089] Here, k represents different categories of the first preset information, and m represents the attribute dimension under each category (m may be equal to or different from n). For example, the first preset information may include weight rules for target attributes in different scenarios, association rules between attributes, etc.

[0090] Furthermore, the video processing system performs attribute matching and weighted calculations as follows:

[0091] The video processing system calculates the matching degree between the first attribute information and the first preset information, specifically: for the attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix of the first preset information The matching degree s between the first attribute information and the first preset information j The calculation formula is:

[0092]

[0093] Wherein, σ and η represent preset adjustment coefficients.

[0094] Furthermore, the video processing system determines the matching degree w between the first attribute information and the first preset information. j Perform weighted processing to obtain weighted attribute information w j , the specific weighted formula is as follows:

[0095]

[0096] Furthermore, the video processing system generates metadata information based on the weighted attribute information. Specifically, the data vector of the metadata information is represented as It can be calculated by the following formula:

[0097]

[0098] in, Represents a function that transforms a preset attribute vector. For example, if it is a rule vector about attribute weights, Can be a vector of attributes that apply weights to the target Functions on (· denotes the multiplication of corresponding elements.) The size of q depends on the definition dimension of the metadata information.

[0099] Furthermore, the generated metadata information may need to be further integrated and adjusted. For example, if the metadata information contains different types of information (such as text descriptions, numerical features, etc.), it can be integrated into a unified format through a specific mapping function. Let the integration function be g, then the final metadata information can be expressed as Representation converts numerical features into specific text descriptions, splices or encodes information of different dimensions, and other operations to meet the subsequent use requirements of metadata information.

[0100] The embodiment of the present application obtains metadata information based on the first attribute information and the first preset information, providing a data basis for the subsequent process, so that the detected target, the category information corresponding to the target, the attribute information or metadata information corresponding to the target, as well as the device information and the acquisition time are labeled and associated and stored in the index database. In subsequent queries, it is only necessary to query a certain keyword to obtain all information associated with the keyword, without having to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0101] S30: Construct a first target index database at a first acquisition time for the video stream data based on the target association information.

[0102] In this embodiment, the acquired target-related information is imported into a database and indexed and stored based on a preset indexing strategy, thereby generating a first target index database constructed for the video stream data acquired at the first acquisition time. This first target index database allows subsequent data queries to retrieve all information associated with a keyword simply by searching for that keyword, eliminating the need to re-analyze the video data, thereby improving video data query efficiency.

[0103] The preset indexing strategy may include creating an index based on the first collection time in the target association information table, so that when querying target association information within a specific time period, relevant records can be quickly located; creating an index based on the category ID in the target association information table, which helps to quickly filter out relevant records when querying all target association information belonging to a certain category; and creating an index based on the device ID in the target association information table to facilitate querying all target association information collected by a certain device. The specific method for establishing the preset indexing strategy is not limited here.

[0104] Another preferred embodiment, please refer to Figure 3 , Figure 3 This is another flow chart of the video data logical processing method provided by the embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiment, and the specific optimization is to explain the implementation process of the above step S30 in detail. When the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information; specifically, when Figure 3 In the above step S130, the above step S310 can be implemented by the following steps S310 to S330:

[0105] S310: Divide the first associated information according to a preset hierarchy to construct a first index database.

[0106] For the first category of the category information and the first associated information corresponding to the first category information, the video processing system obtains the metadata information, device information and first acquisition time in the first associated information, and constructs a first index tree according to the preset hierarchy based on the first category and the metadata information, device information and first acquisition time in the first associated information. Further, the video processing system determines the constructed first index tree as the first index database of the first category. For specific implementation methods, please refer to Figure 4 The corresponding implementation method.

[0107] Among them, the above-mentioned preset levels can be to divide the first information, metadata information, device information and the first acquisition time into layers respectively, such as constructing an index tree with the first acquisition time as the first level, the first information and metadata information as the second level and the device information as the third level, etc. The specific method of establishing the preset levels is not limited to the current example.

[0108] For details, please refer to Figure 4 , Figure 4This is another flow chart of the method for logically processing video data provided by an embodiment of the present application. The embodiment of the present application provides a detailed explanation of the implementation process of the above step S310. Specifically, the implementation process of the above step S310 is explained in detail in the following steps S311 to S313:

[0109] S311. Take the first acquisition time as the root node and the first category as the first-layer child node; determine the first node weight between each target in the first category and the first acquisition time according to the frequency of occurrence of each target in the first category in the first acquisition time; and establish a connection between the root node and the first-layer child nodes based on the first node weight corresponding to each target in the first category.

[0110] Specifically, the target information of this embodiment includes multiple targets, with the first collection time as the root node and the first category as the first-level child node. Based on the occurrence frequency of each target in the first category within the first collection time, the first node weight between each target in the first category and the first collection time is determined, specifically:

[0111] In the embodiment of the present application, the first acquisition time is represented by T1, and the first category set is represented by C1={c 11 ,c 12 ,...,c 1n For Objective C 1i (i=1,2,...,n) the frequency of occurrence f in the first acquisition time T1 1i . Compute Objective C 1i The first node weight w 1i The formula can be expressed as:

[0112]

[0113] Furthermore, the video processing system establishes connections between the first-layer child nodes and the second-layer child nodes based on the first node weight between each target in the first category and the first acquisition time, wherein a larger first node weight indicates that the target information is located closer to the root node (acquisition time) in the first-layer child nodes. In this embodiment of the present application, the first node weight between each target in the first category and the first acquisition time may be sorted by weight using a sorting algorithm, and then connections are established from the root node to each child node in the first-layer child nodes.

[0114] S312. Take the metadata information corresponding to each target in the first category as the second-layer child node; determine the second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; and construct a connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category.

[0115] The metadata information of each target in the first category is used as a second-layer child node. According to the information type of the metadata information of each target in the first category, the second node weight between each target in the first category and its metadata information is determined, specifically:

[0116] The first category of metadata information set in the embodiment of the present application is M 1i ={m 1i1 ,m 1i2 ,...,m 1ik}, k represents the number of metadata information in the metadata information set, and the metadata information type set is Type = {t1, t2, ..., t m}. For metadata information m 1ij (j=1,2,...,k), whose information type is t l (l∈{1,2,..m}), where m represents the number of information types.

[0117] Furthermore, the video processing system calculates the frequency g of the metadata information type in all metadata information of the target. 1il , the specific formula is as follows:

[0118]

[0119] Wherein, Q(·) represents an indicator function, which is used to indicate the number of belonging relationships.

[0120] Furthermore, the video processing system calculates the second node weight between the target and its metadata information. The specific formula is as follows:

[0121]

[0122] Furthermore, the video processing system connects the metadata information node to the corresponding target node based on the second node weight between the target and its metadata information. A higher second node weight indicates that the metadata information is closer to the target (first-layer subnode) in the second-layer subnodes. Similarly, a sorting algorithm can be used to sort the second node weights between the target and its metadata information by weight, and then establish connections from the first-layer subnodes to each subnode in the second-layer subnodes.

[0123] S313. Use the device information corresponding to each target in the first category as a leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct a first index database.

[0124] The device information of each target of the first category is used as a leaf-layer child node.

[0125] In the embodiment of this application, target c 1i Metadata information m 1ij The corresponding device information set is

[0126] D 1ij ={d 1ij1 ,d 1ij2 ,...,d 1ijp}. For nodes with the same device information metadata, they are connected to the same leaf-level child node (device information node). In one embodiment, the connection can be performed through a hash table, using device information as the hash key, and nodes with the same device information metadata are stored in the same hash bucket, with each hash bucket corresponding to a leaf node of device information.

[0127] The embodiment of the present application labels and associates the detected targets, the category information corresponding to the targets, the metadata information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, only a certain keyword needs to be searched to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0128] S320: Divide the second association information according to a preset hierarchy to construct a second index database.

[0129] For target information of the second category, the second associated information corresponds to the target information of the second category. Therefore, the video processing system obtains the attribute information, device information and the first acquisition time in the second associated information, and constructs a second index tree based on the target of the second category, as well as the attribute information, device information and the first acquisition time in the second associated information. Further, the video processing system determines the constructed second index tree as the second index database of the target information of the second category. The specific implementation method can be referred to Figure 5 The corresponding implementation method.

[0130] For details, please refer to Figure 5 , Figure 5 This is another flowchart of the method for logically processing video data provided by an embodiment of the present application. The embodiment of the present application provides a detailed explanation of the implementation process of the above step S320. Specifically, the implementation process of the above step S310 is explained in detail in the following steps S321 to S323:

[0131] S321. Take the first acquisition time as the root node and the second category as the first-layer child node; determine the third node weight between each target in the second category and the first acquisition time based on the frequency of occurrence of each target in the second category in the first acquisition time; and construct a connection between the root node and the first-layer child nodes based on the third node weight of each target in the second category.

[0132] Optionally, with the first acquisition time as the root node and the second category as the first-level child node, the video processing system determines a third node weight between each target in the second category and the first acquisition time based on the frequency of occurrence of each target in the second category within the first acquisition time. Furthermore, the video processing system establishes a connection between the root node and the first-level child nodes based on the third node weight of each target in the second category, as specifically described in step S311.

[0133] S322. Take the attribute information corresponding to each target in the second category as the second-layer sub-node; determine the fourth node weight between each target in the second category and the corresponding attribute information based on the similarity between the attribute information corresponding to each target in the second category and the second preset information; and construct a connection between the first-layer sub-node and the second-layer sub-node based on the fourth node weight of each target in the second category.

[0134] Furthermore, with each first attribute information of the second category as a second-layer child node, the video processing system calculates the similarity between each first attribute information of the second category and the second preset attribute information. Specifically, it can be calculated using the cosine similarity formula and the Euclidean distance formula, which will not be described in detail in the embodiments of this application. Furthermore, the video processing system determines the fourth node weight between each target of the second category and its attribute information based on the similarity between each first attribute information of the second category and the second preset attribute information. Furthermore, the video processing system constructs a connection between the first-layer child nodes and the second-layer child nodes based on the fourth node weight of each target of the second category. The principle refers to the embodiment of step S312.

[0135] S323. Use the device information corresponding to each target in the second category as a leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct a second index database.

[0136] Furthermore, with the device information of each target in the second category as a leaf-layer sub-node, the video processing system connects the second-layer sub-nodes with the same device information to the same leaf-layer sub-node to construct a second index database. The principle refers to the embodiment of step S313.

[0137] The embodiment of the present application labels and associates the detected targets, the category information corresponding to the targets, the attribute information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, it is only necessary to query a certain keyword to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0138] S330: Construct a first target index database at a first acquisition time for the video stream data based on the first index database and / or the second index database.

[0139] The video processing system fuses the first index database and the second index database to generate a first final index database of the target at the first acquisition time. Specifically, the video processing system uses the acquisition time as the standard, fuses the first index database and the second index database at the same acquisition time according to the acquisition time, and obtains the first final index database at the same acquisition time.

[0140] Please refer to Figure 6 , Figure 6 It is a structural diagram of the first target index database provided by an embodiment of the present application; in one embodiment, the first acquisition time is A, the first category includes A1, the second category includes A2, the metadata information of each target in the first category includes A3, A4 and A5, the device information of each target in the first category includes A6 and A7, each first attribute information of the second category includes A8, A9 and A10, and the device information of each target in the second category includes A11. For the first index database of the first category, the acquisition time A is the root node, the target A1 is the first-level child node, the metadata information A3, A4 and A5 are the second-level child nodes, the device information A6 and A7 are the leaf-level child nodes, the node weight between the target A1 and the acquisition time A is 3, the node weights between the target A1 and the metadata information A3, A4 and A5 are 1, 1.5 and 1 respectively, the device information of the metadata information A3 and A4 is the same as A6, and the device information of the metadata information A5 is A7. Therefore, the first index database of the first category refers to Figure 6 shown.

[0141] Please refer to Figure 7 , Figure 7This is another structural diagram of the first target index database provided by the embodiment of the present application. For the second index database of the second category, the acquisition time A is the root node, the target A2 is the first-layer child node, the attribute information A8, A9 and A10 are the second-layer child nodes, the device information A11 is the leaf-layer child node, the node weight between the target A2 and the acquisition time A is 2, the node weights between the target A2 and the attribute information A8, A9 and A10 are 2, 1 and 1.5 respectively, the device information of the attribute information A8, A9 and A10 is the same as A11, and the second index database of the second category of targets is constructed. Further, the first index database and the second index database are fused based on time, and the obtained first target index database is as follows: Figure 7 shown.

[0142] The embodiment of the present application labels and associates the detected targets, the category information corresponding to the targets, the attribute information or metadata information corresponding to the targets, as well as the device information and the acquisition time and stores them in an index database. In subsequent queries, it is only necessary to query a certain keyword to obtain all the information associated with the keyword. There is no need to watch the video data analysis again, thereby improving the query efficiency of the video data.

[0143] The video data logical processing method provided in this embodiment first obtains the target information in the video stream data and the category information and attribute information corresponding to the target information through a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be searched to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of video data.

[0144] Another preferred embodiment, please refer to Figure 8 , Figure 8 This is another flow chart of the method for logically processing video data provided by the embodiment of the present application. The solution provided by the embodiment of the present application can further perform the following steps S40 to S60 after obtaining the first target index database:

[0145] S40: Acquire a second target index database obtained at a second acquisition time.

[0146] The first target index database and the second target index database in the embodiment of the present application have the same database structure. The first acquisition time and the second acquisition time can be different acquisition times of the same video device, the same acquisition time of different video devices, or different acquisition times of different video devices.

[0147] Optionally, the video processing system obtains a second target index database obtained at a second acquisition time.

[0148] S50. Determine the node similarity between the first child node in the first target index database and the second child node in the second target index database; the category information to which the first child node and the second child node belong is the same, and the number of child node layers of the first child node in the first target index database and the number of child node layers of the second child node in the second target index database are the same.

[0149] First, the video processing system traverses the first final index data first target index database library and the second final index database second target index database to find the first child node and the second child node pair that satisfy the same category information and the same number of child node layers, that is, the category information of the first child node in the first final index data first target index database library is the same as the category information of the second child node in the second final index database second target index database, and the number of child node layers of the first child node in the first final index data first target index database library is the same as the number of child node layers of the second child node in the second final index database second target index database.

[0150] In the embodiment of the present application, the first child node N1 is in the first target index database library, and the second child node N2 is in the second target index database. The category to which the first child node N1 and the second child node N2 belong is C, and the number of child node layers is L. Therefore, the node similarity between the first child node in the first target index database library and the second child node in the second target index database is determined by the following formula:

[0151] The node similarity formula for the first category is:

[0152] Sim(N1,N2)=R1*Sim attr (N1,N2)+R2*Sim device (N1,N2) (15)

[0153] The node similarity formula for the second category is:

[0154] Sim(N1,N2)=R3*Sim meta (N1,N2)+R2*Sim device (N1,N2) (16)

[0155] Where R1, R2, and R3 are preset weights, R1+R2+R3=1, Sim(N1, N2) represents the node similarity between the first child node and the second child node, Sim attr (N1, N2) represents attribute similarity, Sim meta (N1, N2) represents metadata similarity, Sim device (N1, N2) represents the device similarity.

[0156] For attribute similarity Sim attr (N1,N2):

[0157] The target attribute vector corresponding to the first child node N1 is The target attribute vector corresponding to the second child node N2 is Consider the importance weight vector of the attribute Therefore, the attribute similarity Sim attr The calculation formula of (N1, N2) can be expressed as:

[0158]

[0159] Therefore, the attribute similarity is made more discriminative in the interval (0,1).

[0160] For metadata similarity Sim meta (N1,N2):

[0161] The source data set corresponding to the first child node N1 is M1={m 11 ,m 12 ,...,m 1k}, the source data set corresponding to the second child node N2 is M2 = {m 21 ,m 22 ,...,m 2k}, therefore, the metadata similarity Sim meta The calculation formula of (N1, N2) can be expressed as:

[0162]

[0163] Among them, v i Represents the preset weight, TF_IDF(·) is used to calculate the importance of metadata in its respective node metadata set. Therefore, for TF_IDF(m 1i ,M1)=TF(m 1i ,M1)*IDF(m 1i ,M1):

[0164]

[0165] Among them, n m1iIndicates m 1i The number of times it appears in M1, N represents the total number of targets, Indicates that it contains m 1i The target number.

[0166] For device similarity Sim device (N1,N2):

[0167] The target attribute vector corresponding to the first child node N1 is D1=(d 11 ,d 12 ,..,d 1p ), the target attribute vector corresponding to the second child node N2 is D2=(d 21 ,d 22 ,..,d 2p ), similarity is measured by calculating the matching degree of device information, so the device similarity Sim device The calculation formula of (N1, N2) can be expressed as:

[0168]

[0169] Wherein, Q(·) represents an indicator function, which is used to indicate the number of belonging relationships.

[0170] S60 : Connect the first subnode in the first target index database and the second subnode in the second target index database based on node similarity to construct a multi-link index database at the first collection time and the second collection time.

[0171] The first target index database and the second target index database have the same database structure.

[0172] Furthermore, the similarity threshold μ (0<μ<1), for each pair of the first child node and the second child node that satisfy the same category information and the same number of child node layers, if the similarity Sim(N1, N2) ≥ μ between the two, a connection is established between the two to construct a multi-link index database at different times.

[0173] In one embodiment, the first target index database is the final index database at the first acquisition time A, and the second target index database is the final index database at the second acquisition time B. For node A1 in the first target index database and node B1 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A8 in the first target index database and node B9 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A10 in the first target index database and node B8 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. Therefore, the final multi-link index database is as follows: Figure 4As shown, Figure 4 This is a schematic diagram of the structure of the multi-link index database provided in an embodiment of the present application.

[0174] Please refer to Figure 9 , Figure 9 It is a structural diagram of the multi-link index database provided by an embodiment of the present application. In one embodiment, the first target index database library is the final index database under the first acquisition time A, and the second target index database is the final index database under the second acquisition time B. For node A1 in the first target index database library and node B1 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A8 in the first target index database library and node B9 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. For node A10 in the first target index database library and node B8 in the second target index database, the similarity between the two nodes is greater than the similarity threshold. Therefore, the final multi-link index database is as follows Figure 9 shown.

[0175] The embodiment of the present application connects the nodes in multiple final index databases whose similarities meet preset conditions to construct a multi-link index database. In subsequent queries, it is only necessary to query a certain keyword to obtain all information associated with the keyword. There is no need to watch the video data analysis again, which improves the query efficiency of video data.

[0176] Figure 10 This is a schematic diagram of the structure of the video data logical processing device provided by the embodiment of the present application. The device is suitable for executing the video data logical processing method provided by the embodiment of the present application. Figure 10 As shown, the device may specifically include: a data acquisition module 410, an information association module 420 and a database construction module 430, wherein:

[0177] The data acquisition module 410 is configured to acquire video stream data acquired by the target video device at the first acquisition time, input the video stream data into a video detection model, and obtain target information in the video stream data and category information and attribute information corresponding to the target information.

[0178] an information association module 420, configured to associate the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information;

[0179] The database construction module 430 is configured to construct a first target index database at the first acquisition time for the video stream data based on the target association information.

[0180] The video data logical processing device provided in the embodiment of the present application first obtains the target information in the video stream data and the category information and attribute information corresponding to the target information through a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be searched to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of video data.

[0181] In one embodiment, the target information includes first information and / or second information; the category information and attribute information corresponding to the first information are respectively the first category and the first attribute, and the category information and attribute information corresponding to the second information are respectively the second category and the second attribute.

[0182] The information association module 420 is specifically used to generate metadata information based on the first attribute and the first preset information; associate the first information with the first category, the metadata information, the device information and the first collection time to obtain first association information; associate the second information with the second category, the second attribute, the device information and the first collection time to obtain second association information; and obtain the target association information based on the first association information and / or the second association information.

[0183] In one embodiment, when the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information.

[0184] The database construction module 430 is used to divide the first associated information according to a preset hierarchy to construct a first index database; divide the second associated information according to a preset hierarchy to construct a second index database; and construct the first target index database for the video stream data at the first acquisition time based on the first index database and / or the second index database.

[0185] In one embodiment, the target information includes multiple targets.

[0186] The database construction module 430 is specifically used to take the first acquisition time as the root node and the first category as the first-layer child node; determine the first node weight between each target in the first category and the first acquisition time according to the frequency of occurrence of each target in the first category within the first acquisition time; construct a connection between the root node and the first-layer child node based on the first node weight corresponding to each target in the first category; take the metadata information corresponding to each target in the first category as the second-layer child node; determine the second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; construct a connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category; take the device information corresponding to each target in the first category as the leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct the first index database.

[0187] In one embodiment, the database construction module 430 is specifically used to take the first acquisition time as the root node and the second category as the first-layer child node; determine the third node weight between each target in the second category and the first acquisition time based on the frequency of occurrence of each target in the second category within the first acquisition time; construct a connection between the root node and the first-layer child node based on the third node weight of each target in the second category; take the attribute information corresponding to each target in the second category as the second-layer child node; determine the fourth node weight between each target in the second category and the corresponding attribute information based on the similarity between the attribute information corresponding to each target in the second category and the second preset information; construct a connection between the first-layer child node and the second-layer child node based on the fourth node weight of each target in the second category; take the device information corresponding to each target in the second category as the leaf-layer child node; connect the second-layer child nodes with the same device information to the same leaf-layer child node to construct the second index database.

[0188] In one embodiment, the apparatus further comprises: a database acquisition module, a similarity determination module, and a node connection module, wherein:

[0189] A database acquisition module, configured to acquire a second target index database of targets at a second acquisition time;

[0190] a similarity determination module, configured to determine a node similarity between a first child node in the first target index database and a second child node in the second target index database; the first child node and the second child node have the same category information, and the first child node has the same number of child node levels in the first target index database as the second child node has in the second target index database;

[0191] A node connection module is used to connect the first subnode in the first target index database and the second subnode in the second target index database based on the node similarity to construct a multi-link index database at the first acquisition time and the second acquisition time; wherein the database structure of the first target index database and the second target index database is the same.

[0192] In one embodiment, the first preset attribute information is an attribute information matrix; the information association module 920 includes a matching degree determination unit, a weighting processing unit, and a metadata generation unit, wherein:

[0193] a matching degree determining unit, configured to determine a matching degree between the first attribute information and the first preset attribute information based on an attribute vector corresponding to the first attribute information and each row vector in the attribute information matrix;

[0194] a weighted processing unit, configured to perform weighted processing on the matching degree between the first attribute information and the first preset attribute information to obtain weighted attribute information between the first attribute information and the first preset attribute information;

[0195] The metadata generating unit is configured to generate the metadata information based on weighted attribute information between the first attribute information and the first preset attribute information.

[0196] In one embodiment, the video detection model includes a feature extraction layer, a target detection layer, a target classification layer, and an attribute recognition layer.

[0197] The data acquisition module 410 is specifically used to perform feature extraction on the video stream data based on the feature extraction layer to obtain video features; perform target detection on the video features based on the target detection layer to obtain the target information; perform category identification on the target information based on the target classification layer to obtain category information corresponding to the target information; perform attribute identification on the target information based on the attribute identification layer to obtain attribute information corresponding to the target information.

[0198] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0199] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video data logical processing method described in any embodiment of the present application.

[0200] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video data logical processing method described in any embodiment of the present application when executed.

[0201] Reference below Figure 11 , Figure 11 FIG1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, which shows a schematic diagram of the structure of a computer system 500 suitable for implementing the electronic device in an embodiment of the present application. Figure 11 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0202] like Figure 11 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0203] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0204] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0205] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, and optical cables, or any suitable combination thereof.

[0206] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] The modules and / or units described in the embodiments of this application may be implemented in software or hardware. The modules and / or units described may also be provided in a processor. For example, a processor may be described as comprising a data acquisition module, an information association module, and a database construction module. The names of these modules do not, in some cases, limit the modules themselves.

[0208] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: obtaining video stream data collected by a target video device at a first collection time, inputting the video stream data into a video detection model, obtaining target information in the video stream data and category information and attribute information corresponding to the target information; associating the target information with the category information, the attribute information, the device information of the target video device, and the first collection time to obtain target association information; and constructing a first target index database for the video stream data at the first collection time based on the target association information.

[0209] According to the technical solution of this embodiment, first, the target information in the video stream data and the category information and attribute information corresponding to the target information are obtained by a video detection model, so that the video detection model can quickly process a large amount of video data in a short time, thereby being able to quickly process the video data and improving the processing efficiency of the video data; then, by associating the target information with the category information, attribute information, device information of the target video device and the first acquisition time, target association information is obtained, and a first target index database is constructed for the video stream data based on the target association information, so that the video stream data can be stored in a target index manner, so that in subsequent queries, only a certain keyword needs to be searched to obtain all information associated with the keyword, and there is no need to watch the video data analysis again, thereby achieving the beneficial effect of improving the query efficiency of video data.

[0210] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for logically processing video data, characterized in that: include: Obtaining video stream data collected by a target video device at a first collection time, inputting the video stream data into a video detection model, and obtaining target information in the video stream data and category information and attribute information corresponding to the target information; Associating the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information; Constructing a first target index database at the first acquisition time for the video stream data based on the target association information; Wherein, when the target information includes first information, the category information and attribute information corresponding to the first information are respectively the first category and the first attribute, and the target association information includes the first association information; the first information is category information containing only people; The constructing a first target index database at the first acquisition time for the video stream data based on the target association information includes: Dividing the first association information according to a preset hierarchy to construct a first index database; The target information includes multiple targets; and the step of dividing the first associated information according to a preset hierarchy to construct a first index database includes: Taking the first collection time as a root node and the first category as a first-layer child node; determining a first node weight between each target in the first category and the first collection time based on the frequency of occurrence of each target in the first category within the first collection time; and establishing a connection between the root node and the first-layer child nodes based on the first node weight corresponding to each target in the first category; Taking metadata information corresponding to each target in the first category as a second-layer child node; determining a second node weight between each target in the first category and the corresponding metadata information according to the information type of the metadata information corresponding to each target in the first category; and establishing a connection between the first-layer child node and the second-layer child node based on the second node weight corresponding to each target in the first category; The device information corresponding to each target in the first category is used as a leaf-layer sub-node; the second-layer sub-nodes with the same device information are connected to the same leaf-layer sub-node to construct the first index database.

2. The video data logical processing method according to claim 1, characterized in that: The target information includes first information and / or second information; the category information and attribute information corresponding to the second information are the second category and the second attribute respectively; The associating the target information with the category information, the attribute information, the device information of the target video device, and the first acquisition time to obtain target association information includes: generating metadata information according to the first attribute and first preset information; associating the first information with the first category, the metadata information, the device information, and the first collection time to obtain first association information; Associating the second information with the second category, the second attribute, the device information, and the first collection time to obtain second associated information; The target association information is obtained according to the first association information and / or the second association information.

3. The video data logical processing method according to claim 2, characterized in that: When the target information includes the first information and / or the second information, the target associated information includes the first associated information and / or the second associated information; Constructing a first target index database at the first acquisition time for the video stream data based on the target association information includes: Dividing the second association information according to a preset hierarchy to construct a second index database; The first target index database at the first acquisition time is constructed for the video stream data based on the first index database and / or the second index database.

4. The video data logical processing method according to claim 3, characterized in that: The target information includes a plurality of targets; and the second associated information is divided according to a preset hierarchy to construct a second index database, including: Taking the first collection time as a root node and the second category as a first-layer child node; determining a third node weight between each target in the second category and the first collection time based on the frequency of occurrence of each target in the second category within the first collection time; and establishing a connection between the root node and the first-layer child nodes based on the third node weight of each target in the second category; Taking the attribute information corresponding to each target in the second category as a second-layer child node; determining a fourth node weight between each target in the second category and the corresponding attribute information based on the similarity between the attribute information corresponding to each target in the second category and the second preset information; and establishing a connection between the first-layer child node and the second-layer child node based on the fourth node weight of each target in the second category; The device information corresponding to each target in the second category is used as a leaf-layer sub-node; the second-layer sub-nodes with the same device information are connected to the same leaf-layer sub-node to construct the second index database.

5. The video data logical processing method according to claim 1, characterized in that: After obtaining the first target index database, the method further includes: Acquiring a second target index database obtained at a second acquisition time; Determining a node similarity between a first child node in the first target index database and a second child node in the second target index database; the first child node and the second child node have the same category information, and the first child node has the same number of child node levels in the first target index database as the second child node has in the second target index database; Connecting a first subnode in the first target index database and a second subnode in the second target index database based on the node similarity to construct a multi-link index database at the first acquisition time and at the second acquisition time; The first target index database and the second target index database have the same database structure.

6. The video data logical processing method according to claim 2, characterized in that: The first preset information is an attribute information matrix; Generating metadata information according to the first attribute and the first preset information includes: determining a matching degree between the first attribute and the first preset information according to the attribute vector corresponding to the first attribute and each row vector in the attribute information matrix; performing weighted processing on the matching degree between the first attribute and the first preset information to obtain weighted attribute information between the first attribute and the first preset information; The metadata information is generated based on weighted attribute information between the first attribute and the first preset information.

7. The video data logical processing method according to claim 1, characterized in that: The video detection model includes a feature extraction layer, a target detection layer, a target classification layer and an attribute recognition layer; The step of inputting the video stream data into a video detection model to obtain target information in the video stream data and category information and attribute information corresponding to the target information, respectively, includes: Performing feature extraction on the video stream data based on the feature extraction layer to obtain video features; Performing target detection on the video features based on the target detection layer to obtain the target information; Performing category identification on the target information based on the target classification layer to obtain category information corresponding to the target information; Attribute recognition is performed on the target information based on the attribute recognition layer to obtain attribute information corresponding to the target information.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the video data logical processing method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for logically processing video data as described in any one of claims 1 to 7 is implemented.

Images