A multimedia object hotness determination method, medium and device

Abstract

The embodiment of the application provides a multimedia object hotness determination method, medium and equipment, the method comprises the following steps: obtaining the access log and content data of a multimedia object, then determining the time behavior hotness representing the activity degree of the multimedia object in the time dimension according to the access log, and determining the semantic feature hotness representing the importance degree of the multimedia object in the semantic dimension according to the content data; and obtaining the comprehensive hotness of the multimedia object by fusing the time behavior hotness and the semantic feature hotness. That is, the comprehensive hotness of the multimedia object is obtained by fusing the dual features of the time dimension and the semantic dimension, and the determination accuracy of the multimedia object hotness is improved.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a method, medium and device for determining the heat of multimedia objects. Background Technology

[0002] As the scale of multimedia data continues to expand, it is often necessary to assess the popularity of multimedia objects in order to effectively manage and utilize data such as images and videos. Popularity refers to a quantitative indicator that measures the degree of attention a multimedia object receives within a specific time window. Popularity assessment can provide a basis for operations such as data retrieval, hierarchical storage, content recommendation, and cache scheduling, thereby optimizing system resource allocation or improving data processing efficiency.

[0003] Currently, the popularity of multimedia objects is generally determined based on access behavior, that is, by counting the number of times a multimedia object is accessed or the time of its most recent access, to quantify its level of attention. However, this method of determining popularity based on access behavior is difficult to accurately reflect the differences in access patterns of multimedia objects over different time periods, resulting in poor accuracy in determining the popularity of multimedia objects. Summary of the Invention

[0004] This application provides a method, medium, and device for determining the popularity of multimedia objects, which can improve the accuracy of determining the popularity of multimedia objects.

[0005] In a first aspect, embodiments of this application provide a method for determining the popularity of a multimedia object, the method comprising: Retrieve access logs and content data for multimedia objects; Based on the access logs, the temporal behavior heat index of the multimedia object is determined, and the temporal behavior heat index characterizes the activity level of the multimedia object in the time dimension; Based on the content data, the semantic feature popularity of the multimedia object is determined, and the semantic feature popularity represents the importance of the multimedia object in the semantic dimension; The overall popularity of the multimedia object is obtained by fusing the temporal behavior popularity and the semantic feature popularity.

[0006] In one feasible implementation, determining the temporal behavior popularity of the multimedia object based on the access log specifically includes: Based on the access log, determine the total number of accesses to the multimedia object, the number of recent accesses within a preset time period, and the recent access time, wherein the recent access time is the time interval between the current access time and the previous access time of the multimedia object; The ratio of the most recent access count to the total access count is used as the access frequency popularity. Based on the most recent access time, the access time popularity is determined, and the access time popularity is negatively correlated with the most recent access time; The time behavior popularity is determined based on the access frequency popularity and the access time popularity.

[0007] In one feasible implementation, determining the access time popularity based on the most recent access time specifically includes: The attenuation coefficient is determined based on the preset half-life of the multimedia object; Based on the most recent access time and the decay coefficient, the access time popularity is determined using a preset exponential decay function.

[0008] In one feasible implementation, determining the time-based behavior popularity based on the access frequency popularity and the access time popularity specifically includes: Obtain the historical frequency popularity and historical time popularity of the multimedia object; Determine the sum of the frequency dispersion of the historical frequency heat and the time dispersion of the historical time heat; The ratio of the frequency dispersion to the sum value is used as the frequency weight; The ratio of the time dispersion to the sum value is used as the time weight; The time behavior popularity is obtained by weighting and summing the access frequency popularity with the frequency weight and the access time popularity with the time weight.

[0009] In one feasible implementation, determining the semantic feature popularity of the multimedia object based on the content data specifically includes: The semantic metrics of the multimedia object are determined. The semantic metrics include at least one of scene category score, event confidence score and content semantic score. The scene category score represents the importance of the business scene to which the multimedia object belongs. The event confidence score represents the detection confidence of the business scene to which the multimedia object belongs. The content semantic score represents the degree of association between the multimedia object and each business scene in the semantic space. The popularity of the semantic features is determined based on the semantic indicators.

[0010] In one feasible implementation, determining the semantic indicators of the multimedia object specifically includes: Determine the probability that the multimedia object belongs to the scenario category of each of the aforementioned business scenarios; Based on the category baseline weights of each business scenario, the probabilities of each scenario category are weighted and summed to obtain the scenario category score.

[0011] In one feasible implementation, determining the semantic indicators of the multimedia object specifically includes: Extract the content semantic vector from the content data; Obtain the semantic anchor vectors for each of the aforementioned business scenarios; Determine the semantic similarity between the content semantic vector and each of the semantic anchor vectors; The semantic score of the content is determined based on the semantic similarity of each item.

[0012] In one feasible implementation, the semantic metrics include at least two of the following: the scene category score, the event confidence score, and the content semantic score. Determining the semantic feature popularity based on the semantic indicators specifically includes: Obtain historical data of the multimedia object for each of the semantic metrics; Based on the historical data, determine the degree of dispersion of the semantic indicators; The semantic index weights of each semantic index are determined based on the degree of dispersion of each semantic index. The semantic feature popularity is obtained by weighting and summing the semantic indicators according to their respective weights.

[0013] Secondly, embodiments of this application provide a multimedia object heat determination device, the device comprising: The acquisition module is used to obtain access logs and content data of multimedia objects. The time behavior heat module is used to determine the time behavior heat of the multimedia object based on the access log, wherein the time behavior heat characterizes the activity level of the multimedia object in the time dimension; The semantic feature popularity module is used to determine the semantic feature popularity of the multimedia object based on the content data. The semantic feature popularity represents the importance of the multimedia object in the semantic dimension. The comprehensive popularity module is used to fuse the comprehensive popularity of the multimedia object based on the time behavior popularity and the semantic feature popularity.

[0014] Thirdly, embodiments of this application provide a multimedia object heat determination device, the device including: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement any of the above-mentioned methods for determining the maximum transmission unit of a network.

[0015] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement any of the above-described methods for determining the heat of multimedia objects.

[0016] Fifthly, embodiments of this application provide a computer program product in which instructions are executed by the processor of an electronic device, causing the electronic device to execute any of the above-described multimedia object heat determination methods.

[0017] This application discloses a method, apparatus, medium, and device for determining the popularity of multimedia objects. It can acquire access logs and content data of multimedia objects, and then determine, based on the access logs, a temporal behavior popularity representing the activity level of the multimedia object in the time dimension, and based on the content data, a semantic feature popularity representing the importance of the multimedia object in the semantic dimension. The application then fuses the temporal behavior popularity and the semantic feature popularity to obtain a comprehensive popularity of the multimedia object. In other words, this application improves the accuracy of determining the popularity of multimedia objects by fusing the dual features of the time and semantic dimensions to obtain a comprehensive popularity.

[0018] Furthermore, this application can determine the popularity of semantic features from multiple semantic indicators such as scene category score, event confidence and content semantic score, which further improves the comprehensiveness and reliability of multimedia object popularity determination. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for determining the popularity of a multimedia object according to an embodiment of this application; Figure 2 This is a schematic diagram of the process for determining overall popularity provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the process of determining the heat of time-based behaviors provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the process of determining the popularity of semantic features provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a multimedia object heat determination device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a multimedia object heat determination device provided in an embodiment of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] In practical applications, modern surveillance systems consist of a massive number of camera devices. Each camera continuously collects and uploads a large number of images or videos for event recording, anomaly detection, and subsequent analysis. The number of images captured by surveillance cameras every day can reach tens of millions, accumulating into a huge amount of multimedia data over a long period of time. Especially in scenarios such as traffic accident monitoring and public safety inspections, multimedia data exhibits high-density and strong continuity in its distribution.

[0024] As the scale of multimedia data continues to expand, it is often necessary to assess and differentiate the popularity of different multimedia objects in order to effectively manage and utilize data such as images and videos. Popularity refers to a quantitative indicator that measures the degree of attention a multimedia object receives within a specific time window. Popularity assessment can provide a basis for operations such as data retrieval, hierarchical storage, content recommendation, and cache scheduling, thereby optimizing system resource allocation or improving data processing efficiency.

[0025] Currently, the popularity of multimedia objects is generally determined based on access behavior, that is, by counting the number of times a multimedia object is accessed or the time of its most recent access, to quantify its level of attention. However, this method of determining popularity based on access behavior is difficult to accurately reflect the differences in access patterns of multimedia objects over different time periods, resulting in poor accuracy in determining the popularity of multimedia objects.

[0026] To address the problems in the prior art, embodiments of this application provide a method, apparatus, device, and computer storage medium for determining the heat of multimedia objects.

[0027] In practical applications, the execution subject of the multimedia object heat determination method in this application embodiment can be a terminal device, such as a desktop computer or laptop computer, or a remote device like a server. Of course, this application embodiment can also adopt an execution subject in the form of software, such as a client or software program installed on a terminal device. The specific type of execution subject corresponding to the technical solution provided in this application embodiment is not strictly limited here, and can be flexibly selected according to the actual application scenario and actual needs.

[0028] The following describes specific embodiments of a method, apparatus, medium, and device for determining the heat of a multimedia object provided in this application. First, a method for determining the heat of a multimedia object is introduced.

[0029] Figure 1 This illustration shows a flowchart of a method for determining the heat of a multimedia object according to an embodiment of this application. Figure 1 As shown, the method includes steps S100 to S103: S100: Obtain the access logs and content data of the multimedia object.

[0030] In one or more embodiments of this application, in order to determine the popularity of a multimedia object from both temporal and semantic dimensions in subsequent steps, this application may acquire two types of basic data in this step: access logs and content data of the multimedia object. The access logs reflect the temporal behavior data of the multimedia object being accessed, while the content data reflects the semantic value of the multimedia object itself.

[0031] Specifically, this application can collect access logs or content data of multimedia objects from multimedia data sources. For example, this application can obtain access logs of the multimedia object through system logs, including but not limited to the access time of each access; this application can extract content data of the multimedia object through image acquisition devices, video surveillance servers, or data storage systems, including but not limited to image frames, video clips, or multimedia files uploaded by users.

[0032] It should be noted that this application does not limit the specific method for obtaining access logs or content data, and can be set according to actual needs. For example, for scenarios with high real-time requirements, streaming reading can be used; for batch processing scenarios, offline database querying can be used. This application does not limit the specific type of multimedia objects. For example, multimedia objects can be images captured by surveillance cameras, user-generated content from short video platforms, or detection images or videos collected by intelligent inspection equipment. The order in which access logs and content data are obtained can be synchronous or asynchronous, and this application does not restrict this. To improve the quality of obtained access logs or content data, this application can also perform data preprocessing on multimedia data, including but not limited to noise reduction, format unification, timestamp extraction, and object ID generation, to facilitate data retrieval or application.

[0033] S101: Based on the access log, determine the temporal behavior heat of the multimedia object, wherein the temporal behavior heat characterizes the activity level of the multimedia object in the time dimension.

[0034] In one or more embodiments of this application, in order to fuse with semantic feature popularity in subsequent steps to obtain the comprehensive popularity of multimedia objects, in this step, this application can determine the temporal behavior popularity of multimedia objects based on access logs.

[0035] It should be noted that this application does not limit the specific method for determining time-based behavior popularity. It can be set according to actual needs, such as using the ratio of recent visits to total visits as time-based behavior popularity, or normalizing the time interval between the last visit and the current visit as time-based behavior popularity. To improve the accuracy of time-based behavior popularity, in one or more embodiments of this application, this application can consider both visit frequency and visit timeliness to determine time-based behavior popularity, as follows: First, this application can determine the total number of accesses to a multimedia object, the number of most recently accessed words within a preset time period, and the most recently accessed time based on the access log. The most recently accessed time is the time interval between the current access time and the last access time of the multimedia object.

[0036] Secondly, the ratio of recent visits to total visits is used as the visit frequency popularity; visit time popularity is determined based on the recent visit time; wherein, the visit time popularity is negatively correlated with the recent visit time.

[0037] Finally, the time-based behavior popularity is determined based on the popularity of access frequency and the popularity of access time.

[0038] In this embodiment, the specific size of the preset time period is not limited and can be set according to business needs. For example, for scenarios with frequent access, the preset time period can be set to 1 hour; for scenarios with sparse access, the preset time period can be set to 24 hours or longer. The time period from the last access time to the current access time is used as the preset time period for the current access, thereby dynamically adapting to the degree of access fluctuation and improving the accuracy of time behavior popularity assessment. This application does not limit the specific method of determining time behavior popularity based on access frequency popularity and access time popularity, and can be set according to actual needs. For example, the two can be normalized and then directly added to obtain the time behavior popularity, or the two can be normalized and then weighted to obtain the time behavior popularity. This application does not limit the specific method of determining access time popularity, and can be set according to actual needs. For example, a linear decay function can be used, and the ratio of the most recent access time to a preset time threshold can be inverted to obtain the access time popularity; or a piecewise function can be used, setting different linear decay rates or access time popularity for different time intervals. In order to accurately reflect the characteristics of the decline in popularity of multimedia objects over time, in one or more embodiments of this application, this application can determine the decay coefficient based on the preset half-life of the multimedia object, and then determine the access time popularity based on the most recent access time and the decay coefficient through a preset exponential decay function.

[0039] For example, multimedia objects Access log is ,in For the duration of this visit, The time frame is the last access time; statistics are collected on multimedia objects within a preset time window. Number of visits within Multimedia objects in the total time period Total number of visits A higher access frequency indicates a higher level of attention a multimedia object receives; the ratio of recent accesses to total accesses is used as the access frequency index.

[0040] Access frequency and popularity The calculation formula is as follows: In formula (1), This indicates the access frequency based on the access logs at the time of this access.

[0041] The normalized range of various types of popularity values ​​is as follows: A higher access time indicates a higher level of attention paid to the multimedia object; after a half-life... The access time of the multimedia object then decays to half of its initial value. Substituting into the exponential decay function, we can obtain the decay coefficient. Therefore, based on the attenuation coefficient and the most recent access time The access time popularity is determined by using an exponential decay function.

[0042] Access time popularity The calculation formula is as follows: In equation (4), This indicates the access popularity based on the access logs at the time of this access.

[0043] S102: Based on the content data, determine the semantic feature heat of the multimedia object, wherein the semantic feature heat characterizes the importance of the multimedia object in the semantic dimension.

[0044] In one or more embodiments of this application, in order to integrate with the time behavior heat in subsequent steps to obtain the comprehensive heat of the multimedia object, in this step, this application can determine the semantic feature heat of the multimedia object based on the content.

[0045] Specifically, this application can extract semantic vectors from content data through a pre-trained visual semantic model, and combine them with semantic anchors of business scenarios to calculate the representation of multimedia objects in the semantic space, thereby quantifying their importance in the semantic dimension.

[0046] It should be noted that this application does not limit the specific method for determining semantic feature popularity, and can be set according to actual needs. For example, the classification confidence score output by the pre-trained model can be directly used as the semantic feature popularity; or the similarity between the multimedia object and the preset semantic anchor point can be used as the semantic feature popularity. In order to more comprehensively reflect the importance of the multimedia object in the semantic dimension, in one or more embodiments of this application, this application can determine the semantic index of the multimedia object, and then determine the semantic feature popularity based on the semantic index.

[0047] In this embodiment, the specific type of semantic indicators is not limited and can be set according to actual needs. For example, semantic indicators may include one or more of scene category scores, event confidence scores, and content semantic scores. Scene category scores characterize the importance of the business scene to which the multimedia object belongs; event confidence scores characterize the detection confidence of the business scene to which the multimedia object belongs; and content semantic scores characterize the degree of association between the multimedia object and each business scene in the semantic space. The specific method for determining each semantic indicator is not limited in this application and can be set according to actual needs. For example, scene category scores can be obtained by inputting the multimedia object into a scene classification model; event confidence scores can be directly output by the event detector of the scene category corresponding to the multimedia object; and content semantic scores can be obtained by calculating the cosine similarity between the content semantic vector and the semantic anchor vectors of each business scene.

[0048] S103: Based on the time behavior popularity and the semantic feature popularity, the comprehensive popularity of the multimedia object is obtained by fusion.

[0049] In one or more embodiments of this application, in order to comprehensively reflect the activity level of a multimedia object in the time dimension and its importance in the semantic dimension, and thus more accurately assess its overall popularity, in this step, this application can fuse the temporal behavior popularity and the semantic feature popularity to obtain the comprehensive popularity of the multimedia object.

[0050] Specifically, this application can normalize the time behavior popularity and semantic feature popularity separately so that the two are on the same dimension, and then perform weighted summation according to the preset weight allocation to obtain the comprehensive popularity.

[0051] It should be noted that this application does not limit the specific fusion method of time behavior popularity and semantic feature popularity. It can be set according to actual needs. For example, a preset fixed weight can be used for weighted summation; or an adaptive weighting method can be used to dynamically determine the corresponding weights of time behavior popularity and semantic feature popularity based on the sample variance.

[0052] In the above-mentioned method for determining the popularity of multimedia objects, this application takes into account both the access activity and semantic importance of multimedia objects, which can more comprehensively reflect the actual popularity of multimedia objects and improve the accuracy of determining the popularity of multimedia objects.

[0053] Figure 2 This is a schematic diagram of the process for determining the overall heat index provided in an embodiment of this application, such as... Figure 2 As shown, the process of determining the overall heat index includes steps S200 to S204: S200: Acquires multimedia objects through acquisition devices such as cameras and snapshot terminals.

[0054] S201: Perform data preprocessing on multimedia objects. Data preprocessing includes, but is not limited to, noise reduction, format standardization, timestamp extraction, and object ID generation.

[0055] S202: Calculate the time behavior popularity based on the number of recent visits, the total number of visits, and the time of the most recent visit.

[0056] S203: Calculate the semantic feature popularity based on scene category, event confidence, content semantic vector, and category baseline weight.

[0057] S204: The comprehensive popularity is obtained by weighted summation based on the popularity of time behavior and the popularity of semantic features.

[0058] Furthermore, in step S101, in order to adapt to the popularity fluctuations of multimedia objects and reasonably integrate access frequency popularity and access time popularity, in one or more embodiments of this application, this application can use a variance normalization method to calculate the weights corresponding to access frequency popularity and access time popularity respectively, so as to reflect the contribution ratio of access frequency popularity and access time popularity to the overall access situation in the total time period, as follows: First, this application can obtain the historical frequency popularity and historical time popularity of a multimedia object, and then determine the sum of the frequency dispersion of the historical frequency popularity and the time dispersion of the historical time popularity. Here, historical frequency popularity and historical time popularity refer to a set of historical sampling data calculated separately based on the access records of the multimedia object within a total time period, used to reflect the popularity fluctuations of the multimedia object at different access times.

[0059] Secondly, the ratio of frequency dispersion to sum value is used as the frequency weight, and the ratio of time dispersion to sum value is used as the time weight.

[0060] Finally, the access frequency popularity and frequency weight, and the access time popularity and time weight are weighted and summed to obtain the time behavior popularity.

[0061] It should be noted that, in order to unify the units of measurement and facilitate calculation and comparison, this application can perform normalization operations on calculation results of various types, including popularity, semantic indicators, and dispersion. This application does not limit the specific type of normalization operation and can set it according to actual needs, such as min-max normalization. This application also does not limit the specific method of measuring dispersion and can set it according to actual needs, such as using statistical measures such as variance, standard deviation, and mean absolute deviation to characterize dispersion.

[0062] Figure 3 This is a flowchart illustrating the process of determining the heat of time-based behaviors, as provided in an embodiment of this application. Figure 3 As shown, the process of determining the heat of time behavior includes steps S300 to S305: S300: Obtain access logs for multimedia objects, including but not limited to object IDs and access time lists.

[0063] S301: Count the number of visits within a preset time window and use the ratio of this number of visits to the total number of visits as the access frequency popularity.

[0064] S302: Calculate access time popularity based on the most recent access time.

[0065] S303: Perform min-max normalization on access frequency popularity and access time popularity.

[0066] S304: Determine the variance of historical frequency popularity and the sum of the variances of historical time popularity, and then use the ratio of the variance of historical frequency popularity to the sum as the frequency weight, and the ratio of the variance of historical time popularity to the sum as the time weight.

[0067] S305: The time behavior popularity is obtained by weighted summation of access frequency popularity and frequency weight, and access time popularity and time weight.

[0068] Using the previous example, we obtain the time-based behavior heat index by weighted summation. The calculation formula is as follows: In equations (5) to (7), For multimedia objects The minimum value corresponding to historical frequency of popularity. For multimedia objects The maximum value of the corresponding historical frequency; For multimedia objects The minimum value of popularity corresponding to historical time period. For multimedia objects The maximum value of popularity corresponding to historical time periods; This represents the variance of historical frequency popularity, i.e., the degree of frequency dispersion. This represents the average historical frequency of popularity. This represents the variance of historical heat, i.e., the degree of temporal dispersion. This represents the average popularity over historical periods.

[0069] In step S102, in order to accurately quantify the importance of the business scenario to which the multimedia object belongs, in one or more embodiments of this application, this application can determine the probability of the multimedia object belonging to each business scenario's scenario category, and then perform a weighted summation of the probability of each scenario category according to the category benchmark weight of each business scenario to obtain the scenario category score.

[0070] Continuing with the previous example, for each multimedia object Its semantic description is defined by Composed of semantic indicators, such as The scenario category score refers to the importance benchmark of the business scenario to which the multimedia object belongs, assuming it is based on... For each business scenario, the formula for calculating the scenario category score is as follows: In equation (8), the first Category benchmark weights for each business scenario The category benchmark weight can be set according to actual business needs, and this application does not restrict it. For example, taking camera capture as an example, the business scenarios with category benchmark weights from large to small are traffic accidents, traffic congestion, and empty streets. Represents multimedia objects Belongs to business scenarios The probability of scene category, i.e., the probability of scene category, has a range of values. If a multimedia object is identified as not belonging to the business scenario. ,but .

[0071] In step S102, this application can, based on the detector of the business scenario to which the multimedia object belongs, perform a detection on the multimedia object. The event confidence level was obtained by conducting the test. .

[0072] In step S102, in order to accurately quantify the degree of association between multimedia objects and various business scenarios in the semantic space, in one or more embodiments of this application, this application can extract the content semantic vector of the content data and obtain the semantic anchor vector of each business scenario; determine the semantic similarity between the content semantic vector and each semantic anchor vector; and determine the content semantic score based on each semantic similarity.

[0073] It should be noted that the content semantic score is used to measure the semantic position and semantic strength of content data in the semantic space, in order to capture objects with strong semantic similarity and typicality. This application does not limit the specific method of obtaining semantic anchor vectors; it can be set according to actual needs, such as using a CLIP text encoder to convert the semantic description of the business scenario into semantic anchor vectors. This application also does not limit the specific method of determining the content semantic score based on each semantic similarity; it can be set according to actual needs, such as directly adding the normalized semantic similarities to obtain the content semantic score; or, according to a preset weight allocation, normalizing the semantic similarities and then weighted summing to obtain the content semantic score. Of course, in order to unify the scale of each semantic anchor vector and facilitate subsequent semantic similarity calculation, each semantic anchor vector needs to be normalized, such as through L2 normalization.

[0074] Following the previous example, this application first constructs a semantic anchor vector for each business scenario, as shown in Table 1: Table 1 Secondly, regarding the first Semantic anchor vectors for each business scenario L2 normalization is performed as follows: In equation (9), This is the semantic anchor vector after L2 normalization.

[0075] At the same time, for each multimedia object This application can use a pre-trained visual semantic model (Contrastive Language Image Pretraining, CLIP) to extract semantic anchor vectors according to the number of semantic anchor vectors in the business scenario. Semantic vectors of content related to each business scenario .in, Represents multimedia objects Video frames or images, This represents the image encoder of the CLIP model. Similarly, to avoid feature scale differences and facilitate subsequent semantic similarity calculation, this application can... Perform L2 normalization.

[0076] The process of determining the content semantic vector is as follows: In equation (11), This is the content semantic vector after L2 normalization.

[0077] Finally, the semantic similarity between the content semantic vector and each semantic anchor vector is determined, and then the content semantic score is determined based on the semantic similarity, as follows: In equations (12) to (14), Score the semantic meaning of the content; This refers to the semantic similarity between the content semantic vector and the semantic anchor vector corresponding to the j-th business scenario. This application allows for pre-setting weights for each business scenario. ; A weight is assigned to the j-th business scenario. This weight can be set according to actual needs; for example, the weight can be set to the same value as the category baseline weight. To facilitate horizontal comparison of the semantic scores of content from different multimedia counterparts, this application can... Normalization is performed to obtain semantic indicators. , The minimum content semantic score. This represents the maximum value of the content semantic score.

[0078] To accommodate the varying contributions of different semantic metrics in popularity calculation, in one or more embodiments of this application, when there are multiple semantic metrics, historical data of the multimedia object on each semantic metric can be obtained. Then, for each semantic metric, the degree of dispersion is determined based on the historical data; based on each degree of dispersion, the semantic metric weight is determined; and then, based on each semantic metric weight, a weighted sum is performed on each semantic metric to obtain the semantic feature popularity. To avoid differences in feature scale, this application can perform standardization processing on each semantic metric, including but not limited to unifying dimensions and normalization processing.

[0079] Following the previous example, this application first addresses the first... This application can use 1-99 percentiles as semantic indicators. The semantic metric is then pruned; secondly, min-max normalization is used to normalize the semantic metric to... The interval is then used to measure the dispersion of semantic indicators, and the weight of semantic indicators is determined based on the proportion of dispersion. Finally, the weighted sum of each semantic indicator after standardization is obtained according to the weight of each semantic indicator to obtain the semantic feature heat.

[0080] The formula for calculating the semantic feature heat is as follows: In equations (15) to (19), Let i be the original value of the multimedia object i on the k-th semantic index; for The value after standardization; The first percentile (lower pruning threshold) of the kth semantic indicator on the training set. The 99th percentile (upper pruning threshold) of the kth semantic indicator on the training set. For the truncation function, Limited to . The variance represents the degree of dispersion of the k-th semantic index. The number of multimedia objects; It represents the average value of the k-th semantic index. The semantic index weight of the k-th semantic index is obtained based on the degree of dispersion normalization; The semantic feature heat of multimedia object i.

[0081] Figure 4 This is a schematic diagram illustrating the process of determining semantic feature popularity as provided in an embodiment of this application. Figure 4 As shown, the process of determining the semantic feature heat includes steps S400 to S403: S400: Retrieves the content data of a multimedia object.

[0082] S401: Determine the semantic metrics of the multimedia object. Step S401 includes calculating the scene category score, calculating the event confidence score, and calculating the content semantic score.

[0083] S402: Standardize the semantic metrics. Step S402 includes collecting the 1st to 99th percentiles and processing all semantic metrics through min-max normalization.

[0084] S403: Determine the semantic index weights, and based on these weights, perform a weighted summation of each semantic index to obtain the semantic feature heat. Step S403 includes calculating the normalized variance of each semantic index, determining the semantic index weights, and obtaining the semantic feature heat through weighted summation.

[0085] In step S103, in order to adapt to the difference in contribution between temporal behavior heat and semantic feature heat, in one or more embodiments of this application, this application can perform weighted summation of temporal behavior heat and semantic feature heat according to preset fusion weights to obtain a comprehensive heat.

[0086] In this embodiment, the application does not limit the specific size of the fusion weight, which can be set according to actual needs. For example, for scenarios with high timeliness requirements, the weight of time behavior popularity can be set to a larger value (such as 0.7); for scenarios where semantic value is more important, the weight of semantic feature popularity can be set to a larger value (such as 0.6).

[0087] Continuing with the previous example, let's analyze the popularity of time-based behaviors. and semantic feature popularity Normalization is performed to obtain and ; Calculate the variance of time-based behavioral heat The variance of semantic feature popularity .in, , Based on the proportion of the variances of the two, the fusion weight is determined. Then, based on the fusion weight, the temporal behavior popularity and the semantic feature popularity are weighted and summed to obtain the comprehensive popularity.

[0088] The formula for calculating the overall popularity is as follows: In equations (20) to (21), The overall popularity of multimedia object i during this access time; This represents the normalized time-based heat of behavior. The normalized semantic feature heat; The weighting for the fusion of time-based behavioral popularity ranges from 0 to 1. The fusion weight for semantic feature popularity ranges from 0 to 1. The variance of time-based behavior popularity reflects the degree of dispersion of time-based behavior popularity in the overall visit; The variance of semantic feature popularity reflects the degree of dispersion of semantic feature popularity in the overall access.

[0089] Using the previous example, the business scenarios are divided into traffic accidents. Empty streets and crowded places Here are three multimedia objects: traffic accident capture (A), empty street capture (B), and pedestrian density capture (C). The number of recent visits for each of these three multimedia objects within a preset time period are as follows: , , The total number of accesses to the three multimedia objects were respectively , , The recent access times of the three multimedia objects are respectively , , The unit is hours; the half-life of all three multimedia objects is set to [value]. The unit is hours.

[0090] The category baseline weights for the three business scenarios are traffic accidents (0.6), empty streets (0.1), and densely populated areas (0.3), respectively. Since the three multimedia objects already have corresponding business scenario labels, each multimedia object is input into the detector of its respective business scenario, and the event confidence scores of the three multimedia objects are A=0.88, B=0.15, and C=0.65, respectively.

[0091] The semantic anchor vectors for each business scenario are respectively the semantic anchor vectors for traffic accidents. Empty street semantic anchor vector Semantic anchor vectors for densely populated areas Using the CLIP model, the content semantic vectors of the three multimedia objects were determined as follows: , , Among them, the first dimension of the semantic anchor vector or content semantic vector indicates whether there are obvious action or accident elements, such as collision, flame, moving objects, etc.; the second dimension indicates whether there are high-attention subjects such as crowds and dense targets in the scene; and the third dimension indicates whether the scene is mainly non-event information such as background, roads, and buildings.

[0092] According to formula (1), the access frequency and popularity of the three multimedia objects are determined as follows: , , According to formula (4), the access time popularity of the three multimedia objects is determined as follows: , , In this example, we assume there are no other historical statistics, and that the maximum value of the historical frequency popularity / historical time popularity of the multimedia object is 1, and the minimum value is 0. Then, according to formula (5), the time behavior popularity of the three multimedia objects is determined as follows: , , .

[0093] The results above show that multimedia object A has a lower overall time behavior popularity due to its lower access frequency; while multimedia object B has a higher overall time behavior popularity due to its higher recent access frequency. However, multimedia object A belongs to the business scenario of traffic accidents, and its business value is obviously higher than that of multimedia object B. If only time behavior popularity is considered, cleaning up cold data may lead to the loss of high business value data.

[0094] Since each multimedia object has a corresponding business scenario label, according to formula (8), the scenario category scores of the three multimedia objects are determined as follows: , , ; Determine the event confidence of the three multimedia objects as , , Based on equations (12) and (13), the content semantic scores of the three multimedia objects are determined as follows: , , Since there are no other data records in this example, according to formula (14), the scene category scores of the three multimedia objects are determined as follows: , , According to formula (15), the standardized values ​​of the semantic indicators of the three multimedia objects are determined as follows: , , .

[0095] According to equation (17), the degree of dispersion of each semantic index is determined as follows: , , According to formula (18), the semantic index weights of each semantic index are determined as follows: , , According to formula (19), the semantic feature heat of each semantic index is determined as follows: , , The popularity of semantic features characterizes the business value of multimedia objects.

[0096] The results above show that although multimedia object A has a low access frequency, it has the highest business value, followed by multimedia object C; and multimedia object B has the lowest business value.

[0097] According to formula (21), the overall popularity of the three multimedia objects is determined as follows: , , .

[0098] In the aforementioned traffic monitoring scenario, the overall popularity of multimedia object A (traffic accident capture), multimedia object B (empty street capture), and multimedia object C (pedestrian density capture) are 0.60515, 0.41654, and 1.59973, respectively. Among these, A and C are multimedia objects with high semantic value, and their overall popularity is higher than that of B. This application can consider both temporal behavioral popularity and semantic feature popularity, effectively distinguishing between high and low business value, and improving the accuracy and reliability of multimedia object popularity assessment.

[0099] Based on the above-described method for determining the popularity of multimedia objects, this application also provides a specific embodiment of a device for determining the popularity of multimedia objects.

[0100] like Figure 5 As shown, Figure 5 This is a schematic diagram of a multimedia object popularity determination device provided in an embodiment of this application. The device 500 includes an acquisition module 501, a time behavior popularity module 502, a semantic feature popularity module 503, and a comprehensive popularity module 504.

[0101] Module 501 is used to obtain access logs and content data of multimedia objects; The time behavior heat module 502 is used to determine the time behavior heat of the multimedia object based on the access log, wherein the time behavior heat characterizes the activity level of the multimedia object in the time dimension; The semantic feature popularity module 503 is used to determine the semantic feature popularity of the multimedia object based on the content data, wherein the semantic feature popularity characterizes the importance of the multimedia object in the semantic dimension; The comprehensive popularity module 504 is used to fuse the comprehensive popularity of the multimedia object based on the time behavior popularity and the semantic feature popularity.

[0102] In one feasible implementation, the time behavior heat module is specifically used to determine, based on the access log, the total number of accesses to the multimedia object, the most recent number of accesses within a preset time period, and the most recent access time, wherein the most recent access time is the time interval between the current access time and the last access time of the multimedia object; to use the ratio of the most recent accesses to the total number of accesses as the access frequency heat; to determine the access time heat based on the most recent access time, wherein the access time heat is negatively correlated with the most recent access time; and to determine the time behavior heat based on the access frequency heat and the access time heat.

[0103] In one feasible implementation, the time behavior heat module can also be used to determine the decay coefficient based on the preset half-life of the multimedia object; and to determine the access time heat based on the recent access time and the decay coefficient through a preset exponential decay function.

[0104] In one feasible implementation, the time behavior heat module can also be used to obtain the historical frequency heat and historical time heat of the multimedia object; determine the frequency dispersion of the historical frequency heat and the sum of the time dispersion of the historical time heat; use the ratio of the frequency dispersion to the sum as the frequency weight; use the ratio of the time dispersion to the sum as the time weight; and perform a weighted summation of the access frequency heat and the frequency weight, and the access time heat and the time weight to obtain the time behavior heat.

[0105] In one feasible implementation, the semantic feature heat module is specifically used to determine the semantic indicators of the multimedia object. The semantic indicators include at least one of scene category score, event confidence score, and content semantic score. The scene category score represents the importance of the business scene to which the multimedia object belongs, the event confidence score represents the detection confidence of the business scene to which the multimedia object belongs, and the content semantic score represents the degree of association between the multimedia object and each business scene in the semantic space. Based on the semantic indicators, the semantic feature heat is determined.

[0106] In one feasible implementation, the semantic feature heat module can also be used to determine the probability of the multimedia object belonging to each of the business scenarios; and to obtain the scenario category score by weighted summation of the probability of each scenario category according to the category benchmark weight of each business scenario.

[0107] In one feasible implementation, the semantic feature popularity module can also be used to extract the content semantic vector of the content data; obtain the semantic anchor vector of each of the business scenarios; determine the semantic similarity between the content semantic vector and each of the semantic anchor vectors; and determine the content semantic score based on the semantic similarity.

[0108] In one feasible implementation, the semantic feature popularity module can also be used to acquire historical data of the multimedia object on each of the semantic indicators; determine the degree of dispersion of the semantic indicators based on the historical data; determine the semantic indicator weight of each semantic indicator based on the degree of dispersion of each semantic indicator; and perform a weighted summation of each semantic indicator based on the weight of each semantic indicator to obtain the semantic feature popularity.

[0109] Figure 6 A schematic diagram of the hardware structure of a multimedia object heat determination device provided in an embodiment of this application is shown.

[0110] A multimedia object heat determination device may include a processor 601 and a memory 602 storing computer program instructions.

[0111] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0112] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one embodiment, memory 602 may include removable or non-removable (or fixed) media, or memory 602 may be non-volatile solid-state memory. Memory 602 may be internal or external to the integrated gateway disaster recovery device.

[0113] In one instance, memory 602 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0114] Memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the multimedia object heat determination method according to one aspect of this application.

[0115] The processor 601 reads and executes computer program instructions stored in the memory 602 to achieve... Figure 1 The multimedia object heat determination method in the illustrated embodiment.

[0116] In one example, a multimedia object heat determination device may further include a communication interface 603 and a bus 604. Wherein, as Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 604 and complete communication with each other.

[0117] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0118] Bus 604 includes hardware, software, or both, that couples components of a multimedia object heat-determining device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0119] Furthermore, in conjunction with the multimedia object heat determination method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the multimedia object heat determination methods in the above embodiments.

[0120] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the multimedia object heat determination methods described in the above embodiments.

[0121] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0122] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0123] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0124] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0125] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for determining the heat of a multimedia object, characterized in that, include: Retrieve access logs and content data for multimedia objects; Based on the access logs, the temporal behavior heat index of the multimedia object is determined, and the temporal behavior heat index characterizes the activity level of the multimedia object in the time dimension; Based on the content data, the semantic feature popularity of the multimedia object is determined, and the semantic feature popularity represents the importance of the multimedia object in the semantic dimension; The overall popularity of the multimedia object is obtained by fusing the temporal behavior popularity and the semantic feature popularity.

2. The method according to claim 1, characterized in that, Based on the access logs, the temporal behavior popularity of the multimedia object is determined, specifically including: Based on the access log, determine the total number of accesses to the multimedia object, the number of recent accesses within a preset time period, and the recent access time, wherein the recent access time is the time interval between the current access time and the previous access time of the multimedia object; The ratio of the most recent access count to the total access count is used as the access frequency popularity. Based on the most recent access time, the access time popularity is determined, and the access time popularity is negatively correlated with the most recent access time; The time behavior popularity is determined based on the access frequency popularity and the access time popularity.

3. The method according to claim 2, characterized in that, Determining the access time popularity based on the most recent access time specifically includes: The attenuation coefficient is determined based on the preset half-life of the multimedia object; Based on the most recent access time and the decay coefficient, the access time popularity is determined using a preset exponential decay function.

4. The method according to claim 3, characterized in that, The time-based behavior popularity is determined based on the access frequency popularity and the access time popularity, specifically including: Obtain the historical frequency popularity and historical time popularity of the multimedia object; Determine the sum of the frequency dispersion of the historical frequency heat and the time dispersion of the historical time heat; The ratio of the frequency dispersion to the sum value is used as the frequency weight; The ratio of the time dispersion to the sum value is used as the time weight; The time behavior popularity is obtained by weighting and summing the access frequency popularity with the frequency weight and the access time popularity with the time weight.

5. The method according to claim 1, characterized in that, Based on the content data, the semantic feature popularity of the multimedia object is determined, specifically including: The semantic metrics of the multimedia object are determined. The semantic metrics include at least one of scene category score, event confidence score and content semantic score. The scene category score represents the importance of the business scene to which the multimedia object belongs. The event confidence score represents the detection confidence of the business scene to which the multimedia object belongs. The content semantic score represents the degree of association between the multimedia object and each business scene in the semantic space. The popularity of the semantic features is determined based on the semantic indicators.

6. The method according to claim 5, characterized in that, Determining the semantic indicators of the multimedia object specifically includes: Determine the probability that the multimedia object belongs to the scenario category of each of the aforementioned business scenarios; Based on the category baseline weights of each business scenario, the probabilities of each scenario category are weighted and summed to obtain the scenario category score.

7. The method according to claim 5, characterized in that, Determining the semantic indicators of the multimedia object specifically includes: Extract the content semantic vector from the content data; Obtain the semantic anchor vectors for each of the aforementioned business scenarios; Determine the semantic similarity between the content semantic vector and each of the semantic anchor vectors; The semantic score of the content is determined based on the semantic similarity of each item.

8. The method according to claim 5, characterized in that, The semantic metrics include at least two of the following: the scene category score, the event confidence score, and the content semantic score. Determining the semantic feature popularity based on the semantic indicators specifically includes: Obtain historical data of the multimedia object for each of the semantic metrics; Based on the historical data, determine the degree of dispersion of the semantic indicators; The semantic index weights of each semantic index are determined based on the degree of dispersion of each semantic index. The semantic feature popularity is obtained by weighting and summing the semantic indicators according to their respective weights.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the multimedia object heat determination method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the multimedia object heat determination method as described in any one of claims 1-8.

Images