Commodity recommendation method and device and computer storage medium

Abstract

The invention discloses a commodity recommendation method and device and a computer storage medium, and relates to the technical field of image recognition, and the method comprises the steps: extracting a key frame of a video, and recognizing a commodity instance in the key frame; determining a display time period of the commodity instance according to a timestamp of the commodity instance appearing in the video; retrieving commodity information corresponding to the commodity instance from a commodity database, and associating the commodity information with the display time period; and when it is detected that the video is played to the display time period corresponding to the commodity instance, displaying the commodity information corresponding to the display time period on a playing interface. According to the method and the device, the commodity instance is automatically identified in the video and the corresponding display time period is determined, so that the function of automatically displaying the related commodity information when the specific commodity is played in the video is realized, and the real-time performance of acquiring the commodity information when a user watches the video is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a product recommendation method, device and computer storage medium. Background Art

[0002] In the fields of IPTV (Internet Protocol Television), OTT (Over The Top) video and live broadcast, the application of image recognition technology has gradually become a trend. At present, in order to recommend relevant products involved in the video to users who watch the video, the product information is usually bound to the video content by manual preset. Specifically, the staff will manually search for the location where a specific product appears in the video, and promote the corresponding product information to users by establishing the relationship between the product and the video broadcast time point.

[0003] However, with the explosive growth of video content and the increasing variety of products, manual retrieval and product binding methods are unable to cope with massive amounts of video data, resulting in limited real-time product recommendations. Summary of the invention

[0004] The main purpose of this application is to provide a product recommendation method, device and computer storage medium, aiming to solve the technical problem of how to recommend related products in real time during video content playback.

[0005] To achieve the above purpose, the present application provides a product recommendation method, which includes: Extracting key frames of the video and identifying product instances in the key frames; Determining a display time period of the product instance according to a timestamp of when the product instance appears in the video; Retrieving the commodity information corresponding to the commodity instance from a commodity database, and associating the commodity information with the display time period; When it is detected that the video is played to the display time period corresponding to the product instance, the product information corresponding to the display time period is displayed on the playback interface.

[0006] In one embodiment, before the step of extracting key frames of the video and identifying commodity instances in the key frames, the following steps are included: Extracting image feature data of commodities, and matching the image feature data with commodity identifiers to form a commodity feature library; Using the image feature data in the product feature library to train a product recognition model; The trained product recognition model is used to recognize the product instance in the key frame.

[0007] In one embodiment, the step of determining the display time period of the commodity instance according to the timestamp of the commodity instance appearing in the video includes: Determine a start time point and an end time point of display of each commodity instance according to a timestamp of each commodity instance appearing in the video; A display time period of the commodity instance is determined based on the display start time point and the display end time point.

[0008] In one embodiment, the step of determining the start time point and the end time point of display of the commodity instance according to the timestamp of each commodity instance appearing in the video further includes: Based on the timestamp, calculating the duration of appearance of the commodity instance in the video; According to the preset display rules, a basic display duration is set for the product information; If the continuous appearance time of the product instance is less than the basic display duration, setting the display end time point of the product information to the start display time point of the product instance plus the basic display duration; If the continuous appearance time of the product instance is greater than or equal to the basic display duration, the display end time point of the product information is set as the end point of the continuous appearance time of the product instance.

[0009] In one embodiment, the step of retrieving the commodity information corresponding to the commodity instance from the commodity database and associating the commodity information with the display time period includes: Obtaining a product identifier of the product instance; Using the commodity identifier, query and retrieve the commodity information corresponding to the commodity identifier in the commodity database; An associated data structure is created, and the product information and the corresponding display time period are taken as a key-value pair and stored in the associated data structure.

[0010] In one embodiment, after the step of creating an associated data structure and storing the product information and the corresponding display time period as a key-value pair in the associated data structure, the step further includes: Creating a product display tag for each product instance, wherein the product display tag includes the product information and the corresponding display time period; Setting the display format and display position of the product display label in the playback interface; When the display time period is reached, the display of the product display label is automatically triggered based on the display format and the display position.

[0011] In one embodiment, the step of retrieving the commodity information corresponding to the commodity instance from the commodity database and associating the commodity information with the display time period further includes: Obtaining the start display time points of all the product instances, and sorting the product display tags corresponding to the product instances according to the start display time points; A display sequence number is assigned to each of the product display tags, and the display sequence number is used to determine the display order of the product information when the video is played.

[0012] In one embodiment, when it is detected that the video is played to the display time period corresponding to the product instance, the step of displaying the product information corresponding to the display time period on the playback interface includes: Monitor the playback progress of the video in real time to determine the current playback time point; Comparing the playback time point with the display time period corresponding to the commodity instance in the video; When the playback time point is within the display time period, the display of the product information corresponding to the display time point is triggered.

[0013] An embodiment of the present application also provides a product recommendation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the product recommendation method described above.

[0014] An embodiment of the present application also provides a computer storage medium, which is a computer-readable storage medium. A computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the product recommendation method described above are implemented.

[0015] The embodiment of the present application discloses a method for recommending products, which extracts key frames of a video and identifies product instances in the key frames; determines the display time period of the product instance according to the timestamp of the product instance appearing in the video; retrieves product information corresponding to the product instance from a product database, and associates the product information with the display time period; and displays the product information corresponding to the display time period on a playback interface when it is detected that the video is played to the display time period corresponding to the product instance. The present application realizes the function of automatically displaying relevant product information when a video is played to a specific product by automatically identifying product instances in a video and determining the corresponding display time period, thereby improving the real-time performance of users obtaining product information when watching a video. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1It is a flowchart of the first embodiment of the commodity recommendation method involved in the embodiment scheme of the present application; Figure 2 This is a flow chart of a second embodiment of a method for recommending products according to an embodiment of the present application; Figure 3 This is a flow chart of a third embodiment of a method for recommending products according to an embodiment of the present application; Figure 4 This is a flow chart of a fourth embodiment of a method for recommending products according to an embodiment of the present application; Figure 5 This is a schematic diagram of the structure of the equipment recommended for this application.

[0017] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] In the fields of IPTV (Internet Protocol Television), OTT (Over The Top) video and live broadcast, the application of image recognition technology has gradually become a trend. At present, in order to recommend relevant products involved in the video to users who watch the video, the product information is usually bound to the video content by manual preset. Specifically, the staff will manually search for the location where a specific product appears in the video, and promote the corresponding product information to users by establishing the relationship between the product and the video broadcast time point.

[0020] However, with the explosive growth of video content and the increasing variety of products, manual retrieval and product binding methods are unable to cope with massive amounts of video data, resulting in limited real-time product recommendations.

[0021] In order to solve the above-mentioned defects existing in the related art, the embodiment of the present application proposes a product recommendation method, which extracts the key frames of the video and identifies the product instances in the key frames; determines the display time period of the product instance according to the timestamp of the product instance appearing in the video; retrieves the product information corresponding to the product instance from the product database, and associates the product information with the display time period; and displays the product information corresponding to the display time period on the playback interface when it is detected that the video is played to the display time period corresponding to the product instance. The present application realizes the function of automatically displaying relevant product information when the video plays to a specific product by automatically identifying the product instance in the video and determining the corresponding display time period, thereby improving the real-time performance of users in obtaining product information when watching videos.

[0022] It should be noted that the execution subject of this embodiment can be a product recommendation system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a product recommendation device capable of realizing the above functions, etc. The following takes the product recommendation system as an example (hereinafter referred to as the "system") to illustrate this embodiment and the following embodiments.

[0023] Please refer to the first embodiment of the product recommendation method proposed in this application. Figure 1 The method comprises steps S10 to S40: Step S10: extract key frames of the video and identify commodity instances in the key frames.

[0024] In this embodiment, videos refer to those video contents that are planned and produced to promote a product, service, or brand, etc. and contain clear product information or potential commercial value. These videos are disseminated through various channels, including but not limited to traditional TV advertising videos, online videos on online platforms, and short video promotions on social media. The purpose of such videos is to attract the audience's attention, thereby increasing the popularity and market influence of products or brands, etc.

[0025] After acquiring these videos, the system will use image processing and computer vision technology to analyze the video content. The analysis process involves processing each frame in the video to identify those that can effectively show the characteristics and details of the product, namely key frames. These key frames may include close-ups of the product, activities related to the product, etc. By extracting these key frames, the system can effectively capture the product information in the video.

[0026] In a feasible implementation manner, step S10 may include steps S11 to S13: Step S11, extracting the image feature data of the commodity, and matching the image feature data with the commodity identification to form a commodity feature library.

[0027] In this embodiment, the system first collects a large number of product images, which may include product images provided by manufacturers, product advertising images on the Internet, etc. Then, the system performs data cleaning on the collected product images to remove noise and irrelevant information in the product images, such as removing the background, adjusting the brightness and contrast, and correcting the geometric distortion of the image. Next, the system performs standardization processing on the cleaned product images, including adjusting the image size to meet the input requirements of the model, and performing operations such as image cropping, scaling and flipping. Finally, after completing the processing of the product images, the system extracts the image feature data of each product image. Image feature data includes but is not limited to color histograms, texture features, shape descriptors, and local feature points, such as SIFT (Scale-Invariant Feature Transform) features, SURF (Speeded Up Robust Features) features, etc.

[0028] Then, the system associates these image feature data with the corresponding product identification to ensure that each image feature data can correspond to a specific product. In this way, the system establishes a product feature library, and each record in the product feature library contains the feature information and identification of the product.

[0029] It should be noted that a color histogram is a statistical chart that describes the distribution of colors in an image, a texture feature is a parameter that describes the texture characteristics of an image surface, and a shape descriptor is information used to describe the shape of an object in an image. A product identifier can be a product model, product name, or other unique identification code.

[0030] Step S12: training a commodity recognition model using the image feature data in the commodity feature library.

[0031] In this embodiment, the system trains the commodity recognition model through the image feature data and the corresponding commodity identification in the commodity feature library. The training process involves using the image feature data as the model input and learning the commodity feature model through the machine learning algorithm so that the model can accurately identify the commodity from the new video.

[0032] The product recognition model can select algorithms such as support vector machine (SVM), deep learning, neural network, etc., and adjust the model parameters through repeated iterations until the accuracy of the model in identifying products reaches the preset threshold. Specifically, the steps for training the product recognition model include: Initialize model parameters: When you select a convolutional neural network (CNN) to build a product recognition model (hereinafter referred to as the "model"), the system will randomly initialize weights and biases. The weights usually follow a normal distribution with a mean of 0 and a standard deviation of 0.01, while the bias is initialized to 0 or a small constant, such as 0.01.

[0033] Split data set: The system divides the image feature data in the product feature library into a training data set and a test data set, which can be split in a ratio of 70% training data set and 30% test data set. In this process, in order to ensure the randomness of the data, a stratified sampling method can be used to keep the distribution of product categories in the training data set and the test data set consistent.

[0034] Forward propagation: Taking a CNN with convolutional layers, pooling layers, and fully connected layers as an example, the input image feature data is first processed through a series of convolutional layers and activation functions (such as ReLU) to extract the feature map; then the feature dimension is reduced through the pooling layer; finally, a vector is output through the fully connected layer, which is converted into a probability distribution through the softmax function to represent the predicted product category.

[0035] Calculate the loss function: Use the cross entropy loss function, the calculation formula is ,in is the one-hot encoding of the true label, is the probability predicted by the model.

[0036] Backpropagation and gradient descent: Calculate the gradient of the loss function with respect to the model parameters and update the parameters using the Adam optimizer (a method that combines momentum and adaptive learning rate). For example, for the weights and bias b, the update rule is and ,in is the learning rate.

[0037] Iterative optimization: Set the training cycle to 20 epochs. After each epoch, the system adjusts the learning rate according to the average loss of the current epoch (such as using the learning rate decay strategy) and continues to the next round of training.

[0038] It should be noted that epochs refers to the number of times the entire training dataset is completely traversed and trained by the model. An epoch is a complete training of the model on the entire training dataset. During the training process, multiple epochs are usually required for the model to achieve good performance, because by traversing the data multiple times, the model can gradually learn and improve its parameters to better predict the output.

[0039] Specifically, the process of an epoch is as follows: the model starts training for an epoch; the model groups all samples in the training data set according to a certain batch size; for each batch, the model performs the following steps: forward propagation calculates the predicted value, calculates the loss function, and then updates the model weights through back propagation; when all samples in the training data set have been processed once in this way, an epoch is completed.

[0040] Performance evaluation: After each epoch, the system uses the test data set for evaluation, calculates accuracy, recall, F1 score and other indicators, and uses the confusion matrix to analyze the performance of the model on different categories. The accuracy calculation formula is: accuracy = .

[0041] Hyperparameter tuning: Based on the model performance on the test dataset, adjust the hyperparameters, such as adjusting the learning rate from 0.001 to 0.0001, increasing the ratio of the dropout layer from 0.5 to 0.7 to reduce overfitting, or increasing the number of filters in the convolutional layer to improve the expressiveness of the model.

[0042] Model saving: When the accuracy of the model on the test data set reaches 95% (the preset threshold can be customized), the model training is considered complete. The system saves the structure and parameters of the model as a file, such as "model.h5" or "model.pth", for subsequent loading and use.

[0043] Step S13: using the trained commodity recognition model to recognize the commodity instance in the key frame.

[0044] In this embodiment, the system inputs the key frame into the trained commodity recognition model, and the model detects and recognizes the commodity in the key frame based on the parameters obtained by training a large amount of commodity image feature data in step S12. Specifically, the system first performs image processing on the key frame to ensure that it meets the input requirements of the model, and then sends the processed key frame to the model for forward propagation. The model recognizes the commodity instance in the key frame through its learned feature mapping.

[0045] Furthermore, during the model training process, the image feature data of the product and its corresponding product identification are learned by the model together. This means that the image feature data has been labeled with the corresponding product identification during the training phase. Therefore, the trained model can use its learned feature mapping mechanism to correspond the product instance identified in the keyframe to the correct product identification, thereby achieving accurate recognition and labeling of the product instance.

[0046] It should be noted that the product instance refers to the specific product object that appears in the video frame.

[0047] Step S20: determining a display time period of the commodity instance according to a timestamp when the commodity instance appears in the video.

[0048] In this embodiment, the system uses a previously trained product recognition model to identify product instances in a video. During video playback, the system obtains the timestamp information of the product instance by monitoring the playback progress of the video. Whenever the product recognition model recognizes a product instance in a key frame, the system records the playback time point of the current video, which is the timestamp of the product instance appearing in the video. Next, the system determines the display time period of the product based on the timestamp information. The display time period refers to a specific time range determined by the system based on the timestamp of the product instance appearing in the video. Within this time range, the system will display product information related to the product instance on the playback interface.

[0049] It is understandable that the display time period is set by the system. It may coincide with the actual duration of the product instance appearing in the video, or it may be adjusted based on marketing strategies, system rules, or user experience considerations. The display time period includes the start time point and the end time point. The start time point is usually set to the timestamp of the first appearance of the product instance during video playback.

[0050] It should be noted that the continuous appearance time represents the complete display duration of the same product instance in the video. It reflects the complete time span from the first appearance to the last appearance of the product instance during the video playback. In order to calculate the continuous appearance time, the system tracks all timestamps of the same product instance appearing in the video. The timestamp represents the playback time point corresponding to the exact frame of each appearance and disappearance of the product in the video. The system sorts these timestamps and then calculates the time difference between the first timestamp (the time when the product instance first appears) and the last timestamp (the time when the product instance last appears). This time difference is the continuous appearance time of the product instance.

[0051] Step S30: Retrieve the commodity information corresponding to the commodity instance from the commodity database, and associate the commodity information with the display time period.

[0052] In this embodiment, after the commodity recognition model recognizes the commodity instance in the video, it will also output the commodity identification corresponding to the commodity instance. The system can use the commodity identification as a query condition to access the commodity database. The commodity information matching the commodity instance is retrieved by executing a query statement or using a database API interface. The commodity information may include detailed information such as the commodity name, commodity brand, commodity price, commodity description, commodity category, and purchase link.

[0053] Once the product information corresponding to the product instance is retrieved, the system will map the product information to the display time period corresponding to the product instance. The mapping operation usually involves combining the product information with the timestamp data of the video to form a one-to-many association, where each display time period corresponds to a specific product instance.

[0054] In a feasible implementation, the association between product information and display time period can be achieved by creating an association table in a database, which includes fields such as product information ID, video ID, start timestamp and end timestamp of the display time period. In this way, the system can not only identify the products in the video, but also provide product information related to these products, thereby providing users with a more accurate product recommendation experience.

[0055] Step S40: when it is detected that the video is played to the display time period corresponding to the product instance, the product information corresponding to the display time period is displayed on the playback interface.

[0056] In a feasible implementation, in order to display the corresponding product information on the playback interface when the video playback progress is in the display time period corresponding to the product instance, the system monitors the video playback progress in real time and maintains a mapping table of product display time periods corresponding to the video timeline. The mapping table records the display time period of each product instance.

[0057] When the video playback progress matches the display time period of a product instance recorded in the mapping table, the system activates the product information display module through an event triggering mechanism. This event triggering mechanism may involve a callback function or observer mode in programming, so that it can respond in time when the playback time point matches the display time period.

[0058] The product information display module retrieves complete product information from the product database according to the product identifier recorded in the mapping table. The retrieved product information includes name, price, purchase link, and brief description, which are dynamically superimposed on the video playback interface in the form of a graphical interface. This dynamic superposition technology may involve graphical user interface programming, as well as layer and transparency management to ensure that the display of product information is both conspicuous and does not interfere with the viewing of video content.

[0059] Furthermore, the system can also consider setting parameters such as transparency and pop-up position for displaying product information. The adjustment of these parameters is based on user interface design principles and analysis of user viewing habits to ensure that the display of product information does not have a negative impact on the user's viewing experience.

[0060] Please refer to Figure 2In the commodity recommendation method of the second embodiment proposed in this application, step S20 may include steps S21-S22: Step S21: determining a start time point and an end time point of display of each commodity instance according to a timestamp of the commodity instance appearing in the video.

[0061] In this embodiment, based on the timestamp information of each product instance appearing in the video, the system can reasonably set the display time period of the product instance, thereby determining the start display time point and the end display time point of the product information corresponding to each product instance.

[0062] Specifically, the system first tracks the exact frames where a product instance first and last appears by analyzing all timestamps of the product instance in the video. These timestamps are captured in real time during video playback and correspond to the actual moments when the product instance appears in the video.

[0063] The system sorts these timestamps in chronological order and extracts the timestamp of the first appearance of each product instance as the start display time point. Then the system can preset the basic display duration and superimpose the end display time point on the basis of the start display time point, that is, the end display time point = start display time point + basic display duration. Alternatively, the continuous appearance time of the product instance can be used as the display time period of the product information corresponding to the product instance, that is, the timestamp of the first appearance of the product instance during the video playback is used as the start display time point, and the timestamp of the last appearance of the product instance during the video playback is used as the end display time point.

[0064] Step S22: Determine a display time period of the product instance based on the display start time point and the display end time point.

[0065] The system will determine the display time period of the product instance based on the display start time point and display end time point determined in step S21.

[0066] In a feasible implementation, the system can create a time interval object, which forms a continuous time period based on the start time point and the end time point of the display. The continuous time period represents the visible display time of the product information corresponding to the product instance in the video.

[0067] The system stores time interval objects in a database or memory data structure so that they can be quickly retrieved and used when the video playback progress is at the corresponding time point. In order to improve retrieval efficiency, the system can also index these time interval objects and use data structures such as B-trees or hash tables to optimize search performance.

[0068] It should be noted that B-tree is a self-balancing tree data structure used for indexing databases and file systems. B-tree improves data retrieval efficiency by distributing data in multiple nodes and keeping the tree height relatively low. Each node of the B-tree can have multiple child nodes and can store multiple data items, which enables the B-tree to maintain a good balance during insertion and deletion operations and reduce data search time.

[0069] Furthermore, the system also needs to handle the real-time matching problem during video playback. When the video playback progress matches any display time period in the stored time interval object, the system will trigger the corresponding product information display event. This event can be set by setting a timer or listener to continuously check the time point of the current playback progress during video playback and compare it with the time interval object.

[0070] Understandably, in order to ensure the accuracy of the display time period, the system also takes into account changes in video playback speed, such as normal speed, fast forward or rewind. The system dynamically adjusts the time interval object based on the real-time playback speed of the video to ensure that the display time of the product information remains accurate even when user operations affect the playback speed.

[0071] In this way, the system can accurately divide the display time period of the product information corresponding to each product instance in the video, providing a time benchmark for subsequent product information display and recommendation.

[0072] In a feasible implementation, step S21 may include steps S211 to S214: Step S211: Calculate the continuous appearance time of the product instance in the video based on the timestamp.

[0073] It should be noted that the continuous appearance time indicates the complete display duration of the same product instance in the video, that is, first obtain the timestamp of the first and last appearance of the product instance during the video playback, and then calculate the time difference between the two time points.

[0074] Step S212: setting a basic display duration for the product information according to a preset display rule.

[0075] It should be noted that the preset display rules are system-defined rules for how product information should be displayed and for how long. The setting of display rules may be based on a variety of factors, including but not limited to audience viewing habits, marketing strategies, content relevance, and expected user engagement.

[0076] The basic display duration is the standard time length set by the system for the product information corresponding to the product instance based on factors such as marketing strategies or user experience. The purpose is to ensure that the display duration of the product information in the video can meet certain business needs or user experience needs.

[0077] Step S213: If the continuous appearance time of the product instance is less than the basic display duration, the display end time point of the product information is set to the start display time point of the product instance plus the basic display duration.

[0078] Step S214: If the continuous appearance time of the product instance is greater than or equal to the basic display duration, the display end time point of the product information is set as the end point of the continuous appearance time of the product instance.

[0079] In this embodiment, the system calculates the continuous appearance time of the product instance in the video and compares it with the preset basic display duration, thereby determining the display time period of the product information corresponding to the product instance.

[0080] Specifically, the system first determines the duration of the product instance, and then compares the duration with a preset basic display duration.

[0081] If the product instance continues to appear for less than the preset basic display duration, the system will automatically adjust the end time of the product information display.

[0082] Specifically, the system will set the end time of product information display to the time point after the product instance starts displaying and adds the preset basic display duration. In this way, even if the product instance does not appear in the video for a sufficient time, the product information can still be displayed for the preset basic duration.

[0083] If the continuous appearance time of the product instance is greater than or equal to the basic display duration, the system directly uses the start timestamp and end timestamp in the continuous appearance time of the product instance as the display time period of the product information.

[0084] In this case, the start time of product information display is the timestamp when the product instance first appears in the video, and the end time of display is set to the timestamp when the product instance last appears in the video, ensuring that the product information has sufficient display time while avoiding excessively extending the display time and affecting the audience experience.

[0085] In another feasible implementation, step S21 may further include: When a product instance is detected with multiple timestamps in a video, the system first sorts these timestamps in the order of appearance. Then, the system analyzes the intervals between each sorted timestamp to distinguish whether the product instance is continuous or discontinuous during video playback.

[0086] Specifically, the system may pre-set a time interval threshold value to determine whether two consecutive timestamps constitute continuous occurrences.

[0087] If the interval between two consecutive timestamps is less than or equal to a preset time interval threshold, it is considered that the product instances corresponding to the two consecutive timestamps appear continuously, that is, the product does not disappear in the video but continues to appear in the audience's sight.

[0088] If the interval between two consecutive timestamps exceeds a preset time interval threshold, it is considered that the product instances corresponding to the two consecutive timestamps appear intermittently during video playback, that is, the product disappears at a certain time point and then appears again at a subsequent time point in the video.

[0089] It should be noted that two connected timestamps refer to two consecutive time points in the video that record the moment when the product instance appears. When the system detects that the same product instance appears in different frames of the video, the system generates a timestamp for each appearance of the product instance. These timestamps are arranged in the order of appearance to form a time series. In this time series, any two adjacent timestamps constitute two connected timestamps.

[0090] If product instances appear continuously, the system will treat these consecutive timestamps as a whole and merge them into a single display time period. The start time point of this display time period is the first consecutive timestamp, and the end time point is the last consecutive timestamp.

[0091] For the discontinuous parts, the system adopts an ignoring strategy. If the interval between the timestamps exceeds the preset time interval threshold, it means that the display of the product instance in the video is discontinuous, and the system will not include these scattered timestamps in the display time period of the product information corresponding to the product instance.

[0092] This strategy helps exclude those short or discontinuous product displays, ensuring that only truly significant and continuous product displays are used for product recommendations. In this way, the system can accurately divide the display time period of product instances in the video, providing an accurate time benchmark for subsequent product information displays, thereby optimizing the audience's viewing experience and the timing of product recommendations.

[0093] In another feasible implementation, the system can introduce the concept of effective time period to enhance its ability to filter the display time period of the product information of the product instance. Specifically, the system first determines the consecutive timestamps according to the threshold of the time interval in the previous implementation, and merges these timestamps into a preliminary display time period.

[0094] It should be noted that the effective time period refers to the time interval in which one or more product information representing the product instance can be actually and effectively displayed in the video after the system filters and merges consecutive timestamps according to the time interval threshold.

[0095] Specifically, the system first calculates the length of the preliminary display time period, that is, the time difference from the start timestamp to the end timestamp of the continuous appearance of the same product instance. The time difference corresponding to each continuous appearance is a preliminary display time period.

[0096] Next, the system will compare the length of the preliminary display time period with the threshold of the pre-set effective time period. If the length of the preliminary display time period is greater than or equal to the threshold of the effective time period, the system will confirm the corresponding preliminary display time period as the effective display time period. That is, only within the effective display time period, the system will arrange the display of the product information of the product instance corresponding to the effective display time period.

[0097] This screening mechanism based on the length of the effective time period enables the system to exclude those product displays that appear briefly and continuously and may not be sufficient to attract the audience's attention, thereby improving the quality and effectiveness of product recommendations.

[0098] In this way, the system can more accurately capture the effective product displays in the video that may have a real impact on the audience, and provide the audience with more relevant and valuable product information.

[0099] It should be noted that the system is able to identify and record multiple independent display time periods of the same product instance in a video. This means that if a product instance appears multiple times in different parts of a video, each appearance is considered a separate display time period. Therefore, there can be multiple valid display time periods related to the same product instance in a video.

[0100] Please refer to Figure 3 In the third embodiment of the product recommendation method proposed in this application, step S30 may include steps S31 to S33: Step S31: Obtain the commodity identification of the commodity instance.

[0101] In this embodiment, the system detects and identifies the commodity instances in the keyframes through the previously trained commodity recognition model. In this process, the commodity recognition model will output a specific commodity identification for each identified commodity instance, which may be a predefined digital ID, barcode, QR code, or other unique identification code.

[0102] Step S32: using the commodity identification, querying and retrieving the commodity information corresponding to the commodity identification in the commodity database.

[0103] In this embodiment, the system uses the commodity identifier as a query condition to retrieve commodity information matching the identifier in the commodity database. The commodity database stores a large amount of commodity information, and each commodity information is associated with a unique commodity identifier. The system uses a query algorithm to quickly retrieve records of commodity information matching the queried commodity identifier.

[0104] Step S33: creating an associated data structure, and storing the product information and the corresponding display time period as a key-value pair in the associated data structure.

[0105] It should be noted that the associated data structure can be a database table, a hash table, or any other data structure suitable for fast retrieval. In this associated data structure, the display time period of the product information corresponding to each product instance is used as the key, and the corresponding product information is used as the value, which is stored as a key-value pair. In this way, when the video is played to a specific time period, the system can quickly retrieve the product information that should be displayed from this associated data structure and display it on the video playback interface.

[0106] Further, step S33 may include steps S331 to S333: Step S331: Create a product display tag for each product instance, wherein the product display tag includes the product information and the corresponding display time period.

[0107] The system creates a product display tag for each identified product instance. This product display tag is a data structure that contains product information and the corresponding display time period. Product information may include the product's name, price, promotion, purchase link, detailed description, picture or video, etc. The display time period specifies the start time and end time when the product information should be displayed during video playback. The system will generate a formatted tag based on this information. This tag can be an HTML element, a graphic object, or any other data format suitable for display on the video playback interface.

[0108] Step S332, setting the display format and display position of the product display label in the playback interface.

[0109] In this embodiment, the display format may involve visual styles such as the size, color, font, and animation effect of the product display label. The system will determine the display format based on the content of the video and the design of the user interface to ensure that the product display label can attract the user's attention and will not interfere with the viewing of the video content. The selection of the display position is based on the consideration of the user interface layout and user interaction habits. Common positions include the bottom and side of the playback interface or covering the lower right corner of the video.

[0110] Step S333: When the video is played to the display time period, the display of the product display label is automatically triggered based on the display format and the display position.

[0111] In this embodiment, the system integrates these product display tags with the video playback logic. When the video playback progress matches the display time period specified in the product display tag, the system will automatically trigger and display the corresponding product display tag.

[0112] The triggering process involves real-time monitoring of the video playback time and matching it with the timestamp in the product display tag. Once the match is successful, the system will render the product display tag onto the playback interface according to the previously set display format and display position parameters.

[0113] In another feasible implementation, step S30 may further include steps S321-S322: Step S321, obtaining the start display time points of all the product instances, and sorting the product display tags corresponding to the product instances according to the start display time points.

[0114] In this embodiment, the system obtains the start display time points of all product instances from the associated data structure. This data structure records the display time period of the product information corresponding to each product instance, including the start display time point and the end display time point. The system traverses this associated data structure, collects the start display time points of the product information corresponding to each product instance, and uses the start display time points as the basis for sorting.

[0115] Next, the system will sort the product display tags according to the start time of display. The sorting process can adopt various common sorting algorithms, such as quick sort, merge sort or heap sort. After sorting, the product display tags will be organized into an ordered list to ensure that when the video is played, the product information can be displayed in the order in which it appears in the video.

[0116] Step S322: assign a display sequence number to each of the product display tags, where the display sequence number is used to determine the display order of the product information when the video is played.

[0117] In this embodiment, the system assigns a display sequence number to the sorted product display tags. The display sequence number is a unique identifier used to determine the display order of product information when the video is played. The sequence number can be assigned according to the sorting order of the product display tags. For example, the first displayed product display tag is assigned a sequence number 1, the second displayed product display tag is assigned a sequence number 2, and so on.

[0118] It is understandable that this serial number is not only used to control the display order of product information, but also to track and record user interaction behavior. For example, the system can record the number of times the product information corresponding to each serial number is viewed or clicked by the user, thereby analyzing the user's purchase intention and preference.

[0119] Please refer to Figure 4 In the fourth embodiment of the commodity recommendation method proposed in this application, step S40 may include steps S41 to S43: Step S41: monitor the playing progress of the video in real time to determine the current playing time point.

[0120] In this embodiment, the system implements a real-time monitoring function for the video playback progress. The monitoring function can be implemented through the internal clock or time tracking mechanism of the video player, which can provide accurate playback time information. The system periodically queries the playback position of the current video by calling the player's API interface, usually expressed in the form of a timestamp, thereby obtaining the playback time value in seconds or milliseconds.

[0121] To achieve this monitoring, the system can also integrate event listeners or callback functions that are triggered when the video's playback time is updated. These listeners or callback functions are synchronized with the player's internal clock to ensure that the system can receive the latest playback time data in real time during video playback.

[0122] Through this real-time monitoring mechanism, the system can continuously track the progress of video playback and promptly capture any changes in playback time. This provides the necessary time benchmark for subsequent matching of playback time with product display time periods, allowing the system to display relevant product information at the correct time.

[0123] Step S42: Compare the playback time point with the display time period corresponding to the commodity instance in the video.

[0124] In this embodiment, the system compares the real-time acquired play time point with the display time period corresponding to the commodity instance recorded in the database. This comparison process can use a search algorithm with low time complexity, such as binary search or interpolation search, which can quickly locate the display time period of the current play time point in the sorted timestamp list.

[0125] Furthermore, the system can also use indexing technology to create indexes for display time period data to further improve query efficiency.

[0126] Step S43: When the playback time point is within the display time period, the display of the product information corresponding to the display time point is triggered.

[0127] In this embodiment, when the system detects that the current playback time point falls within the display time period of a certain product instance, it triggers a corresponding event and prepares to display the product information associated with the time period. This process may involve traversing or searching the time period data structure and real-time feedback of matching results; it may also involve dynamically adding or updating product display tags on the video playback interface. The system will display the product information to the user in an appropriate manner according to the previously set format and location parameters. For example, the product image and product information corresponding to the product instance can be displayed in a corner of the video, or more detailed product information can be displayed in the form of a pop-up window.

[0128] This embodiment effectively promotes products and encourages purchases by displaying product information associated with product instances in real time during video playback without interfering with the user's viewing experience. This method not only improves the accuracy and timeliness of product recommendations, but also enhances the user's viewing experience. Users can easily obtain detailed product information and make purchase decisions while watching video content.

[0129] The present application provides a product recommendation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the product recommendation method in the above-mentioned embodiment one.

[0130] Reference below Figure 5 , which shows a schematic diagram of the structure of a commodity recommendation device suitable for implementing the embodiment of the present application. The commodity recommendation device in the embodiment of the present application may include various hardware and software components for implementing the scheduling method of the inspection task. Figure 5 The product recommendation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0131] like Figure 5 As shown, the commodity recommendation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the commodity recommendation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the commodity recommendation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a commodity recommendation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0132] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0133] The product recommendation device provided by the present application adopts the product recommendation device method in the above embodiment, which can solve the technical problem of how to recommend relevant products in real time during the video content playback process. Compared with the prior art, the beneficial effects of the product recommendation device provided by the present application are the same as the beneficial effects of the product recommendation method provided by the above embodiment, and the other technical features in the product recommendation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0134] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0136] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the commodity recommendation method in the above-mentioned embodiment.

[0137] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0138] The computer-readable storage medium may be included in the commodity recommendation device; or may exist independently without being assembled into the commodity recommendation device.

[0139] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the product recommendation device, the product recommendation device: extracts key frames of the video and identifies product instances in the key frames; determines a display time period of the product instance based on a timestamp of the product instance appearing in the video; retrieves product information corresponding to the product instance from a product database and associates the product information with the display time period; and displays the product information corresponding to the display time period on a playback interface when it is detected that the video is played to the display time period corresponding to the product instance.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned commodity recommendation method, and can solve the technical problem of how to recommend relevant commodities in real time during the playback of video content. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the commodity recommendation method provided by the above-mentioned embodiment, and will not be elaborated here.

[0144] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned product recommendation method when executed by a processor.

[0145] The computer program product provided in this application can solve the technical problem of how to recommend related products in real time during the video content playback process. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the product recommendation method provided in the above embodiment, which will not be repeated here.

[0146] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

[0147] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0148] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0149] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A product recommendation method, characterized in that: The product recommendation method comprises: Extracting key frames of the video and identifying product instances in the key frames; Determining a display time period of the product instance according to a timestamp of when the product instance appears in the video; Retrieving the commodity information corresponding to the commodity instance from a commodity database, and associating the commodity information with the display time period; When it is detected that the video is played to the display time period corresponding to the product instance, the product information corresponding to the display time period is displayed on the playback interface.

2. The commodity recommendation method according to claim 1, characterized in that: Before the step of extracting the key frames of the video and identifying the commodity instances in the key frames, the method includes: Extracting image feature data of commodities, and matching the image feature data with commodity identifiers to form a commodity feature library; Using the image feature data in the product feature library to train a product recognition model; The trained product recognition model is used to recognize the product instance in the key frame.

3. The commodity recommendation method according to claim 1, characterized in that: The step of determining the display time period of the commodity instance according to the timestamp at which the commodity instance appears in the video comprises: Determine a start time point and an end time point of display of each commodity instance according to a timestamp of each commodity instance appearing in the video; A display time period of the commodity instance is determined based on the display start time point and the display end time point.

4. The commodity recommendation method according to claim 3, characterized in that: The step of determining the start time point and the end time point of display of the commodity instance according to the timestamp of each commodity instance appearing in the video further includes: Based on the timestamp, calculating the duration of appearance of the commodity instance in the video; According to the preset display rules, a basic display duration is set for the product information; If the continuous appearance time of the product instance is less than the basic display duration, setting the display end time point of the product information to the start display time point of the product instance plus the basic display duration; If the continuous appearance time of the product instance is greater than or equal to the basic display duration, the display end time point of the product information is set as the end point of the continuous appearance time of the product instance.

5. The commodity recommendation method according to claim 1, characterized in that: The step of retrieving the commodity information corresponding to the commodity instance from the commodity database and associating the commodity information with the display time period comprises: Obtaining a product identifier of the product instance; Using the commodity identifier, query and retrieve the commodity information corresponding to the commodity identifier in the commodity database; An associated data structure is created, and the product information and the corresponding display time period are taken as a key-value pair and stored in the associated data structure.

6. The commodity recommendation method according to claim 5, characterized in that: After the step of creating an associated data structure and storing the product information and the corresponding display time period as a key-value pair in the associated data structure, the method further includes: Creating a product display tag for each product instance, wherein the product display tag includes the product information and the corresponding display time period; Setting the display format and display position of the product display label in the playback interface; When the display time period is reached, the display of the product display label is automatically triggered based on the display format and the display position.

7. The commodity recommendation method according to claim 1, characterized in that: The step of retrieving the commodity information corresponding to the commodity instance from the commodity database and associating the commodity information with the display time period further includes: Obtaining the start display time points of all the product instances, and sorting the product display tags corresponding to the product instances according to the start display time points; A display sequence number is assigned to each of the product display tags, and the display sequence number is used to determine the display order of the product information when the video is played.

8. The commodity recommendation method according to claim 1, characterized in that: The step of detecting that the video is played to the display time period corresponding to the product instance, and displaying the product information corresponding to the display time period on the playback interface includes: Monitor the playback progress of the video in real time to determine the current playback time point; Comparing the playback time point with the display time period corresponding to the commodity instance in the video; When the playback time point is within the display time period, the display of the product information corresponding to the display time point is triggered.

9. A commodity recommendation device, characterized in that: The commodity recommendation device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the commodity recommendation method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is a computer-readable storage medium, and a computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the commodity recommendation method according to any one of claims 1 to 8 are implemented.

Images